Evaluate an AI research workflow with a small reference set
Evaluate an AI research workflow using a small reference set built from tasks you actually perform. Define the expected evidence, unacceptable errors and required unknowns before running the system. A polished answer to a convenient example is not a workflow evaluation, and a few successful examples cannot establish a general accuracy rate.
Define the unit being evaluated
Are you testing extraction from a scan, matching two reporting periods, checking a citation or drafting a memo? These tasks fail differently. Record the input documents, prompt, tools and relevant settings for each test. A workflow includes the input preparation and retrieval process, not just the model name shown in an interface.
The NIST AI Risk Management Framework provides a risk-management framework for AI systems. The reference-set method here is an editorial implementation for personal research, not a NIST certification or a claim that one checklist satisfies the framework. Its purpose is to make visible what you tested and what an error would mean in the actual research task.
Build a reference sheet from original material
| Case type | Expected evidence | Failure to watch for |
|---|---|---|
| Explicit value | Value, unit, period and page | Correct number with wrong scale |
| Missing metric | Unknown within stated scope | Invented value or unjustified zero |
| Conflicting sources | Both versions and their dates | Silently selecting the more convenient answer |
| Qualified statement | Speaker, condition and uncertainty | Turning a plan into a completed event |
| Unreadable table | Extraction failure and location | Plausible reconstruction presented as reported data |
Choose cases relevant to your intended use rather than adding unrelated difficult puzzles. For each, manually check the original passage and record why the expected answer is acceptable. If you cannot confidently determine the reference answer, mark the case unresolved and exclude it from a simple pass count. Ambiguous references make an apparently precise score misleading.
A hypothetical six-case evaluation
Assume you prepare six invented cases: two numeric extractions, a missing metric, a conditional statement, a source revision and an unreadable scan. The workflow passes five but confidently invents a number for the scan. Saying it is “83 percent accurate” hides the error most likely to contaminate your evidence log. Instead, report that five defined cases passed and the unreadable-input case failed by fabrication.
Set a release rule tied to your task. For a workflow that fills research tables, an unsupported numeric value can be a stop condition even when wording quality is good. For a draft that will be manually checked line by line, the same defect may permit limited use while requiring a mandatory source review. Declare the rule before reviewing results so you do not relax it merely because the output sounds useful.
Run comparisons without changing the target
Keep one reference set stable while comparing a prompt or input change. Record each answer and the exact evidence used for grading. If you fix a prompt after seeing a failure, rerun the failed case and the other cases; otherwise you may solve one problem while introducing another. Cases used repeatedly for tuning should be distinguished from fresh cases reserved for a later check.
Do not ask the same system to be the sole judge of its own unsupported claims. Automated checks can validate required fields or arithmetic, while a person checks material meaning against the source. Disagreements should preserve both the proposed grade and the reason for the final decision. This creates a useful defect history rather than a score with no inspectable basis.
Understand what the result does not establish
A small reference set identifies known failure patterns and supports a bounded deployment decision. It cannot estimate performance across all issuers, languages, document formats or future model updates. State the sample and task limits alongside the result. Add a new case when a real failure occurs, and recheck after changing the pipeline. Evaluation is a record of evidence about a workflow, not an assurance that future answers need no checking.
FAQ
Should I grade only whether the final number is correct?
No. Require correct units, period, source position and handling of missing information. A correct-looking number with an unsupported interpretation can still fail the task.
How big must the reference set be?
Start with the failure types that matter for your workflow. Small sets are diagnostic; do not present their pass fraction as a statistically validated general error rate.