A pilot that starts with a demonstration and ends with an accuracy figure leaves the important questions unanswered. Agreeing what a useful result looks like before the test starts is the most valuable part of the exercise.
Define the intended use
State who will use the tool, on what inputs, for what decision, and what happens when it is wrong. A tool that suggests candidate queries for a medical reviewer to accept or reject has a different risk profile from one whose output goes straight into a document.
Choose reference cases deliberately
Include typical cases and the difficult ones: missing data, conflicting dates, unusual toxicity and records that have been corrected. Have qualified clinicians review the same cases independently so there is a reference to compare against, and decide in advance how disagreements will be settled.
Count the errors that matter
Distinguish missed findings from false alarms, and classify each by clinical consequence. A tool that misses a rare but serious event is not equivalent to one that flags an extra harmless discrepancy. Report both, with their severity, rather than a single percentage.
Measure the reviewer’s effort
A tool that produces correct suggestions but needs a long verification step may save less time than it appears to. Time the review with and without the tool, on the same cases.
Keep humans accountable
Decide who reviews every output, what evidence they must see and what the tool is not allowed to do. For data, synthetic or de-identified cases are a sensible starting point; the HHS guidance on de-identification explains the standards that apply to health information in the United States.
Public reference points
Related service: Clinical AI evaluation.