Medical record summary quality checks: the 12-point list
Answer in brief: before you sign a medical record summary, check that it answers the question, that every fact has a page, that the lines your opinion rests on match their pages word for word, and that you searched for what is missing: prior history, referenced records that never arrived, conflicts in side or date. The 12 checks below apply to hand and AI summaries alike; 4 are must-pass.
Limits: We have not timed the listed tools on a shared test file: time figures come from the cited studies and vendor pages, and the worked example is synthetic.
Why the checks look for gaps first
Most people check a summary by reading it and asking whether anything looks wrong. That catches invented facts and typos. It does not catch what isn't there, and in 2 published studies of AI-written clinical summaries, what isn't there was the bigger problem.
The second study, of 12,999 sentences in LLM-written clinical notes, found hallucinations in 1.47% of sentences and omissions of 3.45% of source sentences. Fewer hallucinations, but 44% of them were rated major, and 30% were negations: the note said the opposite of the source (npj Digital Medicine 2025). So the checks below do 2 things: confirm what is there against its page, and search for what should be there.
Both studies are about clinical notes and hospital summaries, not medical-legal files. They show which errors to expect, not how often you will meet them.
The 12 checks, with a score
Tick each check you have done on the summary in front of you. Checks marked must-pass block signing on their own. The score runs in your browser; nothing is saved or sent.
0 of 12 checks done. 4 must-pass checks open: not ready to sign.
How to run the checks in under an hour
Most of the time goes into checks 5 and 9. A practical order:
- List the lines your opinion rests on (check 5). For an IME, that is usually the mechanism, the key imaging, each procedure, every work status change and any prior history. Open each cited page.
- Pick 10 other lines at random (check 6). Number the lines and use any random method. If 1 fails, check every line in that section, since errors cluster where the source was hard to read.
- Search, don't read, for what is missing (checks 8 to 10). Search the file for "ordered", "referred", "pending", "outside records" and the names of tests; confirm each has a result or a flag. Filter the index for dates before the event.
- Scan for side and negation (check 7). Search for "left", "right", "denies" and "negative" on the key findings and compare with the summary.
Assembly Software suggests spot-checking about 10 entries when you review an AI chronology (assemblysoftware.com, read 2026-11-12). Check 6 uses that number as a floor, on top of the targeted check of every line you rely on.
Common failures, by check
| Check | Typical failure | Example (synthetic) |
|---|---|---|
| 3, 5 | A line cites a real page that doesn't support it | "EMG confirmed left S1 radiculopathy (p. 29)"; p. 29 only orders the EMG |
| 7 | Side flipped from a stray note | MRI summarized as "right S1" because 1 PT note said right leg |
| 7 | Negation lost | "Denies numbness" summarized as "numbness" |
| 9 | Prior history missed because it sits in an old PCP note or a pharmacy printout | Naproxen fills for back pain 2 years before the injury |
| 10 | Referenced record never requested | Surgeon's note mentions "outside MRI 2024", not in the file |
| 12 | Only the latest work status recorded | "Full duty" with no note of 5 months of modified duty before it |
The first 3 rows are worked through on a full synthetic file on the worked example page.
Questions about checking a summary
How do you check a medical record summary for accuracy?
Open the cited page for every line an opinion rests on and confirm it says the same thing, spot-check at least 10 other lines at random, and then search separately for what is missing: prior history, referenced records that never arrived, and side or date conflicts.
What is the most common error in AI medical record summaries?
Omission. Physicians reviewing 100 AI hospital course summaries reported omissions in 25, inaccuracies in 20 and hallucinations in 2 (JAMA Network Open 2026). Reading the summary alone won't find omissions; you have to search the file for them.
Do I need to check a summary I wrote by hand?
Yes, with the same list. Hand summaries skip pages late in long files and often lack page numbers, which fails check 3 before anything else.
What does "must-pass" mean here?
The 4 checks that, if skipped, make the summary unsafe to sign whatever else is done: the question answered, a page on every fact, the relied-on lines verified, and a search for prior history.