AI Medical Record Review Vendor Scorecard: What IME and QME Teams Should Check Before Signing?

Before signing with an AI medical record review vendor, an IME or QME team should judge it on four things: accuracy traceable to the source page, turnaround that fits the exam date, independently audited security, and a clean, counted record. Three more decide whether the report holds up: a human check before delivery, a scope that leaves the records summary with the evaluator, and output in your own layout.

An AI vendor’s mistakes don’t end up under its name. They end up under yours. Every medical record review vendor promises accurate summaries, fast turnaround and secure handling, but for an IME/QME the record review sits beneath a signed report. In California, the evaluator verifies the page count under penalty of perjury, and can lose a report entirely, unpaid, if someone else did work the law reserves to the physician.

This post turns those promises into a scorecard you can test yourself, one claim at a time. Start with accuracy because it is the easiest claim to make and the hardest to test. A vendor can put 99% accuracy on its website, but that figure comes from its own testing, on records you never see.

AI gets a medical record wrong in three ways. It can hallucinate, creating a detail the record never contained, such as a diagnosis nobody made. It can misinterpret, reading a real entry the wrong way: a finding credited to a different visit, a restriction applied to the wrong body part, a family history entry mistaken for the patient’s own condition. Or it can omit, leaving out an entry the evaluator needed, such as treatment to the same body part years before the claimed injury.

The errors are measurable. In a 2025 study in npj Digital Medicine, clinicians checked AI-written clinical notes line by line, and found hallucinations in 1.47% of sentences and missing information in 3.45%. The rates look small, percentage wise, but a 400-sentence summary may carry six invented sentences, and nothing in the output tells the evaluator which six. If they reach the report, they reach it under your signature.

Accuracy is only the first place a review can go wrong, though. Some of the most expensive failures have nothing to do with the model, and the costliest is about who did the work.

How can a record review vendor cost you a report?

The first failure is output that nobody checked, or that someone else prepared without the evaluator accounting for it. In California, Labor Code 4628 reserves the review and summary of prior records to the physician who signs the report. Someone else may prepare the initial outline of history and the record excerpts, but the evaluator must review that work and disclose who did it.

Gurrola Martinez v. H & H Wallboard, a WCAB panel decision from August 2025, shows what is at stake. The defense moved to replace the QME after learning that a third-party service had summarized the medical records for his reports. The trial judge found a violation of Labor Code section 4628 and ruled the reports should be stricken. The board sent the case back so the QME could be heard first but called it very likely that the judge had it right and noted that a stricken report goes unpaid. In that case, there were six of them.

The second failure is practical: the records come in messy from employers, carriers, providers and subpoenas, often overlapping and out of order. Duplicate copies inflate the page count, files with no bearing on the dispute crowd in the chronology, and missing records leave gaps nobody sees until the exam or the deposition.

The third failure is a fixed layout. TPAs, IMEs, and QMEs often want their own format. When a tool forces every case into the same structure, the details a particular dispute depends on can end up under headings nobody reads closely, and the evaluator rebuilds the picture by hand before the review can start.

Each of these failures is fixable, and the fix sits in the vendor’s process: who does which part of the work, how the record gets cleaned, and how much say the evaluator has over the output.

What fixes it: a human check, a clear boundary and a layout you control

In a dependable setup, software does the volume work. It converts the scans, removes duplicates, arranges the pages chronologically and by provider, and pulls out entries that the case calls for. This includes prior treatments, index events, imaging findings, and treating physicians’ endpoints and restrictions. Then a person checks the work. A human-in-the-loop review is a workflow in which a qualified reviewer verifies AI output against the source pages. It guards against both hallucination and misinterpretation, because a trained reader can catch what a model states with false confidence and notice what is left out.

The second part of the fix is a boundary. A vendor can sort, de-duplicate, index and excerpt the records. The review and summary that go into the signed report stay with the evaluator. A dependable vendor also supplies the disclosure so that the evaluator can show exactly who did what.

The third part is control over the output. The evaluator should be able to set the layout once for each referral source, and the point of review can change based on the requirements of the vendor rather than accepting a fixed template. The file then arrives ready for review, not for rework.

Four areas to check before you sign

The above three decide whether a report holds up. The four basics below are what every vendor claims, so the point is to test each one yourself.

Accuracy you can confirm yourself

Ignore the headline percentage and test the output directly. Every line should link to the page where it was drawn from. Pick three lines in a sample file and click through to the source. If each one lands on the right page, the vendor’s accuracy is something you can check on every case, which matters more than any number on a sales page. You can also run the test in reverse: pick one page that has a crucial injury or imaging study and check if that is made it into the output. This is how you catch an omission.

Turnaround that fits the exam date

A record set is only useful if it is ready before the evaluation. Ask for the standard turnaround in writing and ask what happens in a rushed case. A rush is when a human check is easiest to cut, and cutting it brings back every risk described above. A good answer keeps the reviewer in place at every speed and lets you move a file up the queue based on priority date.

Security you can see in writing

A full medical history is among the most sensitive data a practice holds. If your practice is a HIPAA covered entity, the Privacy Rule requires written assurances, usually a Business Associate Agreement, from any vendor that handles that information for you. The 2024 ransomware attack on Change Healthcare, a claims and payment processor used across much of the U.S. health system, shows how far a vendor breach can reach: it exposed the data of 192.7 million people, the largest healthcare breach in U.S. history.

Beyond the agreement, look for independent proof. A SOC 2 Type II report is an outside auditor’s review of how a vendor’s security controls actually operated over a period of time, and ISO 27001 is an international standard for managing information security, certified by an external body. These are important security standards to keep in check before making your decisions.

A record that comes back clean

In California, record pages are billing pages. Under the Medical-Legal Fee Schedule, the evaluator verifies under penalty of perjury how many pages of records were reviewed. The DWC’s quality checklist for med-legal reports builds that attestation into the report, and says to get a corrected declaration from the sending party if their page count is wrong. So ask a vendor for a counted page total on a sample file, broken out by source, with duplicates listed separately, and check it against the declaration that came with the records. Then check whether missing or unreadable pages get flagged before the file reaches you, not after the exam.

How WHITE AI checks out on the scorecard

Rapid Care built WHITE AI for IME and QME around the evaluator’s workflow. It handles the record work and draws no conclusions. WHITE AI sorts, indexes and excerpts the records based on date of treatment. On sort and summarize, a clinician-reviewed working summary comes with the excerpts with reference. Opinions on causation and apportionment, and the narrative that ends up in the signed report, remain on the evaluator’s own.

Here is the full scorecard: it has the four basics, then the three checks that help you decide whether a report fully holds up. Use it for any vendor you consider. WHITE AI’s answers are in the last column.

What to check	AI-only record tools	Outsourced summary service	WHITE AI
The four basics
Accuracy traceable to the page	Page links in some tools; run the click-through test	Ask whether entries cite source pages	✓ Every entry hyperlinked to its source page.
Turnaround that fits the exam date	Minutes, with no human check	Often days to weeks on complex files	✓ 24-hour standard; Urgent files same day when page count allows.
Security you can see in writing	Depends on the plan; ask about a BAA and model training	Ask for a BAA and audit reports	✓ SOC 2 Type II, ISO 27001, HIPAA compliant; records never used to train models
A clean, counted record	Ask for a per-source page count	Ask for a per-source page count	✓ Duplicates kept out of the working set and listed; page total per source and overall; missing, illegible and low-confidence pages flagged for human QC
The three that decide it
A human check before delivery	Often none	✓ A reviewer prepares the work	✓ Trained reviewer, then QA, on Sort; clinician review on Sort & Summarize
A scope that leaves the summary with you	Ask what the tool drafts and what you must disclose	Ask who writes the summary; section 4628 reserves it to the evaluator	✓ Sorts, indexes and excerpts; working summary has references based on the source files only.
Output in your layout	Often fixed by the tool	✓ Usually built to your format	✓ Template can be set as per user’s need and individual case-by-case customization can be done too.

How we compared: AI-only tools and outsourced services are assessed as categories, not individual vendors from publicly available information as of October 2026. Individual vendors differ; and current features can change, so verify before relying on any single point.

Score WHITE AI on your own case

A vendor’s mistakes end up under your signature, so test one before you sign with it. Upload your records, run the checks, and see what comes back.