Audit teleradiology report quality: a 12-point scorecard

Most of the teleradiology reports that frustrate your physicians contain no error at all. The appendix was normal, the nodule was measured, nothing was missed; what failed was the report itself. You can measure that failure before you sign. A report quality audit scores a blinded sample of 20 recent final signed reports against 12 fixed criteria: does the impression answer the clinical question, were priors compared, does the follow-up recommendation carry an interval. One clinician can run it in about an hour, and it predicts your physicians' daily experience better than any turnaround statistic or reference call. Ask for the sample in writing and score it with the clinicians who receive the reports.

Why a report quality audit beats a reference call

A report quality audit produces evidence you generated, scored against fixed criteria, comparable across vendors; a reference call produces the vendor's three happiest customers. Turnaround is the easier number to collect. It's also the easier number to present favorably, because whoever holds the timestamps controls the story, and no timestamp records whether the impression answered the question.

The gap shows in how buyers behave. Physician leaders shopping for teleradiology coverage end up asking strangers on clinical forums for recommendations, which is what people do when they have no evaluation tool of their own. Read those same threads and the ordering clinicians are strikingly specific about what fails them: impressions that restate the findings without answering the question, hedges with no most-likely diagnosis, follow-up advice with no interval attached, priors that were sent and clearly never opened. Every one of those failures is visible in a written report before signature. Yet the standard pre-contract check remains the reference call, which can't surface any of it. References are picked by the vendor, they speak in generalities, and they never include the physician who was annoyed last Tuesday. A scored sample fixes all of that at once: you generate the evidence, you score it against criteria that stay fixed from vendor to vendor, and you keep a baseline to re-run at month six on the live account. An hour of one clinician's time buys the single measurement the rest of the sales process leaves vague.

This page turns one line of our teleradiology buyer's guide into a working method. Use the guide for the full RFP; use this for the one question an RFP can't answer on paper.

Ask for 20 reports in your own modality mix

Put the request in writing during the RFP, before pricing settles: 20 complete de-identified reports, in your actual modality mix by percentage, signed within the last 90 days by radiologists eligible to cover your account. Loose wording produces a marketing packet, so spell out the rest too. Ask for at least five STAT and five routine studies, since quality under time pressure is the true test, and for full report text including addenda, which show you how corrections are handled. Recency matters because old samples predate staffing and platform changes, and the mix matters because a vendor strong on plain film can run thin on cross sectional. The vendor de-identifies before sending; that's standard and protects both sides.

Two refusals tell you something. A vendor that declines to provide any sample has declined the only objective pre-contract quality check available. A vendor that hand-picks the reports while ignoring your stated mix and date range has curated the evidence, and the score means less.

Reasonable pushback looks different. Expect redaction, a mutual NDA, and a request for a few business days. A vendor can insist on all three and still meet your mix and recency spec.

Score each report against 12 criteria

Score every report 0, 1 or 2 on each criterion: 2 is a clean pass, 1 is partial, 0 is a fail, for a maximum of 24 per report.

Criteria 1, 2, 6 and 9 carry the most weight in practice, because they change what the receiving clinician does next. They are not house preferences either. The ACR Practice Parameter for Communication of Diagnostic Imaging Findings treats the report as the definitive means of communicating results, and asks that it answer the clinical question the examination was ordered for. It sets a separate expectation of non-routine, direct communication when a finding needs action sooner than the ordinary report cycle allows. Criterion 12 deserves special attention: a report that fails to declare its own status creates a billing and liability problem independent of its clinical content, and our page on preliminary, STAT and final reads explains why.

Set the thresholds before you score

Decide in advance what passing means: across 20 reports, a good service averages 20 of 24 or better, with no single report below 15. A report at 22 to 24 is directly actionable. One at 18 to 21 is acceptable, with minor friction and no safety issue. At 15 to 17 the report is usable, but your clinician is doing work the report should've done. Below 15 is failing, and two or more failing reports out of 20 sampled is a pattern you raise directly with the vendor. One zero on criteria 8, 9 or 11 outranks any average, because those are the criteria where the failure produces a wrong action downstream.

A failing report usually reads like this: the impression repeats the findings, ends with "clinical correlation recommended," names no most-likely diagnosis, gives no follow-up interval, and never mentions the prior study that was sent with it. Every sentence in it is defensible. It just isn't usable, and it will generate a phone call for every study.

Bring the failures to the vendor with report identifiers and the criteria that failed. How a vendor responds to a specific, documented complaint during the sales process is the closest preview you'll get of how it responds to one at month nine.

How often should you re-audit your teleradiology vendor's report quality?

Re-run the same audit every quarter on the live account, because report quality drifts with staffing, volume and ownership changes far faster than turnaround does. Pull 20 reports each quarter at random from your own PACS. Score them against the same 12 criteria, with the same scorers where you can, and trend two numbers: the average and the count of reports below 15. Review the trend with the vendor at the quarterly business review, right beside SLA compliance.

Write the audit into the contract while you still have leverage. A quality clause naming the sample size, the cadence and the response expectation costs nothing at signature and gives you standing later. Pair it with turnaround commitments in our published SLA: AstraRad contracts STAT under 4 hours, with 2-hour and 1-hour STAT on request at a higher multiplier, routine 24 to 48 hours, and stroke protocol under 30 minutes, each measured from last-image arrival to radiologist signature. A serious vendor is comfortable being held to report quality alongside those tiers.

One honest limit: if your facility sends only a handful of studies a month, a quarterly 20-report audit is more governance than the volume justifies. Score the pre-contract sample, then re-audit annually or whenever the vendor changes ownership. And if you're still weighing whether to keep reads in house, the same scorecard works on your own reports; that comparison is usually more honest than the cost model, and our in house versus outsourced analysis covers the rest of the decision.

Route what you find to the right process

An audit surfaces two kinds of problems, service defects and safety events, and they go to different places. Report defects, criteria 1, 2, 3, 6, 7, 10, 11 and 12, go to the vendor as documented service feedback with examples, and they should improve within a quarter. Safety defects, criteria 8 and 9, go to the vendor immediately and to your own risk management: a laterality error or an undocumented critical result callback is an event, not a preference. Suspected interpretive misses go into formal peer review, because the audit was never designed to adjudicate whether a finding was correct. That is the job ACR RADPEER exists to do. It's worth knowing that RADPEER publishes no scoring benchmark, so a vendor claiming its discrepancy rate meets an ACR target is citing something that does not exist.

That last distinction matters when you compare vendors on published quality numbers. A discrepancy rate tells you whether the read was right. The scorecard tells you whether the report was usable. Ask for both, and ask how the vendor's own peer review runs. AstraRad's protocol independently double-reads 1 in 20 signed reports, blind, and every major discrepancy is reviewed at the monthly discrepancy meeting and closed with the reader. Details are on how we work and our radiologists.

For the broader method of pressure-testing what a vendor tells you in a sales process, see verifying teleradiology vendor claims. A report quality audit is the narrow, repeatable piece of that work, and you can run it on us. Book a coverage consultation and ask for 20 de-identified recent reports in your modality mix, signed by the fellowship-trained subspecialists on the panel that would cover your facility. The report is the product. Everything else in a sales process is a description of it.