Season · AI, Trust & Human Oversight · 13 of 16
A new result does not explain what changed.
Concept of the Week · Make the difference understandable
When an AI tool re-analyzes the same recording, people need to understand how the new result differs from the version they already reviewed. Showing both versions together, with human changes clearly identified, supports that judgment without requiring people to reconstruct it from memory. The earlier assessment remains available for comparison while the person decides what should apply now.
Comparison rule: Display the new AI result alongside the previously reviewed version, with differences and human changes clearly identified.
In an outpatient cardiology clinic, a cardiologist reviews a long-term electrocardiogram (ECG) before signing the report, checking a section that an AI analysis tool has marked as possible atrial fibrillation, an irregular heart rhythm. After comparing the recorded channels, the cardiologist changes that section’s label from possible atrial fibrillation to recording artifact, meaning interference in the recorded signal.
They correct an incorrectly imported ECG channel name and request a fresh analysis. Before the rerun, the system warns that it will replace existing classifications. The new analysis again marks the same section as possible atrial fibrillation. Without the earlier reviewed version in view, the cardiologist may need to reconstruct why that section changed before deciding whether to include the new result in the report.
A calmer view could display the previously reviewed version and the new AI result at the same recording time, identifying the original AI label, the human correction, and the new classification. With the differences and the reason for reanalysis visible, the cardiologist can re-evaluate that section before approving the report.
Separating the review status of each version could help the cardiologist see which classification still needs checking before signing the report.
Ship this by designing the comparison with cardiologists, testing whether they can explain each version and make the next review decision in realistic tasks, then revising and retesting until agreed usability and safety criteria are met.
From a cardiologist interpreting a new ECG analysis alone to comparing it directly with the previously reviewed version and its human changes.
Asset & Resource
Design and human factors teams can use the AI Analysis Comparison Review with clinical colleagues when designing how a repeat analysis is presented. Revisit it when analysis inputs, classification, or report review change.
Show both versions: Display the new AI result alongside the previously reviewed version at the same point in the recording.
Explain why the analysis was repeated: Identify which input or setting changed and why a new result was requested.
Separate AI output from human changes: Identify the original AI label, the recorded correction, and the new AI result.
Highlight meaningful differences: Make classification changes clear while keeping the source signal available for assessment.
Keep the next decision explicit: Show what remains unresolved and let the cardiologist record the current judgment before report approval.
Use the AI Analysis Comparison Review alongside the interface and realistic review tasks, including interrupted comparisons. Check whether cardiologists can explain what changed, which assessment each label represents, and what remains to be decided. Use any misunderstandings to revise and re-evaluate the design.
The Guidelines for Human-AI Interaction recommend making AI errors easy to correct and making the consequences of user actions clear. The AI Analysis Comparison Review applies these general principles to a proposed ECG comparison, but cardiologists still need to evaluate its usefulness.
Light Wisdom & Reflection
“What’s past is prologue”
Which earlier judgment would a colleague need to see before deciding differently?
Mindful Practice
At home, look at a well-used recipe with a handwritten adjustment in the margin.
What does the note bring back that the printed instructions do not?
How does it feel to follow it today?
Next time, notice how a small trace of an earlier choice can invite attention without deciding for you.
Best,
Andreas Walden
Share this with someone designing how people compare AI results with work they have already reviewed.
For work passed to another reviewer, revisit how to keep unchecked statements visible at handover.