Season · AI, Trust & Human Oversight · 10 of 16
A quick answer can leave little room to learn.
Concept of the Week · Leave room for your own thinking
To keep using their judgment, people need opportunities to form and explain their own assessment. When an assistant provides the first explanation, their work can shift from generating possibilities to reviewing someone else's answer.
Design rule: Make room for an initial assessment before showing suggestions, while keeping help immediately available.
In an intensive care unit (ICU), a physician reviews a patient's falling blood pressure to determine the cause while the team continues urgent care. A diagnostic AI assistant suggests sepsis as a possible explanation before the physician has worked through the available findings. The physician may focus on confirming that suggestion rather than generating alternatives. Checks for other causes of the falling blood pressure could receive less attention, leaving the initial assessment incomplete.
A different interface could show the observations and results first, with space to record a provisional hypothesis, its basis, and what remains uncertain. The physician could then reveal the assistant's suggestions and compare them with their own assessment. Supporting and conflicting findings would help them decide what to reconsider or investigate next. Help must also be available immediately without an entry; recording a hypothesis must never delay assessment, escalation, or treatment.
The assistant can gather relevant information and suggest possibilities to examine. Its first assessment should use the clinical data separately from the physician's newly recorded hypothesis, so the comparison does not simply echo that hypothesis. The anchoring effect means sticking too closely to an initial explanation. It can also affect the physician's view, so revising that view must be easy.
Keeping the initial assessment visible during comparison lets the physician explain what changed their judgment rather than reconstructing it afterward.
Ship this by designing and prototyping the optional assessment field and immediate help, evaluating them with ICU physicians using realistic simulated tasks, revising based on feedback and observed difficulties, then retesting against defined usability and safety criteria before release.
From an ICU physician seeing a suggested explanation first to forming an initial assessment from the findings, with assistance available on request.
Asset & Resource
Design and human factors teams can use this Reasoning Review to decide where an assistant enters a reasoning task; revisit it when the task, model, or suggestion timing changes.
Name the thinking: Identify what the person should notice, connect, and explain themselves.
Make space for a first view: Show the source information beside a short, optional assessment field.
Keep help within reach: Let people request suggestions immediately, including when they cannot form a hypothesis.
Support a real comparison: Keep both assessments visible, preserve conflicting evidence, and make revision easy.
Check what remains independent: Evaluate later performance on different cases without suggestions, separately from assisted task completion.
Use the review to spot missed alternatives, unclear evidence, and extra documentation work during realistic simulated tasks with representative users. Revise the prototype from those findings and retest against defined usability and safety criteria. Assess later independent reasoning separately from successful use of the assistant.
Google's pedagogical framework gives the design rationale: actively forming an explanation and reflecting on it should remain part of the work. Its learning principles inform this proposal, not a rule to withhold clinical support.
Now available · UX template for Episode 42
I'm happy to share my first CalmDesign UX template, Work With and Without AI, for the last episode, CALM: Design for the Day AI Fails. More will follow.
The fillable template helps human factors and design teams design and evaluate how people complete a task when AI is unavailable. It includes a blank page for your use scenario and a working example of a physician preparing for morning rounds in the ICU.
Your confirmed CalmDesign subscription already gives you access. Enter the email address that receives this newsletter to get a sign-in link. No new subscription is needed.
Light Wisdom & Reflection
“For the things we have to learn before we can do them, we learn by doing them.”
Which part of the thinking does your interface still leave to the person using it?
Mindful Practice
When an everyday question comes to mind, pause for thirty seconds before looking it up or asking someone.
What do you remember about it?
What might the answer be?
Next time, notice what comes to mind when you give yourself a moment.
Best,
Andreas Walden
Share this with someone designing how assistants support human judgment.
Continue with CALM: Design for the Day AI Fails.