By Mehdi HassanHow do you explain AI decisions to users in the interface?
Explain AI decisions by answering four questions in plain language, right next to the result: why this, based on what, how sure, and what can I do. Most users don't need to know how the model works. They need enough to decide whether to trust this particular result and what to do next.
Explainability is often treated as a technical feature: feature weights, attention maps, confidence scores. Those help ML engineers. For non-technical users, the explanation is a piece of writing, and it should be written by someone who writes for a living, working with the people who know the model. That's the angle of this guide. It's part of our hub on AI product UX.
What does explainable AI UX need to show?
Explainable AI UX needs to show four things: the main reason for the result, the evidence behind it, how confident the system is, and how the user can change or challenge the outcome. Each answers a question users ask, often silently, when an AI decides something for them.
User question | What to show | Example |
|---|---|---|
Why did the AI do this? | The one or two reasons that mattered most | Flagged because the amount is far above this supplier's usual invoices |
Based on what? | The inputs or sources, linked | Based on your last 12 invoices from this supplier |
How sure is it? | A named confidence level and why | Needs review: the invoice date is unclear |
What can I do? | How to change, override or challenge it | Mark as expected, or change the usual amount |
What would change it? | The input that would flip the result | This wouldn't be flagged under your set threshold |
Not every result needs all five. A suggested tag needs a short reason at most. A declined application, a flagged transaction or a medical triage result needs the full set, and may have legal requirements around it. Match the depth of the explanation to the stakes of the decision.
How do you write AI explanations in plain language?
Write AI explanations as short sentences that name the specific reason, use the user's words for things, and avoid model vocabulary. Because the model weighted features X and Y explains nothing to a non-technical user. Because this order ships to a new address and is larger than your usual orders explains everything.
- Lead with the reason, not the mechanism. Recommended because you often book morning flights, not Based on your preference profile.
- Name real things. Use the user's labels for fields, records and people.
- Cap the list. Show the top one to three reasons. More looks thorough and reads as noise.
- Avoid false precision. Don't show 87.3% confidence when the number isn't meaningful to the user.
- Stay neutral. Explanations describe; they don't sell the AI or apologize for it.
Explanation templates you can adapt
Recommendation: Suggested because [main reason]. [Optional second reason]. Not right? [Action].
Flag or alert: Flagged because [specific difference from normal]. [Evidence link]. Mark as expected or Review.
Classification: Tagged as [label] because it mentions [key phrase] and came from [source].
Low confidence: I'm not sure about [specific part] because [reason]. Please check [where].
Decision with a threshold: This was [outcome] because [value] is above your limit of [limit]. Change the limit in [settings link].
No reason available: This result is based on overall patterns, not one specific reason. See what it used.
How should an AI confidence indicator work?
An AI confidence indicator should use a few named levels tied to an action, such as Ready, Needs review and Not sure, each paired with a short reason. Users can act on Needs review: two sources disagree. Most can't act on 0.73, because a bare number doesn't say what to check or how good is good enough.
Keep levels few and meaningful, and make sure each one changes what the interface does. Ready might allow one-click use. Needs review might highlight the uncertain part and ask for a check. Not sure might leave a field empty rather than guessing. Pair color with icons and words so the levels work for color-blind users. Calibrate the levels with your ML team, so Ready actually means the system is reliably right on that kind of task. Our guide to AI product dashboard design shows how these levels sit next to sources and controls on a dashboard.
Where should AI explanations appear in the UI?
AI explanations should appear right next to the result, as a short line with an option to see more. Put the main reason in view and the full detail one click away. Explanations hidden in a separate screen or a help article don't get read when the user needs them.
Inline reason
A short line under or beside the result: Suggested because you often work with this client. Always visible for decisions with consequences.
Why? link or tooltip
For lower-stakes results, a small Why? that reveals the reason on demand keeps the screen clean.
Detail panel
For high-stakes decisions, a panel with all reasons, linked evidence and the confidence level. Our guide to AI citations UI covers how to show evidence.
Action next to the explanation
Override, correct or change the setting, placed where the explanation is, so understanding leads straight to control.
The same structure works in agent products, where the question is why the AI took an action rather than why it produced a result. There, the explanation lives in the action log and on the approval card.
How do you show users what would change the AI's result?
Show what would change the result by naming the input or setting that drove the decision, and letting users adjust it. This kind of explanation, often called counterfactual, is the most actionable: it tells users not only why, but what to do about it.
Pros
- Turns an explanation into a clear next action
- Helps users correct wrong inputs the AI relied on
- Makes thresholds and rules visible and adjustable
Cons
- Not every model can produce reliable what-would-change answers
- Can expose rules you'd rather keep private, such as fraud thresholds
- Needs careful wording to avoid sounding like a promise
Use it where the driving input is clear and safe to share: a spending threshold, a missing field, a date range, a preference. Avoid it where revealing the rule would help someone game the system. In those cases, explain the category of reason instead, and offer a way to ask for human review.
How do you test whether explanations work?
Test explanations by showing them to people who have never seen the model and asking them to say, in their own words, why the AI did what it did and what they'd do next. If they can't, the explanation isn't working, however accurate it is.
Run these tests with real results, including wrong and borderline ones, since explanations matter most there. Watch for users over-trusting a confident explanation of a wrong result; that's the riskiest failure. Store explanation patterns and confidence levels as shared components with copy rules, which our guide to design systems for AI products covers. If you want help writing and designing explanation patterns, our $3,000 brand & design sprint explores them in Figma as part of a fixed-scope project.
