Algorithm Aversion and Trust Repair: What Happens After Your AI Gets It Wrong (2026)

Users abandon an AI faster than they abandon a human who makes the identical mistake. This is not a perception problem you can fix with better copy, and it is not solved by making the model more accurate — it persists even when people have watched the algorithm outperform the human. It is a well-replicated finding with a specific name, a specific mechanism, and — unusually for a behavioural finding — a specific, cheap intervention that works.

The intervention is not what most teams reach for. It is not more explanation, more confidence display, or a better apology. It is giving the user a small amount of control over the output. In the canonical experiment, that single change raised the share of people willing to use an imperfect algorithm from 32% to 76% — and the size of the control made almost no difference.

This guide covers what the research actually shows, what it means for AI product design, and how to research trust repair after your own AI fails.

The finding: algorithm aversion

Dietvorst, Simmons, and Massey named the effect in Journal of Experimental Psychology: General (2015, 144(1), 114–126). Across five studies, participants saw an algorithm make forecasts, a human make forecasts, both, or neither, then chose whose predictions to tie their own incentives to.

The result is counterintuitive in a way that matters commercially. In the authors' words, people are "especially averse to algorithmic forecasters after seeing them perform, even when they see them outperform a human forecaster." The mechanism they identify: "people more quickly lose confidence in algorithmic than human forecasters after seeing them make the same mistake."

Read the sequencing carefully, because it inverts the usual product intuition. Participants who had no information about the algorithm were generally happy to use it. Exposure to its performance is what destroyed adoption — including exposure to performance that was better than the human alternative.

Two consequences follow directly:

1. Your demo is not your product. Adoption measured before users have seen a failure tells you almost nothing about adoption after. Pilot enthusiasm that collapses in month two is the expected pattern, not a surprise.
2. Accuracy improvements have asymmetric returns. Going from 85% to 92% accuracy does not buy you proportionally more trust, because the remaining failures are still weighted far more heavily than a human's identical failures would be.

The intervention: give people a little control

The same authors published the constructive counterpart in Management Science (2018, 64(3), 1155–1170), and the numbers are strong enough to change a roadmap.

In Study 1, participants forecast students' standardised maths test scores. They were told plainly that the statistical model was imperfect — off by 17.5 percentiles on average. The only manipulation was whether they could adjust its output.

A 41-point swing, from an affordance rather than a model improvement.

Study 2 is the one that should change how you scope the feature. With 816 participants, the researchers varied how much adjustment was permitted — 10, 5, or 2 percentiles:

The three adjustment conditions did not differ significantly (χ²(2, N = 623) = 0.42, p = 0.809). As the authors put it, willingness to use the model "was not detectably altered by imposing an 80% reduction of the amount by which they could adjust."

This is the single most actionable finding in the algorithm-aversion literature. People are not seeking sufficient control. They are seeking some control. A tightly bounded edit affordance buys you nearly all of the adoption benefit of an unbounded one.

And it does not cost you accuracy — it gains you accuracy. Because more people used the model, average absolute error fell from 22.34 in the cannot-change condition to 20.87, 20.10, and 20.75 in the adjust-by-10, -5, and -2 conditions. The constrained conditions also kept people closer to the model: participants who could adjust freely deviated by 8.18 percentiles on average, while those constrained to ±10 deviated by only 4.71.

The design implication is precise: let users edit the AI's output, and bound the edit. Bounded editing outperforms both no editing (adoption collapses) and unbounded editing (people drift too far from a model that was better than them).

Where repair stops working: the three-strike ceiling

Adoption is one problem. Recovery after a visible failure is another, and here the news is harder.

Esterwood and Robert (Computers in Human Behavior, 2023, vol. 142) ran 240 participants through a task with a robot colleague that made mistakes, testing four repair strategies drawn from the interpersonal trust literature: apologies, denials, explanations, and promises of trustworthiness.

The headline result: after three violations, none of the strategies fully restored trustworthiness. As Robert summarised it, the results support the adage "three strikes and you're out."

Two secondary findings are useful:

• Denial is consistently the worst strategy. Denying a failure the user observed damages trust further. This should be obvious and yet ships regularly, in the form of error messages that reframe the system's failure as the user's input problem.
• Apologies, explanations, and promises perform similarly after repeated interactions — which means the choice between them matters less than whether you have any coherent repair at all.

Follow-up work integrating theory of mind (Scientific Reports, 2023) found that repair effectiveness is moderated by how users perceive the system's mind: apologies work better when the system is perceived as capable of emotion, explanations work better when users attribute intentionality to it. Practically: the more your product presents the AI as an agent with a persona, the more an apology is expected; the more it presents as a tool, the more an explanation is expected. Mismatching these reads as insincere.

The operational takeaway is a budget, not a technique. You get roughly three visible failures per user before repair stops working. That reframes the reliability conversation: the question is not "what is our error rate" but "how many of our errors are visible and attributed to the AI, per user, per month?" A system with a 5% error rate that surfaces failures gracefully may spend fewer strikes than a 2% system that fails loudly.

Designing the failure itself

Google's People + AI Guidebook is the strongest practitioner source here. Its central reframe: "The trick isn't to avoid failure, but to find it and make it just as user-centered."

Three of its principles map directly onto the research above:

• Return control. "When an AI system fails, often the easiest path forward is to let the user take over." This is the same lever as bounded editing — the user needs a route to the outcome that does not depend on the model.
• Give a path forward. "Focusing on what your users can do after the system fails empowers them while maintaining usefulness."
• Own it plainly. "Address mistakes with humanity and humility, and explain the system's limits while inviting people forward." This is the anti-denial principle stated as copy guidance.

The Guidebook also notes that "many of the errors your users experience require their feedback in order to improve the system" — which is the honest case for asking, at the moment of failure, rather than inferring from telemetry.

What to measure

Most teams measure trust with a satisfaction question, which is close to useless here because satisfaction is dominated by whether this task succeeded. Four better measures:

Attribution is the one teams overlook and the one that predicts churn. A user who concludes "I phrased it badly" costs you nothing. A user who concludes "this thing makes things up" has generalised from one failure to your whole product — and, per the algorithm-aversion research, that generalisation happens faster for AI than it would for a human colleague.

Do not use per-session measurement. The three-strike effect is cumulative across a relationship. A dashboard that resets every session will report healthy numbers right up until the user leaves.

How to research this without waiting for a disaster

You do not need an incident to study trust repair. Three approaches work:

1. Critical incident interviews. Ask users to recall the last time the AI got something wrong, then walk through what they did next and what they concluded. The critical incident technique is well suited here because it anchors on a specific remembered event rather than a general attitude, which is where survey data goes vague.

2. Seeded-failure studies. In a controlled session, let users encounter a realistic wrong output, then interview immediately. This is the only reliable way to observe the moment of attribution, because in the wild it passes in seconds and is rationalised afterwards.

3. Repair-strategy comparison. Show matched cohorts different repair copy after the same failure — an apology, an explanation, a plain path forward — and measure willingness to rely again rather than satisfaction. Given that apologies, explanations, and promises perform similarly in the research, what you are usually testing is whether your phrasing avoids the denial trap, not which category wins.

Timing is the constraint on all three. Interview someone a week after a failure and you get a tidied, coherent narrative that has already resolved the ambiguity you were trying to observe.

How Koji helps

Trust research fails on logistics more than on method. The window in which a user can accurately report what they concluded is hours, not weeks — and a traditional round of recruiting, scheduling, moderating, transcribing, and analysing takes two to three weeks. By the time the study reports, the participants are describing a memory of a feeling.

• Trigger an AI-moderated interview at the failure event — when a user rejects, heavily edits, or abandons an AI output — so you capture attribution while it is live.
• Voice interviews for exactly this topic. Trust talk is hedged in writing and much more direct out loud; the hesitation before "…yeah, I don't really rely on it" is data a text field throws away.
• Automatic thematic analysis across the whole cohort, so you can separate the users who blamed themselves from the users who generalised to the product — the distinction that predicts churn.
• Structured questions alongside the conversation. Koji's six types — open_ended, scale, single_choice, multiple_choice, ranking, and yes_no — let one instrument carry the reliance-again scale metric, a single_choice attribution question, and the open_ended probe that explains both. The structured questions guide covers how to sequence them so the scale question does not anchor the open answer.
• A customisable AI consultant briefed on the specific failure mode, so it probes what the user concluded rather than whether they were satisfied.
• Real-time reporting, so repair copy can be tested and changed inside a release cycle.

The legacy alternative — a satisfaction survey emailed the following week — measures the wrong construct at the wrong time. It asks about satisfaction when the predictive variable is willingness to rely again, and it asks after the user has already rewritten the memory.

Common mistakes

1. Treating trust as an accuracy problem. The research is explicit that exposure to failure drives aversion even when the algorithm outperforms the human alternative.
2. Shipping an all-or-nothing AI. Framing use as a binary is precisely the condition that produced 32% adoption.
3. Over-scoping the edit affordance. An 80% reduction in adjustment range had no detectable effect on willingness to use. Ship the small version.
4. Denying observed failures. The consistently worst repair strategy, and error copy that blames the user's phrasing is a denial.
5. Measuring per session. The three-strike ceiling is cumulative across the relationship.
6. Measuring satisfaction instead of future reliance. Satisfaction tracks whether this task worked; reliance predicts next quarter.
7. Interviewing a week later. By then you are studying a reconstructed narrative, not an attribution.

Frequently asked questions

What is algorithm aversion?

Algorithm aversion is the tendency to abandon an algorithmic forecaster faster than a human one after seeing it make the same mistake. It was named by Dietvorst, Simmons and Massey in the Journal of Experimental Psychology: General in 2015, across five studies. The crucial detail is that it appears after people see the algorithm perform — including when they have seen it outperform the human — so it is not simple unfamiliarity or technophobia.

How do you overcome algorithm aversion?

Give users a small, bounded ability to modify the output. In Dietvorst et al. (Management Science, 2018), the share of participants choosing to use an imperfect model rose from 32% to 73–76% when they could adjust its forecasts. Study 2 found willingness was statistically indistinguishable whether people could adjust by 10, 5, or 2 percentiles, so a tightly constrained edit affordance captures nearly all of the benefit.

Does letting users edit AI output make the results worse?

In the research, it made them better — because the alternative was people not using the model at all. Average absolute error was 22.34 when the model could not be changed versus 20.10–20.87 when it could. Constraining the edit also kept users closer to the model: unconstrained users deviated by 8.18 percentiles on average against 4.71 for those limited to ±10. Bounded editing beats both no editing and unlimited editing.

How many mistakes can an AI make before users give up?

Roughly three visible failures, on current evidence. Esterwood and Robert (Computers in Human Behavior, 2023) had 240 participants work with a robot that erred, and after three violations none of the tested repair strategies — apology, denial, explanation, or promise — fully restored trustworthiness. Treat visible, AI-attributed failures per user as a budget rather than a rate.

What is the best way to apologise for an AI error?

Avoid denial, which is consistently the worst-performing strategy, and pair the acknowledgement with a concrete path forward. Apologies, explanations, and promises perform similarly in the research, so the category matters less than the execution. Match the register to your product: research integrating theory of mind found apologies land better when users perceive the system as capable of emotion, while explanations land better when they attribute intentionality — so a persona-led assistant and a plain tool call for different copy.

Should we measure trust with a satisfaction survey?

No. Satisfaction is dominated by whether the current task succeeded, which is not what you are trying to learn. Measure willingness to rely on the feature again, attribution of the failure, and behavioural return rate after a failure. Track these per user cumulatively rather than per session, because the three-strike effect accumulates across the relationship and per-session dashboards will look healthy until the user leaves.

How soon after a failure should we interview a user?

Within hours. Attribution — whether the user concluded the AI is unreliable, or that they phrased it badly — is formed quickly and rationalised into a tidier story afterwards. A week later you are studying a reconstructed narrative. This timing requirement is what makes AI-moderated interviews practical for trust research: they can be triggered at the failure event and run immediately, rather than waiting on scheduling.

Related Resources

• Structured Questions in AI Interviews — the six question types and how to sequence a scale metric with its open-ended explanation
• AI Over-Reliance and Automation Bias — the opposite failure, where users trust the AI too much
• AI Explainability Testing — whether explanations actually help users calibrate
• Critical Incident Technique — the interview method best suited to studying remembered failures
• AI Failure Mode Analysis — anticipating which failures will be visible and attributed
• AI Incident Postmortems — investigating failures with user evidence
• AI-to-Human Handoff — the escalation path that gives users a route around a failure
• Staged Rollout for AI Features — limiting how many users spend a strike on a bad release

---

Find out what your users concluded the last time your AI got it wrong. Koji gives you 10 free credits — enough to run your first AI-moderated trust interviews and see the analysis the same day.