Participant Confidence Is Not Accuracy: How Interviewer Feedback Inflates Certainty (2026)

Answer first: how confident a participant sounds tells you something about accuracy only at the first telling, before anyone has reacted to their answer. Once an interviewer affirms an account - "great, that is really helpful", "yes, exactly", "a lot of people say that" - participants become more certain, and they also start remembering that they saw more clearly and paid more attention than they did. In the classic experiment, every one of 352 witnesses had picked the wrong person, and a single sentence of confirming feedback still inflated their reported certainty. So capture confidence at the first telling, keep your reactions neutral, and give low initial confidence far more weight than later certainty.

This matters because research teams routinely use confidence as a filter. The customer who says "I am absolutely sure that is why we churned" gets quoted in the readout; the one who says "I think it might have been the pricing, but I am not certain" gets softened into a maybe. That ranking is only as good as the conditions under which the confidence was formed.

The experiment: "Good, you identified the suspect"

Gary Wells and Amy Bradfield's 1998 study ("Good, you identified the suspect: Feedback to eyewitnesses distorts their reports of the witnessing experience", Journal of Applied Psychology 83, 360-376) was designed to isolate the effect of a single remark.

• Participants watched a security video and tried to identify the gunman from a photospread.
• The actual gunman was not in the photospread, so every witness who made an identification was wrong. The analysis covered 352 of them.
• After identifying someone, each witness received confirming feedback ("Good, you identified the actual suspect"), disconfirming feedback, or no feedback.
• Witnesses were then asked about their experience of witnessing.

Confirming feedback strongly inflated how certain witnesses said they had been, and it also changed their reports of the witnessing itself: how good their view had been, how much attention they had paid, how clear their memory was, and how quickly they had made the identification. None of those things had changed. Only the feedback had.

Read that list again with a research interview in mind. How certain am I? How well did I see it? How closely was I paying attention? Those are exactly the judgements a researcher uses to decide which participant's account to trust.

It is not a one-off

The effect has been replicated many times. Douglass and Steblay's meta-analysis ("Memory distortion in eyewitnesses: a meta-analysis of the post-identification feedback effect", Applied Cognitive Psychology 20, 859-869, 2006) pooled 20 experimental tests with more than 2,400 participant-witnesses. It found large effects of confirming feedback on retrospective certainty, view and attention, and smaller effects on so-called objective measures such as how long the person was in view.

The pattern of effect sizes is itself useful. Feedback distorts subjective, self-assessed judgements most. Those are also the judgements a participant makes about their own account in an interview: how sure they are, how well they remember, how representative their experience was.

The twist: initial confidence is informative

It would be easy to conclude that participant confidence is worthless. The more recent evidence says otherwise, and the difference is timing.

John Wixted and Gary Wells reviewed the field in "The Relationship Between Eyewitness Confidence and Identification Accuracy: A New Synthesis" (Psychological Science in the Public Interest 18(1), 2017). Their conclusion was that confidence can be a useful indicator of accuracy when it is recorded under clean conditions at the moment of first identification - and loses that value once the witness has been exposed to repeated questioning, coaching or confirming feedback. The conditions they recommend include making sure the person administering the procedure does not know which answer is the expected one. On low confidence, they were emphatic:

"Instead of being ignored, an initial expression of low confidence should take center stage -- overshadowing all other considerations -- when a jury's goal is to evaluate the reliability of a suspect ID." - John T. Wixted and Gary L. Wells, 2017

For a research team, the translation is direct. First-telling confidence is data. Later confidence is partly a record of how you reacted. And a participant who was unsure the first time and is certain by the end of the session has told you something about your interview, not about their memory.

Where confirming feedback enters a research interview

Moderators give confirming feedback all the time, mostly to build rapport. Common forms:

Each of these is a small "good, you identified the suspect". Their effect accumulates: by the end of a session, the participant's confidence in the themes you reacted to has risen, and so has their sense of how clearly they remember them. If you then ask a scale question - "How confident are you that this was the main reason?" - you are measuring partly their memory and partly your own reactions.

Rapport is still important; see building rapport in interviews. The goal is rapport that acknowledges the person without evaluating the content: "Thank you, that is clear" rather than "That is exactly what we needed."

A protocol for capturing confidence before you contaminate it

1. Keep the moderator blind to the hypothesis where you can. Wixted and Wells recommend that the person administering the procedure should not know the expected answer. In research terms: the interviewer should know the topic, not the answer you hope for. See blind analysis for the analysis-side version of the same idea.
2. Ask for confidence right after the first free-recall answer. "How sure are you about that?" or a short rating scale, asked immediately, before any follow-up that signals interest.
3. Use neutral acknowledgements. Replace evaluative feedback ("great", "exactly") with receipt-only feedback ("thank you", "understood", "go on"), used evenly across every answer.
4. Never tell participants how their answer compares. No "others said the same", no "that matches what we are seeing".
5. Weight low first-telling confidence heavily. If a participant hedged at first telling, report the finding with the hedge, even if they sounded certain by the end.
6. Do not re-ask confidence after stimulus. Once you have shown a concept or a fix, later confidence ratings reflect the stimulus.

A worked example

A product team interviews 12 churned customers. The moderator, who believes onboarding was the problem, responds warmly whenever onboarding comes up. At the end of each session participants rate "How confident are you that onboarding was a major reason you left?" The average is high, and the readout says churned customers are confident onboarding drove churn.

Rerun it with the protocol: a moderator who does not know the hypothesis, confidence captured immediately after each participant's first unprompted reason, and neutral acknowledgements throughout. Suppose now only a minority name onboarding first, and several of those hedge when asked how sure they are. The data has not got worse. The first study was measuring the moderator's enthusiasm. (This example is illustrative; the mechanism is the one measured in the studies above.)

How Koji helps

Human moderators cannot fully control their own feedback - they nod, smile, lean in and say "great" without noticing, and they do it differently in every session. Legacy survey tools avoid the problem only by removing the conversation entirely, which loses the reasoning behind every rating. Koji's AI-moderated interviews keep the conversation and standardise the feedback.

• Consistent, neutral acknowledgement. Koji's interviewer is a customizable AI consultant. You can instruct it to acknowledge without evaluating, and it applies that instruction identically in the first interview and the five-hundredth. There is no warm reaction to the answers you were hoping for.
• Blind by design. You decide what goes into the interviewer's instructions. Give it the topic and the questions, and keep your hypothesis out, and the interviewer genuinely does not know which answer you expect.
• Confidence captured at first telling. Place a scale question directly after an open_ended recall question in the study flow, and every participant rates their certainty at the same point, before any follow-up. Koji supports six structured question types - open_ended, scale, single_choice, multiple_choice, ranking and yes_no - covered in the structured questions guide.
• Hedges survive into the analysis. Koji's automatic thematic analysis links each theme to the verbatim quotes behind it, so "I think it might have been pricing, but I am not sure" reaches your report with its hedge intact instead of being summarised into a confident bullet.
• Quality scoring. Every Koji interview receives a 1-5 quality score, so thin or low-effort conversations are visible before anyone quotes them.

You do not need a background in eyewitness psychology to get this right. You need an interviewer that reacts the same way to every answer - which is much easier to guarantee in software than in people.

Frequently asked questions

Is a confident participant more likely to be right?

Only when confidence is recorded at the first telling under clean conditions. Wixted and Wells (2017) concluded that initial confidence can indicate accuracy, but confidence expressed after repeated questioning, coaching or confirming feedback is much less informative.

What is the post-identification feedback effect?

It is the finding that telling witnesses their answer was correct inflates their later reports of how certain they were, how good their view was and how much attention they paid. Wells and Bradfield (1998) demonstrated it with 352 witnesses who had all made a wrong identification.

How big is the effect?

Large. Douglass and Steblay's 2006 meta-analysis pooled 20 experimental tests and more than 2,400 witnesses and found large effects on retrospective certainty, view and attention, with smaller effects on more objective measures.

Does this mean I should stop building rapport?

No. Rapport keeps participants talking openly. The change is to acknowledge the person rather than evaluate the content: say thank you and ask them to go on, rather than telling them an answer is exactly what you needed.

When should I ask participants how confident they are?

Immediately after their first unprompted answer, before follow-up questions, stimuli or any sign of which answers interest you. Confidence asked at the end of a session partly reflects how the interviewer reacted during it.

What should I do with low-confidence answers?

Report them with the hedge intact and give them weight. Wixted and Wells argue that an initial expression of low confidence should take center stage when judging reliability. A participant who was unsure at first and certain by the end has usually been influenced by the session.

Related Resources

• The Misinformation Effect - how researcher words become part of participant memory
• Memory Conformity in Group Research - why the confident co-witness usually wins
• Interviewer Bias - the wider set of ways the moderator shapes the data
• Demand Characteristics - participants answering what they think you want
• Blind Analysis - keeping the expected answer away from the people handling the data
• Structured Questions Guide - capturing confidence with a scale question at the right moment