Demand Characteristics: When Participants Tell You What They Think You Want

Demand characteristics are the cues in a study that let participants guess what you are trying to find — and then, consciously or not, change their behavior to give it to you. A user says a prototype is "great" because they can tell you built it. A survey respondent infers what a question is "about" and answers to fit. The result is research that confirms your hypothesis not because it is true, but because you accidentally told participants what answer you were hoping for.

This guide explains what demand characteristics are, where they come from, how they differ from related biases, what the classic research shows, and how a neutral AI moderator reduces (but does not magically erase) the problem.

What are demand characteristics?

The term was coined by psychologist Martin Orne in his classic 1962 paper On the Social Psychology of the Psychological Experiment (American Psychologist, 17, 776-783). Orne argued that a study is never neutral measurement — it is "a special form of social interaction." Participants want to be helpful and want the study to succeed, so they scan every available cue for the hypothesis and then behave to validate it. As Orne put it, a participant "will behave in an experimental context in a manner designed to play the role of a 'good subject'."

That is the heart of it. Demand characteristics are the sum of the cues — the setting, the researcher's tone and body language, the wording and ordering of questions, even rumors about what the study is "really" testing — from which a participant reconstructs your intent. Once they have a guess, it is almost impossible for them to un-see it.

Just how far will a cooperative participant go? In one of Orne's demonstrations, subjects were handed sheets of arithmetic and told that after finishing each sheet they should tear it into at least 32 pieces and start another. The task was pointless and endless, yet participants persisted for hours with little drop in effort; in one run the experimenter gave up after five and a half hours. People will do remarkable things when they believe it serves the research.

Demand characteristics vs. social desirability vs. the Hawthorne effect

These three biases are cousins, and conflating them leads to the wrong fix. The distinction is about what the participant is responding to:

• Demand characteristics: the participant infers the researcher's hypothesis from situational cues and shifts behavior to help confirm it. Trigger: perceived experimental intent.
• Social desirability bias: the participant answers to look good against broader social norms — likeable, normal, moral — independent of any guess about your hypothesis. Trigger: self-image in front of others.
• Hawthorne effect: behavior changes simply because the person knows they are being observed, without any hypothesis-guessing or specific direction. Trigger: observation itself.

The Hawthorne effect is worth a footnote, because its own evidence is a cautionary tale about believing tidy stories. When Levitt and List recovered and reanalyzed the original illumination-experiment data (American Economic Journal: Applied Economics, 2011), they found the dramatic productivity swings everyone repeats were largely "fictional" — only subtle effects survived rigorous analysis. Even our textbook example of observation-driven bias was partly a demand characteristic of how the story got told.

Why it matters for product and customer research

Demand characteristics are not a lab curiosity — they are the reason so much customer research is politely useless. In a usability session, the moderator is read as the authority in the room. As Nielsen Norman Group notes, "many participants will not want to disagree." So they nod, they say the design is intuitive, they rate the concept a 4 — and then they never use the product.

This is exactly the failure Rob Fitzpatrick built The Mom Test around. His blunt verdict: "Opinions are worthless." People give compliments and agree with your idea to be nice, especially when they can tell it is your idea. What matters is not what they say about your hypothetical, but what they have actually done. Ask a leading question and, as NN/g warns, respondents "are prone to simply mimic the words of the interviewer" — you hear your own hypothesis echoed back and mistake it for validation.

How to reduce demand characteristics

1. Ask about past behavior, not hypotheticals. "Walk me through the last time you tried to solve this" beats "Would you use a feature that does X?" The past is fact; the future is flattery. This is the core move of the Mom Test.
2. Write neutral, non-leading questions. Never embed the answer you want, or your product's own terminology, in the question. (See avoiding leading questions.)
3. Conceal the true hypothesis. Do not tell participants which design is yours or what you expect to find. A single-blind setup removes the target they would otherwise aim at.
4. Reduce rapport pressure. Warmth is good for candor but bad when it creates social debt. Stay neutral, do not react to answers, and never signal approval or disappointment.
5. Prefer behavioral and unobtrusive measures. What people do — task success, real usage, actual purchases — is far harder to fake than what they say.

The modern approach: a neutral moderator at scale

Most demand characteristics enter through the moderator. Tone, a raised eyebrow, an encouraging "great," an inconsistent follow-up, the visible hope that you will like the answer — these are the cues participants read. Koji removes that channel by design.

Because every Koji interview is AI-moderated, there is no authority figure in the room to please and no visible disappointment to avoid. The AI interviewer delivers identically neutral wording to every participant, asks non-leading follow-ups, and never rewards a "right" answer with a smile. It probes past behavior in the participant's own words — "what did you actually do?" — rather than fishing for approval of a hypothetical. And because Koji uses six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, and yes_no), you can anchor claims to concrete choices and then let the AI dig into the reasoning behind them.

Two honest caveats, because credibility matters more than a sales line:

• AI does not eliminate demand characteristics — it relocates and reduces them. Participants can still infer purpose from the content of your questions and perform for a perceived audience (the company behind the study, or the sense of being recorded). Social desirability and the good-subject impulse do not vanish just because the moderator is software.
• A biased prompt is a biased moderator. If you write leading questions into the AI, it will ask them leadingly to everyone. The advantage is that this bias is centralized, auditable, and fixable once — edit the script and every future participant gets the neutral version — rather than being reintroduced, invisibly, by every human interviewer in every session.

The defensible claim is not immunity. It is consistency plus the removal of moderator-driven cues, at a scale no human panel can match — and Koji makes that discipline the default rather than something only a trained researcher can pull off.

A worked example

A team tests a redesigned onboarding flow with eight moderated sessions. Seven participants call it "much clearer" and the team ships it — then activation drops. What went wrong? The moderator had introduced each session with "we've been working hard on making this simpler, I'd love your thoughts," handing every participant the hypothesis on a plate. The polite, cooperative response was to confirm it. Re-run as neutral AI interviews that ask participants to complete a real task and then describe, unprompted, where they got stuck, the same flow surfaces three concrete friction points. The difference was not the design — it was the demand characteristics baked into how the humans framed the study.

Where demand characteristics hide

They rarely announce themselves. The most common entry points:

• The recruiting message. "Help us test our exciting new feature" primes participants to be enthusiastic before the session even starts.
• The order of questions. An earlier question can reveal what the study is "about," changing how people answer everything after it.
• The reaction to answers. A moderator who lights up at positive feedback and goes quiet at criticism teaches participants, within minutes, which answers are rewarded.
• The hypothetical. "Would you pay for this?" invites a generous guess; "what did you pay for the last tool that solved this?" invites a fact.
• The prototype reveal. The moment a participant knows a design is yours, praise inflates and honesty deflates.

Audit each of these before you field a study. If you cannot remove a cue, at least measure whether it moved the results — for example, by comparing responses across differently framed versions.

Common mistakes to avoid

• Treating agreement as validation. A room full of "yes" is often a room full of politeness. Look for evidence of past behavior and real commitments, not compliments.
• Debriefing too early. Explaining your goal at the start converts the whole session into a hypothesis-confirmation exercise.
• Assuming AI makes you immune. A neutral moderator removes one channel of cues, not all of them. Keep writing neutral questions.

Related resources

• Structured Questions Guide — the six question types and when to use each
• Social Desirability Bias — answering to look good to others
• The Mom Test — asking questions even a biased participant answers honestly
• Avoiding Leading Questions — question wording that does not signal an answer
• Avoiding Bias in Interviews — moderator habits that skew results
• Hawthorne Effect — behavior change from being observed
• Research Bias Guide — the umbrella guide to bias in research