The Curse of Knowledge in User Research

The curse of knowledge is the difficulty of imagining what it's like not to know something you know. In user research it creeps into the questions you write, the tasks you set, the way you read answers and the reports you hand over. You can't think your way out of it. You catch it by testing your materials on people who don't share your knowledge, before the real sessions start.

This guide covers where the idea comes from, the evidence that experts badly misjudge novices, where the curse shows up at each stage of a study, and a checklist for catching it.

What the curse of knowledge is

The term entered research through a 1989 paper by economists Colin Camerer, George Loewenstein and Martin Weber in the Journal of Political Economy. Economic models often assumed that a better-informed person could easily predict how a less-informed person would judge something. Their experiments showed that isn't true. Once people knew the real outcome, they couldn't set that knowledge aside when guessing what others without it would think.

Steven Pinker later made it the centrepiece of his book on writing, The Sense of Style (2014). His summary is the clearest one: "The main cause of incomprehensible prose is the difficulty of imagining what it's like for someone else not to know something that you know."

Swap "prose" for "interview questions" or "usability tasks" and you have the researcher's version of the problem.

The tapping study: 50% expected, 2.5% achieved

The best-known demonstration came from Elizabeth Newton, a Stanford psychology PhD student, in 1990. She split people into tappers and listeners. Tappers picked a well-known song and tapped its rhythm on a table. Listeners had to name the song.

Before the listeners guessed, tappers predicted they would get it right about half the time. Across 120 songs, listeners named 3. That's 2.5%.

The tappers weren't careless. They could hear the melody in their heads while they tapped, and couldn't un-hear it. To the listeners it was just knocking. That is exactly the position a product team is in when it writes a task like "Set up a workspace integration" for someone who has never seen the product.

Experts are the worst judges of novices

You might expect the people who know a product best to be the best at predicting where newcomers will struggle. Pamela Hinds tested this in 1999. She asked people who sold mobile phones, people with some experience of the phones and people with none to estimate how long a novice would take to learn to use one. Then she had novices learn it from the written instructions.

The experts underestimated badly: on average the novices took about twice as long as the experts predicted. People with intermediate experience came closer. Hinds called it the curse of expertise.

Two more findings matter for anyone hoping to simply try harder:

• Warnings don't fix it. In studies of the curse of knowledge, explicitly warning people about it hasn't stopped them projecting what they know.
• Feedback only helps partly. In Camerer, Loewenstein and Weber's market experiments, trading reduced the curse by roughly 50%, but did not remove it.

For research teams the takeaway is uncomfortable. The more senior and experienced the person writing the discussion guide, the more likely the guide is to assume things participants don't know.

Where it shows up in a research study

The curse of knowledge doesn't stay in one place. It turns up at each stage of a study, usually in a slightly different form.

Writing questions. Internal feature names, acronyms and team jargon slip into questions: "How do you use Smart Sync?" A participant who has never noticed the feature's name will guess, agree politely or describe something else.

Writing tasks. Usability tasks that use the words on the button give the answer away, and tasks that assume a mental model ("Go to the admin area") test whether people share yours, not whether they can do the job. See usability test tasks and scenarios for how to phrase them.

Screening participants. Screener questions written in insider terms filter for people who already talk like your team, which quietly skews the sample towards experts.

Moderating. When you know the answer, a participant's confused silence feels like a pause rather than a sign of trouble, so you rescue them too early. Or you rephrase the question by explaining how the feature works, which changes what you're testing.

Analysing. You fill gaps in a participant's explanation with what you know the product does, and code the answer as "understood the feature" when they didn't. Their workaround looks like an edge case because you know the intended path.

Reporting. Findings written in the research team's shorthand ("JTBD drift in the onboarding cohort") don't land with the people who need to act on them. The curse works on your stakeholders too.

How it differs from the false consensus effect

The two are often mixed up because both involve projecting yourself onto others.

• The false consensus effect is about preferences and choices: assuming others would decide what you'd decide.
• The curse of knowledge is about information: assuming others know what you know.

In practice they stack. A team that knows the product well (curse of knowledge) and also assumes users want what the team wants (false consensus) can design a feature that only makes sense to itself.

A checklist for catching it

You can't remove what you know, so the defence is process. Run this before a study goes into the field:

1. Strip internal names. Replace feature names, acronyms and team jargon with plain descriptions of what the thing does.
2. Use the participant's words. If support tickets or reviews describe a feature a certain way, use their phrasing.
3. Describe goals, not paths. Write tasks as outcomes ("Share this report with a colleague"), not as the steps or labels you know.
4. Ask a newcomer to read it. Someone outside the team should read the guide aloud and say where they had to guess.
5. Pilot with a real participant. One pilot session catches more than any amount of internal review. See pilot studies.
6. Note where you rescued someone. After each session, list the moments you explained something. Each one is a finding.
7. Code what was said, not what you know. If a participant didn't name a step, don't assume they took it.
8. Write findings for a newcomer. Test the summary on someone outside research before you share it.

Testing your questions on people who don't know

Most of the checklist comes down to one habit: put your materials in front of someone who doesn't know what you know, and watch.

Photo by Vitaly Gariev on Unsplash

A pilot session is the most reliable version. Recruit one or two people who match your participants but haven't seen the guide. Run the session exactly as planned and listen for three things: questions they ask you to repeat, answers that describe something other than what you meant, and long pauses before they answer. Each is a place where your knowledge filled a gap their knowledge couldn't.

A cheaper version is the "read it back" test. Give a colleague from another team your interview questions and ask them to explain, in their own words, what each question is asking. Where their explanation differs from your intent, rewrite.

Cognitive interviewing is the formal method for this. It was built for survey questions, but it works just as well for an interview guide.

How Koji helps

Koji runs AI-moderated interviews by voice or text. It doesn't make your team forget what it knows, but it makes the newcomer test cheap and gives participants room to say things in their own words.

• Test the guide as a participant would. The Preview tab lets you take your own interview by text or voice, exactly as a participant would. Previews never count as responses or appear in the report, so you can run one with a colleague from another team at the keyboard and watch where they stumble.
• Spot jargon in the questions. The question editor shows every question with its type and follow-up setting in one place, so a quick read for internal names is easy before you publish.
• Let participants explain in their own words. Open questions get AI follow-ups, up to three per question, so a vague or puzzled answer gets another question instead of a moderator explaining the product.
• Keep the quotes verbatim. Findings in the report link to the supporting responses, so anyone can check what participants actually said rather than the team's reading of it.
• Pilot cheaply. Run a handful of interviews, read them, adjust the questions and carry on. You don't need to rebook a moderator for each round.

Related Resources

• The false consensus effect: why you are not your user
• Writing interview questions
• Cognitive interview guide
• Pilot studies in user research
• Plain language research questions
• Structured questions guide

Sources

• Camerer, C., Loewenstein, G., & Weber, M. (1989). The curse of knowledge in economic settings: An experimental analysis. Journal of Political Economy, 97(5), 1232–1254.
• Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance. Journal of Experimental Psychology: Applied, 5(2), 205–221.
• Newton, E. L. (1990). The rocky road from actions to intentions. Doctoral dissertation, Stanford University.
• Pinker, S. (2014). The Sense of Style: The Thinking Person's Guide to Writing in the 21st Century. Viking.