{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-24T01:37:40.557Z"},"content":[{"type":"documentation","id":"801c68fc-eb49-4d32-bc66-db478bd7f3d3","slug":"hypothetical-questions-interviews","title":"Why Hypothetical Questions Ruin User Interviews (and What to Ask Instead)","url":"https://www.koji.so/docs/hypothetical-questions-interviews","summary":"Hypothetical questions — \"Would you use this?\", \"How much would you pay?\" — produce unreliable user-research data because people cannot predict their own behavior, answer to please you (social desirability, the core Mom Test lesson), and imagine idealized future selves. The result is false positives that make weak ideas look validated. The fix is to convert future-tense questions into past-tense ones that ask for a specific, recent, real instance: instead of \"Would you use a tool that did X?\" ask \"Tell me about the last time you needed X — what did you do?\" Past behavior is evidence; future intention is a guess. A narrow exception is concept testing, where you ground reactions in real context and pair them with real-stakes signals. Audit guides for would/will/if-we future framing and rewrite. AI interviews (Koji) help by probing for concrete examples on every open-ended answer and by capturing real frequencies and choices as structured question types instead of inviting speculation.","content":"# Why Hypothetical Questions Ruin User Interviews (and What to Ask Instead)\n\n**Short answer:** Hypothetical questions — \"Would you use this?\", \"How much would you pay?\", \"Would you recommend us?\" — produce some of the least reliable data in user research. People are genuinely bad at predicting their own future behavior, and they answer hypotheticals with optimism, politeness, and imagination rather than fact. The fix is to **replace speculation with memory**: instead of asking what someone *would* do, ask what they *did* — \"When did you last face this problem? Walk me through what you actually did.\" Past behavior is evidence; future intention is a guess. AI interview platforms like Koji help enforce this by probing for concrete, recent examples on every open-ended answer, so a vague hypothetical gets pulled back to a real story automatically.\n\nThis guide explains why hypothetical questions mislead, how to spot them, and how to rewrite your interview guide around real behavior.\n\n## What Counts as a Hypothetical Question\n\nA hypothetical question asks the participant to imagine, predict, or speculate rather than report something that actually happened. Common forms:\n\n- **Future intention:** \"Would you use this feature?\" / \"Will you upgrade?\"\n- **Imagined preference:** \"Which of these would you prefer?\" (with no real stakes)\n- **Speculative pricing:** \"How much would you pay for this?\"\n- **Idealized self:** \"How often would you use a tool like this?\"\n- **Leading hypotheticals:** \"If we added X, would that make you happy?\"\n\nThey feel productive because participants always have an answer. That's exactly the problem: the answer is invented on the spot, not retrieved from experience.\n\n## Why Hypotheticals Produce Bad Data\n\nThree well-documented biases make hypothetical answers unreliable:\n\n1. **People can't predict their own behavior.** The gap between stated intention and actual behavior is one of the most consistent findings in social science. \"Yes, I'd definitely use that\" routinely becomes \"I never got around to it.\"\n2. **Social desirability and politeness.** Participants want to be helpful and kind, especially to the person who built the thing. Asked \"Would you use this?\", they say yes to please you. This is the central lesson of *The Mom Test* — your mom will lie to you about your business idea because she loves you. See [the Mom Test for user interviews](/docs/mom-test-user-interviews).\n3. **Optimism and imagination bias.** When people imagine their future selves, they picture an idealized version — more disciplined, more organized, more willing to change habits than they really are. Hypotheticals sample that fantasy, not reality.\n\nThe result: hypothetical questions generate false positives. They make weak ideas look validated and send teams off to build features that test well in conversation and fail in the market.\n\n## The Core Fix: Trade the Future for the Past\n\nThe single most powerful move in interviewing is to **convert future-tense questions into past-tense ones.** Memory of real events is grounded in fact; speculation about the future is not. A few rewrites:\n\n| Hypothetical (weak) | Past-behavior (strong) |\n| --- | --- |\n| \"Would you use a tool that did X?\" | \"Tell me about the last time you needed X. What did you do?\" |\n| \"How much would you pay for this?\" | \"What do you currently spend solving this? What tools have you paid for?\" |\n| \"Would this feature be useful?\" | \"Walk me through the last time you hit this problem.\" |\n| \"How often would you use this?\" | \"How many times did you do this in the past week?\" |\n| \"Would you switch from your current tool?\" | \"Have you switched tools before? What triggered it?\" |\n\nNotice the pattern: every strong version asks for a **specific, recent, real instance**. The more concrete and recent the example, the more reliable the data — and the more useful detail you get (the trigger, the workaround, the cost, the emotion).\n\n## When a Hypothetical Is Actually OK\n\nThere's a narrow exception. Concept and reaction testing legitimately need to show people something that doesn't exist yet. The trick is to **ground the reaction in real context, not abstract preference.** Instead of \"Would you use this?\", show the concept and ask \"How does this compare to how you solve this today?\" or \"When in your last project would this have helped — or not?\" You're still anchoring to real behavior, just using the concept as a prompt. And rather than trusting stated intent, pair the reaction with a real-stakes signal where you can — a [fake-door or smoke test](/docs/fake-door-testing-guide), a willingness to be a beta user, an actual sign-up.\n\n## How to Audit Your Interview Guide\n\nBefore fielding a study, scan every question and flag any that:\n\n- Contain \"would,\" \"will,\" \"could,\" or \"if we\" pointing at the future.\n- Ask for a prediction, preference, or estimate about something the participant hasn't actually done.\n- Could be answered by someone with zero real experience of the problem.\n\nRewrite each flagged question to ask for a past instance. This single pass dramatically improves data quality and is one of the most common fixes in our [customer interview mistakes](/docs/customer-interview-mistakes-2026) guide. For building strong questions from scratch, see [customer interview questions examples](/docs/customer-interview-questions-examples).\n\n## How AI Interviews Keep Conversations Grounded\n\nEven skilled interviewers slip into hypotheticals under time pressure, and they can't probe every answer deeply. This is where an AI interviewer helps structurally. Koji's AI is built to chase concrete evidence: when a participant gives a vague or speculative answer, it follows up for a real example — \"Can you tell me about a specific time that happened?\" — on every open-ended question, without fatigue or the politeness reflex that makes human moderators accept the easy answer.\n\nYou can also design hypotheticals *out* of the study upfront using the right question types. Koji's six structured types — `open_ended`, `scale`, `single_choice`, `multiple_choice`, `ranking`, and `yes_no` — let you capture real preferences and frequencies as structured data (\"How many times last week?\" as a `scale`; \"Which did you actually use?\" as `single_choice`) instead of inviting open-ended speculation. See the [structured questions guide](/docs/structured-questions-guide). Combined with consistent probing for specifics, this keeps every interview anchored to what people *did*, not what they imagine they might do — which is the whole point of talking to customers in the first place. For the broader set of biases to watch, see [avoiding bias in interviews](/docs/avoiding-bias-in-interviews).\n\n## Related Resources\n\n- [Structured Questions in AI Interviews](/docs/structured-questions-guide) — capture real behavior as structured data\n- [The Mom Test for User Interviews](/docs/mom-test-user-interviews)\n- [Avoiding Bias in Interviews](/docs/avoiding-bias-in-interviews)\n- [Customer Interview Mistakes to Avoid](/docs/customer-interview-mistakes-2026)\n- [Customer Interview Questions: Examples That Work](/docs/customer-interview-questions-examples)\n- [How to Conduct User Interviews](/docs/how-to-conduct-user-interviews)\n","category":"Interview Techniques","lastModified":"2026-06-23T03:30:34.551081+00:00","metaTitle":"Why Hypothetical Questions Ruin User Interviews (Fix Guide)","metaDescription":"Hypothetical questions like \"Would you use this?\" produce unreliable data. Learn why future-tense and speculative questions mislead, how to rewrite them as past-behavior questions, and how AI interviews keep conversations grounded in real evidence.","keywords":["hypothetical questions interviews","would you use this question","asking about future behavior","past behavior interview questions","user interview question mistakes","speculative survey questions","mom test questions"],"aiSummary":"Hypothetical questions — \"Would you use this?\", \"How much would you pay?\" — produce unreliable user-research data because people cannot predict their own behavior, answer to please you (social desirability, the core Mom Test lesson), and imagine idealized future selves. The result is false positives that make weak ideas look validated. The fix is to convert future-tense questions into past-tense ones that ask for a specific, recent, real instance: instead of \"Would you use a tool that did X?\" ask \"Tell me about the last time you needed X — what did you do?\" Past behavior is evidence; future intention is a guess. A narrow exception is concept testing, where you ground reactions in real context and pair them with real-stakes signals. Audit guides for would/will/if-we future framing and rewrite. AI interviews (Koji) help by probing for concrete examples on every open-ended answer and by capturing real frequencies and choices as structured question types instead of inviting speculation.","aiPrerequisites":["Basic experience writing or conducting user interviews","An interview guide or question list to improve"],"aiLearningOutcomes":["Recognize hypothetical questions and the biases that make them unreliable","Convert future-tense questions into grounded past-behavior questions","Handle concept testing without trusting stated intent","Audit an interview guide for speculative questions","Use AI probing and structured question types to keep interviews grounded in real evidence"],"aiDifficulty":"beginner","aiEstimatedTime":"10 minutes"}],"pagination":{"total":1,"returned":1,"offset":0}}