{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-10-10T12:04:01.944Z"},"content":[{"type":"documentation","id":"df544d88-226d-46e6-a434-82c1e7146f6f","slug":"double-barreled-questions","title":"Double-Barreled Questions: How to Spot and Fix Them in Surveys and Interviews","url":"https://www.koji.so/docs/double-barreled-questions","summary":"A double-barreled question asks about two or more things but accepts only one answer, for example \"How satisfied are you with the speed and reliability of the app?\". It stops participants expressing mixed views, makes answers impossible to interpret, and lowers validity: Menold (2020) found single-element versions measured better, and Margolis (2022) found more than one-third of White Americans classified as evangelical by a double-barreled question rejected that label. Fix it by splitting into one question per concept, picking the right format for each, and pre-testing aloud. Koji's brief assistant checks every draft question for this and splits double-barreled ones automatically.","content":"## What Is a Double-Barreled Question?\n\nA **double-barreled question** asks about two (or more) things but accepts only one answer. \"How satisfied are you with the speed and reliability of our app?\" is the classic shape: a participant who finds the app fast but flaky has no honest way to respond, and you have no way to tell which half their answer refers to.\n\nThe fix is almost always the same: **split it into one question per idea**, then ask each one on its own. The rest of this guide covers how to spot double-barreled questions (including the subtle versions that slip past reviewers), what the research says about the damage they do, and how to rewrite them in surveys and interviews.\n\n## Why Double-Barreled Questions Ruin Your Data\n\nA double-barreled question fails in three ways at once.\n\n1. **Participants can't express a mixed view.** If someone likes the food but not the service, any single rating they give is wrong for one half of the question.\n2. **You can't interpret the answer.** A 3 out of 5 might mean \"both were average\", \"one was great and one was terrible\", or \"I only thought about the first part\". Your analysis has to guess.\n3. **Different people answer different questions.** Some respondents silently pick one element and ignore the other. Two people giving the same score may be rating different things.\n\nPew Research Center's guidance on writing survey questions describes questions \"that ask respondents to evaluate more than one concept (known as double-barreled questions)\" and uses the example of asking how much confidence people have in a president \"to handle domestic and foreign policy\". Its recommendation is direct: \"it would be more effective to ask two separate questions, one about domestic policy and another about foreign policy.\"\n\n### What the research shows\n\nDouble-barreled questions are not just a style problem. Survey methodologists have measured the effects:\n\n- **Validity drops.** In two randomized experiments published in the _Journal of Official Statistics_, Natalja Menold (2020) compared double-barreled items with versions that kept only one of the two elements. Metric and scalar measurement invariance did not hold across versions, meaning the double-barreled item was not measuring the same thing as its parts, and at least one single-element version showed higher validity.\n- **Respondents need more help.** The same paper reports that in a telephone survey, respondents asked interviewers for clarification more often on the double-barreled versions than on their separated parts.\n- **Speed is a warning sign either way.** Bassili and Scott's 1996 _Public Opinion Quarterly_ study showed that response latency can flag problem questions, and later work cites it for respondents spending longer on double-barreled items. Menold's experiments found the opposite pattern can also appear: response times were the same or shorter for the double-barreled versions, which fits the idea that many respondents simply answer one half and ignore the other.\n- **Whole populations get misclassified.** Michele Margolis (2022, _Public Opinion Quarterly_) studied the long-used survey question asking whether people describe themselves as \"born-again or evangelical\". Among White Americans, **more than one-third** of those classified as evangelical by that double-barreled question actually rejected the evangelical label, and the same was true for **just under two-thirds** of African Americans. A single \"or\" changed who counted as what, and changed the political picture drawn from the data.\n\nThat last example matters for product teams too. Questions like \"Are you a founder or product leader?\" in a screener or segmentation question quietly merge groups you may later want to compare.\n\n## How to Spot a Double-Barreled Question\n\nMost are easy to catch once you know what to look for. Some are not.\n\n### Signal 1: \"and\" or \"or\" joining two concepts\n\nRead every question and stop at each **and**, **or**, **as well as**, or **both**. Ask: could someone feel differently about the thing before the conjunction and the thing after it? If yes, it's double-barreled.\n\n| Double-barreled                                      | Split version                                                                        |\n| ---------------------------------------------------- | ------------------------------------------------------------------------------------ |\n| How easy and enjoyable was onboarding?               | How easy was onboarding? / How enjoyable was onboarding?                             |\n| Was our support team fast and helpful?               | How quickly did support resolve your issue? / How helpful was the answer you got?    |\n| Do you use our app at work or at home?               | Where do you use the app? (multiple choice: At work / At home / Both / Elsewhere)    |\n| How satisfied are you with pricing and plan options? | How satisfied are you with the price? / How well do the plan options fit your needs? |\n\nNot every \"and\" is a problem. \"How satisfied are you with salt and pepper shakers?\" is fine if the shakers are one product. The test is whether the two parts can be judged separately.\n\n### Signal 2: A built-in reason or condition\n\n\"Would you recommend us **because of** our pricing?\" asks two things: whether they'd recommend you, and whether pricing is the reason. Same with \"If the export feature were faster, would you use it more?\", which bundles an assumption (the feature is slow) with a prediction.\n\n### Signal 3: One-and-a-half-barreled questions\n\nThe _SAGE Encyclopedia of Survey Research Methods_ describes a sneakier variant: the question names one concept, but the **answer options introduce a second**. For example:\n\n> How would you rate the new dashboard?\n> ○ Useful and easy to read ○ Useful but cluttered ○ Not useful\n\nNow a participant who finds it easy to read but not useful has nowhere to go. Check your answer options with the same \"and/or\" test you apply to the question text.\n\n### Signal 4: Compound behavior questions\n\n\"How often do you plan and review your sprint?\" assumes planning and reviewing happen together. If one happens weekly and the other monthly, any frequency answer is wrong.\n\n### Signal 5: Mixed time frames\n\n\"How was your experience during and after setup?\" asks about two periods. People who had a rough setup and a smooth first month will average them in their heads, or pick one.\n\n## How to Fix a Double-Barreled Question: A Step-by-Step Method\n\n### Step 1: Name the decision each part informs\n\nBefore splitting, ask what you'll do with each answer. If \"speed\" feeds a performance roadmap and \"reliability\" feeds an incident review, they clearly need separate questions. If you genuinely don't care about one half, delete it rather than splitting.\n\n### Step 2: Write one question per concept\n\nKeep the wording parallel so answers are comparable:\n\n- \"How satisfied are you with how fast the app loads?\"\n- \"How satisfied are you with how reliably the app works?\"\n\n### Step 3: Choose the right question type for each part\n\nSplitting often reveals that the parts need different formats. \"Do you use the app at work or at home?\" is really one multiple-choice question with sensible options, not two yes/no questions. Satisfaction with two features may be two scale questions. \"Why\" behind either belongs in an open-ended follow-up.\n\n### Step 4: Keep the survey short after splitting\n\nSplitting adds questions. If your survey grows past what people will finish, cut lower-priority items instead of re-merging. A shorter set of clean questions beats a longer set of ambiguous ones.\n\n### Step 5: Pre-test with real people\n\nRead each question aloud to two or three people from your audience and ask them to explain what it means in their own words. If they paraphrase only half of it, or hesitate, you've found a problem. This is the core of cognitive interviewing, and it catches double-barreled items that look fine on paper.\n\n## Double-Barreled Questions in Interviews\n\nIn a live interview, a skilled moderator can recover from a double-barreled question by noticing the participant answered only one half and asking about the other. But that relies on the moderator spotting it in the moment, every time, across dozens of sessions. Many don't.\n\nInterview guides are also prone to a specific form: the **stacked question**. \"Tell me about the last time you onboarded a teammate. How long did it take, what went wrong, and what would you change?\" The participant usually answers the last part they heard and forgets the rest.\n\nBetter practice:\n\n- Ask one question, wait for the full answer, then ask the next.\n- Put the follow-ups in your guide as **separate probes**, not as a list inside one question.\n- When you need a rating, ask the rating first, then ask why, rather than combining \"How would you rate X and why?\" into one breath.\n\n## A Quick Checklist Before You Launch\n\nRun each question through these checks:\n\n- [ ] Does it contain \"and\", \"or\", \"as well as\", or \"both\" joining two things someone could judge differently?\n- [ ] Does it build in a reason, cause, or condition (\"because\", \"if\", \"since\")?\n- [ ] Do the answer options introduce a second concept?\n- [ ] Does it cover two time periods or two separate behaviors?\n- [ ] Could two people give the same answer while thinking about different things?\n- [ ] Have you read it aloud to someone in your audience?\n\nIf any box is ticked, split or rewrite.\n\n## How Koji Helps\n\nKoji is an AI research platform that runs interviews by text or voice and turns them into reports. Several parts of the product are designed to keep double-barreled questions out of your study.\n\n**The brief assistant checks questions before you see them.** When you describe your research goal, Koji's research brief assistant drafts the interview plan and checks every question against a question-design checklist before presenting it: one idea per question (never double-barreled), not leading, not loaded, no double negatives, and no undefined terms. If a draft combines two ideas, it is split into two questions. You can then ask the assistant to change any question in plain language, and it tells you exactly what it changed.\n\n**Structured question types match each half to the right format.** Once a question is split, you can make each part the right type: open-ended, scale, single choice, multiple choice, ranking, or yes/no. A \"work or home\" question becomes one clear multiple-choice question; a \"speed and reliability\" question becomes two scale questions with their own endpoint labels. See the [structured questions guide](/docs/structured-questions-guide) for how each type works.\n\n**Follow-ups happen one at a time.** Instead of stacking three prompts into one question, Koji's AI interviewer asks a question, listens to the answer, then follows up. You set how deep it probes on each question (up to three follow-ups), and for scale questions it can anchor a follow-up to the participant's own score, asking what would change it. That keeps the \"rate it and tell me why\" pattern as two separate turns.\n\n**Preview catches what the checklist misses.** The Preview tab lets you take your own interview, by text or voice, exactly as a participant would. Previews are free and never count as responses. Reading your questions aloud in a voice preview is a quick way to hear a stacked question before your participants do.\n\n**Reports show each answer on its own.** Because each structured question is stored separately, the report shows its own chart (a distribution with mean and median for scales, frequency bars for choice questions), with participant quotes linked to the answers. You never have to guess which half of a merged question a number refers to.\n\nCompared with a static survey tool, the difference is less about catching the mistake once and more about not having to catch it at all: the checklist runs every time the plan is drafted or edited.\n\n## Related Resources\n\n- [Structured Questions Guide](/docs/structured-questions-guide): every question type Koji supports and when to use each\n- [Avoiding Leading Questions](/docs/avoiding-leading-questions): the other wording mistake that skews answers\n- [Survey Question Wording Guide](/docs/survey-question-wording-guide): plain, specific, neutral wording rules\n- [Cognitive Interview Guide](/docs/cognitive-interview-guide): pre-test questions with real people before launch\n- [Questionnaire Design Guide](/docs/questionnaire-design-guide): build a full questionnaire from goals to launch\n- [Open-Ended vs Closed-Ended Questions](/docs/open-ended-vs-closed-ended-questions): pick the right format after you split\n\n## Sources\n\n- Pew Research Center, _Writing Survey Questions_ (pewresearch.org/writing-survey-questions)\n- Menold, N. (2020). Double Barreled Questions: An Analysis of the Similarity of Elements and Effects on Measurement Quality. _Journal of Official Statistics_, 36(4), 855–886.\n- Bassili, J. N., & Scott, B. S. (1996). Response Latency as a Signal to Question Problems in Survey Research. _Public Opinion Quarterly_, 60(3), 390–399.\n- Margolis, M. F. (2022). Born Again but Not Evangelical? _Public Opinion Quarterly_, 86(3), 621 onward.\n- Lavrakas, P. J. (Ed.). _Encyclopedia of Survey Research Methods_, entry \"Double-Barreled Question\". SAGE.\n","category":"Research Methods","lastModified":"2026-10-10T04:55:16.292071+00:00","metaTitle":"Double-Barreled Questions: Examples and How to Fix Them","metaDescription":"A double-barreled question asks about two things but allows one answer. See examples, the research on how they hurt data quality, and how to split them.","keywords":["double-barreled questions","double barreled question examples","double-barreled question survey","compound questions","survey question wording","questionnaire design"],"aiSummary":"A double-barreled question asks about two or more things but accepts only one answer, for example \"How satisfied are you with the speed and reliability of the app?\". It stops participants expressing mixed views, makes answers impossible to interpret, and lowers validity: Menold (2020) found single-element versions measured better, and Margolis (2022) found more than one-third of White Americans classified as evangelical by a double-barreled question rejected that label. Fix it by splitting into one question per concept, picking the right format for each, and pre-testing aloud. Koji's brief assistant checks every draft question for this and splits double-barreled ones automatically.","aiPrerequisites":["Basic familiarity with surveys or user interviews"],"aiLearningOutcomes":["Define a double-barreled question and explain why it produces uninterpretable data","Spot five forms of double-barreled questions, including one-and-a-half-barreled answer options","Rewrite compound questions into single-concept questions with the right format","Avoid stacked questions in interview guides"],"aiDifficulty":"beginner","aiEstimatedTime":"10 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}