{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-07T15:14:07.036Z"},"content":[{"type":"documentation","id":"59425176-3779-434c-8abd-7696f31c60c4","slug":"choice-ranking-questions-guide","title":"Choice and Ranking Questions in AI Interviews: Capture Preference Data at Scale","url":"https://www.koji.so/docs/choice-ranking-questions-guide","summary":"Koji supports four choice-style structured question types: single choice (frequency bar chart), multiple choice (stacked frequency chart), ranking (ranked list with average position), and yes/no (pie chart). All work conversationally in text and voice mode with automatic AI probing for qualitative context.","content":"Choice and ranking questions let you collect structured preference data within an AI interview — the kind of quantitative signal that tells you *which* option resonates, not just *that* something resonates. In Koji, these question types work conversationally in both voice and text modes, and their responses aggregate automatically into charts in your research report.\n\nThis guide covers four of Koji''s six structured question types: single choice, multiple choice, ranking, and yes/no. For scale questions (NPS, CSAT, ratings), see the [Scale Questions Guide](/docs/scale-questions-guide). For the full overview of all six types, see the [Structured Questions Guide](/docs/structured-questions-guide).\n\n## The Four Choice-Style Question Types\n\n| Type | How It Works | Report Visualization |\n|---|---|---|\n| **Single choice** | Participant picks exactly one option | Frequency bar chart |\n| **Multiple choice** | Participant picks one or more options | Stacked frequency chart |\n| **Ranking** | Participant orders options by preference | Ranked list with average position |\n| **Yes/No** | Binary question | Pie/donut chart |\n\nEach type has a different widget in text mode and a different conversational delivery in voice mode. All produce structured values that aggregate automatically across your interviews.\n\n## Single Choice Questions\n\nSingle choice questions present a list of options and ask the participant to pick exactly one. They''re ideal when you need to understand which of several things is most important, most used, or most preferred.\n\n**Common use cases:**\n- \"Which of the following best describes why you switched to our product?\"\n- \"What is your primary use case for this feature?\"\n- \"Which competitor were you using before?\"\n- \"What was the biggest barrier to getting started?\"\n\n### Text Mode: Radio Buttons\n\nIn text mode, a radio button widget appears after the AI asks the question. Participants tap or click exactly one option. The response is captured immediately as the structured value, and the AI continues the conversation.\n\n### Voice Mode: Conversational Selection\n\nIn voice mode, the AI reads the question and lists the options aloud:\n\n> \"What was the primary reason you chose our product? Was it the pricing, the ease of setup, the AI capabilities, or something else?\"\n\nThe participant responds verbally, and Koji maps their answer to the closest matching option. If they mention something outside the list, and `allowOther` is enabled, it can be captured as a free-text response.\n\n### Configuration\n\n| Setting | What It Does |\n|---|---|\n| `options` | The list of choices (2–10 options recommended) |\n| `allowOther` | Adds a free-text \"Other\" option |\n\nThe `allowOther` option is powerful for discovery research: present your top hypotheses as options, but still capture answers you hadn''t anticipated.\n\n### Report: Frequency Bar Chart\n\nSingle choice responses aggregate into a **frequency bar chart** in the report. Each bar represents one option, showing the percentage and count of participants who selected it. At a glance, you see the distribution of preferences across your study.\n\n## Multiple Choice Questions\n\nMultiple choice questions let participants select all options that apply. They''re ideal when a participant might have several valid answers simultaneously.\n\n**Common use cases:**\n- \"Which of the following features do you use regularly? Select all that apply.\"\n- \"What were your reasons for canceling? Check all that apply.\"\n- \"Which research methods does your team currently use?\"\n- \"What communication channels do you prefer for support?\"\n\n### Text Mode: Checkboxes\n\nIn text mode, a checkbox widget appears with all options. Participants can select as many as they like and then confirm their selections.\n\n### Voice Mode: Flexible Multi-Select\n\nThe AI asks the question and explains that the participant can mention multiple options:\n\n> \"Which of these features do you use regularly? The options are: automated reports, voice interviews, CSV export, or the API. You can mention as many as apply.\"\n\nThe participant might respond: \"I mostly use automated reports and the API.\" Koji extracts both as selected options.\n\n### Configuration\n\nSame as single choice:\n- `options` — The list of available choices\n- `allowOther` — Enables a free-text answer for responses outside the list\n\nThe structured value for multiple choice is an array of strings (e.g., `[\"Automated reports\", \"API\"]`).\n\n### Report: Stacked Frequency Chart\n\nMultiple choice responses aggregate into a **stacked frequency chart** showing how often each option was selected. Since participants can choose multiple options, percentages may add up to more than 100%. The report shows both count and percentage for each option.\n\n## Ranking Questions\n\nRanking questions ask participants to order a list of options by preference, priority, or importance. They produce the richest data of all the choice types — showing not just *which* options matter, but in *what order*.\n\n**Common use cases:**\n- \"Please rank these features from most to least important to you.\"\n- \"Order these benefits from what matters most to what matters least.\"\n- \"Rank these pain points by how severely they affect your workflow.\"\n\n### Text Mode: Drag-and-Drop\n\nIn text mode, a drag-and-drop widget lets participants reorder items by dragging them up or down the list. The interface is intuitive and fast.\n\n### Voice Mode: Ordered Response\n\nIn voice mode, the AI reads the options and asks participants to order them verbally:\n\n> \"I''ll read you five features, and I''d like you to rank them from most important to least important for your workflow: reporting, voice interviews, the embed widget, CSV export, and the API. How would you order them?\"\n\nParticipants respond by listing them in order, and Koji extracts the sequence as an ordered array.\n\n### Report: Ranked List with Average Position\n\nRanking responses aggregate into a **ranked list with average position** across all participants. Each option shows its mean rank — making it easy to see which features are consistently prioritized and which are consistently deprioritized.\n\nFor example, if \"AI reports\" averages position 1.3 out of 5, it''s clearly the most valued feature across your user base. If \"CSV export\" averages position 4.7 out of 5, it''s consistently lowest priority.\n\n## Yes/No Questions\n\nYes/No is the simplest structured question type. It asks a binary question with exactly two possible responses.\n\n**Common use cases:**\n- \"Do you currently use this feature?\"\n- \"Have you recommended our product to someone in the last 3 months?\"\n- \"Were you able to complete your task successfully?\"\n- \"Have you experienced this specific problem?\"\n\n### Text Mode: Two Buttons\n\nIn text mode, two clear tap targets appear: Yes and No. The response is captured immediately.\n\n### Voice Mode: Binary Response\n\nThe AI asks the question and listens for a yes or no response. Natural language variants (\"I have,\" \"not really,\" \"definitely,\" \"not at all\") all map correctly to the binary value.\n\n### Report: Pie/Donut Chart\n\nYes/No responses aggregate into a **pie or donut chart** showing the percentage split. \"68% of participants have experienced this problem\" is a powerful data point for prioritization discussions.\n\n## Adding Probing to Choice Questions\n\nEvery choice and ranking question can have follow-up probing configured — asking the AI to explore the *why* behind the selection.\n\nFor a single choice question like \"What was your primary reason for switching?\", you might configure the probing instruction: \"After they answer, ask them to tell you more about what triggered that specific decision.\" The AI then naturally probes into the story behind the selection.\n\nFor a yes/no question like \"Have you experienced this problem?\", a powerful probing pattern is: if yes, \"Tell me about the last time it happened.\" If no, \"What has made that a non-issue for you?\"\n\nFor ranking questions, after the participant gives their order, the AI can probe: \"You ranked [top item] first — can you tell me more about why that one is your priority?\"\n\nThis is the core advantage of AI interviews over traditional surveys: collect the structured, aggregatable data you need *and* the qualitative context that explains it — in the same conversation. See the [AI Probing Guide](/docs/ai-probing-guide) for details on configuring follow-up behavior.\n\n## Configuration Reference\n\n| Setting | Applies To | Description |\n|---|---|---|\n| `options` | Single choice, Multiple choice, Ranking | Array of option strings (2–10 recommended) |\n| `allowOther` | Single choice, Multiple choice | Adds a free-text \"Other\" entry |\n| `maxFollowUps` | All types | Number of follow-up probing questions (0–3) |\n| `instructions` | All types | Custom AI probing instructions for this question |\n\n## Best Practices\n\n**Keep option lists short.** Five to seven options is optimal for both cognitive load and voice delivery. More than ten options in a voice interview is hard to follow.\n\n**Use single choice for \"pick the most important\" questions.** Forced choice reveals true priorities — when someone can''t select everything, you learn what actually matters most.\n\n**Use multiple choice with `allowOther` for discovery.** Present your top hypotheses as options, but give participants an escape hatch. \"Other\" responses often surface insights you hadn''t anticipated.\n\n**Use ranking when relative priority matters.** If you need to know not just what users value, but in what order, ranking produces far richer data than single choice.\n\n**Always add probing to choice questions.** The structured answer tells you *what*. The probing conversation tells you *why*. A researcher who only reads the chart is missing half the data.\n\n**Use yes/no as a conversation opener, not a standalone data point.** \"Have you experienced this problem?\" followed by \"Tell me about the last time it happened\" is far more valuable than a yes/no response in isolation.\n\n## Frequently Asked Questions\n\n**Can I use choice questions in voice mode?**\nYes. In voice mode, Koji''s AI reads the options aloud and accepts verbal responses. For ranking questions, the AI reads the list and asks the participant to order them verbally. All responses are extracted and stored as structured values.\n\n**What is the maximum number of options I can add?**\nThere is no hard technical limit, but we recommend keeping option lists to five to seven items for text mode and five or fewer for voice mode. Longer lists increase cognitive load and reduce response quality, especially in voice interviews where participants cannot see the options.\n\n**How does Koji handle the \"Other\" option in analysis?**\nWhen `allowOther` is enabled and a participant selects \"Other\" or describes something not on the list in voice mode, Koji captures their free-text response as a qualitative note. In the report, \"Other\" appears as a separate bar in the frequency chart, and the free-text responses are shown as supporting quotes.\n\n**Can I combine a ranking question with an open-ended question on the same topic?**\nAbsolutely — this is a powerful pattern. The ranking question shows priority order across your whole study; an open-ended question explores the reasoning. For example: \"Rank these features by importance\" followed by \"Tell me more about why your top-ranked feature is your priority.\"\n\n**How are multiple choice responses shown in the report?**\nEach option appears as a bar in the frequency chart, showing the number and percentage of participants who selected it. Because participants can select multiple options, percentages can exceed 100%. The chart clearly shows which options resonate most broadly.\n\n**What happens if a participant gives an ambiguous ranking in voice mode?**\nIf the ordering is unclear (\"I''d say A and B are tied for first\"), the AI asks a clarifying follow-up to establish a clear order. The final extracted ranking is stored as an ordered array.\n\n## Related Resources\n\n- [Structured Questions in AI Interviews](/docs/structured-questions-guide) — Overview of all six question types\n- [Scale Questions Guide](/docs/scale-questions-guide) — How to use NPS, CSAT, and rating scales in your studies\n- [AI Probing Guide](/docs/ai-probing-guide) — How Koji''s AI follow-up questioning works\n- [Interview Mode Guide](/docs/interview-mode-guide) — Structured, exploratory, and hybrid interview modes\n- [Generating Research Reports](/docs/generating-research-reports) — How your study data becomes a research report\n- [Understanding Themes and Patterns](/docs/understanding-themes-patterns) — How qualitative themes are extracted alongside your quantitative data\n\n## Further reading on the blog\n\n- [B2C User Research: How to Understand Consumer Behavior at Scale (2026)](/blog/b2c-user-research-guide-2026) — B2C user research is systematically underinvested at most consumer companies. While B2B teams run structured customer discovery as a matter \n- [How to Run Customer Exit Interviews: The Complete Guide (2026)](/blog/customer-exit-interviews-guide-2026) — Customer exit interviews reveal the real reasons customers churn — not the polished answer they gave on your cancellation form. Here is how \n- [Google Forms to AI Interviews: A Complete Migration Guide](/blog/google-forms-to-ai-interviews) — Why teams are moving beyond Google Forms and how to convert your existing forms into AI-powered research conversations in 30 seconds.\n\n<!-- further-reading:blog -->\n","category":"Study Design","lastModified":"2026-06-05T03:21:10.009258+00:00","metaTitle":"Choice and Ranking Questions in AI Interviews: Preference Data at Scale | Koji","metaDescription":"Learn how to use single choice, multiple choice, ranking, and yes/no questions in Koji AI interviews. Collect structured preference data with automatic report charts across all participants.","keywords":["multiple choice interview questions","ranking questions user research","single choice survey AI","preference ranking research tool","yes no questions interview","choice questions AI research"],"aiSummary":"Koji supports four choice-style structured question types: single choice (frequency bar chart), multiple choice (stacked frequency chart), ranking (ranked list with average position), and yes/no (pie chart). All work conversationally in text and voice mode with automatic AI probing for qualitative context.","aiPrerequisites":["Familiarity with creating Koji studies","Understanding of the Structured Questions overview"],"aiLearningOutcomes":["Use single choice questions to identify top preferences across participants","Configure multiple choice questions with allowOther for discovery research","Design ranking questions to reveal priority order","Combine choice questions with AI probing to capture both quantitative and qualitative data","Read frequency charts and ranked lists in your research report"],"aiDifficulty":"beginner","aiEstimatedTime":"9 minutes"}],"pagination":{"total":1,"returned":1,"offset":0}}