Choice and Ranking Questions in AI Interviews: Capture Preference Data at Scale

Choice and ranking questions let you collect structured preference data within an AI interview — the kind of quantitative signal that tells you which option resonates, not just that something resonates. In Koji, these question types work conversationally in both voice and text modes, and their responses aggregate automatically into charts in your research report.

This guide covers four of Koji''s six structured question types: single choice, multiple choice, ranking, and yes/no. For scale questions (NPS, CSAT, ratings), see the Scale Questions Guide. For the full overview of all six types, see the Structured Questions Guide.

The Four Choice-Style Question Types

Each type has a different widget in text mode and a different conversational delivery in voice mode. All produce structured values that aggregate automatically across your interviews.

Single Choice Questions

Single choice questions present a list of options and ask the participant to pick exactly one. They''re ideal when you need to understand which of several things is most important, most used, or most preferred.

Common use cases:

• "Which of the following best describes why you switched to our product?"
• "What is your primary use case for this feature?"
• "Which competitor were you using before?"
• "What was the biggest barrier to getting started?"

Text Mode: Radio Buttons

In text mode, a radio button widget appears after the AI asks the question. Participants tap or click exactly one option. The response is captured immediately as the structured value, and the AI continues the conversation.

Voice Mode: Conversational Selection

In voice mode, the AI reads the question and lists the options aloud:

"What was the primary reason you chose our product? Was it the pricing, the ease of setup, the AI capabilities, or something else?"

The participant responds verbally, and Koji maps their answer to the closest matching option. If they mention something outside the list, and allowOther is enabled, it can be captured as a free-text response.

Configuration

The allowOther option is powerful for discovery research: present your top hypotheses as options, but still capture answers you hadn''t anticipated.

Report: Frequency Bar Chart

Single choice responses aggregate into a frequency bar chart in the report. Each bar represents one option, showing the percentage and count of participants who selected it. At a glance, you see the distribution of preferences across your study.

Multiple Choice Questions

Multiple choice questions let participants select all options that apply. They''re ideal when a participant might have several valid answers simultaneously.

Common use cases:

• "Which of the following features do you use regularly? Select all that apply."
• "What were your reasons for canceling? Check all that apply."
• "Which research methods does your team currently use?"
• "What communication channels do you prefer for support?"

Text Mode: Checkboxes

In text mode, a checkbox widget appears with all options. Participants can select as many as they like and then confirm their selections.

Voice Mode: Flexible Multi-Select

The AI asks the question and explains that the participant can mention multiple options:

"Which of these features do you use regularly? The options are: automated reports, voice interviews, CSV export, or the API. You can mention as many as apply."

The participant might respond: "I mostly use automated reports and the API." Koji extracts both as selected options.

Configuration

Same as single choice:

• options — The list of available choices
• allowOther — Enables a free-text answer for responses outside the list

The structured value for multiple choice is an array of strings (e.g., ["Automated reports", "API"]).

Report: Stacked Frequency Chart

Multiple choice responses aggregate into a stacked frequency chart showing how often each option was selected. Since participants can choose multiple options, percentages may add up to more than 100%. The report shows both count and percentage for each option.

Ranking Questions

Ranking questions ask participants to order a list of options by preference, priority, or importance. They produce the richest data of all the choice types — showing not just which options matter, but in what order.

Common use cases:

• "Please rank these features from most to least important to you."
• "Order these benefits from what matters most to what matters least."
• "Rank these pain points by how severely they affect your workflow."

Text Mode: Drag-and-Drop

In text mode, a drag-and-drop widget lets participants reorder items by dragging them up or down the list. The interface is intuitive and fast.

Voice Mode: Ordered Response

In voice mode, the AI reads the options and asks participants to order them verbally:

"I''ll read you five features, and I''d like you to rank them from most important to least important for your workflow: reporting, voice interviews, the embed widget, CSV export, and the API. How would you order them?"

Participants respond by listing them in order, and Koji extracts the sequence as an ordered array.

Report: Ranked List with Average Position

Ranking responses aggregate into a ranked list with average position across all participants. Each option shows its mean rank — making it easy to see which features are consistently prioritized and which are consistently deprioritized.

For example, if "AI reports" averages position 1.3 out of 5, it''s clearly the most valued feature across your user base. If "CSV export" averages position 4.7 out of 5, it''s consistently lowest priority.

Yes/No Questions

Yes/No is the simplest structured question type. It asks a binary question with exactly two possible responses.

Common use cases:

• "Do you currently use this feature?"
• "Have you recommended our product to someone in the last 3 months?"
• "Were you able to complete your task successfully?"
• "Have you experienced this specific problem?"

Text Mode: Two Buttons

In text mode, two clear tap targets appear: Yes and No. The response is captured immediately.

Voice Mode: Binary Response

The AI asks the question and listens for a yes or no response. Natural language variants ("I have," "not really," "definitely," "not at all") all map correctly to the binary value.

Report: Pie/Donut Chart

Yes/No responses aggregate into a pie or donut chart showing the percentage split. "68% of participants have experienced this problem" is a powerful data point for prioritization discussions.

Adding Probing to Choice Questions

Every choice and ranking question can have follow-up probing configured — asking the AI to explore the why behind the selection.

For a single choice question like "What was your primary reason for switching?", you might configure the probing instruction: "After they answer, ask them to tell you more about what triggered that specific decision." The AI then naturally probes into the story behind the selection.

For a yes/no question like "Have you experienced this problem?", a powerful probing pattern is: if yes, "Tell me about the last time it happened." If no, "What has made that a non-issue for you?"

For ranking questions, after the participant gives their order, the AI can probe: "You ranked [top item] first — can you tell me more about why that one is your priority?"

This is the core advantage of AI interviews over traditional surveys: collect the structured, aggregatable data you need and the qualitative context that explains it — in the same conversation. See the AI Probing Guide for details on configuring follow-up behavior.

Configuration Reference

Best Practices

Keep option lists short. Five to seven options is optimal for both cognitive load and voice delivery. More than ten options in a voice interview is hard to follow.

Use single choice for "pick the most important" questions. Forced choice reveals true priorities — when someone can''t select everything, you learn what actually matters most.

Use multiple choice with allowOther for discovery. Present your top hypotheses as options, but give participants an escape hatch. "Other" responses often surface insights you hadn''t anticipated.

Use ranking when relative priority matters. If you need to know not just what users value, but in what order, ranking produces far richer data than single choice.

Always add probing to choice questions. The structured answer tells you what. The probing conversation tells you why. A researcher who only reads the chart is missing half the data.

Use yes/no as a conversation opener, not a standalone data point. "Have you experienced this problem?" followed by "Tell me about the last time it happened" is far more valuable than a yes/no response in isolation.

Frequently Asked Questions

Can I use choice questions in voice mode? Yes. In voice mode, Koji''s AI reads the options aloud and accepts verbal responses. For ranking questions, the AI reads the list and asks the participant to order them verbally. All responses are extracted and stored as structured values.

What is the maximum number of options I can add? There is no hard technical limit, but we recommend keeping option lists to five to seven items for text mode and five or fewer for voice mode. Longer lists increase cognitive load and reduce response quality, especially in voice interviews where participants cannot see the options.

How does Koji handle the "Other" option in analysis? When allowOther is enabled and a participant selects "Other" or describes something not on the list in voice mode, Koji captures their free-text response as a qualitative note. In the report, "Other" appears as a separate bar in the frequency chart, and the free-text responses are shown as supporting quotes.

Can I combine a ranking question with an open-ended question on the same topic? Absolutely — this is a powerful pattern. The ranking question shows priority order across your whole study; an open-ended question explores the reasoning. For example: "Rank these features by importance" followed by "Tell me more about why your top-ranked feature is your priority."

How are multiple choice responses shown in the report? Each option appears as a bar in the frequency chart, showing the number and percentage of participants who selected it. Because participants can select multiple options, percentages can exceed 100%. The chart clearly shows which options resonate most broadly.

What happens if a participant gives an ambiguous ranking in voice mode? If the ordering is unclear ("I''d say A and B are tied for first"), the AI asks a clarifying follow-up to establish a clear order. The final extracted ranking is stored as an ordered array.

Related Resources

• Structured Questions in AI Interviews — Overview of all six question types
• Scale Questions Guide — How to use NPS, CSAT, and rating scales in your studies
• AI Probing Guide — How Koji''s AI follow-up questioning works
• Interview Mode Guide — Structured, exploratory, and hybrid interview modes
• Generating Research Reports — How your study data becomes a research report
• Understanding Themes and Patterns — How qualitative themes are extracted alongside your quantitative data