Structured Questions in AI Interviews

Structured questions are what make Koji fundamentally different from both traditional surveys and standard interview tools. They let you embed quantitative data collection — scales, ratings, multiple choice, ranking — directly inside a natural AI-powered conversation. Every structured response is automatically followed up by the AI, so you capture both the number and the reasoning behind it in a single interaction.

The Problem with Traditional Approaches

Research has traditionally forced a painful choice: run a survey to collect structured, chartable data (but miss the "why"), or run an interview to understand motivations and context (but lose the ability to aggregate and compare across participants). Most teams end up running both — doubling the cost, timeline, and participant burden.

Koji's structured questions eliminate that tradeoff. A single AI interview captures quantitative metrics that aggregate into charts and distributions, alongside rich qualitative context from conversational follow-up. You get the benchmarkable numbers your stakeholders want and the deep human insight your research needs — from the same conversation.

How It Works

When you add a structured question to your study, two things happen during every interview:

1. The participant provides a structured response — a number on a scale, a selected option, a ranked list, or a yes/no answer. In text mode, this appears as an interactive widget (slider, radio buttons, checkboxes, drag-to-rank) embedded naturally in the chat. In voice mode, the AI handles everything conversationally — it asks the question, hears the answer, and extracts the structured value.

2. The AI immediately probes for the "why" — based on the response, the AI follows up with contextual probing. A participant who gives a low satisfaction score gets different follow-up questions than one who gives a high score. This probing is what transforms a data point into an insight.

Every response is captured twice: as a structured value (the number, choice, or ranking) for aggregate reporting, and as qualitative context from the follow-up conversation. When you generate a report, structured questions produce charts — distribution charts for scales, bar charts for choices, ranked lists for rankings — alongside AI-synthesized qualitative insights explaining the patterns in the data.

The Six Question Types

Koji supports six structured question types, each optimized for a different kind of data:

Open Ended — The default question type. Pure qualitative, no structured value. The AI asks the question and probes for depth. Best for exploratory discovery where you don't yet know the right categories or scales.

Scale — A numeric rating (e.g., 1–5, 1–10, NPS 0–10). You define the range and can add labels for the endpoints (e.g., "Very Unlikely" to "Very Likely"). Reports show a distribution chart with average and median. Best for NPS, CSAT, satisfaction scores, and any metric you want to track over time.

Single Choice — Pick one option from a list. Reports show a frequency bar chart. Best for segmentation questions like "which of these describes your role?" or "which feature do you use most?"

Multiple Choice — Pick any number of options from a list. You can also enable an "Other" free-text option for responses you didn't anticipate. Reports show a stacked frequency chart. Best for "what challenges do you face?" (select all that apply).

Ranking — Order a list of items by preference or priority. In text mode, participants drag items into their preferred order. Reports show average position for each item. Best for feature prioritization and preference ordering.

Yes/No — A binary question. Reports show a pie or donut chart. Best for confirmation questions ("Have you tried X?") that open up more nuanced follow-up conversation.

Anchor Probing: The Power of Scale Questions

Scale questions have a unique probing feature called anchor probing. When enabled, after a participant gives their rating, the AI asks a question like:

"You said 7 — what would need to change to make it a 9 or 10?"

This technique is powerful because it:

• Surfaces specific, actionable improvements rather than vague dissatisfaction
• Anchors the follow-up in the participant's own evaluation framework
• Reveals the gap between current experience and ideal experience
• Produces insights that map directly to product or service improvements

Anchor probing works on any scale question and is especially valuable for NPS, satisfaction, and likelihood-to-recommend metrics. You can enable it in the probing configuration for any scale question in the brief editor.

Step-by-Step Guide

1. Open your study's research brief From your study dashboard, open the research brief editor and navigate to the Questions tab. For details on the editor, see Editing the Brief Manually.

2. Add a new question Click the Add Question button. Enter your question text and select the question type from the dropdown.

3. Configure type-specific options For Scale questions, set the min/max values and optional endpoint labels (e.g., "Very Unlikely" to "Very Likely"). For Choice or Ranking questions, add your option list. You can enable "Allow Other" on choice questions to capture write-in responses.

4. Set AI probing depth Each question has a probing configuration. Set the maximum number of follow-up questions the AI will ask (0 = no probing, 1–3 = increasing depth). You can add specific probing instructions, like "If the participant gives a low score, ask what would need to change." For scale questions, consider enabling anchor probing.

5. Order your questions Drag questions into the sequence that makes sense for the conversation flow. Open-ended discovery questions work well at the start; structured ratings fit naturally toward the end once rapport is established.

6. Publish your study Once you're satisfied with your question mix, publish the study as normal. Question IDs are assigned at publish time and remain stable across the life of the study, ensuring accurate data aggregation.

"Prefer Not to Answer"

Every structured question includes a "Prefer not to answer" option. Participants can always skip any question they're uncomfortable answering. Skipped responses are tracked separately from answered responses in your report data, so your aggregate statistics (averages, distributions, frequencies) remain accurate and are not skewed by missing data.

This is deliberate — research ethics require that participants never feel forced to answer any question. The "prefer not to answer" option is always available and cannot be disabled.

Key Things to Know

• Text vs. voice behavior: In text mode, quantitative questions render as interactive widgets — sliders, radio buttons, checkboxes, or drag-to-rank lists. In voice mode, all questions are handled conversationally and the AI extracts structured values from spoken answers. Both modes capture the same data.

• It's a conversation, not a survey: Unlike traditional survey tools, structured questions in Koji are embedded within a natural, flowing AI conversation. The AI builds rapport, asks open-ended questions for context, and then introduces structured questions at natural conversation points. Participants experience a conversation with data capture moments, not a form with chat bolted on.

• Question IDs are stable: Each question has a stable identifier assigned at publish time that flows through from the brief to the interview to the report. This means your reports accurately aggregate data across all interviews, even if question text is lightly edited.

• AI probing follows structured answers: After a participant selects their answer via widget (or speaks it in voice mode), the AI adapts its follow-up based on what they selected. A participant who gives a low satisfaction score gets different follow-up questions than one who gives a high score.

• Reports automatically visualize structured data: When you generate a research report, each structured question gets the right visualization: distribution charts for scales, bar charts for choices, ranked lists for rankings — no manual chart-building required.

• Sections help with longer studies: If your study has many questions, use section grouping to organize them (e.g., "Background," "Product Experience," "NPS"). This helps the AI present questions in a coherent, conversational arc.

Tips and Best Practices

• Put qualitative questions first: Build rapport and let participants share naturally before asking them to rate anything. Open-ended questions early in the interview produce richer context for the structured answers that follow.
• Use scale questions for benchmarks: If you're tracking a metric over time (NPS, CSAT, feature satisfaction), use a Scale question with consistent parameters so you can compare across studies.
• Don't over-structure: Resist the urge to turn every question into multiple choice. Structured questions are most powerful when used sparingly — they anchor key metrics while open-ended questions carry the qualitative depth. Most studies benefit from 1–3 structured questions mixed with several open-ended ones.
• Always enable probing on quantitative questions: A scale or choice question without AI follow-up is just a survey. The real value is in the "why" — configure at least one follow-up for every quantitative question.
• Enable anchor probing on key scales: For your most important scale questions (NPS, overall satisfaction), turn on anchor probing. The "what would change your score?" follow-up produces some of the most actionable insights in any study.
• Mix and match for richer reports: A study that ends with an NPS score and immediately probes the reason behind it gives you both the benchmarkable number and the insight — something no traditional survey tool can deliver.
• Keep choice lists focused: For single and multiple choice questions, aim for 3–7 clear, mutually exclusive options. More than that and participants spend too much time reading instead of thinking. Use "Allow Other" as a catch-all.

Related Articles

• Understanding the Research Brief — how structured questions fit into the overall brief
• Editing the Brief Manually — using the question editor to configure structured questions
• Text Interview Experience — how structured questions appear as interactive widgets
• Voice Interview Experience — how structured questions work in voice mode
• Choosing a Methodology — which methodologies pair best with structured data capture
• Publishing Your Study — how question IDs are assigned at publish time
• Generating Research Reports — how structured data becomes charts and visualizations
• AI-Generated Insights — how qualitative follow-ups are synthesized

Frequently Asked Questions

Q: Do structured questions work in both voice and text mode? A: Yes, but they work differently. In text mode, quantitative questions appear as interactive widgets — sliders, radio buttons, checkboxes, or drag-to-rank lists. In voice mode, the AI handles all questions conversationally and extracts structured values from spoken responses. Both modes capture identical data for reporting.

Q: Can I mix structured and open-ended questions in the same study? A: Absolutely — this is the recommended approach. Open-ended questions explore freely and build rapport; structured questions anchor key metrics. A typical study might start with 2–3 open-ended discovery questions, then include 1–2 scale or choice questions toward the end.

Q: How does the AI use structured question data in reports? A: When you generate a research report, each structured question produces a chart (distribution for scales, bar chart for choices) alongside AI-synthesized qualitative context from follow-up conversations. You get both the numbers and the reasoning in a single report.

Q: Can participants skip a structured question? A: Yes. A "Prefer not to answer" option is always available on every structured question and cannot be disabled. Skipped questions are tracked separately from answered questions in the report, so your aggregate data stays accurate.

Q: How many structured questions should I include in a study? A: Most studies benefit from 1–3 structured questions mixed with several open-ended ones. More than 5 structured questions can make the conversation feel survey-like, which reduces the qualitative depth participants share.

Q: What is the difference between structured questions in Koji and a regular survey? A: In a regular survey, structured questions stand alone — participants select an answer and move on. In Koji, every structured question is embedded in a natural AI conversation and followed by intelligent probing. The AI adapts its follow-up based on the specific response given, capturing both the quantitative data point and the qualitative reasoning behind it. It's the difference between knowing your NPS is 7 and knowing your NPS is 7 because the onboarding was confusing but the support team was excellent.

Q: What is anchor probing? A: Anchor probing is a feature specific to scale questions. After a participant gives a rating, the AI asks what would need to change to improve that rating (e.g., "You said 7 — what would make it a 9?"). This surfaces specific, actionable insights about the gap between current and ideal experience. You can enable it in the probing configuration for any scale question.

Further reading on the blog

• The 11 Best AI Survey Tools in 2026 (Ranked & Reviewed) — We tested 11 AI-powered survey platforms head-to-head — from AI question generation to automatic open-ended analysis to fully AI-moderated c
• Koji vs Insight7: AI Customer Research Platforms Compared (2026) — Insight7 analyzes recorded calls. Koji runs the interviews itself — AI-moderated voice conversations from €29/month. Compare features, prici
• Koji vs Listen Labs: AI Interview Platforms Compared (2026) — Listen Labs raised $69M and powers enterprise research at brands across the Fortune 500. Koji is the accessible AI-native alternative starti

<!-- further-reading:blog -->