How to Customize AI Interview Questions: A Step-by-Step Guide

What customizing interview questions actually means in Koji

Customizing interview questions in Koji means editing the core question list inside your study brief and setting how the AI interviewer asks, follows up on, and probes each one. Unlike a static survey form where you write a fixed list of questions and hope respondents stay on track, Koji turns your questions into a conversational script that adapts to each participant in real time.

You write the question once. The AI handles follow-ups, clarifications, scale anchoring and probing for specifics on its own, so no moderator has to sit in on every session.

This guide walks through the full customization workflow: editing core questions, choosing question types, setting options and scales, tuning AI behavior, and validating the guide before launch.

Quick answer: how to customize questions in 5 steps

1. Open your study and click into the Research Brief.
2. Scroll to the Questions section and click any question to edit its text, type or follow-up settings. You can also ask the research assistant in plain words ("shuffle the options on question 3", "make question 2 a 0 to 10 scale").
3. Pick a question type: open-ended for stories, scale for ratings, single or multiple choice for selections, ranking for prioritization, yes/no for binary signals, or a price test to find what people would pay.
4. Set Follow-Up Questions (None, or 1 to 3 follow-ups). Higher values mean the AI probes deeper on each answer.
5. Preview the interview to test the conversational flow, then publish.

The rest of this guide explains what each setting does, when to use it, and how to avoid common mistakes.

The question types Koji supports

Every question has a type that decides how the AI asks it, how the text interview shows it, and how the report charts it. Picking the right type up front is the most important part of customization.

Open-ended

Free-form qualitative answers. The AI probes for specifics, examples and emotional context. Use for "tell me about the last time…", "walk me through how you…", or any question where the story matters more than a number. Reports show thematic summaries with verbatim quotes.

Scale

Numeric ratings for NPS, CSAT, satisfaction, importance or agreement. Pick the range from the scale menu on the question: the four common ranges come first (1 to 5 for satisfaction, 1 to 7 for agreement, 0 to 10 for NPS, and 1 to 10), followed by shorter ranges from 1 to 2 up to 1 to 9. Give each end a label ("Very dissatisfied", "Very satisfied") in the Low label and High label fields. The labels move with the ends, so if you change a 1 to 5 scale to 1 to 7, your labels stay on the lowest and highest points. When you ask the research assistant to add or reword a scale question, it sets both end labels too, and keeps the ones you wrote when it rewords the question. Reports show distribution charts. Turn on Score-aware probing to have the AI follow up differently on high and low scores ("You gave it a 7. What would push it to a 10?").

Single choice

Pick one option from a list. Use for role, primary use case, or any mutually exclusive selection. Voice mode reads the options aloud; text mode shows a radio list. Reports show frequency bars.

Multiple choice

Pick one or more from a list. Use for "which of these have you tried", "which features matter most", or any non-exclusive selection. By default participants pick any number. Reports show stacked frequency. See "Setting how many options people pick" below for exact and ranged limits.

Ranking

Drag-to-rank an ordered list. Use for prioritization, preference order, or buying-criteria importance. Reports show average rank position and full rank distributions.

Yes / no

Binary signal. Use for awareness checks or definitive decisions. Reports show pie or donut charts.

Price test (Gabor-Granger)

Enter a ladder of 3 to 10 prices and Koji asks each participant how likely they are to buy at one price at a time, moving up or down until it finds the highest price they would still buy at, then asks why. The report shows the share who would buy at each price and the revenue each price implies. Set one up with the price test setup guide, or read the Gabor-Granger pricing method for the method behind it.

Koji tip: Most great interview guides mix 2-3 open-ended questions for the texture with 3-5 structured questions for the comparable metrics. See the structured questions guide for in-depth examples.

Setting how many options people pick

Under the options of a multiple choice question, a menu sets how many a participant picks:

• Any number (the default).
• Exact number: "Choose exactly 3". Use it for "pick your top three".
• Range: set an "at least" and an "at most", such as 2 to 4, or only an upper limit ("Choose up to 2").

Exact numbers and ranges need at least three options. In a text interview the participant sees the rule ("Choose 3", "Choose up to 3", "Choose 2 to 4") and the answer has to fit it. In a voice interview the AI asks them to adjust when they pick too many or too few. Pick limits are not available on screening questions, where each option already decides who qualifies.

You can set the same thing by asking the research assistant: "make question 4 pick exactly three" or "let people choose up to two".

Setting the order options appear in

People pick the first options they see more often than the ones further down. For single choice, multiple choice and ranking questions, the order menu under the options gives you three settings:

• Order as shown: everyone sees the list as you wrote it. Use it for ordered answers such as frequency, agreement, price bands or time ranges.
• Randomize order: each participant sees the options in a different order. Use it for unordered lists such as brands, features, reasons or tools.
• Randomize except last: shuffles the list but keeps the last 1, 2 or 3 options in place, for "Other" or "None of the above". Keeping the last 1 needs at least three options, and each extra kept option needs one more.

Each participant keeps their order if they refresh the page, and reports always list the options in your original order. The research assistant applies these settings when you ask ("shuffle the options but keep None of the above last"). For when randomizing helps and when it hurts, read the survey randomization guide.

How to edit core question text

Click any question in the brief to edit it inline. Good question text follows three rules:

1. Anchor to past behavior, not hypothetical futures. "Tell me about the last time you tried to…" beats "Would you ever use…". This is the Mom Test principle, and Koji's interviewer follows it when it probes.
2. One idea per question. Compound questions ("How easy was it AND would you recommend it?") confuse both humans and the AI. Split them.
3. Use the participant's language, not yours. "Software you use to track customer interviews" beats "research repository solution".

Keep the scale framing out of the question text. Write "How likely are you to recommend us to a colleague?" and set the range and end labels on the question, rather than adding "(1 = not at all, 10 = very)" to the wording.

For starter wording, browse customer interview questions examples or generate a draft with the AI interview questions generator.

Tuning AI follow-up probing per question

Every question has a Follow-Up Questions setting that controls how far the AI follows up:

• None (skip): ask the question, accept the first answer, move on. Use for quick factual and yes/no questions where probing wastes time.
• 1 follow-up (the default): one natural follow-up. Catches surface-level answers and asks for one example or one piece of reasoning.
• 2 follow-ups: best for the 2-3 most important questions in the guide.
• 3 follow-ups: deep probing. Use sparingly, only for the single most strategic question, because it adds noticeably to the length of every interview.

You can also write custom follow-up guidance such as "If they mention a workaround, ask how often they use it". The AI uses it when deciding what to probe. See probing and follow-up questions for examples.

Customizing for voice vs text interviews

Koji runs the same customized guide in voice mode (real spoken conversation) and text mode (chat-style messaging). One key difference:

• Voice mode asks every question conversationally. Choice options are read aloud. The participant speaks naturally.
• Text mode shows structured questions as interactive widgets: buttons or sliders for scales, radio buttons for single choice, checkboxes for multiple choice, drag-to-rank for ranking. Open-ended questions appear as normal chat prompts.

Most teams customize one guide and let respondents pick the modality. Compare voice vs text interviews for when each mode shines.

Customizing a template guide for a new audience

Templates are the fastest way to start. Pick from research interview templates, then customize:

1. Swap the persona language. Replace "B2B SaaS buyer" with "ecommerce shopper" everywhere in question text.
2. Re-rank the questions. Move the most strategic question to position 3-4 (after rapport, before fatigue).
3. Adjust probing depth. A 15-minute discovery interview can afford 2 follow-ups on a few questions; a 5-minute pulse check should stay at 1 across the board.
4. Check the option settings. Lists copied from a template keep their order and pick limits, so review them for the new audience.

Testing before you launch

Always preview your customized guide before sharing the link. From the study page:

1. Click Preview Interview to run yourself through the guide in either modality.
2. Try giving deliberately vague answers and watch how the AI probes. If probing feels too aggressive, lower the follow-ups. If it gives up too quickly, raise them.
3. Check that every required question is reached even with a slow conversation.
4. In text mode, check that each structured question shows the right widget, the pick rule and the scale labels.

A 5-minute preview catches most guide issues before you spend credits on real participants.

Common customization mistakes

• Too many core questions. 10+ questions cause drop-off. Cut to 5-7 strong ones and let probing do the rest.
• Mixing screening into the interview. Use screener questions up front to filter respondents and keep the core guide focused on insight gathering.
• Setting every question to 3 follow-ups. This balloons interview length to 25-30 minutes. Pick your top 2-3 questions for deep probing.
• Shuffling an ordered list. Randomize brands and features, never "Daily, Weekly, Monthly".
• Editing live without preview. Always preview after changes; widget rendering and probing behavior can surprise you.

Related Resources

• Structured Questions Guide: full reference for every question type
• Survey Randomization Guide: when to shuffle options
• Writing Interview Questions: principles for question wording
• AI Interview Questions Generator: draft a guide from a research goal
• Customer Interview Questions Examples: proven starter questions
• Probing and Follow-Up Questions: how AI follow-ups work
• AI Interviewer Tuning Guide: deeper AI behavior controls