How to Customize AI Interview Questions: A Step-by-Step Guide

What customizing interview questions actually means in Koji

Customizing interview questions in Koji means editing the core question list inside your study brief and setting how the AI interviewer asks, follows up on, and probes each one. Unlike a static survey form where you write a fixed list of questions and hope respondents stay on track, Koji turns your questions into a conversational script that adapts to each participant in real time.

You write the question once. The AI handles follow-ups, clarifications, scale anchoring, and probing for specifics — automatically. This is what makes platforms like Koji 10x faster than traditional research tools where every probe has to be manually moderated by a human researcher.

This guide walks through the full customization workflow: editing core questions, choosing question types, tuning AI behavior, and validating the guide before launch.

Quick answer: how to customize questions in 5 steps

1. Open your study and click into the Research Brief.
2. Scroll to the Questions section and click any question to edit text, type, or probing settings.
3. Pick a question type — open-ended for stories, scale for ratings, single/multiple choice for forced selection, ranking for prioritization, yes/no for binary signals.
4. Set follow-up depth (0 to 3) — higher values mean the AI probes deeper on each answer.
5. Preview the interview to test the conversational flow, then publish.

The rest of this guide explains what each setting does, when to use it, and how to avoid common mistakes.

The 6 question types Koji supports

Every customized question carries a type that determines how the AI asks it, how the text UI renders it, and how the report visualizes it. Picking the right type up front is the most important part of customization.

Open-ended

Free-form qualitative answers. The AI probes for specifics, examples, and emotional context. Use for "tell me about the last time…", "walk me through how you…", or any question where the story matters more than a number. Reports show thematic summaries with verbatim quotes.

Scale

Numeric ratings on a configurable range (1-5, 0-10, 1-100). Use for NPS, CSAT, satisfaction, importance, agreement. Set scaleMin, scaleMax, and optional anchor labels. Reports show distribution charts. Turn on the anchor probe to have the AI ask "You gave it a 7 — what would push it to a 10?" automatically.

Single choice

Pick one option from a list. Use for plan tier, role, primary use case, or any mutually exclusive selection. Voice mode reads the options aloud; text mode renders a radio widget. Reports show frequency bars.

Multiple choice

Pick one or more from a list. Use for "which of these have you tried", "which features matter most", or any non-exclusive selection. Reports show stacked frequency.

Ranking

Drag-to-rank an ordered list. Use for prioritization, preference order, or buying-criteria importance. Reports show average rank position and full rank distributions.

Yes / no

Binary signal. Use for screening, awareness checks, or definitive decisions. Reports show pie/donut charts.

Koji tip: Most great interview guides mix 2-3 open-ended questions for the texture with 3-5 structured questions for the comparable metrics. See the structured questions guide for in-depth examples.

How to edit core question text

Click any question in the brief to inline-edit. Good question text follows three rules:

1. Anchor to past behavior, not hypothetical futures. "Tell me about the last time you tried to…" beats "Would you ever use…". This is the Mom Test principle, and Koji's methodology frameworks embed it directly into the AI's probing logic.
2. One idea per question. Compound questions ("How easy was it AND would you recommend it?") confuse both humans and the AI. Split them.
3. Use the participant's language, not yours. "Software you use to track customer interviews" beats "research repository solution".

For starter wording, browse customer interview questions examples or generate a draft with the AI interview questions generator.

Tuning AI follow-up probing per question

Every question has a probing setting that controls how aggressively the AI follows up. The settings:

• maxFollowUps: 0 — Just ask the question, accept the first answer, move on. Use for screener and yes/no questions where probing wastes time.
• maxFollowUps: 1 (default) — One natural follow-up. Catches surface-level answers and asks for one example or one piece of reasoning.
• maxFollowUps: 2 — Two probes. Best for the 2-3 most important questions in the guide.
• maxFollowUps: 3 — Deep probing. Use sparingly; only for the single most strategic question because it adds 60-90 seconds of conversation each time.

You can also pass custom instructions like "If they mention a workaround, ask how often they use it" — the AI uses these as guardrails when deciding what to probe. See probing and follow-up questions for examples.

Customizing for voice vs text interviews

Koji runs the same customized guide in voice mode (real spoken conversation) and text mode (chat-style messaging). One key difference:

• Voice mode asks every question conversationally. Choice options are read aloud. The participant speaks naturally.
• Text mode renders structured questions as interactive widgets — sliders for scales, radio buttons for single choice, checkboxes for multiple choice, drag-to-rank for ranking. Open-ended questions appear as normal chat prompts.

Most teams customize one guide and let respondents pick the modality. Compare voice vs text interviews for when each mode shines.

Customizing a template guide for a new audience

Templates are the fastest way to start. Pick from research interview templates, then customize:

1. Swap the persona language. Replace "B2B SaaS buyer" with "ecommerce shopper" everywhere in question text.
2. Re-rank the questions. Move the most strategic question to position 3-4 (after rapport, before fatigue).
3. Adjust probing depth. A 15-minute discovery interview can afford maxFollowUps: 2 on a few questions; a 5-minute pulse check should stay at 1 across the board.
4. Update the methodology framework. Switching from a Mom Test guide to a Jobs to be Done guide changes the AI's entire probing posture — use the framework toggle inside the brief.

Testing before you launch

Always preview your customized guide before sharing the link. From the study page:

1. Click Preview Interview to run yourself through the guide in either modality.
2. Try giving deliberately vague answers — watch how the AI probes. If probing feels too aggressive, lower maxFollowUps. If it gives up too quickly, raise it.
3. Check that every required question is reached even with a slow conversation.
4. If you're using a structured question type, verify the widget renders correctly in text mode.

A 5-minute preview catches 90% of guide issues before you spend credits on real participants.

Common customization mistakes

• Too many core questions. 10+ questions cause drop-off. Cut to 5-7 strong ones and let probing do the rest.
• Mixing screening into the interview. Use screener questions up front to filter respondents — keep the core guide focused on insight gathering.
• Setting every question to maxFollowUps: 3. This balloons interview length to 25-30 minutes. Pick your top 2-3 questions for deep probing.
• Editing live without preview. Always preview after changes; widget rendering and probing behavior can surprise you.

Related Resources

• Structured Questions Guide — full reference for the 6 question types
• Writing Interview Questions — principles for question wording
• AI Interview Questions Generator — draft a guide from a research goal
• Customer Interview Questions Examples — proven starter questions
• Probing and Follow-Up Questions — how AI follow-ups work
• AI Interviewer Tuning Guide — deeper AI behavior controls
• Editing the Brief Manually — full brief-editing reference