You already have a discussion guide. Some of its questions are going to get you polite, vague or made-up answers, and you have a few days before the sessions start. This post is about fixing that: twelve common faults in interview questions, each with a before and after rewrite and why the new version works. For the full method of writing questions from scratch, read our reference guide, How to Write Great Interview Questions.
The short version: almost every bad interview question fails in one of three ways. It hands the participant an answer (leading, loaded, presupposing), it asks them to imagine instead of remember (hypothetical, future, "usually"), or it asks two things or nothing in particular (double-barreled, too broad, yes/no). The fix is nearly always the same move: ask about one specific thing that already happened, in neutral words.
If you want to try this on your own guide now, paste your questions into a Koji study and ask the research assistant to rewrite any question that is leading or hypothetical. It changes the question you name and tells you what it changed, so you can compare before and after.
Why small wording changes matter
Wording changes answers more than most teams expect. In Loftus and Palmer's classic experiment, people who watched the same car crash estimated the speed at 40.8 mph on average when asked how fast the cars were going when they "smashed" into each other, and 34.0 mph when the verb was "hit". A week later, 32% of the "smashed" group remembered seeing broken glass, against 14% of the "hit" group. There was no broken glass in the film (Journal of Verbal Learning and Verbal Behavior, 1974).
A follow-up by Loftus and Zanni found that asking "Did you see the broken headlight?" made people about twice as likely to say yes (17%) as asking "Did you see a broken headlight?" (7%), again when there was none (Bulletin of the Psychonomic Society, 1975).
Surveys show the same effect at scale. In January 2003, Pew Research Center found 68% of Americans favoured military action in Iraq to end Saddam Hussein's rule. When the question added "even if it meant that U.S. forces might suffer thousands of casualties", support fell to 43% (Pew Research Center, 2003). Same topic, one added clause, 25 points.
An interview is more forgiving than a survey, because you can follow up. But the first question in each section sets the frame for everything after it, so it is worth getting right.
Twelve rewrites

1. Leading
Before: "How frustrating is it when exports fail?"
After: "Tell me about the last time you exported a report. What happened?"
The original assumes exports fail and that it is frustrating. The rewrite lets them tell you whether either is true.
2. Hypothetical
Before: "Would you use a feature that summarised your calls?"
After: "What did you do with your notes after your last call?"
People are poor at predicting what they would do and kind about your ideas. What they did last week is a fact.
3. Future price
Before: "Would you pay a monthly fee for this?"
After: "What do you pay for today to deal with this, if anything?"
A yes to a hypothetical price costs nothing. What they already spend, in money or time, is the real anchor. If you need price evidence, use a pricing method built for it, not one interview question.
4. Double-barreled
Before: "How do you plan and track your sprints?"
After: "How do you plan a sprint?" Then, separately: "How do you keep track once it starts?"
Two questions in one get one answer, and you will not know which part it answered.
5. Presupposing
Before: "What do you dislike about the dashboard?"
After: "When did you last open the dashboard? What were you looking for?"
The original assumes they use it and dislike something. As Loftus and Zanni showed, people go along with what a question takes for granted. Our guide to presupposition in interview questions goes deeper.
6. Loaded words
Before: "How much time do you waste on manual reporting?"
After: "How long did your last report take, from start to finish?"
"Waste" and "manual" tell them what you think. Ask for the number and let them judge it.
7. Closed yes or no
Before: "Is onboarding easy?"
After: "Walk me through your first day with the product."
A yes or no ends the topic. A walk-through gives you the steps, and the friction shows up on its own.
8. "Usually"
Before: "How do you usually choose a supplier?"
After: "Tell me about the last supplier you chose. How did it go?"
"Usually" invites a tidy average that may never have happened. One real instance has detail you can trust.
9. Asking for the solution
Before: "What features would you like us to add?"
After: "What did you have to do outside the product last week to get the job done?"
Participants are experts on their problems, not on your roadmap. Workarounds show you the gap without asking them to design the fix.
10. Too broad
Before: "Tell me about your workflow."
After: "Tell me about the last time you handed a project to a client."
A broad question gets a broad answer. Pick the moment you care about and ask about one instance of it.
11. Jargon
Before: "How do you handle stakeholder alignment in your discovery cadence?"
After: "Before you start a new project, who do you talk to, and about what?"
If they would not say it out loud, do not ask it. Their own words are also data: note what they call things.
12. "Why" that sounds like blame
Before: "Why didn't you finish setting it up?"
After: "What was happening when you stopped setting it up?"
"Why didn't you" puts people on the defensive. Asking about the moment gets the reason without the excuse.
A worked example: one guide, before and after
Here is a short guide for a study on why trial users go quiet, as a team first wrote it:
- "Did you like the product during your trial?"
- "Why didn't you upgrade?"
- "Would more templates have helped?"
- "How do you usually manage your projects and team?"
- "What would make you come back?"
And rewritten:
- "What were you hoping to get done when you signed up?"
- "Tell me about the last time you logged in. What were you trying to do?"
- "What did you use instead, after the trial ended?"
- "Tell me about the last project you started. How did you set it up?"
- "Is there anything I should have asked about your trial and didn't?"
Each original question had at least one fault from the list: closed (1), blame (2), hypothetical and leading (3), double-barreled and "usually" (4), future (5). Every rewrite asks about one real moment, and none of them mentions a feature.
Rewriting follow-ups
Your probes need the same care as your main questions. "So that was annoying?" is a leading probe. "And how did you feel about that?" can push people into an emotion they did not have. Short, neutral follow-ups work best: "What happened next?", "Can you give me an example?", "What did you do then?", "Tell me more about that." Our guide to probing and follow-up questions has more examples, and avoiding leading questions covers the survey side.
A last check before you run the guide
Read every question out loud, then run it against this list.
Photo by Aaron Burden on Unsplash

- Does it ask about one specific thing that already happened?
- Is it one question, not two?
- Could someone answer it honestly without agreeing with you?
- Are there any loaded words (waste, struggle, frustrating, easy)?
- Does it assume something you have not established?
- Would the participant use these words themselves?
- Does it avoid naming your feature or idea?
- Can it be answered with more than yes or no?
Then do one practice run with a colleague, or a test interview, and listen for where the answers go flat. A flat answer usually points to the question before it.
Rewriting questions in Koji
In Koji, your guide lives in the study's questions editor, where each question has its own follow-up setting. You can edit any question by hand, or ask the research assistant to change one ("make question 3 about the last time they did it, not what they would do"); it changes the question you name, leaves the rest of the guide alone and tells you what it changed.

When the study runs, Koji's AI interviewer asks your questions and follows up on what each person actually says, with neutral probes, in voice or text. You can preview the interview yourself before it goes live, which is the quickest way to hear a question that does not land. Each interview then gets a quality rating, so answers that went nowhere are easy to spot.
Start a free study and paste in the guide you are working on, or read how Koji's AI follow-up probing works first.


