Why a Vague Question Gets a Short Answer: Specificity and the Length Contract (2026)

If your interviews keep producing thin answers, the most likely cause is not disengaged participants. It is that your questions told them a short answer would be sufficient.

Answer length is largely a property of the question. A question carries an implied contract about how much detail the asker needs, the participant reads that contract accurately, and then supplies roughly that much. Teams who believe they have a participant-quality problem usually have a question-specificity problem, and the two require opposite fixes: one sends you back to recruiting, the other sends you back to the guide, which is cheaper and works.

The maxim that governs answer length

The underlying principle comes from the philosopher H. P. Grice, whose account of conversation treats cooperative exchange as governed by a small number of maxims. In the Stanford Encyclopedia of Philosophy formulation, the Cooperative Principle asks a speaker to "Contribute what is required by the accepted purpose of the conversation," and the maxim of quantity reduces to a single instruction: "Be as informative as required."

The operative words are as required. Not maximally informative. Not as informative as possible. As informative as the exchange appears to demand. A cooperative participant who says four words is not being unhelpful, they are being precisely as helpful as your question asked them to be. Over-answering is a violation of the same maxim, which is why nobody volunteers a ten-minute narrative in response to a yes-or-no question and why it would be slightly strange if they did.

The other three maxims matter here too. Relation - "Be relevant" - means the participant is continuously inferring what you are trying to find out and answering that rather than the literal words. Manner - "Be perspicuous; so avoid obscurity and ambiguity, and strive for brevity and order" - means a participant facing an ambiguous question will often resolve the ambiguity silently and answer the version they guessed, giving you a confident answer to a question you did not ask.

How a question signals the size of the answer it wants

Four properties of a question set the expected answer length, and all four are under your control.

The first row is the highest-leverage change most teams can make. How do you usually asks for a summary, and a summary is correctly short. A specific past episode asks for a narrative, and a narrative is correctly long. The participant did not become more forthcoming; the contract changed.

The fourth row is worth dwelling on. Stating what you are trying to learn is not the same as leading. A leading question tells the participant which answer you want. Stating your purpose tells them which kind of information is relevant, which is exactly what the maxim of relation says they are already trying to infer. If you leave them to guess your purpose, they will guess, and you have no record of what they guessed.

The most expensive question in product research

The single worst offender is the open sweep at the end: anything else? or any other thoughts?

Under the maxim of quantity that question communicates that the required contribution has already been made and you are performing a courtesy. It also gives the participant no criterion for what would count as relevant, so the safest cooperative response is no, I think that covers it. Teams then conclude there was nothing else, when what happened is that the question told them nothing else was needed.

The repair is to give the sweep a criterion and a scope: We have talked about setup and reporting. What about the parts of the week we have not touched - what else takes up your time? That question tells the participant what has been covered, what has not, and what would count as a relevant contribution.

Ambiguity does not produce hesitation, it produces confident wrong answers

This is the part that costs the most and is the hardest to see, because an answer to a misread question looks exactly like an answer to the right one.

A 2015 study in Frontiers in Psychology (volume 6, article 1578) by Frederick Conrad, Michael Schober, Matt Jans, Rachel Orlowski, Daniel Nielsen and Rachel Levenstein tested this directly. Seventy-three participants were interviewed by an on-screen virtual agent using questions drawn from US government surveys, answering on the basis of fictional scenarios so that comprehension accuracy could be measured against the official definitions. The agents varied in two ways: how capable they were of dialogue, and how much facial animation they had.

The finding that matters: "Respondents answered more accurately with the high-dialog-capability agents, requesting clarification more often particularly for ambiguous scenarios."

Two things follow. First, accuracy improved specifically where the scenarios were ambiguous - which is where a participant who cannot ask is forced to guess. Second, the mechanism was the participant asking for clarification, not the interviewer being warmer. The same study reports that "Greater interviewer facial animation did not affect response accuracy." Social warmth changed how natural the agent seemed and how much participants acknowledged it; it did not make the answers more correct. The ability to resolve an ambiguous question did.

For anyone choosing between a static form and a conversational interview, that is the substantive argument. A form cannot answer a question about what a question means. Every ambiguity in a form is resolved privately by the respondent, and you never learn which reading they used.

A practical rewrite pass

Run this over an existing guide. It takes about twenty minutes and usually doubles the median answer length.

1. Find every question containing usually, generally, typically or in general. Replace with a specific recent episode. These words are explicit instructions to summarise.
2. Find every question answerable in one word. Decide deliberately whether you want the one word. Sometimes you do, in which case make it a structured item rather than an open one so it can be counted properly.
3. Find every question with two readings. If you can imagine a participant asking do you mean X or Y, either split it or make sure the format allows them to ask.
4. Give the final sweep a scope and a criterion. Never end on anything else.
5. Add one sentence of purpose near the top. What you are trying to understand, stated plainly, without indicating what you hope to hear.

Where specificity stops helping

Specificity has a ceiling, and passing it causes a different problem.

A question can be so narrow that it excludes the answer you needed. How long did the export take? forecloses the discovery that the participant never got as far as exporting. Narrow questions are precise instruments for confirming things and poor instruments for finding them, which is why a discovery guide should open wide and narrow as it goes.

Specificity also does not overcome recall limits. Anchoring on a specific past event works because episodes are better remembered than averages, but an event six months ago is still reconstructed rather than recalled.

And length is not quality. A long answer produced by a participant who is enjoying talking is not more informative than a short precise one. The goal is answers proportioned to what you actually need, which sometimes means shorter.

How Koji handles this

• AI follow-up that reopens an under-specified answer. When a participant gives a summary where you needed an episode, Koji asks the follow-up a moderator would - for the specific instance, the next step, the thing that happened before - without you having to anticipate every branch in the guide.
• Participants can ask what a question means. Because a Koji interview is a conversation rather than a form, an ambiguous item can be clarified in the moment, which is the mechanism the Conrad and Schober study found to drive accuracy.
• Six structured question types for the items that should be short. open_ended, scale, single_choice, multiple_choice, ranking and yes_no let you collect the facts that belong in a count as countable data, so your open questions can be reserved for the places where narrative is the point. The structured questions guide explains how Koji asks each type in voice and in text.
• Identical wording across the whole sample. Answer-length differences between participants reflect the participants, not a moderator who phrased it more invitingly on a Thursday.
• Full transcripts for measurement. Koji keeps verbatim answers, so you can compare median answer length per question and find the specific items that are under-specified rather than guessing.

That last point is the closest thing to a diagnostic. Sort your questions by median answer length across the sample. The short ones at the bottom are rarely the boring topics; they are usually the vague questions.

Frequently asked questions

Why do my user interviews produce such short answers?

Most often because the questions ask for summaries rather than episodes. Words like usually and generally instruct the participant to compress, and under the conversational maxim of quantity a cooperative person supplies as much as the question appears to require and no more. Replacing how do you usually handle this with walk me through the last time you did this changes the implied contract and typically produces several times the detail from the same person.

Is asking a more specific question the same as leading the participant?

No. A leading question signals which answer you want. A specific question signals what kind of information is relevant and how much of it is needed. You can be extremely specific about scope - a particular task, a particular week, a particular step - without indicating any preferred conclusion. The two axes are independent, and the best questions are narrow in scope and neutral in direction.

Should I tell participants what the research is about?

Usually yes, at the level of purpose rather than hypothesis. Participants are inferring your purpose regardless, because the maxim of relation means they are trying to give you relevant information. Telling them you want to understand where people get stuck is useful and not leading. Telling them you suspect the onboarding is confusing is leading. If you say nothing, they guess, and you have no record of the guess.

Does an AI interviewer help with under-specified answers?

Yes, in two ways. Koji follows up automatically when an answer is thinner than the question needed, asking for the specific episode or the next step rather than repeating the question. And because a Koji interview is conversational, participants can ask what an ambiguous question means instead of silently picking a reading, which is the mechanism shown to improve comprehension accuracy in the virtual-agent research.

Can a question be too specific?

Yes. A question narrow enough to presuppose part of the answer can exclude what you needed to learn - asking how long the export took forecloses finding out the participant never reached the export. Narrow questions confirm well and discover poorly, so open wide early in a guide and narrow as you go.

How do I find the under-specified questions in an existing guide?

Measure median answer length per question across your completed interviews, then look at the bottom of the list. Short medians usually indicate a vague question rather than an unimportant topic. Cross-check by reading three answers to each low scorer: if they are all summaries rather than stories, the question asked for a summary.

Related Resources

• Structured Questions in AI Interviews - the six question types and when a short answer is the right answer
• How to Write User Interview Questions - the broader question-writing workflow
• Why Hypothetical Questions Ruin User Interviews - a different failure of question design
• What Participants Mean But Never Say - the answer side of the same conversational maxims
• How to Avoid Leading Questions - the direction axis that specificity is independent of
• The Mom Test - why specific past behaviour beats general opinion