Understanding Quality Scores

Every completed interview in Koji receives a quality score from 1 to 5. This score tells you how useful the interview is for your research — and it directly affects your billing, because only interviews scoring 3 or above count toward your monthly limit. Reports also filter to this same threshold, ensuring your aggregate analysis is built from substantive data.

How Quality Scoring Works

After each interview concludes, Koji's AI automatically evaluates the conversation across multiple dimensions to produce a single quality score. The evaluation happens immediately upon interview completion and takes just a few seconds — there is no manual trigger required.

The score reflects the overall research value of the interview — not the participant's intelligence or effort, but how much actionable data the conversation produced. A high-quality interview is one that gives you clear, detailed, and relevant information you can use to make decisions.

The 1-5 Scale Explained

Quality scores range from 1 to 5. Here's what each score means in practice:

The Five Quality Dimensions

Koji evaluates each interview across five specific dimensions that combine into the overall score:

Relevance

How well the participant's responses addressed the research questions defined in your study brief. On-topic, focused answers score higher than tangential or off-topic responses.

Depth

The level of detail and elaboration in responses. Multi-sentence answers with explanations, examples, and reasoning score higher than brief or surface-level replies.

Coverage

How thoroughly the interview covered the key topics and questions in your study. Interviews where the participant engaged with most or all research areas score higher.

Completion

Whether the participant completed the full interview or dropped off early. Finishing the entire conversation contributes positively to the score.

Structured Quality

For studies using structured questions — scales, multiple choice, rankings, yes/no — this dimension evaluates how thoughtfully the participant engaged with those interactive elements. Did they provide considered ratings and selections, or did they rush through? Did they explain their reasoning in follow-up probes?

Why Quality Scores Matter

Quality scores serve three important purposes:

1. Research Data Quality

Not all interviews are equally valuable. A five-minute conversation where a participant gave one-word answers tells you far less than a twenty-minute deep dive. The quality score helps you quickly identify which interviews deserve your attention first and which ones might need to be supplemented with additional data collection.

When you're reviewing results, sorting by quality score helps you prioritize your time. Start with the highest-scoring interviews to get the strongest signals, then work your way down.

2. Fair Billing Through the Quality Gate

Here's something that works strongly in your favor: interviews scoring below 3 don't count toward your monthly interview limit. This is Koji's quality gate, and it exists to protect you.

If a participant rushes through your interview giving minimal answers, or if someone provides off-topic responses, you shouldn't have to pay for that. The quality gate ensures you're only billed for interviews that actually deliver research value.

Learn more about how this works in How the Quality Gate Works.

3. Report Filtering

When you generate a research report, Koji filters to only include interviews with a quality score of 3 or above. This ensures your aggregate analysis, theme detection, and recommendations are built from substantive data rather than being diluted by low-quality responses. The results page shows a report-eligible count so you know exactly how many interviews will feed into your report.

Improving Your Quality Scores

While you can't control how individual participants respond, you can set the stage for better interviews:

• Write a clear study brief: The better your research objectives and target questions are defined, the more relevant the AI interviewer's questions will be. Clear briefs lead to focused conversations.

• Target the right participants: People with direct experience in your research topic naturally provide richer, more detailed responses. A product user will give you better feedback about your product than someone who's never used it.

• Use structured questions strategically: Adding structured questions like scales and choice selections gives participants concrete ways to express their views, which can boost the structured quality dimension of the score.

• Set expectations upfront: Your study description (what participants see before starting) should explain what the interview is about and roughly how long it takes. Prepared participants tend to give more thoughtful answers.

• Design for engagement: Studies with interesting, relevant topics naturally produce better interviews. If your research questions connect to things participants genuinely care about, they'll be more willing to share detailed responses.

Viewing Quality Scores

You can see quality scores in several places:

• Responses tab: Each interview card displays its quality score prominently, making it easy to scan across all interviews at a glance.
• Analysis Drawer: When you click an interview card, the drawer shows the quality score alongside other insights.
• Insights view: On the full transcript page, the Insights sidebar view displays the quality score with its dimensional breakdown (relevance, depth, coverage, completion, structured quality).
• Study results overview: Aggregate quality statistics including average score and distribution appear on the Experience tab.

Key Things to Know

• Scores are final: Quality scores are calculated once after the interview completes and don't change. This ensures consistency in your billing and analysis.
• Scores are not participant ratings: A low score doesn't mean the participant was "bad." It means the conversation didn't produce enough usable research data for various possible reasons.
• You can't manually override scores: The scoring is automated to ensure objectivity and consistency across all interviews.
• Analysis is automatic: You do not need to click a button to trigger scoring. It happens immediately when an interview completes.

Related Articles

• How the Quality Gate Works — Understanding why low-quality interviews don't count toward your limits
• AI-Generated Insights — What analysis Koji produces from your interviews
• Viewing Interview Transcripts — How to read and navigate interview conversations
• Structured Questions Guide — How structured question types affect the quality score
• Generating Research Reports — How quality filtering shapes your reports

Frequently Asked Questions

Q: Can a participant retake an interview to improve the quality score? A: Each interview submission is scored independently. If you share the link again with the same participant, they could complete a new interview, which would receive its own quality score.

Q: Do low-quality interviews still generate insights? A: Yes, Koji generates AI insights for every completed interview regardless of its quality score. However, the insights from lower-quality interviews will naturally be less detailed and less reliable.

Q: What if I think a quality score is wrong? A: The scoring evaluates objective factors like response depth, relevance, coverage, completion, and structured quality. If an interview has a lower score than expected, review the transcript — you may find that while the participant said some useful things, overall depth or completeness was limited.

Q: Does the quality score affect report generation? A: Yes. Research reports only include interviews scoring 3 or above. This quality filter ensures your aggregate analysis is built from substantive data. The results page shows the report-eligible count so you can see how many interviews qualify.

Q: What's the average quality score across all Koji interviews? A: Quality scores vary significantly by study topic, participant recruitment, and study design. Well-designed studies with targeted participants typically see average scores above 3.

Further reading on the blog

• Koji vs Listen Labs: AI Interview Platforms Compared (2026) — Listen Labs raised $69M and powers enterprise research at brands across the Fortune 500. Koji is the accessible AI-native alternative starti
• Koji vs Looppanel: End-to-End AI Research vs Analysis-Only (2026) — Looppanel is a great AI analysis tool — but it only handles half the workflow. You still need a separate platform to actually run interviews

<!-- further-reading:blog -->