{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-05-18T13:49:44.660Z"},"content":[{"type":"documentation","id":"867eea81-89a2-47d6-8a36-aa5822c4ab58","slug":"understanding-quality-scores","title":"Understanding Quality Scores","url":"https://www.koji.so/docs/understanding-quality-scores","summary":"Koji automatically scores every interview from 1 to 5 across five dimensions: relevance, depth, coverage, completion, and structured quality. Only interviews scoring 3 or above count toward billing and are included in research reports. Analysis is auto-triggered on completion.","content":"Every completed interview in Koji receives a quality score from 1 to 5. This score tells you how useful the interview is for your research — and it directly affects your billing, because only interviews scoring 3 or above count toward your monthly limit. Reports also filter to this same threshold, ensuring your aggregate analysis is built from substantive data.\n\n## How Quality Scoring Works\n\nAfter each interview concludes, Koji's AI automatically evaluates the conversation across multiple dimensions to produce a single quality score. The evaluation happens immediately upon interview completion and takes just a few seconds — there is no manual trigger required.\n\nThe score reflects the overall research value of the interview — not the participant's intelligence or effort, but how much actionable data the conversation produced. A high-quality interview is one that gives you clear, detailed, and relevant information you can use to make decisions.\n\n## The 1-5 Scale Explained\n\nQuality scores range from 1 to 5. Here's what each score means in practice:\n\n| Score | Rating | What It Means |\n|-------|--------|---------------|\n| **1** | Poor | The interview produced very little usable data. The participant may have given one-word answers, gone off-topic, or disengaged early. |\n| **2** | Below Average | Some relevant information was shared, but responses lacked depth. There may be a few useful data points, but not enough for confident analysis. |\n| **3** | Good | A solid interview with meaningful responses. The participant engaged with most questions and provided enough detail to identify themes and insights. This is the threshold for billing and report inclusion. |\n| **4** | Very Good | A strong interview with detailed, thoughtful responses. The participant shared personal experiences, examples, and reasoning behind their opinions. |\n| **5** | Excellent | An exceptional interview with rich, nuanced data. The participant went deep on multiple topics, provided vivid examples, and offered perspectives you didn't anticipate. |\n\n## The Five Quality Dimensions\n\nKoji evaluates each interview across five specific dimensions that combine into the overall score:\n\n### Relevance\nHow well the participant's responses addressed the research questions defined in your study brief. On-topic, focused answers score higher than tangential or off-topic responses.\n\n### Depth\nThe level of detail and elaboration in responses. Multi-sentence answers with explanations, examples, and reasoning score higher than brief or surface-level replies.\n\n### Coverage\nHow thoroughly the interview covered the key topics and questions in your study. Interviews where the participant engaged with most or all research areas score higher.\n\n### Completion\nWhether the participant completed the full interview or dropped off early. Finishing the entire conversation contributes positively to the score.\n\n### Structured Quality\nFor studies using [structured questions](/docs/structured-questions-guide) — scales, multiple choice, rankings, yes/no — this dimension evaluates how thoughtfully the participant engaged with those interactive elements. Did they provide considered ratings and selections, or did they rush through? Did they explain their reasoning in follow-up probes?\n\n## Why Quality Scores Matter\n\nQuality scores serve three important purposes:\n\n### 1. Research Data Quality\n\nNot all interviews are equally valuable. A five-minute conversation where a participant gave one-word answers tells you far less than a twenty-minute deep dive. The quality score helps you quickly identify which interviews deserve your attention first and which ones might need to be supplemented with additional data collection.\n\nWhen you're reviewing results, sorting by quality score helps you prioritize your time. Start with the highest-scoring interviews to get the strongest signals, then work your way down.\n\n### 2. Fair Billing Through the Quality Gate\n\nHere's something that works strongly in your favor: **interviews scoring below 3 don't count toward your monthly interview limit**. This is Koji's quality gate, and it exists to protect you.\n\nIf a participant rushes through your interview giving minimal answers, or if someone provides off-topic responses, you shouldn't have to pay for that. The quality gate ensures you're only billed for interviews that actually deliver research value.\n\nLearn more about how this works in [How the Quality Gate Works](/docs/how-the-quality-gate-works).\n\n### 3. Report Filtering\n\nWhen you [generate a research report](/docs/generating-research-reports), Koji filters to only include interviews with a quality score of 3 or above. This ensures your aggregate analysis, theme detection, and recommendations are built from substantive data rather than being diluted by low-quality responses. The results page shows a **report-eligible count** so you know exactly how many interviews will feed into your report.\n\n## Improving Your Quality Scores\n\nWhile you can't control how individual participants respond, you can set the stage for better interviews:\n\n- **Write a clear study brief**: The better your research objectives and target questions are defined, the more relevant the AI interviewer's questions will be. Clear briefs lead to focused conversations.\n\n- **Target the right participants**: People with direct experience in your research topic naturally provide richer, more detailed responses. A product user will give you better feedback about your product than someone who's never used it.\n\n- **Use structured questions strategically**: Adding [structured questions](/docs/structured-questions-guide) like scales and choice selections gives participants concrete ways to express their views, which can boost the structured quality dimension of the score.\n\n- **Set expectations upfront**: Your study description (what participants see before starting) should explain what the interview is about and roughly how long it takes. Prepared participants tend to give more thoughtful answers.\n\n- **Design for engagement**: Studies with interesting, relevant topics naturally produce better interviews. If your research questions connect to things participants genuinely care about, they'll be more willing to share detailed responses.\n\n## Viewing Quality Scores\n\nYou can see quality scores in several places:\n\n- **Responses tab**: Each interview card displays its quality score prominently, making it easy to scan across all interviews at a glance.\n- **Analysis Drawer**: When you click an interview card, the drawer shows the quality score alongside other insights.\n- **Insights view**: On the full transcript page, the Insights sidebar view displays the quality score with its dimensional breakdown (relevance, depth, coverage, completion, structured quality).\n- **Study results overview**: Aggregate quality statistics including average score and distribution appear on the Experience tab.\n\n## Key Things to Know\n\n- **Scores are final**: Quality scores are calculated once after the interview completes and don't change. This ensures consistency in your billing and analysis.\n- **Scores are not participant ratings**: A low score doesn't mean the participant was \"bad.\" It means the conversation didn't produce enough usable research data for various possible reasons.\n- **You can't manually override scores**: The scoring is automated to ensure objectivity and consistency across all interviews.\n- **Analysis is automatic**: You do not need to click a button to trigger scoring. It happens immediately when an interview completes.\n\n## Related Articles\n\n- [How the Quality Gate Works](/docs/how-the-quality-gate-works) — Understanding why low-quality interviews don't count toward your limits\n- [AI-Generated Insights](/docs/ai-generated-insights) — What analysis Koji produces from your interviews\n- [Viewing Interview Transcripts](/docs/viewing-interview-transcripts) — How to read and navigate interview conversations\n- [Structured Questions Guide](/docs/structured-questions-guide) — How structured question types affect the quality score\n- [Generating Research Reports](/docs/generating-research-reports) — How quality filtering shapes your reports\n\n## Frequently Asked Questions\n\n**Q: Can a participant retake an interview to improve the quality score?**\nA: Each interview submission is scored independently. If you share the link again with the same participant, they could complete a new interview, which would receive its own quality score.\n\n**Q: Do low-quality interviews still generate insights?**\nA: Yes, Koji generates AI insights for every completed interview regardless of its quality score. However, the insights from lower-quality interviews will naturally be less detailed and less reliable.\n\n**Q: What if I think a quality score is wrong?**\nA: The scoring evaluates objective factors like response depth, relevance, coverage, completion, and structured quality. If an interview has a lower score than expected, review the transcript — you may find that while the participant said some useful things, overall depth or completeness was limited.\n\n**Q: Does the quality score affect report generation?**\nA: Yes. Research reports only include interviews scoring 3 or above. This quality filter ensures your aggregate analysis is built from substantive data. The results page shows the report-eligible count so you can see how many interviews qualify.\n\n**Q: What's the average quality score across all Koji interviews?**\nA: Quality scores vary significantly by study topic, participant recruitment, and study design. Well-designed studies with targeted participants typically see average scores above 3.\n\n## Further reading on the blog\n\n- [Koji vs Listen Labs: AI Interview Platforms Compared (2026)](/blog/koji-vs-listen-labs-2026) — Listen Labs raised $69M and powers enterprise research at brands across the Fortune 500. Koji is the accessible AI-native alternative starti\n- [Koji vs Looppanel: End-to-End AI Research vs Analysis-Only (2026)](/blog/koji-vs-looppanel-2026) — Looppanel is a great AI analysis tool — but it only handles half the workflow. You still need a separate platform to actually run interviews\n\n<!-- further-reading:blog -->\n","category":"Reports & Analysis","lastModified":"2026-05-13T00:25:38.788654+00:00","metaTitle":"Quality Scores — Koji Docs","metaDescription":"Understand how Koji rates interview quality on a 0-5 scale. Learn what makes high-quality interviews and why only scores of 3+ count toward billing.","keywords":["interview quality score","qualitative research quality","quality gate","interview evaluation","research data quality","Koji scoring"],"aiSummary":"Koji automatically scores every interview from 1 to 5 across five dimensions: relevance, depth, coverage, completion, and structured quality. Only interviews scoring 3 or above count toward billing and are included in research reports. Analysis is auto-triggered on completion.","aiPrerequisites":["creating-your-first-study"],"aiLearningOutcomes":["Understand the 0-5 quality scoring scale","Identify what factors contribute to higher quality scores","Design studies that encourage higher-quality interviews","Know how quality scores affect billing through the quality gate"],"aiDifficulty":"beginner","aiEstimatedTime":"7 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}