{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-11-14T17:55:52.997Z"},"content":[{"type":"documentation","id":"1d668e04-ba64-4c60-b0ff-b63e3115d2c8","slug":"how-the-quality-gate-works","title":"How the Quality Gate Works","url":"https://www.koji.so/docs/how-the-quality-gate-works","summary":"Koji’s quality gate automatically scores every conversation on a 1-5 scale. Conversations scoring below 3 are completely free, with no credits consumed. Reports also filter to only include conversations scoring 3 or above. This applies on every plan (Free, Insights, Interviews, Enterprise) and to every credit source: plan credits, credit packs and auto-reload top-ups.","content":"## The Quality Gate: Never Pay for Bad Data\n\nKoji’s quality gate is a built-in protection that ensures you **only spend credits on meaningful conversations**. Any interview that scores below 3 on Koji’s 1-5 quality scale is completely free — no credits consumed, no charges incurred.\n\nThis applies to **every plan** and every way of buying credits: plan credits, credit packs and auto-reload top-ups. It covers text and voice interviews, whether the participant finished or left partway through. Phone interviews work differently: they are charged per connected minute.\n\n---\n\n## How Quality Scoring Works\n\nEvery text and voice interview on Koji is automatically scored on a **1–5 scale** based on the depth and quality of the interaction. The app shows each score as one of four levels:\n\n| Score | Level in the app | Credits charged? | In your report? |\n|-------|------------------|------------------|-----------------|\n| 1–2 | **Shallow** | No — free | No |\n| 3 | **Fair** | Yes | Yes |\n| 4 | **Good** | Yes | Yes |\n| 5 | **Excellent** | Yes | Yes |\n\nThe cut-off is the same everywhere: a score has to be 3 or higher to count. A score in between, such as 2.5, is Shallow for billing and for the report alike. If the analysis of an interview fails and never produces a score, it is treated the same way: free, and left out of the report.\n\nThe scoring evaluates factors like:\n- **Response depth** — Did the participant give substantive, detailed answers?\n- **Conversation completeness** — Did the interview cover the key topics?\n- **Engagement level** — Was the participant actively engaged throughout?\n- **Information value** — Did the conversation generate actionable insights?\n\n### What the participant actually said decides the score\n\nSince September 15, 2026, thin or nonsense answers bring the score down. Someone who clicks through the multiple-choice questions and answers every follow-up with a few dots no longer gets a high score, so that interview stays out of your report and you aren’t charged for it. This applies to interviews from that date on; earlier scores stay as they were.\n\nShort answers are not the same as thin answers. A reply like “twice a week” or a single rating counts when it really answers the question. When most of the questions got a real, relevant answer and the interview stayed on topic, it scores at least 3. [Understanding Quality Scores](/docs/understanding-quality-scores) explains how each question is judged.\n\n### Interviews that end early\n\nA participant who leaves partway through doesn’t automatically get a free pass. Since November 9, 2026, a partial interview is scored like any other: if it scores 3 or more, it is billed and goes into your report, because the answers it holds are usable. A partial interview that scores below 3 is free and stays out.\n\nDropped interviews and interviews you archive never count toward the report. You can read more about why people stop in [Partial Interviews and Breakoff Analysis](/docs/partial-interviews-breakoff-analysis).\n\n---\n\n## Why the Quality Gate Matters\n\n### Budget Protection\nWithout a quality gate, every conversation would cost credits — even ones where:\n- Responses were too brief or off topic to be useful\n- The participant answered with filler or nonsense\n- The participant didn’t engage meaningfully with the questions\n\nWith Koji’s quality gate, these low-value interactions are filtered out automatically. You only pay for conversations that actually contribute to your research.\n\n### Real Impact on Your Budget\n\nConsider an example study with 50 total conversations:\n- 40 conversations score 3+ (Fair, Good or Excellent) — credits charged\n- 10 conversations score below 3 (Shallow) — **completely free**\n\nIn that example, that’s a **20% saving** on your [credit budget](/docs/understanding-usage-limits), applied automatically.\n\n---\n\n## Quality Gate Across All Plans\n\n| Plan | Quality Gate | Effect |\n|------|-------------|--------|\n| **Free** | Active | Low-quality conversations don’t consume your 10 sign-up credits |\n| **Insights** | Active | Low-quality conversations don’t consume your 29 monthly credits |\n| **Interviews** | Active | Low-quality conversations don’t consume your 79 monthly credits |\n| **Enterprise** | Active | Low-quality conversations don’t consume your committed credits |\n| **Credit packs and auto-reload** | Active | Low-quality conversations don’t consume purchased credits |\n\n### Quality Gate on Every Credit\nThe quality gate applies no matter where your credits came from: your plan, a credit pack, or an auto-reload top-up. If a conversation scores below 3 on the 1-5 scale, no credits are drawn from your balance. You are never billed for low-quality interactions, and Koji has no overage billing. When your balance runs out, new interviews pause until the workspace owner adds credits. See [Running Out of Credits, Explained](/docs/overage-billing-explained) for packs, auto-reload and the monthly spending cap.\n\n---\n\n## Quality Gate in Reports\n\nThe quality gate doesn’t just protect your budget — it also protects your research quality. **Reports include completed and partial interviews that score 3 or above.** Shallow conversations are excluded from report aggregation, theme analysis, and statistical summaries. This ensures your research findings are built on substantive data rather than being diluted by low-effort responses.\n\nYou can still read every Shallow response in the Responses tab, but it won’t affect your aggregated insights.\n\n---\n\n## What Affects Conversation Quality\n\nSeveral factors influence whether a conversation meets the quality threshold:\n\n### Participant-Side Factors\n- **Engagement** — participants who take the interview seriously produce higher scores\n- **Substance** — real, on-topic answers count, even short ones; filler and nonsense pull the score down\n- **Relevance** — responses that address the actual questions score higher\n- **Completion** — finishing the full interview improves the score\n\n### Study Design Factors (You Control These)\n- **Clear, open-ended questions** — well-crafted questions encourage detailed responses\n- **Appropriate interview length** — not too short (insufficient depth) or too long (fatigue)\n- **Good probe points** — AI follow-up prompts that encourage elaboration\n- **Relevant guardrails** — keeping conversations on topic improves quality\n- **[Structured questions](/docs/structured-questions-guide)** — mixing in scale, choice, and ranking questions helps maintain engagement and produces richer data\n\n### Tips to Maximize Quality Scores\n1. **Write open-ended questions** — \"Tell me about...\" instead of \"Do you like...?\"\n2. **Set up probe points** — guide the AI to dig deeper on key topics\n3. **Keep interviews focused** — 5-8 core questions is the sweet spot\n4. **Use guardrails** — prevent conversations from going off-topic\n5. **Target the right participants** — better targeting = better engagement\n6. **Use [voice interviews](/docs/voice-interview-experience)** — voice conversations tend to produce richer, more detailed responses\n7. **Add [structured questions](/docs/structured-questions-guide)** — scale and choice questions keep participants engaged between open-ended exploration. Pair them with open questions, because the score depends on what people say in their own words too.\n\n---\n\n## Viewing Quality Scores\n\nYou can see how your study did against the quality gate in its results:\n\n1. Navigate to your study and open the **Responses** tab.\n2. The **Quality Evaluation** card shows how scored responses split across Shallow, Fair, Good and Excellent, and its caption says how many are in the report (for example \"7 of 10 in the report\").\n3. Each response shows its level in the **Evaluation** column, which you can sort by.\n4. When a study has Shallow responses, an evaluation filter appears above the table: **All evaluations**, **Fair and above** or **Shallow only**.\n5. Click a response to open its analysis, or export the table as CSV to get each response’s 1-5 score.\n\n---\n\n## The Quality Gate and Research Integrity\n\nThe quality gate improves your research in several ways:\n\n- **Budget protection** — you only spend credits on conversations that met the quality threshold\n- **Report accuracy** — reports aggregate only interviews scoring 3 or above, so your insights are built from substantive data\n- **Targeting feedback** — a large Shallow group may mean you need to adjust your participant criteria or recruitment source\n- **Study design insights** — patterns in Shallow conversations reveal where to refine your questions\n\n## Related Articles\n\n- [Understanding Quality Scores](/docs/understanding-quality-scores): how each interview is scored and how to read the Quality Evaluation card\n- [Partial Interviews and Breakoff Analysis](/docs/partial-interviews-breakoff-analysis): why participants stop and what partial interviews tell you\n- [Running Out of Credits, Explained](/docs/overage-billing-explained): packs, auto-reload and the spending cap\n- [Understanding Usage Limits](/docs/understanding-usage-limits): what each interview type costs\n- [Structured Questions Guide](/docs/structured-questions-guide): question types that keep participants engaged","category":"Billing & Plans","lastModified":"2026-11-14T05:09:23.956672+00:00","metaTitle":"How the Quality Gate Works — Koji","metaDescription":"Koji's quality gate ensures conversations scoring below 3/5 are completely free. Never pay credits for low-quality interviews — protection built into every plan.","keywords":["koji quality gate","interview quality scoring","free low quality interviews","research budget protection","conversation quality"],"aiSummary":"Koji’s quality gate automatically scores every conversation on a 1-5 scale. Conversations scoring below 3 are completely free, with no credits consumed. Reports also filter to only include conversations scoring 3 or above. This applies on every plan (Free, Insights, Interviews, Enterprise) and to every credit source: plan credits, credit packs and auto-reload top-ups.","aiPrerequisites":["understanding-quality-scores"],"aiLearningOutcomes":["Understand how the quality gate protects your interview budget","Know the threshold and how quality is evaluated","Maximize the quality gate benefit through study design","Distinguish between quality filtering and research bias"],"aiDifficulty":"beginner","aiEstimatedTime":"7 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}