How the Quality Gate Works

The Quality Gate: Never Pay for Bad Data

Koji's quality gate is a built-in protection that ensures you only spend credits on meaningful conversations. Any interview that scores below 3 on Koji's 1-5 quality scale is completely free — no credits consumed, no charges incurred.

This applies to every plan and every way of buying credits, including pay as you go.

---

How Quality Scoring Works

Every conversation on Koji is automatically scored on a 1–5 scale based on the depth and quality of the interaction:

The scoring evaluates factors like:

• Response depth — Did the participant give substantive, detailed answers?
• Conversation completeness — Did the interview cover the key topics?
• Engagement level — Was the participant actively engaged throughout?
• Information value — Did the conversation generate actionable insights?

---

Why the Quality Gate Matters

Budget Protection

Without a quality gate, every conversation would cost credits — even ones where:

• A participant abandoned the interview early
• Responses were too brief to be useful
• The participant didn't engage meaningfully with the questions

With Koji's quality gate, these low-value interactions are filtered out automatically. You only pay for conversations that actually contribute to your research.

Real Impact on Your Budget

Consider a typical research study with 50 total conversations:

• ~40 conversations score 3+ (quality threshold met) — credits charged
• ~10 conversations score below 3 — completely free

That's a 20% savings on your credit budget, automatically applied.

---

Quality Gate Across All Plans

Quality Gate on Every Credit

The quality gate applies no matter how you bought your credits: monthly plan, credit pack, or pay as you go. If a conversation scores below 3 on the 1-5 scale, no credits are drawn from your balance. You are never billed for low-quality interactions, period. And when your balance runs out, nothing is charged automatically: you top up with pay as you go, a pack, or a plan.

---

Quality Gate in Reports

The quality gate doesn't just protect your budget — it also protects your research quality. Reports only include conversations that score 3 or above. Low-scoring conversations are excluded from report aggregation, theme analysis, and statistical summaries. This ensures your research findings are built on substantive, high-quality data rather than being diluted by low-effort responses.

You can still view individual low-scoring transcripts in the Responses tab for reference, but they won't affect your aggregated insights.

---

What Affects Conversation Quality

Several factors influence whether a conversation meets the quality threshold:

Participant-Side Factors

• Engagement — participants who take the interview seriously produce higher scores
• Response length — one-word answers typically result in lower scores
• Relevance — responses that address the actual questions score higher
• Completion — finishing the full interview improves the score

Study Design Factors (You Control These)

• Clear, open-ended questions — well-crafted questions encourage detailed responses
• Appropriate interview length — not too short (insufficient depth) or too long (fatigue)
• Good probe points — AI follow-up prompts that encourage elaboration
• Relevant guardrails — keeping conversations on topic improves quality
• Structured questions — mixing in scale, choice, and ranking questions helps maintain engagement and produces richer data

Tips to Maximize Quality Scores

1. Write open-ended questions — "Tell me about..." instead of "Do you like...?"
2. Set up probe points — guide the AI to dig deeper on key topics
3. Keep interviews focused — 5-8 core questions is the sweet spot
4. Use guardrails — prevent conversations from going off-topic
5. Target the right participants — better targeting = better engagement
6. Use voice interviews — voice conversations tend to produce richer, more detailed responses
7. Add structured questions — scale and choice questions keep participants engaged between open-ended exploration

---

Viewing Quality Scores

You can see quality scores for each conversation in your study dashboard:

1. Navigate to your study
2. Open the Responses tab
3. Each conversation shows its quality score (1-5)
4. Filter by score to review which conversations met the threshold
5. Low-scoring conversations are marked and clearly shown as free

---

The Quality Gate and Research Integrity

The quality gate improves your research quality in two ways:

• Budget protection — you only spend credits on conversations that met the quality threshold
• Report accuracy — reports aggregate only conversations scoring 3 or above, ensuring your insights are built from substantive data
• Targeting feedback — consistently low scores may indicate you need to adjust your participant criteria
• Study design insights — patterns in low-scoring conversations reveal opportunities to refine your questions