How the Quality Gate Works

The Quality Gate: Never Pay for Bad Data

Koji's quality gate is a built-in protection that ensures you only spend credits on meaningful conversations. Any interview that scores below 3 out of 5 on Koji's quality scale is completely free — no credits consumed, no charges incurred.

This applies to every plan, including overage conversations.

---

How Quality Scoring Works

Every conversation on Koji is automatically scored on a 0–5 scale based on the depth and quality of the interaction:

The scoring evaluates factors like:

• Response depth — Did the participant give substantive, detailed answers?
• Conversation completeness — Did the interview cover the key topics?
• Engagement level — Was the participant actively engaged throughout?
• Information value — Did the conversation generate actionable insights?

---

Why the Quality Gate Matters

Budget Protection

Without a quality gate, every conversation would cost credits — even ones where:

• A participant abandoned the interview early
• Responses were too brief to be useful
• The participant didn't engage meaningfully with the questions

With Koji's quality gate, these low-value interactions are filtered out automatically. You only pay for conversations that actually contribute to your research.

Real Impact on Your Budget

Consider a typical research study with 50 total conversations:

• ~40 conversations score 3+ (quality threshold met) → credits charged
• ~10 conversations score below 3 → completely free

That's a 20% savings on your credit budget, automatically applied.

---

Quality Gate Across All Plans

Quality Gate During Overage

The quality gate also applies during overage billing. If you've exceeded your included credits and a conversation scores below 3/5, no overage charge is generated. You are never billed for low-quality interactions, period.

---

What Affects Conversation Quality

Several factors influence whether a conversation meets the quality threshold:

Participant-Side Factors

• Engagement — participants who take the interview seriously produce higher scores
• Response length — one-word answers typically result in lower scores
• Relevance — responses that address the actual questions score higher
• Completion — finishing the full interview improves the score

Study Design Factors (You Control These)

• Clear, open-ended questions — well-crafted questions encourage detailed responses
• Appropriate interview length — not too short (insufficient depth) or too long (fatigue)
• Good probe points — AI follow-up prompts that encourage elaboration
• Relevant guardrails — keeping conversations on topic improves quality

Tips to Maximize Quality Scores

1. Write open-ended questions — "Tell me about..." instead of "Do you like...?"
2. Set up probe points — guide the AI to dig deeper on key topics
3. Keep interviews focused — 5-8 core questions is the sweet spot
4. Use guardrails — prevent conversations from going off-topic
5. Target the right participants — better targeting = better engagement
6. Use voice interviews — voice conversations tend to produce richer, more detailed responses

---

Viewing Quality Scores

You can see quality scores for each conversation in your study dashboard:

1. Navigate to your study
2. Open the Responses tab
3. Each conversation shows its quality score
4. Filter by score to review which conversations met the threshold
5. Low-scoring conversations are marked and clearly shown as free

---

The Quality Gate and Research Integrity

The quality gate isn't just about saving money — it also improves your research quality. By automatically flagging low-quality conversations, Koji helps you:

• Focus analysis on meaningful data — your reports are built from quality conversations only
• Identify targeting issues — consistently low scores may indicate you need to adjust your participant criteria
• Improve study design — patterns in low-scoring conversations reveal opportunities to refine your questions