Scale Questions in AI Interviews: Measure NPS, CSAT, and Ratings Automatically

Scale questions give you the best of both worlds in customer research: a clean, chartable number that aggregates across hundreds of responses, plus the qualitative "why" behind it — all captured in a single AI-powered conversation. While traditional surveys collect a rating and stop there, Koji''s scale questions pair every numeric response with automatic AI probing to surface the reasoning behind the score.

This guide explains how scale questions work in Koji, how to configure them for NPS, CSAT, effort scores, and custom satisfaction ratings, and how your results appear in your research report.

What Are Scale Questions?

Scale questions are one of Koji''s six structured question types. They ask participants to rate something on a numeric scale — like satisfaction from 1 to 5, or likelihood to recommend from 0 to 10. Unlike open-ended questions, which generate qualitative themes, scale questions produce a structured numeric value that Koji aggregates across all your interviews into a distribution chart.

Koji''s complete question type library:

You can learn about all six types in the Structured Questions Guide. This guide goes deep on scale questions specifically.

How Scale Questions Work in Koji

Text Mode: Interactive Widgets

In a text-based interview, the AI delivers the scale question conversationally, then a visual widget appears for the participant to respond. The widget type adapts to your scale range:

• Buttons (default for ranges of 10 or fewer points) — Tappable number buttons, perfect for 1–5 CSAT or 0–10 NPS
• Slider — For wider ranges (more than 10 points), a drag slider gives more precise control

The participant selects a number, Koji immediately captures it as the structured value, and the AI continues the conversation — asking follow-up questions based on their rating.

Voice Mode: Fully Conversational

In a voice interview, there are no widgets. The AI speaks the question aloud and listens for the participant''s verbal response. For an NPS question, this sounds like:

"On a scale of 0 to 10, how likely are you to recommend us to a friend or colleague?"

The participant responds verbally ("I''d say about a 7"), and Koji extracts the numeric value from speech. The AI then follows up naturally, tailored to that specific rating.

Voice mode makes scale questions feel like a genuine conversation — not a clinical survey. Conversational delivery produces more honest and considered responses than form-based rating scales.

Configuring Scale Questions

When you set up a scale question in your study, you can configure three key parameters:

Adding Endpoint Labels

Labels transform a raw number into a meaningful rating. Always label at least the two endpoints:

NPS (0–10): 0 = "Not at all likely" / 10 = "Extremely likely"

CSAT (1–5): 1 = "Very dissatisfied" / 5 = "Very satisfied"

Customer Effort Score (1–7): 1 = "Very easy" / 7 = "Very difficult"

Labels appear in the widget (text mode) and are woven into how the AI phrases the question (voice mode), so participants always understand what they''re rating — without needing a preamble.

AI Probing on Scale Answers

This is where Koji separates from every traditional survey tool.

After a participant submits a rating, the AI doesn''t just record the number and move on. With anchor probing enabled, the AI follows up based on the specific value given:

• High rating (8–10/10): "What''s been driving that positive experience?"
• Mid-range (5–7/10): "You gave us a 6 — what would need to change for that to be a 9 or 10?"
• Low rating (1–4/10): "What''s been the biggest source of frustration?"

This "anchor" approach — using the participant''s own rating as the conversational starting point — converts a number into an actionable insight. Instead of knowing that 40% of users rate satisfaction at 3/5, you discover why: "The product is solid but the onboarding documentation is nearly impossible to find."

Custom Probing Instructions

You can write specific instructions for how the AI should probe each scale question. For example:

• "If they rate below 7, ask specifically about what moment in their journey caused the frustration."
• "If they rate 9 or 10, ask what single thing would be impossible for them to live without."

This level of control is what lets platforms like Koji deliver research-grade insights at survey-level scale.

Probing Depth

The maxFollowUps setting controls how many follow-up questions the AI may ask per scale question:

• 0 — Ask the question, record the rating, no probing
• 1 — One follow-up (the default)
• 2–3 — Deeper probing, best for questions where the "why" is the core insight

How Scale Results Appear in Your Report

Koji''s research report automatically aggregates scale answers across all completed interviews.

Distribution Chart

Every scale question gets a distribution chart showing how responses spread across the scale values. At a glance, you can see whether ratings cluster near the top, the bottom, or split across the range. This is far more informative than a single average score.

Summary Statistics

Alongside the distribution chart:

• Mean — The average rating across all respondents
• Median — The midpoint value
• Response count — Number of participants who answered

Qualitative Context

Below the chart, Koji surfaces representative quotes from the probing follow-ups. If 12 out of 15 participants who rated 4/10 mentioned "onboarding was confusing," that pattern surfaces immediately — without you needing to read every transcript.

Common Scale Question Configurations

Net Promoter Score (NPS)

NPS is the most common use of a 0–10 scale in research.

• Scale: 0–10
• Labels: 0 = "Not at all likely" / 10 = "Extremely likely"
• Question: "On a scale of 0 to 10, how likely are you to recommend [product] to a friend or colleague?"
• Probing: Enable anchor — the AI follows up based on the specific score given

Pair your NPS question with an open-ended question to capture the narrative behind the score.

Customer Satisfaction (CSAT)

CSAT measures satisfaction with a specific interaction, product, or experience.

• Scale: 1–5
• Labels: 1 = "Very dissatisfied" / 5 = "Very satisfied"
• Question: "How satisfied were you with [interaction]?"
• Probing: "What would have made this experience better?"

Customer Effort Score (CES)

CES measures how easy it was for a customer to accomplish something.

• Scale: 1–7
• Labels: 1 = "Very easy" / 7 = "Very difficult"
• Question: "How easy was it to [complete the task]?"
• Probing: For high-effort ratings: "Where specifically did you hit friction?"

Custom Satisfaction Rating

• Scale: 1–10
• Labels: 1 = "Very poor" / 10 = "Excellent"
• Question: "How would you rate [aspect] on a scale of 1 to 10?"
• Probing: Anchor to specific rating

Scale Questions vs. Traditional Survey Rating Scales

Traditional surveys like those built in SurveyMonkey or Typeform present a scale as a static form element — respondents select a number and click Next. There''s no follow-up, no probing, and no way to understand what drove the rating.

With platforms like Koji, the number tells you what; the probing conversation tells you why. Both in the same research session, at scale.

Best Practices

Always add endpoint labels. Without them, participants interpret the scale differently, making your data inconsistent across respondents.

Use standard ranges for benchmark metrics. NPS is always 0–10; CSAT is typically 1–5. Deviating from standards makes benchmarking against industry data harder.

Enable anchor probing. The follow-up is where the real insight lives. A 6/10 NPS score is a data point; "I gave you a 6 because the mobile app crashes every time I export data" is an actionable finding.

Limit scale questions per study. Three to four quantitative questions (scale, choice, yes/no) alongside five to eight open-ended questions is a good balance. Too many scales and you lose the depth that makes AI interviews valuable.

Mix with open-ended questions. Start with quantitative (to benchmark), then follow with open-ended (to understand). See the Interview Mode Guide for structural patterns that work well.

Frequently Asked Questions

What is the difference between a scale question and a Likert question? A Likert question uses fixed agreement statements ("Strongly agree / Strongly disagree") with labeled response options. Koji''s scale question is more flexible — you define the numeric range, the endpoint labels, and what exactly is being rated. This makes it suitable for NPS (0–10), CSAT (1–5), CES (1–7), or any custom metric.

Can participants skip a scale question? Yes. In text mode, participants can tap "Skip." In voice mode, they can verbally decline. Skipped responses are flagged in the report and excluded from aggregation — maintaining data quality without forcing responses on unwilling participants.

How many scale questions should I add to one study? Three to four quantitative questions total (across scale, choice, and yes/no types) is a healthy ceiling. More than that, and the interview starts to feel like a survey rather than a conversation, which reduces completion rates and response quality.

How does Koji handle scale questions in voice mode? Koji''s AI speaks the question aloud and listens for a numeric response. It understands natural phrasing like "I''d say about a seven" and extracts the number. If the response is ambiguous, the AI politely asks for clarification before moving on.

Can I see which participant gave which rating? Yes. In your study''s Recruit tab, each participant row links to their full transcript. You can see the exact rating alongside the complete conversation. The research report aggregates ratings across all participants for your distribution charts.

Does probing work differently for high vs. low ratings? When anchor probing is enabled, Koji''s AI is aware of the specific rating and tailors its follow-up accordingly — asking high raters about positive drivers and low raters about friction points. Custom probing instructions give you even more control over how the AI responds to specific rating ranges.

Related Resources

• Structured Questions in AI Interviews — Overview of all six question types
• Interview Mode Guide — Structured, exploratory, and hybrid patterns
• NPS Follow-Up Interviews — Running NPS research at scale with AI
• Setting Up Voice Interviews — How voice mode works technically
• Generating Research Reports — How Koji builds your report from interview data
• Insights Chat Guide — Ask natural language questions about your research data