Back to blog
Research

How Many Customer Interviews Does a Founder Need? Validation vs Discovery Sample Sizes

Discovery stops when new interviews stop telling you anything new, usually 12 to 20 per segment. Validation stops when you pass or fail a bar you set before the first call. Here are the numbers, the studies behind them and the decision rules for a founder.

Koji

Koji Team

Product · · 11 min read

You are a founder with no researcher, a short runway and a question you need answered. Before you book a single call, you want to know how many conversations it will take.

The answer depends on which of two jobs you are doing. Discovery (finding out what the problem really is, who has it and how they cope) is done when new interviews stop telling you anything new: usually 12 to 20 conversations with one kind of customer. Validation (checking whether a specific bet holds, like "ops leads at 20-person agencies will pay to fix this") is done when you hit or miss a pass bar you wrote down before the first call: often 10 to 15 conversations to see the signal, and 30 or more once you want a share you can quote.

This post gives a founder the numbers and the decision rules for both. For the full sample size reference across every research method, see our guide How Many User Interviews Do You Need?.


Discovery or validation? Know which number you need

DiscoveryValidation
The questionWhat is the problem, who has it, how do they cope today?Does this specific bet hold for this specific customer?
When you stopWhen new interviews stop adding new themes (saturation)When you clearly pass or fail a bar you set in advance
Typical number12 to 20 per segment10 to 15 to see a signal, 30+ to quote a share
What counts as evidenceRepeated stories, workarounds, the words people useCommitments: a second call, a pilot, a deposit, a letter of intent
Biggest riskToo broad a segment, so answers never convergeCounting compliments as demand

Most early founders start with discovery and move to validation once the same problem keeps coming up. If you are not sure which you are doing, you are doing discovery.

How to set a validation pass bar

Write it down before the first call, so you cannot move it afterwards. For example:

  • "Of 12 ops leads at agencies with 10 to 50 people, at least 6 describe losing a day or more a month to this in the last quarter."
  • "At least 3 of those 6 agree to a paid pilot or a follow-up with their manager."

The exact bar is your call. What matters is that it is about past behavior and commitments, not about whether people liked the idea. If you hit it, widen the sample. If you miss it, change the segment or the problem, not the bar.


The "5 Users" Myth: Where It Came From and Why It Doesn't Always Apply

You've probably heard the famous claim: "Testing with 5 users uncovers 85% of usability problems." This insight, popularized by Jakob Nielsen of the Nielsen Norman Group, has become one of the most cited (and misapplied) guidelines in product research.

Here's the thing: Nielsen's research was specifically about usability testing: watching users interact with an interface to find problems. It was never meant to apply to all types of customer research.

What Nielsen Actually Said

Nielsen's original recommendation came with important caveats that often get overlooked:

  1. It applies to finding interface issues, not understanding customer needs, motivations, or behaviors
  2. It assumes a homogeneous user group: if you have distinct user segments, you need 5 per segment
  3. It's meant for iterative testing: test 5 users, fix issues, test another 5, repeat

Nielsen himself noted that you would need about 15 users to find all the usability problems in a design, but that it is better to run 3 tests with 5 users each, fixing the problems you find between rounds.

Why Customer Interviews Are Different

Customer interviews aren't about finding interface bugs. They're about understanding:

  • What problems customers are trying to solve
  • How they currently approach those problems
  • What motivates their decisions
  • What barriers they face
  • What language they use to describe their needs

This type of discovery research requires enough conversations to identify patterns and validate that those patterns are real, not just individual opinions. And that typically requires more than 5 interviews.


Understanding Data Saturation: When Have You Heard Enough?

Rather than fixating on a magic number, experienced researchers focus on data saturation: the point where new interviews stop revealing new insights.

Think of it like panning for gold. The first few pans bring up plenty of nuggets. But eventually, each new pan yields less and less. When you're consistently coming up empty, you've likely found what there is to find.

How to Recognize Saturation

You're approaching saturation when:

  • Themes repeat consistently across interviews
  • New interviews confirm what you've already heard rather than adding new dimensions
  • You can predict how customers will respond to certain questions
  • Your research questions feel answered with confidence

What Research Shows About Saturation

Three studies founders can lean on:

  • Griffin and Hauser (1993, Marketing Science) found that 20 to 30 interviews uncovered roughly 90 to 95% of customer needs in their product category.
  • Guest, Bunce and Johnson (2006, Field Methods) found that most themes in a set of 60 interviews had appeared within the first 12, with the broad themes visible after about 6.
  • Hennink and Kaiser (2022, Social Science & Medicine), a systematic review of saturation studies, found that 9 to 17 interviews reached saturation in most studies with a fairly similar group of people and a focused question.
Population TypeTypical Saturation Point
Homogeneous, focused scope9-12 interviews
Moderately diverse15-20 interviews
Highly diverse or broad scope25-30+ interviews

The key insight? Saturation depends heavily on how diverse your customers are and how broad your research questions are. For a founder, that means the fastest way to need fewer interviews is a narrower segment.


Sample Size Recommendations by Research Type

Different research goals require different sample sizes. Here's a practical framework:

Table of customer interview sample sizes by research type, from 5 to 8 for usability testing to 30 to 50 or more for market research.
Start from your research goal, then add interviews for each segment you need to compare. · Download or share

Usability Testing: 5-8 Participants

If you're testing whether customers can complete specific tasks in your product, 5-8 participants per user segment will reveal most usability issues. This is where Nielsen's original guidance applies.

When this works:

  • Testing specific features or flows
  • Identifying interface problems
  • Validating design solutions

Discovery Interviews: 12-20 Participants

When you're trying to understand customer problems, needs, and behaviors, plan for more conversations. Discovery research requires identifying patterns that hold true across multiple customers.

When this applies:

  • Understanding customer problems and pain points
  • Exploring how customers currently solve problems
  • Identifying unmet needs
  • Validating problem hypotheses

Segmented Research: 8-12 Per Segment

If you're researching across distinct customer segments (e.g., enterprise vs. SMB, new users vs. power users), you need enough interviews within each segment to identify segment-specific patterns.

The math:

  • 3 segments x 10 interviews each = 30 total interviews
  • This ensures you can distinguish real segment differences from individual variation

Concept Validation: 15-25 Participants

When testing new product concepts or value propositions, you need enough feedback to feel confident about patterns. Positive feedback from 3 people isn't validation; it could be coincidence. Set your pass bar before you start (see above), and count commitments, not compliments.

Market Research: 30-50+ Participants

For broader market understanding, like sizing opportunities or understanding competitive dynamics, you need larger samples to draw meaningful conclusions.

Consider these benchmarks:

  • 30 minimum for directional insights
  • 50+ for more confident conclusions
  • 100+ to support quantitative analysis

The Traditional Constraint: Why Teams Interview Too Few

Here's the uncomfortable truth: most teams conduct fewer interviews than they should. Not because they don't understand the value, but because of practical constraints.

Time Investment

A single 45-minute interview takes far more than 45 minutes. Count the scheduling back and forth, the call itself, writing up notes and the time to pull patterns together across calls. For a founder doing this alongside everything else, 20 interviews can easily take more than a working week.

Access Challenges

Finding and recruiting the right customers takes effort:

  • Identifying qualified participants
  • Getting responses to outreach
  • Scheduling around availability
  • Managing no-shows

Cost Considerations

Researcher time, participant incentives and the opportunity cost of a founder's week all add up. At those economics, teams make trade-offs. They interview fewer customers than ideal, prioritize only the highest-stakes research, or skip customer interviews altogether.


How AI Changes the Sample Size Equation

AI interview tools change the economics of customer research, because the interviewer's time is no longer the limit.

What AI Interviewers Enable

With AI handling the conversation:

  1. Interviews run any time: customers take part on their schedule, not yours
  2. No scheduling coordination: send a link, get responses
  3. Automated transcription and analysis: each interview is summarised as it comes in
  4. Consistent interview quality: every conversation follows your guide
  5. Lower marginal cost: each additional interview costs little beyond the tool and any incentive

That doesn't just mean more interviews. It means:

  • Confidence in patterns: when most of 50 customers describe the same problem, you can act on it
  • Segment-level insights: interview enough customers to understand each segment
  • Continuous learning: run ongoing research, not just periodic projects
  • Numbers next to stories: a structured question asked of everyone gives you a share you can report alongside the quotes

When Human Interviews Still Matter

AI interviewers are powerful, but they're not always the right choice:

  • Sensitive topics requiring careful rapport-building
  • Complex exploratory research with unpredictable directions
  • High-stakes stakeholder interviews where personal connection matters
  • Deeply technical discovery requiring expert follow-up questions

For a founder, there is one more: your first handful of discovery calls. Do those yourself. They are how you learn the language of the problem. Then use AI interviews to reach the 20, 30 or 50 people a validation round needs.


A Practical Framework for Determining Your Sample Size

Rather than memorizing numbers, use this decision framework:

Step 1: Define Your Research Type

Research TypeMinimum SampleRecommended Sample
Usability testing58 per segment
Discovery interviews1215-20
Concept validation1520-25
Segment research8/segment12/segment
Market research3050+

Step 2: Assess Audience Diversity

  • Homogeneous audience (similar contexts, needs, behaviors): Use minimum samples
  • Moderately diverse (some variation in use cases): Add 25-50%
  • Highly diverse (distinct segments, varied needs): Double the minimum or segment your research

Step 3: Consider Your Confidence Needs

  • Exploratory/directional insights: Minimum samples acceptable
  • Informing significant decisions: Recommended samples
  • High-stakes or contentious topics: Add 20-30% buffer

Step 4: Plan for Attrition

Recruit 15-20% more than your target to account for:

  • No-shows
  • Technical issues
  • Poor-quality responses
  • Participants who don't match criteria

With Koji, screening questions run before every interview, whatever route people came in by, and anyone who doesn't qualify is turned away. When you recruit from Koji's panel, interviews that were screened out or rejected for quality don't count toward your study's participant limit, and credits set aside for a recruitment that aren't used come back at the end.

Step 5: Monitor Saturation as You Go

Don't commit to a rigid number upfront. Instead:

  1. Start with your minimum viable sample
  2. Analyze in batches (every 5-10 interviews)
  3. Track whether new themes are emerging
  4. Stop when you're consistently hearing familiar patterns

Common Sample Size Mistakes (And How to Avoid Them)

Mistake 1: Stopping at 5 Because "Nielsen Said So"

The fix: Recognize that 5 users is for usability testing, not discovery research. For understanding customer needs, start with 12 minimum.

Mistake 2: Moving the Validation Bar After the Calls

The fix: Write the pass bar down before the first call. If you change it after seeing the results, you are no longer testing anything.

Mistake 3: Not Accounting for Segments

The fix: If you're researching across segments, you need critical mass within each segment, not just total interviews. 20 interviews across 4 segments means only 5 per segment, which isn't enough.

Mistake 4: Treating Each Study in Isolation

The fix: Build cumulative knowledge. If you interviewed 15 enterprise customers last quarter, you might only need 8 more for a focused follow-up study.

Mistake 5: Prioritizing Quantity Over Quality

The fix: More interviews only helps if they're well-conducted. Ensure your interview guide is solid, your participants match your criteria, and your analysis is rigorous.


Making Customer Research Sustainable

The real goal isn't to nail the "right" sample size for a single study. It's to build a habit where customer insights keep informing decisions.

Shift from Projects to Programs

Instead of occasional deep-dive studies, consider:

  • Continuous discovery: regular small-batch interviews every week
  • Always-on feedback channels: interviews embedded in your product
  • Rapid research sprints: quick-turn studies for specific questions

Use the Right Tool for Each Job

NeedBest Approach
Broad pattern identificationAI interviews (30-50+)
Deep explorationHuman interviews (8-12)
Quick validationAI interviews (15-20)
Sensitive topicsHuman interviews (10-15)
Ongoing pulseAI interviews (continuous)

Key Takeaways

  1. Know which job you are doing. Discovery stops at saturation, usually 12 to 20 per segment. Validation stops when you pass or fail a bar you set in advance.

  2. The "5 users" rule is for usability testing, not customer interviews.

  3. Saturation matters more than arbitrary numbers. Monitor when new interviews stop revealing new insights, and stop there.

  4. A narrower segment needs fewer interviews. Diverse populations require larger samples.

  5. Count commitments, not compliments, when you validate.

  6. AI changes the economics. When interviews cost less, you can afford the sample sizes that give you real confidence in your findings.


Start Talking to More Customers Today

Run your first discovery calls yourself. When you are ready for a validation round, put the same questions into a Koji study and send one link: the AI interviewer runs every conversation by voice or text, probes the same way each time, and the report shows the themes with the quotes behind them.

Koji can recruit the participants too: describe your audience in Panel recruit and see the cost before you launch, or import your own customer list so each person gets a personal link, and the interviews land in the same study. You get 10 free credits when you sign up, enough for ten text interviews.

See how Koji works | Start free

For the full reference on sample sizes by method, see How Many User Interviews Do You Need?. To run the calls themselves, see The Mom Test: How to Talk to Customers Without Being Misled.


Sources: Jakob Nielsen, "Why You Only Need to Test with 5 Users", Nielsen Norman Group (2000); Abbie Griffin and John Hauser, "The Voice of the Customer", Marketing Science (1993); Greg Guest, Arwen Bunce and Laura Johnson, "How Many Interviews Are Enough?", Field Methods (2006); Monique Hennink and Bonnie Kaiser, "Sample sizes for saturation in qualitative research: A systematic review of empirical tests", Social Science & Medicine (2022).

Run your first AI-moderated study in 10 minutes

100 free credits on signup. No credit card required.

GDPR compliantEU or US data residencyNo AI training on your data
Koji

Koji Team

Product

Share this article

Keep reading