You are a founder with no researcher, a short runway and a question you need answered. Before you book a single call, you want to know how many conversations it will take.
The answer depends on which of two jobs you are doing. Discovery (finding out what the problem really is, who has it and how they cope) is done when new interviews stop telling you anything new: usually 12 to 20 conversations with one kind of customer. Validation (checking whether a specific bet holds, like "ops leads at 20-person agencies will pay to fix this") is done when you hit or miss a pass bar you wrote down before the first call: often 10 to 15 conversations to see the signal, and 30 or more once you want a share you can quote.
This post gives a founder the numbers and the decision rules for both. For the full sample size reference across every research method, see our guide How Many User Interviews Do You Need?.
Discovery or validation? Know which number you need
| Discovery | Validation | |
|---|---|---|
| The question | What is the problem, who has it, how do they cope today? | Does this specific bet hold for this specific customer? |
| When you stop | When new interviews stop adding new themes (saturation) | When you clearly pass or fail a bar you set in advance |
| Typical number | 12 to 20 per segment | 10 to 15 to see a signal, 30+ to quote a share |
| What counts as evidence | Repeated stories, workarounds, the words people use | Commitments: a second call, a pilot, a deposit, a letter of intent |
| Biggest risk | Too broad a segment, so answers never converge | Counting compliments as demand |
Most early founders start with discovery and move to validation once the same problem keeps coming up. If you are not sure which you are doing, you are doing discovery.
How to set a validation pass bar
Write it down before the first call, so you cannot move it afterwards. For example:
- "Of 12 ops leads at agencies with 10 to 50 people, at least 6 describe losing a day or more a month to this in the last quarter."
- "At least 3 of those 6 agree to a paid pilot or a follow-up with their manager."
The exact bar is your call. What matters is that it is about past behavior and commitments, not about whether people liked the idea. If you hit it, widen the sample. If you miss it, change the segment or the problem, not the bar.
The "5 Users" Myth: Where It Came From and Why It Doesn't Always Apply
You've probably heard the famous claim: "Testing with 5 users uncovers 85% of usability problems." This insight, popularized by Jakob Nielsen of the Nielsen Norman Group, has become one of the most cited (and misapplied) guidelines in product research.
Here's the thing: Nielsen's research was specifically about usability testing: watching users interact with an interface to find problems. It was never meant to apply to all types of customer research.
What Nielsen Actually Said
Nielsen's original recommendation came with important caveats that often get overlooked:
- It applies to finding interface issues, not understanding customer needs, motivations, or behaviors
- It assumes a homogeneous user group: if you have distinct user segments, you need 5 per segment
- It's meant for iterative testing: test 5 users, fix issues, test another 5, repeat
Nielsen himself noted that you would need about 15 users to find all the usability problems in a design, but that it is better to run 3 tests with 5 users each, fixing the problems you find between rounds.
Why Customer Interviews Are Different
Customer interviews aren't about finding interface bugs. They're about understanding:
- What problems customers are trying to solve
- How they currently approach those problems
- What motivates their decisions
- What barriers they face
- What language they use to describe their needs
This type of discovery research requires enough conversations to identify patterns and validate that those patterns are real, not just individual opinions. And that typically requires more than 5 interviews.
Understanding Data Saturation: When Have You Heard Enough?
Rather than fixating on a magic number, experienced researchers focus on data saturation: the point where new interviews stop revealing new insights.
Think of it like panning for gold. The first few pans bring up plenty of nuggets. But eventually, each new pan yields less and less. When you're consistently coming up empty, you've likely found what there is to find.
How to Recognize Saturation
You're approaching saturation when:
- Themes repeat consistently across interviews
- New interviews confirm what you've already heard rather than adding new dimensions
- You can predict how customers will respond to certain questions
- Your research questions feel answered with confidence
What Research Shows About Saturation
Three studies founders can lean on:
- Griffin and Hauser (1993, Marketing Science) found that 20 to 30 interviews uncovered roughly 90 to 95% of customer needs in their product category.
- Guest, Bunce and Johnson (2006, Field Methods) found that most themes in a set of 60 interviews had appeared within the first 12, with the broad themes visible after about 6.
- Hennink and Kaiser (2022, Social Science & Medicine), a systematic review of saturation studies, found that 9 to 17 interviews reached saturation in most studies with a fairly similar group of people and a focused question.
| Population Type | Typical Saturation Point |
|---|---|
| Homogeneous, focused scope | 9-12 interviews |
| Moderately diverse | 15-20 interviews |
| Highly diverse or broad scope | 25-30+ interviews |
The key insight? Saturation depends heavily on how diverse your customers are and how broad your research questions are. For a founder, that means the fastest way to need fewer interviews is a narrower segment.
Sample Size Recommendations by Research Type
Different research goals require different sample sizes. Here's a practical framework:

Usability Testing: 5-8 Participants
If you're testing whether customers can complete specific tasks in your product, 5-8 participants per user segment will reveal most usability issues. This is where Nielsen's original guidance applies.
When this works:
- Testing specific features or flows
- Identifying interface problems
- Validating design solutions
Discovery Interviews: 12-20 Participants
When you're trying to understand customer problems, needs, and behaviors, plan for more conversations. Discovery research requires identifying patterns that hold true across multiple customers.
When this applies:
- Understanding customer problems and pain points
- Exploring how customers currently solve problems
- Identifying unmet needs
- Validating problem hypotheses
Segmented Research: 8-12 Per Segment
If you're researching across distinct customer segments (e.g., enterprise vs. SMB, new users vs. power users), you need enough interviews within each segment to identify segment-specific patterns.
The math:
- 3 segments x 10 interviews each = 30 total interviews
- This ensures you can distinguish real segment differences from individual variation
Concept Validation: 15-25 Participants
When testing new product concepts or value propositions, you need enough feedback to feel confident about patterns. Positive feedback from 3 people isn't validation; it could be coincidence. Set your pass bar before you start (see above), and count commitments, not compliments.
Market Research: 30-50+ Participants
For broader market understanding, like sizing opportunities or understanding competitive dynamics, you need larger samples to draw meaningful conclusions.
Consider these benchmarks:
- 30 minimum for directional insights
- 50+ for more confident conclusions
- 100+ to support quantitative analysis
The Traditional Constraint: Why Teams Interview Too Few
Here's the uncomfortable truth: most teams conduct fewer interviews than they should. Not because they don't understand the value, but because of practical constraints.
Time Investment
A single 45-minute interview takes far more than 45 minutes. Count the scheduling back and forth, the call itself, writing up notes and the time to pull patterns together across calls. For a founder doing this alongside everything else, 20 interviews can easily take more than a working week.
Access Challenges
Finding and recruiting the right customers takes effort:
- Identifying qualified participants
- Getting responses to outreach
- Scheduling around availability
- Managing no-shows
Cost Considerations
Researcher time, participant incentives and the opportunity cost of a founder's week all add up. At those economics, teams make trade-offs. They interview fewer customers than ideal, prioritize only the highest-stakes research, or skip customer interviews altogether.
How AI Changes the Sample Size Equation
AI interview tools change the economics of customer research, because the interviewer's time is no longer the limit.
What AI Interviewers Enable
With AI handling the conversation:
- Interviews run any time: customers take part on their schedule, not yours
- No scheduling coordination: send a link, get responses
- Automated transcription and analysis: each interview is summarised as it comes in
- Consistent interview quality: every conversation follows your guide
- Lower marginal cost: each additional interview costs little beyond the tool and any incentive
That doesn't just mean more interviews. It means:
- Confidence in patterns: when most of 50 customers describe the same problem, you can act on it
- Segment-level insights: interview enough customers to understand each segment
- Continuous learning: run ongoing research, not just periodic projects
- Numbers next to stories: a structured question asked of everyone gives you a share you can report alongside the quotes
When Human Interviews Still Matter
AI interviewers are powerful, but they're not always the right choice:
- Sensitive topics requiring careful rapport-building
- Complex exploratory research with unpredictable directions
- High-stakes stakeholder interviews where personal connection matters
- Deeply technical discovery requiring expert follow-up questions
For a founder, there is one more: your first handful of discovery calls. Do those yourself. They are how you learn the language of the problem. Then use AI interviews to reach the 20, 30 or 50 people a validation round needs.
A Practical Framework for Determining Your Sample Size
Rather than memorizing numbers, use this decision framework:
Step 1: Define Your Research Type
| Research Type | Minimum Sample | Recommended Sample |
|---|---|---|
| Usability testing | 5 | 8 per segment |
| Discovery interviews | 12 | 15-20 |
| Concept validation | 15 | 20-25 |
| Segment research | 8/segment | 12/segment |
| Market research | 30 | 50+ |
Step 2: Assess Audience Diversity
- Homogeneous audience (similar contexts, needs, behaviors): Use minimum samples
- Moderately diverse (some variation in use cases): Add 25-50%
- Highly diverse (distinct segments, varied needs): Double the minimum or segment your research
Step 3: Consider Your Confidence Needs
- Exploratory/directional insights: Minimum samples acceptable
- Informing significant decisions: Recommended samples
- High-stakes or contentious topics: Add 20-30% buffer
Step 4: Plan for Attrition
Recruit 15-20% more than your target to account for:
- No-shows
- Technical issues
- Poor-quality responses
- Participants who don't match criteria
With Koji, screening questions run before every interview, whatever route people came in by, and anyone who doesn't qualify is turned away. When you recruit from Koji's panel, interviews that were screened out or rejected for quality don't count toward your study's participant limit, and credits set aside for a recruitment that aren't used come back at the end.
Step 5: Monitor Saturation as You Go
Don't commit to a rigid number upfront. Instead:
- Start with your minimum viable sample
- Analyze in batches (every 5-10 interviews)
- Track whether new themes are emerging
- Stop when you're consistently hearing familiar patterns
Common Sample Size Mistakes (And How to Avoid Them)
Mistake 1: Stopping at 5 Because "Nielsen Said So"
The fix: Recognize that 5 users is for usability testing, not discovery research. For understanding customer needs, start with 12 minimum.
Mistake 2: Moving the Validation Bar After the Calls
The fix: Write the pass bar down before the first call. If you change it after seeing the results, you are no longer testing anything.
Mistake 3: Not Accounting for Segments
The fix: If you're researching across segments, you need critical mass within each segment, not just total interviews. 20 interviews across 4 segments means only 5 per segment, which isn't enough.
Mistake 4: Treating Each Study in Isolation
The fix: Build cumulative knowledge. If you interviewed 15 enterprise customers last quarter, you might only need 8 more for a focused follow-up study.
Mistake 5: Prioritizing Quantity Over Quality
The fix: More interviews only helps if they're well-conducted. Ensure your interview guide is solid, your participants match your criteria, and your analysis is rigorous.
Making Customer Research Sustainable
The real goal isn't to nail the "right" sample size for a single study. It's to build a habit where customer insights keep informing decisions.
Shift from Projects to Programs
Instead of occasional deep-dive studies, consider:
- Continuous discovery: regular small-batch interviews every week
- Always-on feedback channels: interviews embedded in your product
- Rapid research sprints: quick-turn studies for specific questions
Use the Right Tool for Each Job
| Need | Best Approach |
|---|---|
| Broad pattern identification | AI interviews (30-50+) |
| Deep exploration | Human interviews (8-12) |
| Quick validation | AI interviews (15-20) |
| Sensitive topics | Human interviews (10-15) |
| Ongoing pulse | AI interviews (continuous) |
Key Takeaways
-
Know which job you are doing. Discovery stops at saturation, usually 12 to 20 per segment. Validation stops when you pass or fail a bar you set in advance.
-
The "5 users" rule is for usability testing, not customer interviews.
-
Saturation matters more than arbitrary numbers. Monitor when new interviews stop revealing new insights, and stop there.
-
A narrower segment needs fewer interviews. Diverse populations require larger samples.
-
Count commitments, not compliments, when you validate.
-
AI changes the economics. When interviews cost less, you can afford the sample sizes that give you real confidence in your findings.
Start Talking to More Customers Today
Run your first discovery calls yourself. When you are ready for a validation round, put the same questions into a Koji study and send one link: the AI interviewer runs every conversation by voice or text, probes the same way each time, and the report shows the themes with the quotes behind them.
Koji can recruit the participants too: describe your audience in Panel recruit and see the cost before you launch, or import your own customer list so each person gets a personal link, and the interviews land in the same study. You get 10 free credits when you sign up, enough for ten text interviews.
See how Koji works | Start free
For the full reference on sample sizes by method, see How Many User Interviews Do You Need?. To run the calls themselves, see The Mom Test: How to Talk to Customers Without Being Misled.
Sources: Jakob Nielsen, "Why You Only Need to Test with 5 Users", Nielsen Norman Group (2000); Abbie Griffin and John Hauser, "The Voice of the Customer", Marketing Science (1993); Greg Guest, Arwen Bunce and Laura Johnson, "How Many Interviews Are Enough?", Field Methods (2006); Monique Hennink and Bonnie Kaiser, "Sample sizes for saturation in qualitative research: A systematic review of empirical tests", Social Science & Medicine (2022).


