Survey Sample Size: How Many Responses Do You Really Need? (2026 Guide)

Answer-first (BLUF): For most product and marketing surveys, 384 responses give you a ±5% margin of error at 95% confidence for any population larger than ~20,000 — and that number barely moves whether you have 50,000 users or 50 million. For directional decisions you can act on quickly, 100–200 responses are often enough. For statistical comparisons between segments, you need 384 per segment (not total). And for qualitative depth — the kind of "why" that no sample size formula can capture — switch from surveys to AI-moderated interviews, where 15–30 conversations consistently outperform 1,000 multiple-choice answers.

The one-paragraph version (if you only read this)

If you need a number right now: use n = 384 for a one-time decision-grade survey on a population of any reasonable size, with 95% confidence and a ±5% margin of error. If you're comparing two groups (e.g., free vs. paid users), use 384 per group. If your population is under 1,000, use a finite-population correction (formulas below). And remember: the quality of your sample matters more than the quantity — 100 well-recruited responses crush 5,000 self-selected ones every time.

What sample size actually means

In survey research, sample size is the number of responses you collect from a defined target population. The reason it matters: you're using the sample to draw conclusions about the entire population, and the math of statistical inference says you can only be so confident in those conclusions based on how many people you ask.

Three numbers drive every sample size calculation:

1. Confidence level — the probability that your sample's answer is within your margin of error of the true population answer. 95% is the standard in product research; 90% is acceptable for directional work; 99% is reserved for high-stakes regulatory or clinical contexts.
2. Margin of error — the plus-or-minus accuracy you'll tolerate. ±5% is standard for product surveys; ±3% for more rigorous work; ±10% for early-stage exploration.
3. Population size — the total number of people in the group you want to learn about. Here's the surprising part: above ~20,000 people, the population size stops affecting the required sample size meaningfully.

A fourth number — expected response variance (often called p) — also matters. Researchers conservatively assume p = 0.5 (maximum variance) when they don't know the population's answer distribution. This gives the largest required sample size and is the safe default.

The formula every researcher should memorize

For an infinite (or very large) population:

n = (z² × p × (1-p)) / e²

Where:

• n = required sample size
• z = z-score for your confidence level (1.645 for 90%, 1.96 for 95%, 2.576 for 99%)
• p = expected proportion (use 0.5 when unknown — most conservative)
• e = margin of error in decimal form (0.05 for ±5%)

Plugging in the standard values (95% confidence, ±5% margin, p = 0.5):

n = (1.96² × 0.5 × 0.5) / 0.05²
n = (3.8416 × 0.25) / 0.0025
n = 384.16

That's where the famous 384 number comes from. It's the universal "good enough" sample size for any large population at standard rigor.

For smaller populations: the finite population correction

If your population is under ~20,000, apply the finite population correction:

n_adjusted = n / (1 + ((n - 1) / N))

Where N is your total population size. For a population of 500, the corrected sample is 217 (not 384) — a meaningful savings for B2B research on small named-account lists.

Standard sample size reference table

What about statistical power?

Sample size formulas above answer the question "how precisely can I estimate one number?" If you're comparing groups (A/B testing, segment differences, pre/post analysis), you need statistical power analysis instead.

The 80% power rule of thumb: Most researchers set statistical power at 0.80 — meaning if there really is a difference between groups, you'll detect it 80% of the time. According to peer-reviewed methodology research, "the minimum power of a study required is ideally 80%, which is a commonly accepted benchmark in research methodology."

The sample size you need for an A/B comparison depends on:

• Effect size (how big a difference you care about detecting)
• Significance level (alpha, usually 0.05)
• Power (usually 0.80)
• Baseline rate (your control group conversion or response rate)

For a typical product survey comparing two segments where you want to detect a 5-percentage-point difference at 95% confidence and 80% power, you need roughly 385 responses per group (770 total). To detect a smaller 2-point difference, that jumps to 2,400 per group.

This is why "we got 500 responses, let's slice it ten ways" almost always produces underpowered analyses. Each slice needs to clear the per-group sample size threshold.

Sample size benchmarks by use case

Formulas give you statistical floors. Real-world benchmarks tell you what working researchers actually use:

The most common sample size mistakes

1. Confusing total sample with per-segment sample

If you're going to slice your survey by industry, role, or company size, every slice you care about needs to clear the sample size threshold independently. A 400-person survey split across 5 industries gives you 80 per industry — underpowered for anything but the broadest claims.

2. Treating self-selected respondents as a random sample

The sample size formula assumes random sampling. A pop-up survey on your homepage isn't random — it overweights frequent visitors. A LinkedIn poll skews toward your network. Calculate your sample size for the question you can actually answer (e.g., "what do my homepage visitors think") not the one you wish you could ("what do users think").

3. Ignoring response rate when sizing distribution

If you need 400 completed responses and your typical email response rate is 5%, you need to send invitations to 8,000 people. Plan distribution backward from completes.

4. Defaulting to "as many as we can get"

This isn't cost-free. Long surveys with too many respondents:

• Inflate cost and incentive spend
• Make analysis slower
• Tempt you into over-slicing
• Can introduce more noise as quality declines past the optimal sample

Decide your target sample, hit it, and stop.

5. Forgetting that quality > quantity

A 100-response survey from a well-screened panel of your actual customer ICP will out-predict a 5,000-response survey from a Facebook ad. Sample size is the floor for statistical confidence; sample quality is the ceiling on insight.

When sample size is the wrong question

Sample size formulas assume you're measuring something you can already define — a known metric like NPS, a known choice like preference between concepts, a known proportion like "% who would buy at price X."

If you're still trying to understand what to measure — what users actually care about, why they churn, what frustrates them about your category — sample size becomes a distraction. You need depth, not breadth. The right tool isn't a 1,000-person survey; it's 15–30 qualitative interviews.

Research from Nielsen Norman Group and others has consistently shown that roughly 5 user interviews surface ~85% of the major usability issues in a flow, and 15–30 conversations reach thematic saturation for most discovery questions. For exploratory work, you don't need more respondents — you need richer conversations with fewer.

This is where AI-moderated interview platforms have completely rewritten the trade-off.

The modern AI-native approach with Koji

The historical reason teams over-relied on surveys was simple: interviews were expensive. Recruiting, scheduling, moderating, transcribing, and analyzing 30 interviews cost more than running a 1,000-person survey — so PMs picked the survey, even when the question called for depth.

AI-moderated platforms like Koji collapse the interview cost curve and change the calculus:

• AI moderates interviews 24/7. A 30-person interview study that used to take 4–6 weeks now finishes in days, with Koji's AI conducting and probing each conversation in real time.
• Hybrid structured + open-ended in one session. Koji supports all 6 structured question types (scale, single_choice, multiple_choice, ranking, yes_no, open_ended) inside the same interview. You get survey-quality numbers and interview-depth context from every respondent — no need to choose.
• Automatic thematic analysis. Instead of manually coding 30 transcripts (40+ hours of work), Koji surfaces themes, sentiment, and quotes automatically. You spend your time on interpretation, not data entry.
• Real-time reporting as responses come in. Watch themes emerge while the study is still in field. Decide during the study whether you've reached saturation, instead of guessing at the start.
• Sample size flexibility. Because each interview costs a fraction of traditional moderated research, you can comfortably run 50–200 person interview studies that previously would have been replaced by a thin survey.

While traditional survey tools like SurveyMonkey and Qualtrics require you to pick "wide and shallow" or "narrow and deep" — and then plug in a sample size calculator to figure out wide-and-shallow — AI-native platforms like Koji let you have both at once. The sample size question itself shifts: instead of "how many responses do I need to be confident?" it becomes "how many conversations do I need to understand the why?"

That's a better question.

How to choose your sample size in 5 steps

1. Write down your decision. What will you do differently based on the result? If the answer is "nothing meaningful," reduce your sample size — you're over-investing.
2. Identify your target population. B2B buyers at Series B SaaS companies? Free users of your iOS app? Decision matters: it changes both the formula and the recruiting strategy.
3. Pick your confidence level and margin of error. Defaults: 95% and ±5%. Only deviate with a reason.
4. List the comparisons you need to make. Every group you'll compare needs its own sample size, not a slice of one total.
5. Plan distribution for 3–5× your target n based on expected response rate, screening attrition, and quality removals.

Quick sample size cheat sheet

• One-time directional read: 100–200 responses
• Decision-grade single metric: 384 responses
• Two-segment comparison: 384 per segment (768 total)
• Pricing or conjoint: 300–500 responses
• Brand tracking wave: 300–500 per wave
• Qualitative depth: 15–30 AI-moderated interviews (skip the survey)
• Anything involving slicing more than 5 ways: rethink the study design — you probably need a different methodology

Related Resources

• Structured questions guide — Get survey-grade numbers and interview-grade depth in one session
• How many user interviews you need — The qualitative counterpart to this guide
• Survey design best practices — Get the most out of every response
• Qualitative vs quantitative research — Picking the right method, not just the right sample size
• Mixed methods research guide — When you genuinely need both
• How to increase survey response rates — Practical tactics for hitting your target n

---

Sources: Memon et al., "Sample Size for Survey Research: Review and Recommendations," Journal of Applied Structural Equation Modeling (2020); Cochran, "Sampling Techniques" (1977); Hair et al., "A Primer on Partial Least Squares Structural Equation Modeling" (2017); Qualtrics Sample Size Calculator methodology documentation; CloudResearch sample size guide.