{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-29T14:02:03.970Z"},"content":[{"type":"blog","id":"19cb914d-fce2-4d61-b09a-a628051d6e22","slug":"how-many-users-usability-testing-2026","title":"How Many Users Do You Need for Usability Testing? The \"5 Users\" Rule, Revisited (2026)","url":"https://www.koji.so/blog/how-many-users-usability-testing-2026","summary":"How many users do you need for usability testing? For qualitative testing, 5 users per audience segment uncovers ~85% of usability problems (Nielsen & Landauer, based on a 31% per-user problem-discovery rate). The formula N=1-(1-L)^n means lower discovery rates need more users: 9 users at L=20%, 18 at L=10%. Faulkner showed 5-user runs range from 55% to 95% detection. For quantitative metrics (task success, time-on-task, SUS) you need 20-40 participants at ~90% confidence; a task-completion rate above 78% is above average. Test 5 per distinct segment, not 5 total. In 2026, AI-moderated interviews (Koji) remove the per-participant cost, making larger, better-segmented studies as fast and cheap as a 5-user test, with automatic thematic analysis and one-click reports.","content":"**TL;DR:** For *qualitative* usability testing, **5 users per audience segment uncovers roughly 85% of usability problems** — the classic finding from Jakob Nielsen and Tom Landauer, based on an average problem-discovery rate of 31% per user. But \"5 users\" only holds when each problem is fairly easy to hit. For low-frequency issues, complex flows, or multiple distinct user groups you need more (9 users at a 20% discovery rate, 18 at 10%). For *quantitative* metrics — task-success rate, time-on-task, SUS benchmarking — you need **20–40 participants**, not 5. The real rule isn't a number; it's *match your sample to your goal*. And when you need interview-level depth at survey-level scale, AI-moderated interviews let you reach 30–50 users in a week for a fraction of the cost.\n\n## The short answer\n\nHow many users you need depends entirely on what kind of answer you want:\n\n- **Finding usability problems (qualitative):** 5 users per segment finds ~85% of issues. Run 2–3 small rounds instead of one big one.\n- **Measuring usability (quantitative):** 20–40 users to get statistically meaningful task-success rates, completion times, or System Usability Scale scores.\n- **Multiple distinct audiences:** 5 users *per group* — a B2B tool with admins, end users, and buyers needs ~15, not 5.\n- **Subtle or rare problems:** 10–18+ users, because each individual user has a low chance of triggering the issue.\n\nIf you only remember one thing: the famous \"5 users\" number is a rule for *discovering problems in a single, homogeneous group* — not a universal answer for every research question.\n\n## Where the \"5 users\" rule comes from\n\nThe rule traces back to a 1993 study by Jakob Nielsen and Thomas Landauer, popularized in Nielsen's widely cited 2000 article \"Why You Only Need to Test with 5 Users.\" They built a mathematical model showing that a qualitative test with 5 participants typically surfaces about **85% of the usability problems** in an interface — and that the cost-benefit curve flattens fast after that. Testing a sixth, seventh, or eighth user mostly re-finds problems you've already seen.\n\nThe strategic takeaway was never \"only ever test 5 people.\" It was \"run *more, smaller* tests.\" Three rounds of 5 users — test, fix, re-test — beats one round of 15, because you catch and repair problems iteratively instead of documenting them all at once and shipping the same flawed design.\n\n## The formula behind 5 users (and why it breaks)\n\nThe model is simple: **N = 1 − (1 − L)^n**, where N is the proportion of problems found, n is the number of users, and **L is the average probability that any single user encounters any given problem.**\n\nNielsen and Landauer measured an average L of **31%** across their projects. Plug that in and 5 users find ~85%. But L is an *average* — and it's the assumption that quietly breaks the rule:\n\n- If L = **31%**, 5 users find ~85%.\n- If L = **20%**, you need about **9 users** to reach 85%.\n- If L = **10%** (subtle, low-frequency problems), you need roughly **18 users.**\n\nAnd the variance is real. In a landmark re-examination, researcher Laura Faulkner found that while 5 users averaged 85% problem detection, individual runs of 5 ranged from as low as **55%** to as high as **95%** depending on which 5 people you happened to recruit. With a single small sample, you might catch nearly everything — or miss nearly half. More users narrows that gap. This is the same diminishing-returns logic behind [data saturation in qualitative research](/docs/data-saturation-qualitative-research): you keep sampling until new sessions stop surfacing new themes.\n\n## Qualitative vs quantitative: two different sample sizes\n\nThis is where most \"how many users\" debates go wrong — people argue about a number without agreeing on the *type* of study.\n\n**Qualitative usability testing** answers *\"what's broken and why?\"* You watch people attempt tasks, hear them think aloud, and find problems to fix. Here, 5 per group is genuinely efficient.\n\n**Quantitative usability testing** answers *\"how usable is it, numerically?\"* — task-success rate, error rate, time-on-task, or a [System Usability Scale](/docs/system-usability-scale-guide) score you can benchmark over time. Numbers need a bigger sample so a single outlier doesn't swing the result. The consensus range is **20–40 participants**, with analysis typically reported at 90% confidence intervals. For context on what \"good\" looks like, a task-completion rate above **78%** is considered above average, and many teams require ~80% success before calling a feature launch-ready. See our [usability metrics guide](/docs/usability-metrics-guide) for the full benchmark set.\n\nTrying to compute a reliable task-success percentage from 5 users is like calling an election from 5 voters — the math simply isn't there.\n\n## How many users by research goal\n\n| Your goal | Study type | Recommended sample |\n|---|---|---|\n| Find the obvious usability problems | Qualitative | 5 per segment |\n| Catch subtle / low-frequency issues | Qualitative | 10–18 |\n| Test multiple distinct user groups | Qualitative | 5 per group |\n| Benchmark task-success / time-on-task | Quantitative | 20–40 |\n| Compare two designs (A/B) | Quantitative | 20–50+ per variant |\n| Continuous discovery / \"why\" behind behavior | Interviews | 15–30+ |\n\nFor the underlying method and a step-by-step plan, see the full [usability testing guide](/docs/usability-testing-guide) and our [moderated vs unmoderated research](/blog/moderated-vs-unmoderated-research-2026) breakdown.\n\n## The hidden assumption that wrecks the 5-user rule: segments\n\nThe single most common way teams misapply \"5 users\" is by ignoring **audience segmentation.** The rule assumes one homogeneous user group. But if your product serves meaningfully different people — say, a B2B platform with administrators, daily end users, and economic buyers — each group hits different problems. Five admins tell you almost nothing about what confuses end users. You need 5 *per distinct segment*, which is why a serious study often lands at 15–20 even when each individual group only needs 5.\n\n## Why teams are rethinking sample size in 2026\n\nThe \"5 users\" rule was built for an era when every additional participant meant another scheduled session, another moderator hour, and another transcript to code by hand. The cost curve was steep, so 5 was pragmatic. That economic constraint is collapsing.\n\nWith **AI-moderated interviews and usability sessions**, the marginal cost of one more participant approaches zero. You can run 30, 40, or 50 sessions in parallel and have every transcript [analyzed thematically](/docs/ai-usability-testing-guide) the moment it finishes — so the trade-off that made 5 attractive (more users = more cost and more manual analysis) largely disappears. You get the qualitative depth of small-N testing *and* enough scale to trust the numbers.\n\n## How Koji changes the math\n\n[Koji](/) is an AI-native customer research platform that runs **AI-moderated voice and text interviews** at scale. Instead of choosing between \"5 users, fast\" and \"40 users, expensive,\" you get both:\n\n- **Depth at scale:** Launch a study and have 30–50 people complete an AI-moderated session in days, each one probed with intelligent follow-ups — no moderator fatigue, no scheduling Tetris.\n- **Structured + open in one study:** Combine all six structured question types — open_ended, scale, single_choice, multiple_choice, ranking, and yes_no — so you capture *quantitative* signals (a 0–10 ease rating, task success) and the *qualitative \"why\"* in the same session.\n- **Automatic thematic analysis:** Every transcript is coded automatically, so a 40-person study takes the same analysis time as a 5-person one — minutes, not weeks.\n- **One-click reports:** Go from raw sessions to a shareable insight report without manually tagging a single transcript.\n\nThe \"5 users\" rule was a smart workaround for an expensive process. In 2026, the smarter move is to size your sample to your actual question — and use [AI-moderated interviews](/docs/ai-moderated-interviews) to make bigger, better-segmented studies as cheap and fast as the old 5-user test.\n\n## Ready to run usability research without the sample-size compromise?\n\nKoji lets you go from question to insight in hours, not weeks — with no research expertise required and no moderator bias. Talk to enough of the *right* users to be confident, and let AI handle the moderation and analysis. [Start with Koji](/) and turn usability testing from a staffing problem into a one-click workflow.\n\n*Related reading: [How Many Customer Interviews Do You Really Need?](/blog/how-many-customer-interviews-do-you-really-need) · [Usability Testing: The Complete Guide for Product Teams (2026)](/blog/usability-testing-guide-2026) · [Best Usability Testing Tools in 2026](/blog/best-usability-testing-tools-2026)*","category":"Research","lastModified":"2026-06-28T03:19:49.09029+00:00","metaTitle":"How Many Users Do You Need for Usability Testing? (2026 Guide)","metaDescription":"Five users find ~85% of usability problems at a 31% discovery rate — but you need 9-18 for subtle issues, 5 per segment for multiple audiences, and 20-40 for quantitative benchmarking. How to size any usability study by goal in 2026.","keywords":["how many users for usability testing","usability testing sample size","5 users rule","nielsen 5 users","number of usability test participants","usability testing how many participants","qualitative vs quantitative usability testing","usability test sample size 2026"],"aiSummary":"How many users do you need for usability testing? For qualitative testing, 5 users per audience segment uncovers ~85% of usability problems (Nielsen & Landauer, based on a 31% per-user problem-discovery rate). The formula N=1-(1-L)^n means lower discovery rates need more users: 9 users at L=20%, 18 at L=10%. Faulkner showed 5-user runs range from 55% to 95% detection. For quantitative metrics (task success, time-on-task, SUS) you need 20-40 participants at ~90% confidence; a task-completion rate above 78% is above average. Test 5 per distinct segment, not 5 total. In 2026, AI-moderated interviews (Koji) remove the per-participant cost, making larger, better-segmented studies as fast and cheap as a 5-user test, with automatic thematic analysis and one-click reports.","aiKeywords":["how many users for usability testing","usability testing sample size","5 user rule","nielsen landauer formula","qualitative usability testing","quantitative usability testing","task completion rate","system usability scale","usability test segments","ai moderated usability testing","data saturation","ai customer research"],"aiContentType":"guide","faqItems":[{"answer":"For qualitative usability testing, 5 users per audience segment uncovers roughly 85% of usability problems, based on Nielsen and Landauer's model with an average 31% per-user problem-discovery rate. Run 2-3 small rounds rather than one large one. For quantitative usability metrics (task-success rate, time-on-task, SUS), you need 20-40 participants.","question":"How many users do you need for usability testing?"},{"answer":"Yes, for qualitative problem discovery in a single homogeneous user group. But it assumes each problem has about a 31% chance of being found per user. For subtle, low-frequency issues you need more (9 users at a 20% discovery rate, 18 at 10%), and for multiple distinct audiences you need 5 per group, not 5 total.","question":"Is the 5-user rule for usability testing still valid in 2026?"},{"answer":"20-40 participants, typically reported at 90% confidence intervals. Quantitative metrics like task-success rate and time-on-task need a larger sample so one or two outliers don't skew the numbers. A task-completion rate above 78% is considered above average.","question":"How many users do you need for quantitative usability testing?"},{"answer":"Because of variability and segmentation. Laura Faulkner's research found that individual runs of 5 users detected anywhere from 55% to 95% of problems depending on who was recruited. And the rule assumes one user group — if your product serves distinct audiences (e.g., admins vs end users), each needs its own 5 participants.","question":"Why do some studies say 5 users isn't enough?"},{"answer":"AI-moderated interviews remove the per-participant cost and manual analysis time that made 5 users attractive. Platforms like Koji let you run 30-50 sessions in parallel with automatic thematic analysis and one-click reports, so you get qualitative depth and quantitative scale without the old trade-off.","question":"How does AI-moderated testing change usability sample size?"}],"relatedTopics":["usability testing sample size","5 user rule","usability test participants","qualitative usability testing","quantitative usability testing","usability metrics","task completion rate","ai moderated usability testing","user research sample size","continuous discovery"]}],"pagination":{"total":1,"returned":1,"offset":0}}