Quantitative User Research: Methods, Examples, and When to Use Them

What is quantitative user research? (TL;DR)

Quantitative user research is the systematic collection of numerical data about user behavior, preferences, or attitudes to identify patterns, measure outcomes, and validate hypotheses at scale. Where qualitative research answers "why?", quantitative research answers "what, how often, and how many?" — and lets you state findings with statistical confidence rather than narrative interpretation.

In 2026, quantitative methods are no longer optional even for design-led teams. According to the User Interviews 2025 State of User Research Report, the median researcher runs 2 mixed-methods, 3 qualitative, and 1 quantitative study every six months — meaning every researcher is expected to be at least quant-literate. And per Lyssna's 2025 Research Synthesis Report, surveys are used by 83% of research teams and remain the second-most common research method after interviews (92%).

This guide covers the nine core quantitative UX research methods, when to use each one, how many participants you need, and how AI-native research is collapsing the wall between qualitative depth and quantitative scale.

Quantitative vs. qualitative research: the one-line distinction

The two are complementary, not competitive. Nielsen Norman Group's landmark "When to Use Which UX Research Methods" framework places virtually every method on a spectrum between qualitative-attitudinal and quantitative-behavioral — and the strongest research programs deliberately use both. For a deeper dive see our qualitative vs. quantitative research guide.

When to use quantitative user research

Quantitative methods earn their cost when you need to:

• Measure the size or impact of a known problem
• Compare two or more design options statistically
• Validate an insight uncovered through qualitative work
• Prioritize which of several issues affects the most users
• Benchmark experience over time or against competitors
• Predict behavior at scale (cohort analysis, propensity)

Do not lead with quantitative research when the problem space is unclear or the hypothesis is unformed — quant is great at testing answers and terrible at generating questions.

The 9 core quantitative user research methods

1. Surveys

What it is. A structured questionnaire delivered to a sample of users to measure attitudes, behaviors, satisfaction, or preferences at scale.

When to use. Whenever you need to estimate the proportion of users who think, feel, or do something. Surveys are the workhorse method — flexible enough to run as in-product intercepts, post-purchase emails, or full panel studies.

Sample size. For an estimate within ±5% margin at 95% confidence on a binary question, you need roughly 385 responses for any population over ~10,000.

Limitations. Surveys measure what users say, not what they do. They are also vulnerable to self-selection bias and respondent fatigue. See our survey fatigue guide for details.

2. A/B testing (split testing)

What it is. A controlled experiment where two or more design variants are randomly assigned to user groups, and the difference in a target metric (conversion, engagement, retention) is measured.

When to use. When you have two or more concrete options and a measurable success metric. Best for tactical optimization (button color, copy, layout) and validation of bigger changes once they're built.

Sample size. Driven by your baseline conversion rate and the minimum effect you want to detect. A 2% absolute lift on a 10% baseline conversion typically needs ~17,000 users per variant to reach statistical significance.

Limitations. Tells you which variant won but rarely why. Pair with qualitative research to interpret unexpected results. See our A/B testing vs. user research comparison for the trade-offs.

3. Product analytics

What it is. Passive measurement of user behavior inside your live product — page views, feature usage, conversion funnels, retention curves, drop-off points.

When to use. Always. Analytics is the foundation of behavioral quant research and the source of most "we noticed X" hypotheses that other methods then explore.

Sample size. Whatever your product produces — usually thousands or millions of events.

Limitations. Tells you what users did but never why. A 30% drop-off on step 3 of onboarding could be confusion, intentional skip, or technical bug — analytics can't distinguish.

4. Card sorting (closed and hybrid)

What it is. Participants group content items into categories and label them. Open card sorts let participants invent labels; closed card sorts use predefined categories. Quantitative analysis measures the percentage of participants who created similar groupings.

When to use. When you're designing or restructuring information architecture (navigation, taxonomy, content hubs).

Sample size. Per Nielsen Norman Group, 30+ participants is the threshold for quantitatively reliable card-sort patterns.

Limitations. Tests organization in isolation, not in the context of real tasks. See our card sorting guide for full methodology.

5. Tree testing

What it is. Participants are given a navigation tree (no visual design) and asked to find specific items. Quantitative outputs include success rate, time to find, directness, and first-click accuracy.

When to use. To validate or compare information architecture before investing in design and build. Often paired with card sorting (card sort to design IA, tree test to validate it).

Sample size. 50+ participants per tree is the standard for quantitative confidence.

Limitations. Tests the IA only — not visual design, copy, or interaction. See our tree testing guide.

6. System Usability Scale (SUS)

What it is. A standardized 10-question Likert questionnaire that produces a single 0–100 usability score. SUS has been used in over 1,300 published studies, making it the most-validated usability metric in the field.

When to use. When you need a comparable, benchmarkable usability score over time, across products, or against industry norms. Industry average SUS is ~68; a score above 80 is considered "above average."

Sample size. 30+ participants for stable scores; smaller samples can produce wide confidence intervals.

Limitations. A single composite score — useful for benchmarking, weak for diagnosing specific issues. See our SUS guide.

7. Single Ease Question (SEQ) and task-level metrics

What it is. A single 7-point post-task rating ("Overall, how difficult or easy was this task?"). Often paired with completion rate, time on task, and error count.

When to use. Inside any moderated or unmoderated usability test where you want comparable difficulty scores across tasks or designs.

Sample size. 15+ for stable averages at the task level.

Limitations. Rates perceived ease, not actual ease — useful but should be triangulated with completion rate. See our SEQ guide.

8. Preference and ranking studies

What it is. Participants choose between two or more design options or rank a list. Modern variants include MaxDiff (forced trade-offs across many items) and conjoint analysis (decomposing preferences across feature combinations).

When to use. Pricing research, feature prioritization, message testing, brand positioning. MaxDiff is dramatically more discriminating than rating scales when you need to rank many items.

Sample size. 100–300 for stable rankings; conjoint typically needs 200+.

Limitations. Stated preference often diverges from revealed preference — what users say they want isn't always what they choose. See our preference testing guide, MaxDiff guide, and conjoint analysis guide.

9. Net Promoter Score (NPS), CSAT, and CES

What it is. Three of the most common standardized customer-experience metrics:

• NPS ("how likely are you to recommend?") — relationship-level loyalty
• CSAT ("how satisfied?") — interaction-level satisfaction
• CES ("how easy was it?") — effort to complete a goal

When to use. As ongoing tracking metrics tied to specific touchpoints. Most useful when paired with an open-ended follow-up that captures the why.

Sample size. Hundreds per cohort for stable scoring; thousands for trend detection.

Limitations. All three are lagging indicators and notoriously sensitive to wording, channel, and timing. See our NPS guide, CSAT guide, and CES guide.

Quantitative research sample-size cheat sheet

Common quantitative research mistakes

• Running quant before qual. You can't measure something if you don't know what to measure. Almost every successful survey is preceded by 6–10 interviews that surface what to ask about.
• Ignoring statistical significance. A 5% lift in conversion across 80 users is noise. Design tests with the sample size your effect size requires.
• Cherry-picking the metric. "Engagement is up 20%!" — but conversion is flat. Pick a primary metric before running the test.
• Treating the score as the insight. SUS=72 isn't actionable. SUS=72, with the lowest sub-scores on questions 4 and 10, is.
• Forgetting the comparison. A standalone CSAT of 4.1/5 means nothing without a baseline, benchmark, or trend line.
• Survey-only research. Surveys answer the questions you thought to ask. Pair with interviews to surface the questions you didn't.

Mixed methods: where quant gets its meaning

The modern best practice is mixed-methods research — pairing quantitative measurement with qualitative depth in the same study. NN/g recommends combining behavioral analytics (what users do) with attitudinal interviews (what users think) to triangulate findings.

Classic mixed-methods workflow:

1. Discover with 5–10 qualitative interviews → identify candidate problems
2. Quantify with a survey or analytics → measure prevalence and impact
3. Validate with A/B test or usability test → confirm intervention works
4. Track with NPS / SUS / CSAT → monitor outcome over time

See our mixed-methods research guide for end-to-end examples.

How AI-native research closes the quant/qual gap

For decades, quantitative and qualitative research lived in separate tools, separate teams, and separate timelines. Quant teams shipped a survey in days; qualitative teams shipped a synthesis in weeks. AI-native research platforms are dissolving that wall.

When an AI moderator can run structured questions and adaptive open-ended probes in the same conversation, every interview becomes a mixed-methods study. Per Lyssna's 2025 research, 54.7% of researchers now use AI-assisted analysis — and the most advanced workflows are already running thousands of moderated conversations per month with full thematic analysis applied automatically.

How Koji unifies quant and qual in one study

Koji is built on the premise that quantitative and qualitative data should come from the same conversation, not separate tools. Here is how that plays out:

• Six structured question types in every study — open_ended, scale, single_choice, multiple_choice, ranking, yes_no. Scale and ranking questions produce quantitative distributions; open_ended questions produce qualitative depth, all in the same interview. See our structured questions guide.
• AI-moderated probing. When a respondent picks "3 out of 5" on a scale, the AI consultant adaptively asks why — turning a single quant data point into a quote-backed insight.
• Real-time aggregation. Distributions, charts, and theme clusters update live as responses come in. You see a histogram and the supporting verbatims side by side.
• Quality scoring. Every interview is rated 1–5 on response depth, so low-effort responses don't pollute your quant or qual analysis.
• Statistical confidence indicators. Reports show sample sizes and surface when a finding is durable vs. early-signal.

While traditional setups require Qualtrics for the survey, Dovetail for the qualitative analysis, and a researcher to sit between them, Koji collapses the entire pipeline into a single AI-native workflow — typically delivering both quantitative and qualitative findings in 24–48 hours.

Quantitative research vs. analytics: what's the difference?

A common confusion: isn't product analytics already quantitative research? Sort of, but with a key distinction:

• Analytics is passive observation of behavior in your live product. It tells you what your existing users did.
• Quantitative research is active measurement — typically a designed study with a hypothesis, a control, and a defined sample. It tells you what all potential users would likely do, or how a change would shift behavior.

Analytics is the cheapest source of quantitative signal you have, and most teams underuse it. But it can't answer questions about non-users, alternative designs, or root cause — that's where the nine methods above earn their keep.

Related Resources

• Qualitative vs. Quantitative Research
• Mixed Methods Research Guide
• Structured Questions Guide
• Survey Design Best Practices
• System Usability Scale Guide
• A/B Testing vs. User Research
• How to Analyze Open-Ended Survey Responses