{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-28T01:38:52.863Z"},"content":[{"type":"documentation","id":"b7687d2d-c175-4dce-98bb-5da33828c17a","slug":"user-interviews-vs-usability-testing","title":"User Interviews vs Usability Testing: When to Use Each (and How They Work Together)","url":"https://www.koji.so/docs/user-interviews-vs-usability-testing","summary":"User interviews are generative research - they explore what to build by uncovering needs, motivations, and problems, usually with no product present. Usability testing is evaluative research - it watches users attempt tasks on a prototype or product to find friction. Use interviews early (discovery) and usability testing after a design exists; the strongest teams loop between them. About 5 users find 85% of usability problems (Nielsen and Landauer), while interviews continue until themes repeat. Koji runs both generative interviews and evaluative task-based sessions on one AI-moderated platform with six structured question types and automatic synthesis.","content":"**Short answer (BLUF):** Use **user interviews** to learn *what to build* and **usability testing** to learn *whether what you built works*. Interviews are **generative** research — they explore needs, motivations, and problems before a design exists. Usability testing is **evaluative** research — it watches people attempt tasks on a prototype or product to find where the design breaks ([NN/g; Dovetail](https://dovetail.com/blog/generative-evaluative-research/)). They answer different questions, run at different moments, and the strongest teams use both — often in the same week. Here is how to tell them apart and when to reach for each.\n\n## The core difference: generative vs. evaluative\n\nThe cleanest way to separate the two is the research stage they belong to:\n\n- **User interviews are generative (discovery).** You sit with a person and ask about their world: what they are trying to accomplish, what frustrates them, how they solve the problem today. There is usually no product in front of them. The goal is to *generate* direction — problems worth solving, unmet needs, the language users actually use. This is the home of [customer discovery](/docs/customer-discovery-interviews) and [jobs-to-be-done](/docs/jobs-to-be-done-interviews).\n- **Usability testing is evaluative.** You give a person a specific *task* on a real interface (\"sign up and invite a teammate\") and watch where they hesitate, misclick, or give up. The goal is to *evaluate* a design and find friction. This is [usability testing](/docs/usability-testing-guide), often using the [think-aloud protocol](/docs/think-aloud-protocol).\n\nAs the Nielsen Norman Group frames it, generative methods like interviews shape *what* you build, while evaluative methods like usability testing measure *how well* the thing you built actually works ([NN/g](https://www.nngroup.com/articles/which-ux-research-methods/)).\n\n## Side-by-side comparison\n\n| | User interviews | Usability testing |\n|---|---|---|\n| Research type | Generative / discovery | Evaluative |\n| Question answered | \"What should we build, and why?\" | \"Does this design work?\" |\n| When in the process | Before design; ongoing discovery | After a prototype or product exists |\n| What you show the user | Nothing — you ask about their life | A prototype, mockup, or live product |\n| Structure | Open conversation, semi-structured | Task-based scenarios |\n| Primary output | Needs, pain points, opportunities, JTBD | Friction points, task success, severity |\n| Typical sample | 5–15+ for themes | ~5 users finds ~85% of issues |\n| Risk if skipped | You build the wrong thing | You ship the right thing, broken |\n\nA key planning number sits in that table: for usability testing, **5 users uncover roughly 85% of an interface's usability problems** ([Nielsen & Landauer, NN/g](https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/)), which is why evaluative rounds stay small and frequent. As Jakob Nielsen put it:\n\n> \"The best results come from testing no more than 5 users and running as many small tests as you can afford.\" — Jakob Nielsen, [Nielsen Norman Group](https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/)\n\nInterviews scale differently: because you are looking for the range of needs and motivations rather than counting task failures, you typically keep going until themes repeat — often [more than 5](/docs/how-many-interviews-enough).\n\n## When to use which\n\n**Reach for user interviews when:**\n\n- You are exploring a new problem space or audience.\n- You do not yet know what to build, or whether the problem is real.\n- You need the *why* behind a metric (\"why is activation dropping?\").\n- You are validating a problem before committing design resources. (See [problem vs. solution interviews](/docs/problem-interviews-vs-solution-interviews).)\n\n**Reach for usability testing when:**\n\n- You have a prototype, mockup, or live flow.\n- You want to find where users get stuck completing a real task.\n- You are choosing between two designs or iterating toward a fix.\n- You need to validate that a redesign actually improved task success.\n\n**The trap to avoid:** running a usability test when you actually have a discovery question. If you show users a polished design and they \"like it,\" you have learned almost nothing about whether you are solving a real problem. Conversely, asking people in an interview whether they *would* use a feature is notoriously unreliable — watch them attempt the task instead. Match the method to the question.\n\n## How they work together\n\nThe two are a loop, not a choice:\n\n1. **Interview** to find a problem worth solving.\n2. **Design** a solution.\n3. **Usability test** the design to find friction.\n4. **Interview again** when the test surfaces a deeper \"why\" you did not expect.\n\nThe classic example: a usability test shows users abandoning checkout at the shipping step. The test tells you *where*. A quick follow-up interview tells you *why* — they expected free shipping. You need both lenses.\n\n## Run both with Koji\n\nMost teams use one tool for interviews and another for usability testing, then stitch the insights together by hand. [Koji](/docs/ai-moderated-interviews) runs **both** generative interviews and evaluative, task-based sessions on one AI-moderated platform:\n\n- **One AI moderator, two jobs.** Run an open-ended discovery interview to explore a problem, or attach a prototype link and have the AI run a task-based [usability session](/docs/usability-testing-guide) with real-time, [non-leading](/docs/avoiding-leading-questions) probing. Both happen by [voice or text](/docs/voice-vs-text-interviews), asynchronously, at the participant's convenience.\n- **Six structured question types for either mode.** Combine `open_ended`, `scale`, `single_choice`, `multiple_choice`, `ranking`, and `yes_no` — a generative \"walk me through how you do this today\" sits beside an evaluative SEQ `scale` and a `yes_no` task-success check. See the [structured questions guide](/docs/structured-questions-guide). Each aggregates into the right chart automatically.\n- **Dozens in parallel, synthesized automatically.** Whether discovery or usability, themes, quotes, and quantified answers compile into a live report as sessions finish — no re-watching recordings. ([Turning interviews into insights](/docs/turning-interviews-into-insights).)\n- **A quality gate on every session.** Each interview is scored 1–5, so low-effort responses are flagged rather than diluting your findings.\n\nWhile a traditional setup forces you to choose a generative tool *or* an evaluative one — and a researcher to moderate each session live — an AI-native platform like Koji lets you move between *what to build* and *whether it works* without switching tools or waiting on calendars. And you do not need a research background to run either.\n\n## Frequently asked questions\n\n**Is a user interview the same as a usability test?** No. A user interview is generative — it explores needs and problems, usually with no product present. A usability test is evaluative — it watches a user attempt tasks on a specific design to find friction.\n\n**Which should I do first?** Almost always interviews. Discovery interviews tell you *what* to build; usability testing then checks whether your design of that thing works. Testing a great design for the wrong problem is wasted effort.\n\n## A worked example\n\nSay activation is dropping in a B2B SaaS onboarding flow. Here is how the two methods divide the work:\n\n1. **Usability test (evaluative):** You give five new users the task \"set up your workspace and invite a teammate.\" Four of five stall at the integration step. The test tells you *where* the flow breaks — precisely and quickly.\n2. **User interview (generative):** You then ask those users — and a few who churned — about their first week. You learn that the integration is not just confusing; many do not understand *why* they would connect it at all. The interview tells you *why*, and reframes the problem from \"fix the integration UI\" to \"explain the value of integrating in the first place.\"\n\nNeither method alone gets you there. The usability test without the interview leads you to polish a step users do not understand the purpose of. The interview without the test leaves you guessing where the friction actually lives.\n\n## A simple cadence for both\n\nYou do not have to choose one method per quarter. A sustainable rhythm looks like:\n\n- **Continuous discovery interviews** — a steady trickle (even 2–3 a week) so you always have a live read on customer needs and language. See [continuous discovery](/docs/continuous-discovery-user-research).\n- **Usability testing every design iteration** — small 5-user rounds whenever a flow changes, not just before launch.\n- **Targeted interviews after each test** — whenever a usability result surprises you, follow up with a short interview to capture the *why*.\n\nThe teams that ship the best products are not the ones that pick the \"right\" method once. They are the ones that loop between *what to build* and *whether it works* fast enough that the two never drift apart.\n\n## Mistakes teams make choosing between them\n\n- **Using a usability test to validate an idea.** Showing users a polished design and asking if they like it is not validation — it is confirmation bias. Validate the *problem* with interviews first.\n- **Trusting interview claims about future behavior.** \"Would you use this?\" is notoriously unreliable. If behavior is the question, watch a task instead of asking a hypothetical.\n- **Treating \"5 users\" as a universal rule.** Five is the sweet spot for evaluative usability rounds. Generative interview sample sizes depend on how many segments and how much variation you are studying — keep going until themes repeat.\n\n## Related resources\n\n- [User Interview Guide](/docs/user-interview-guide)\n- [Usability Testing: The Complete Guide](/docs/usability-testing-guide)\n- [Generative vs. Evaluative Research](/docs/generative-vs-evaluative-research)\n- [Structured Questions Guide](/docs/structured-questions-guide)\n- [How Many Interviews Is Enough?](/docs/how-many-interviews-enough)\n- [AI-Moderated Interviews](/docs/ai-moderated-interviews)","category":"Research Methods","lastModified":"2026-06-27T03:23:56.343253+00:00","metaTitle":"User Interviews vs Usability Testing: When to Use Each (2026)","metaDescription":"User interviews are generative (what to build); usability testing is evaluative (does it work). A side-by-side comparison, when to use each, and how to run both on one AI platform.","keywords":["user interviews vs usability testing","usability testing vs user interviews","generative vs evaluative research","user research methods comparison","when to use usability testing","interviews or usability testing"],"aiSummary":"User interviews are generative research - they explore what to build by uncovering needs, motivations, and problems, usually with no product present. Usability testing is evaluative research - it watches users attempt tasks on a prototype or product to find friction. Use interviews early (discovery) and usability testing after a design exists; the strongest teams loop between them. About 5 users find 85% of usability problems (Nielsen and Landauer), while interviews continue until themes repeat. Koji runs both generative interviews and evaluative task-based sessions on one AI-moderated platform with six structured question types and automatic synthesis.","aiPrerequisites":["Basic familiarity with the product development process"],"aiLearningOutcomes":["Tell the difference between generative interviews and evaluative usability testing","Choose the right method for a given research question","Combine both methods in a single discovery-to-validation loop"],"aiDifficulty":"beginner","aiEstimatedTime":"12 minutes"}],"pagination":{"total":1,"returned":1,"offset":0}}