{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-07-12T12:39:40.204Z"},"content":[{"type":"documentation","id":"de0ea692-278e-454a-b340-69227dd446c2","slug":"solution-validation-guide","title":"Solution Validation: How to Test a Solution Before You Build It (2026)","url":"https://www.koji.so/docs/solution-validation-guide","summary":"Solution validation proves with real customer evidence that a specific proposed solution solves a validated problem, is usable, and creates enough value that people will adopt or pay for it. It comes after problem validation and before full engineering. Because roughly half of product ideas fail and 64-80% of shipped features are rarely or never used, validating the solution cheaply — via solution interviews, prototype and usability tests, fake-door demand tests, and concierge or Wizard-of-Oz MVPs — prevents building the wrong thing. Weight actions over compliments, quantify reactions with structured questions, and use AI-moderated platforms like Koji to run deep validation at survey scale in days instead of weeks.","content":"## What Is Solution Validation? (Short Answer)\n\nSolution validation is the process of proving — with evidence from real customers — that a specific proposed solution actually solves a validated problem, is usable, and creates enough value that people will adopt or pay for it. It happens **after** you have confirmed the problem is real (problem validation) and **before** you invest full engineering effort in building the finished product.\n\nPut simply: problem validation asks *\"is this a real, painful problem?\"* Solution validation asks *\"is this particular solution the right way to solve it?\"* Skipping the second question is one of the most expensive mistakes in product development.\n\nThe stakes are well documented. In *Inspired*, Silicon Valley Product Group founder Marty Cagan states the first inconvenient truth of product work bluntly:\n\n> \"At least half of our ideas are just not going to work.\"\n> — Marty Cagan, *Inspired*\n\nIf half of your solution ideas will fail, the only question is whether you discover that in a few days of validation or after a few months of engineering. Solution validation is the discipline that moves the discovery to the cheap end of that spectrum.\n\n---\n\n## Solution Validation vs. Problem Validation vs. Idea Validation\n\nThese three terms are used interchangeably, and that confusion causes teams to skip steps. Here is the correct sequence:\n\n1. **Idea / problem validation** — Is this a real, frequent, painful problem for a specific group of people? (No solution proposed yet.)\n2. **Solution validation** — Does *this specific solution* solve that problem in a way customers find valuable and usable, and will they switch to it?\n3. **Feature validation** — Once the solution is live, which specific features and refinements move the needle?\n\nThe mistake most teams make is jumping from \"the problem is real\" straight to \"so let us build our idea.\" But a validated problem does not mean your solution is correct. Customers can have a genuine, painful problem *and* reject your particular answer to it. Solution validation is the bridge that catches that failure before it becomes sunk cost.\n\n---\n\n## Why Solution Validation Matters: The Cost of Building the Wrong Thing\n\nThe classic Standish Group research on software feature usage found that in a typical product, **45% of features are never used and another 19% are rarely used** — meaning roughly **64% of what teams build delivers little or no value**. Pendo's later product-benchmark analysis reached a strikingly similar conclusion, reporting that **around 80% of features in the average software product are rarely or never used**.\n\nRead those numbers again. The majority of engineering effort in the industry goes toward solutions that never earn their keep. Most of that waste is not a coding problem — the features worked. It is a *validation* problem: teams built solutions no one actually wanted, or that solved the problem worse than the customer's existing workaround.\n\nSolution validation attacks four distinct risks that Marty Cagan defines for any product idea:\n\n- **Value risk** — Will customers choose to use or buy it? (The most common killer.)\n- **Usability risk** — Can users figure out how to use it?\n- **Feasibility risk** — Can we actually build it with the time, skills, and technology we have?\n- **Viability risk** — Does it work for our business (legal, sales, finance, brand)?\n\nSolution validation research primarily de-risks **value** and **usability** — the two risks only your customers can answer.\n\n---\n\n## Methods for Validating a Solution (Cheapest to Richest)\n\nYou do not need to build the real thing to validate it. Match the method to the risk and the fidelity you can afford:\n\n### 1. Solution Interviews\nShow the customer your proposed approach — described, sketched, or storyboarded — and probe whether it fits how they actually work. The Mom Test rule still applies: anchor every reaction in the customer's real past behavior, not hypothetical praise. Ask *\"walk me through the last time you dealt with this\"* before you ever show your concept.\n\n### 2. Prototype & Usability Testing\nPut a clickable prototype in front of users and give them a realistic task. Watch where they hesitate, misclick, or give up. This is the fastest way to expose usability risk before a single production line is written.\n\n### 3. Fake-Door / Painted-Door Tests\nAdvertise the solution (a button, a landing page, a pricing tier) as if it exists and measure how many people try to engage. Genuine click-through and sign-up intent is far stronger evidence than a survey response.\n\n### 4. Concierge & Wizard-of-Oz MVPs\nDeliver the solution manually behind the scenes (concierge) or simulate the automated experience with humans pulling the levers (Wizard of Oz). Customers get the real outcome; you validate demand and value without building the engine.\n\n### 5. Demand & Preference Tests\nA/B tests, landing-page conversion tests, and structured preference questions (\"which of these two approaches would you reach for first, and why?\") quantify relative appeal across a larger sample.\n\nThe guiding principle: **spend the least you can to learn the most you must.** Reserve full engineering for solutions that survive cheaper tests.\n\n---\n\n## How to Run a Solution Validation Study\n\nA rigorous solution validation study combines depth (why) and breadth (how much). A repeatable structure:\n\n1. **State the riskiest assumption.** What must be true for this solution to succeed? Usually it is \"customers will switch from their current workaround to this.\"\n2. **Recruit the right people.** Talk only to people who genuinely have the validated problem — never a convenience sample of colleagues.\n3. **Anchor in current behavior.** Before showing your solution, capture how they solve the problem today, how long it takes, and what it costs them. This is your baseline.\n4. **Expose the solution and observe.** Present the prototype or concept and give a real task or scenario. Capture reactions, confusion, and — critically — evidence of intent to act.\n5. **Quantify with structured questions.** Layer in scale, ranking, and single-choice questions to turn qualitative reactions into comparable numbers.\n6. **Reach saturation.** Keep interviewing until new sessions stop revealing new objections or usability failures.\n\n### Signals of Genuine Validation vs. False Positives\n\n| Real validation | False positive |\n|---|---|\n| \"Can I start using this today?\" | \"That is a cool idea.\" |\n| Asks about price, timeline, migration | Polite, non-committal enthusiasm |\n| Tries to complete the task unaided | Needs you to explain every step |\n| Describes abandoning their current workaround | \"I could see someone using this\" |\n| Offers to pay, pre-order, or refer | \"Sure, I would probably try it\" |\n\nThe single most important habit: **discount compliments, weight actions.** A signed-up email, a completed task, or an offer to pay outweighs a hundred \"I love it\"s.\n\n---\n\n## The Modern Approach: Validating Solutions at Scale with AI\n\nTraditional solution validation forces a painful trade-off. Deep, moderated interviews give you rich \"why\" but take weeks to schedule, run, and synthesize — so most teams talk to five people and hope. Surveys scale but strip out the nuance that reveals *why* a solution lands or fails.\n\nAI-native platforms like **Koji** collapse that trade-off. Instead of manually scheduling and moderating each session, you deploy an **AI-moderated interview** that talks to dozens or hundreds of qualified customers in parallel — over voice or text — and adapts its follow-up questions in real time, probing exactly like a skilled researcher would when someone says \"I am not sure I would switch.\"\n\nThree capabilities make Koji especially suited to solution validation:\n\n- **AI-moderated depth at survey scale.** Every participant gets a genuine, adaptive conversation, so you capture the *why* behind a reaction without spending your week in back-to-back calls. Teams using AI-assisted research routinely compress time-to-insight from weeks to days.\n- **Six structured question types for quantifiable evidence.** Blend open-ended probing with `scale`, `single_choice`, `multiple_choice`, `ranking`, and `yes_no` questions in the same session — so you can rank two solution concepts, measure purchase intent on a scale, and still hear the reasoning behind the numbers. See the [structured questions guide](/docs/structured-questions-guide) for how to combine them.\n- **Automatic thematic analysis and real-time reporting.** Koji clusters objections, usability failures, and demand signals across every conversation automatically, so the pattern — \"eight of twenty testers stalled at the same step\" — surfaces in minutes instead of a weekend of tagging transcripts.\n\nWhile traditional tools like SurveyMonkey can tell you *what* percentage preferred concept A, an AI-native platform like Koji tells you *why* — and does it fast enough to run before your next sprint planning. You no longer need a research team or a PhD in methodology to validate a solution properly; you need a clearly framed assumption and an afternoon.\n\n---\n\n## Common Solution Validation Mistakes to Avoid\n\n- **Validating the solution before the problem.** If the problem is not real, a \"validated\" solution is a well-tested answer to a question no one asked.\n- **Pitching instead of probing.** The moment you start selling, participants start being polite. Your job is to learn, not to convince.\n- **Leading questions.** \"Would you love a faster way to do this?\" invites a yes from everyone. Ask about past behavior instead.\n- **Confusing interest with commitment.** Enthusiasm is free; a pre-order, a completed task, or a real sign-up is evidence.\n- **Testing with the wrong people.** Friends, colleagues, and anyone without the actual problem will give you flattering, useless data.\n- **Stopping at one method.** Triangulate — pair a prototype test with a fake-door demand test to confirm both usability *and* value.\n\n---\n\n## Related Resources\n\n- [Structured Questions Guide](/docs/structured-questions-guide) — the six question types that turn solution reactions into quantifiable evidence\n- [Problem Validation: How to Prove a Problem Is Worth Solving](/docs/problem-validation-guide) — the step that must come first\n- [Feature Validation Guide](/docs/feature-validation-guide) — validating specifics once your solution is live\n- [Solution Interviews Guide](/docs/solution-interviews-guide) — the conversation technique for testing solutions\n- [Fake-Door Testing Guide](/docs/fake-door-testing-guide) — measuring demand before you build\n- [Riskiest Assumption Test](/docs/riskiest-assumption-test-guide) — how to isolate what to validate first\n- [The Mom Test Methodology](/docs/mom-test-methodology) — keeping validation conversations honest\n- [Customer Validation Guide](/docs/customer-validation-guide) — the full validation lifecycle\n\nSolution validation is where good ideas prove they deserve engineering time — and where bad ones die cheaply. Run it before you build, weight actions over compliments, and use AI-moderated research to reach enough of the right people to trust the answer.","category":"Research Methods","lastModified":"2026-07-11T03:17:30.446106+00:00","metaTitle":"Solution Validation: How to Test a Solution Before You Build It (2026) | Koji","metaDescription":"Solution validation proves a proposed solution actually solves the problem and delivers value before you build. Learn solution interviews, prototype and fake-door testing, the signals that matter, and how to validate at scale with AI.","keywords":["solution validation","how to validate a solution","solution validation vs problem validation","validate solution before building","solution validation methods","concept validation","solution interviews","fake door test","prototype testing","product validation"],"aiSummary":"Solution validation proves with real customer evidence that a specific proposed solution solves a validated problem, is usable, and creates enough value that people will adopt or pay for it. It comes after problem validation and before full engineering. Because roughly half of product ideas fail and 64-80% of shipped features are rarely or never used, validating the solution cheaply — via solution interviews, prototype and usability tests, fake-door demand tests, and concierge or Wizard-of-Oz MVPs — prevents building the wrong thing. Weight actions over compliments, quantify reactions with structured questions, and use AI-moderated platforms like Koji to run deep validation at survey scale in days instead of weeks.","aiPrerequisites":["A validated problem (see problem validation)","Basic familiarity with product discovery"],"aiLearningOutcomes":["Distinguish solution validation from problem and feature validation","Choose the right validation method for each product risk","Run a solution validation study that combines depth and quantifiable evidence","Tell genuine validation signals from polite false positives","Validate solutions at scale with AI-moderated research and structured questions"],"aiDifficulty":"intermediate","aiEstimatedTime":"14 minutes"}],"pagination":{"total":1,"returned":1,"offset":0}}