Interviews, Fake Door, Landing Page or Pre-Sales: Which Validation Test to Run First

Short answer: run the cheapest test that can kill your riskiest assumption. For most new ideas that means problem interviews first (days, almost free), because if the problem isn't real or painful nothing else matters. Once 8 to 10 people in one segment describe the same costly problem in their own words, test demand with a landing page or fake door. Then test money with pre-sales or a letter of intent. Interviews tell you whether the problem exists and how people talk about it. They can't tell you whether people will pay, so never stop at interviews when the risk is willingness to pay.

If you're deciding this week: write down the one assumption that would sink the idea if it were wrong, find it in the table below, and run that test. If you have nobody to talk to yet, start with finding people to interview with no audience.

Why the order matters

Most new products don't fail because they were badly built. In CB Insights' 2021 analysis of 111 startup post-mortems, 35% of founders named "no market need" as a reason they failed, second only to running out of cash at 38%. Founder-reported reasons are imperfect evidence, but the pattern is consistent: teams build before they know the problem is worth solving.

Alberto Savoia, author of The Right It, puts the rule in one line: "Make sure you are building The Right It before you build It right." Each validation test answers a different version of "is this the right it?", and they cost very different amounts of time and money. Running them in the wrong order wastes both: a landing page for a problem nobody has gets you clicks from curious people, not customers.

The four kinds of risk, and the test for each

Every startup idea carries four risks. Name the biggest one first.

Test 1: problem interviews

What it tests: whether the problem is real, frequent and painful enough, and who has it worst.

How to run it: talk to 5 people in each of 2 or 3 candidate segments, then go deeper in the segment with the sharpest pain. Ask only about the past: "Tell me about the last time this happened. What did you do? What did it cost you?" Don't describe your solution. Our guide to how many user interviews you need covers sample sizes, and the Mom Test method covers the questions.

Pass signal: people describe the problem without prompting, have already tried to solve it (a spreadsheet, a hack, a paid tool they dislike), and can name what it costs them.

Why it misleads: people are polite and bad at predicting their own behaviour. Rob Fitzpatrick's warning in The Mom Test is the one to remember: "Compliments are the fool's gold of customer learning: shiny, distracting, and entirely worthless." The research on intentions backs him. In Webb and Sheeran's 2006 meta-analysis of experiments, a medium-to-large change in what people intended to do (d = 0.66) produced only a small-to-medium change in what they actually did (d = 0.36). "I'd definitely use that" is a weak signal about the future, so use interviews for the past and the present only.

Test 2: landing page or fake door

What it tests: whether strangers act on your promise. A landing page measures sign-ups for a product that doesn't exist yet. A fake door puts a button for a non-existent feature inside a real product and counts the clicks.

How to run it: one page, one promise, one call to action, written in the words people used in your interviews. Send traffic from the segment you interviewed, not friends. Our smoke test guide covers set-up and ethics, and the founder playbook test your landing page before you launch shows how to check the page makes sense before you pay for traffic.

Pass signal: set the bar before you start, and set it from a benchmark, not hope. Unbounce's 2024 Conversion Benchmark Report, built on more than 57 million conversions across over 41,000 landing pages, puts the median conversion rate at 6.6% across industries and 3.8% for SaaS. Those are pages for real products, so a test page converting well below them is a warning sign, and one at or above them is worth following up.

Why it misleads: a sign-up costs nothing. Clicks tell you the promise is interesting, not that the product is needed. A page that confuses people also fails for reasons that have nothing to do with demand, which is why a quick first-impression test of the page is worth doing before you buy traffic.

Test 3: pre-sales or a letter of intent

What it tests: whether people will pay, at your price, before the product exists.

How to run it: offer early access at a real price with a clear delivery date and a refund promise. In B2B, ask for a signed letter of intent that names a price and a start date, or a paid pilot. Fitzpatrick calls these commitments: time, reputation or money that a person gives up, as opposed to compliments that cost them nothing.

Pass signal: some people pay, or sign, without a big discount. Even a handful of paying customers from a cold audience is stronger evidence than a hundred interview compliments.

Why it misleads: a deep discount turns a price test into a bargain test. And a "no" can mean the wrong segment rather than the wrong product, which is why you run interviews first. If you're unsure whether to show a price at all, the playbook on contact us or a published price walks through that test.

Test 4: concierge or Wizard of Oz

What it tests: whether you can deliver the value before you build the software. In a concierge MVP you do the work by hand, openly. In a Wizard of Oz test the customer sees a product while a person works behind it.

When to run it: after demand and money look real, when the biggest open risk is whether the outcome can be delivered at all, or what the product actually needs to do.

Which test to run first

Use the questions in order:

1. Have 8 to 10 people in one segment described the same costly problem without prompting? If not, run problem interviews.
2. Have strangers from that segment signed up from a page that states the promise? If not, run a landing page or fake door test.
3. Has anyone paid, put down a deposit or signed a letter of intent? If not, run a pre-sale.
4. Can you deliver the outcome by hand for a few customers? If you're not sure, run a concierge test before you build.

If you already have strong evidence for one step, skip it. A B2B founder with ten buyers who already pay for a bad workaround can go straight to a letter of intent. The point is to spend the least to learn the most, not to tick every box.

Common mistakes

• Starting with the landing page. Without interviews you don't know whose problem it is or what words they use, so the page tests your copy, not the idea.
• Stopping at interviews. Interviews tell you the problem is real. They don't tell you people will pay.
• Counting compliments. "Love it" and "I'd use that" are not evidence. Count past behaviour and commitments.
• Moving the bar after the test. Write the pass signal down before you start.
• Testing everyone. One segment per test, or a weak result in one hides a strong result in another.

Running the interviews with real people

Interviews are the first test and the one most founders put off, because finding people and lining up calls takes time. The method is simple; the logistics are what slow it down.

Photo by JESHOOTS.COM on Unsplash

How Koji helps

Koji runs the first test, problem interviews, without the scheduling, and feeds the later ones:

• Write the interview from one sentence. Describe what you want to learn ("whether freelance designers struggle to chase late invoices") and the research assistant drafts a brief with questions about past behaviour, which you edit.
• AI-moderated interviews by voice or text. Share one link, and each person answers when it suits them while Koji asks the follow-up questions (none, or 1 to 3 per question, with your guidance). Nobody has to be on a call at the same time, so interviews run in parallel.
• Find people when you have no audience. Panel recruitment lets you describe who you need and see the cost before you launch; launching a recruitment needs a paid plan. Screening questions keep out people who don't have the problem.
• Test the page before you buy traffic. Attach an image of your landing page to a question, show it for 5 seconds, then ask what people remember and who they think it's for.
• Read the pattern, with the quotes. The report groups the problems people describe into themes, with the share of participants behind each and the quotes underneath, so you can see whether 8 to 10 people really did describe the same costly problem.

Koji is the right tool for the interview step and for checking a page. It isn't a substitute for the later tests: a landing page with real traffic and a real pre-sale still give you evidence an interview can't.

Related Resources

• Structured Questions Guide
• Problem Validation Guide
• Riskiest Assumption Test
• Startup Idea Validation
• Smoke Tests and Fake Door Tests
• Waitlist Validation

Sources

• CB Insights, "The Top 12 Reasons Startups Fail", 2021 (analysis of 111 startup post-mortems).
• Unbounce, Conversion Benchmark Report, 2024 (more than 57 million conversions across over 41,000 landing pages).
• Thomas L. Webb and Paschal Sheeran, "Does Changing Behavioral Intentions Engender Behavior Change? A Meta-Analysis of the Experimental Evidence", Psychological Bulletin 132(2), 2006.
• Rob Fitzpatrick, The Mom Test, 2013.
• Alberto Savoia, The Right It: Why So Many Ideas Fail and How to Make Sure Yours Succeed, 2019.