You can use AI to validate a startup idea, but only for the parts of the job that don't need a customer. A chatbot can sharpen your assumptions, draft your questions and summarise what people tell you. It cannot tell you whether anyone will pay, because it has never met your customers. By the end of this guide you'll know where AI helps, where it fools you, and how to get real evidence from about 10 to 15 people in a week.
When you need this
You have an idea, maybe a Notion page of it, and you've done the thing everyone does now: pasted it into a chatbot and asked "is this a good idea?" It said yes, with a SWOT analysis. You feel better, and you've learned nothing you can bet on.
The sign you need this guide: you can't name the last three people who described this problem to you in their own words. If you can't, you have an opinion, not evidence.
The cost of guessing is large and boring. You build for months, then find that the people you imagined were never looking for a fix. Most expensive mistakes are about demand, and demand lives with people, not models.
What the research says about AI and real customers
Two things are true at once, and you should hold both.
First, the newest results on AI stand-ins for customers are better than they were. A 2025 preprint from Maier and colleagues asked language models for purchase intent on product concepts. When the models were asked for a 1 to 5 score directly, the match with real survey distributions was poor. When the models first wrote a free-text opinion and that text was converted to a score, the match jumped [2]. The chart below shows the gap on a 0 to 1 similarity scale.
Second, the same year, Verasight compared AI-generated survey answers with a nationally representative survey of more than 1,500 US adults. On political questions the average error was 3.0 percentage points. On coffee questions it was 19.8. On daily coffee drinking, the real figure was 56% and the AI-generated answers said 91% [1]. The model did well where the internet has talked about the topic for years, and badly where it hadn't.
What that means for you: a new idea in a niche market is exactly the place where the model has the least to go on. The Maier study tested packaged consumer goods against real panel data, which is a different world from "scheduling software for dental clinics". Both studies are preprint or industry-report evidence rather than peer-reviewed trials, so read them as signals, not proof.
There is a second, older reason to want real people: stated intent runs high. A meta-analysis of 77 studies on willingness to pay found people overstate what they'd pay by about 21% on average, and indirect methods like conjoint overstated more than direct questions [3]. If real people overstate, a model trained on what people say overstates too.
And there is evidence that testing beats thinking. In a randomised trial of Italian startups, founders trained to form hypotheses and test them abandoned more weak ideas, pivoted in a more focused way and earned more on average than those who relied on intuition [4]. The point is the testing, not the training course.
The method in brief
Use AI as an assistant for your thinking and real conversations as your evidence. In practice that is three layers.
- AI for the desk work. List your assumptions, find the riskiest one, draft questions, spot leading wording, tidy transcripts and pull themes. Check any market-size number it gives you by hand.
- Real people for the evidence. 10 to 15 interviews with people who match your target, asked about their past behaviour. See the Mom Test guide and customer discovery interviews for the method.
- A commitment test for the truth. Something that costs the person something: a pilot date, a deposit, an introduction, a pre-order. Smoke tests and the guide to which validation test to run first cover the options.
If you want the longer argument on AI personas, see synthetic users in research. Here is the short version: use them to rehearse and brainstorm, never to decide.
Step by step: five days
Day 1. Write the idea as a bet. One sentence: "[Type of person] has [problem] often enough to pay about [amount] to fix it." Ask a chatbot to list the five assumptions that must be true, then pick the one that would hurt most if wrong. Don't ask it for a verdict.
Day 1. Pick one narrow group. Not "small businesses", but "owners of two to five dentist clinics". If you're unsure who, start with finding your first customer segment.
Day 2. Find 15 people. Your network, communities where the group already posts, LinkedIn messages that ask for 15 minutes and say what you want to learn, not what you want to sell. How to find 20 people to interview with no audience has the scripts.
Days 3 and 4. Interview. Aim for 10 to 15 conversations. Ask about the last time the problem happened, what they did, what it cost and what they already pay for. How many user interviews explains why you'll start hearing repeats within a dozen or so in a narrow group, and why you should stop earlier if the first five say nothing.
Day 5. Decide. Sort what you heard into the three signals below, then run one commitment test.
What to ask
These questions stay on what happened, not on what people would do. Take them as a starting bank and keep the wording plain.
Do not describe your idea until the last ten minutes. If you do, people try to be kind. How to avoid leading questions shows what to cut.
A worked example
Say you're a two-person team with an idea for scheduling software that cuts no-shows at dental clinics. A chatbot says the market is large, no-shows cost clinics a lot, and your idea is promising. None of that is evidence.
You write the bet: "Practice managers at clinics with two to five dentists lose enough to no-shows to pay for a fix." The riskiest assumption turns out to be that practice managers, not owners, feel the pain. You find 14 practice managers through two dental forums and a friend's clinic, and interview them.
What you hear: 9 of 14 describe a specific no-show last month and what it cost them. Of those 9, 6 already tried text reminders, and 4 pay for software that includes them. Only 2 say they'd look at another tool. Three offer an intro to the owner, which is a stronger signal than any compliment.
What you decide: the pain is real, but most have a partial fix. Your idea needs to beat text reminders, so the next test is narrower: the 3 clinics without any reminder tool. You ask them for a 30-day pilot. This is made-up data to show the shape of the decision, not a result from a real study.
Reading the results and deciding
Look for three signals in your notes:
- Behaviour: people describe a specific recent episode, not "it would be nice".
- Pain with cost: they can say what it cost in hours or money.
- Commitment: they give something: time, an intro, data, a deposit, a pilot date.
If most of your 10 to 15 have the first two and a few give the third, narrow and test further. If you only hear compliments, you're in polite-answer territory: stop and change the segment or the problem. If the same people keep saying "we've solved it with a spreadsheet", ask what the spreadsheet fails at before you write any code.
One rule of thumb that I use, not a published threshold: treat any AI summary as a draft, and don't write down a finding unless you can point to two or more transcripts that say it.
Run it this week with Koji
You can do all of this by hand with a calendar, a call recorder and a spreadsheet, and many founders do. Koji is one way to get the interviews done without scheduling every call. It runs interviews with real people by text or voice, so you get 10 to 15 transcripts rather than synthetic answers. Here is the setup.
- Start from the idea. On the home screen, type your riskiest assumption as the research question. The research assistant drafts a brief with a goal and open questions. Edit it with the question bank above, and ask the assistant to remove any leading question.
- Screen for your segment. Add a screening question so only the right role gets through, for example practice managers.
- Add a concept or a price. Attach an image of your landing page or mockup to any question. Show it once for 3, 5, 10 or 15 seconds as a first-impression test, then let Koji ask what people remember. For pricing, the price test question takes a ladder of 3 to 10 prices and reads best from about 30 interviews. It's directional, so confirm with real payment.
- Send it to real people. Share the interview link with your list or community, or recruit from Koji's panel (launching a panel recruitment needs a paid plan).
- Read the report with the evidence. Each finding links to the quotes behind it. Read three full transcripts yourself before you trust the summary.
Check the interview before it goes out
Take your own interview first, by text and by voice. A preview is free, doesn't count as a response and never shows up in the report, so you can fix the wording first.
If you want to try it, start with one study on the free account: five interviews, one segment, the questions above.
Common mistakes
- Asking the chatbot to judge the idea. It will find reasons it could work, because it is built to be helpful.
- Using AI answers as customers. Fine for rehearsal. Not for a decision.
- Taking a compliment as demand. "Cool idea" costs nothing to say.
- Interviewing friends and fellow founders. They are kind, and they aren't buyers.
- Skipping the commitment test. Interviews tell you about pain, not about paying.
When research will mislead you
Interviews tell you what people say and remember, which is not the same as what they will do. The 21% average overstatement in stated willingness to pay is the reason to pair what you hear with a deposit, a pilot or a pre-order [3]. Small samples can hide a segment you didn't talk to. And an AI interviewer or summariser can miss tone, so read the transcripts. Can you trust AI interviewers? covers how Koji limits that.
Where this fits
Before this, decide which problem to solve first and find a first segment. After it, price before you have customers and test your landing page. The whole path is in the founder's research playbook, and Koji for founders shows the setup.
FAQ
Can ChatGPT validate my startup idea? It can help you list assumptions, draft interview questions and summarise notes. It can't validate demand, because it has no access to what your customers do or pay for. Treat its verdict as a prompt for what to test.
Are synthetic users good enough to validate an idea? For rehearsal, sometimes. For a decision, not yet. Error was 19.8 points on a topic with thin coverage in one 2025 comparison [1], and results depend on the method [2]. Check any synthetic result against real people.
How many real people do I need? Start with 10 to 15 in one narrow segment. If the first five say nothing specific, change the segment before you do more.
Can an AI interview my customers? Yes, and the people answering are real. The interviewer is the AI; the evidence is human. Take a preview yourself first.
How long does it take? About a week: a day to write the bet, a day or two to find people, two days of interviews and a day to decide.
What if everyone is polite? Ask about the last time they hit the problem, and ask for a small commitment. Politeness without commitment is a no.
Next step
Write your idea as a one-sentence bet, pick the assumption that would hurt most, and book five conversations this week.
References
- Verasight, Synthetic Sampling Report for Market Research (coffee), 2025. verasight.io/reports/coffee-llm
- Maier, B. F. et al., LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, arXiv:2510.08338, 2025. arxiv.org/abs/2510.08338
- Schmidt, J. and Bijmolt, T., Accurately measuring willingness to pay for consumer goods: a meta-analysis of the hypothetical bias, Journal of the Academy of Marketing Science, 2020. marketingcenter.de/en/publication/151167
- Camuffo, A., Cordova, A., Gambardella, A. and Spina, C., A Scientific Approach to Entrepreneurial Decision Making: Evidence from a Randomized Control Trial, Management Science, 2020. doi.org/10.1287/mnsc.2018.3249
Further reading
- The Mom Test method: the questions that keep people from being polite.
- Synthetic users in research: where AI personas help and where they mislead.
- Which validation test to run first: interviews, fake door, landing page or pre-sales.
