Koji vs Synthetic Users: Real Customer Voices vs AI Personas (2026)

Synthetic Users is one of the most-discussed names in AI research right now. The pitch is irresistible: skip recruitment, skip scheduling, skip the messy work of getting real humans on a call. Generate AI personas, "interview" them, get a report in minutes.

It is fast. It is cheap. And for some narrow use cases — early hypothesis generation, desk research, identifying what to investigate next — it can save real time.

But it is also being marketed as a replacement for talking to actual customers, and that is where the conversation gets serious. The Nielsen Norman Group, ACM Interactions, and a growing body of academic literature have all raised the same fundamental concern: synthetic users cannot be validated, because they have no connection to observed reality. Insight built on synthetic data is an assumption built on top of other assumptions, all the way down.

Koji takes the opposite approach. The AI is the interviewer, but the participants are real customers — your customers, recruited through your own channels or Koji's panel, talking about their actual experience. Same speed advantages, none of the validity concerns.

This guide covers exactly where Synthetic Users fits, where it does not, and how Koji handles the same job differently.

---

What Synthetic Users Actually Does

Synthetic Users is an AI platform that generates synthetic participants — LLM-based agents calibrated to demographic, psychographic, and OCEAN (Big Five personality) profiles. You describe the participant you want to "interview," set up a discussion guide, and the platform runs an AI-versus-AI conversation: one agent asks questions, another agent plays the participant, a third agent critiques.

The output looks like a real research transcript. There is a participant name, a personality profile, follow-up probing, and a thematic summary. For €2–€27 per interview (with a small RAG add-on if you want the agents to reference your own documents), you can generate dozens of these in an afternoon.

Where this is genuinely useful:

• Discussion guide stress-testing. Before you spend money recruiting real people, run your guide against a few synthetic personas to find leading questions or confusing wording.
• Hypothesis brainstorming. Generating a wide funnel of "what might users think?" hypotheses before deciding which ones to test with real people.
• Internal alignment exercises. When stakeholders cannot agree on who the user is, synthetic personas can structure the disagreement before you commit to recruiting.
• Desk research at scale. Sketching market segments quickly when no real-data alternative exists.

These are real jobs and synthetic users can help with them. The problem starts when teams use synthetic personas as if they were participants — making roadmap, pricing, or positioning decisions on data that has no grounding in actual customer behavior.

---

The Validity Problem

The critique of synthetic users is not "AI is bad." The critique is specific and methodologically rigorous, and it deserves a clear hearing.

Synthetic users praise everything

A widely-cited finding from research published in 2025–2026 is that synthetic users care about everything equally and praise concepts without criticism. Real customers have sharp preferences, surprising blind spots, and load-bearing objections. Synthetic agents are trained to be helpful and fluent, which means they tend to validate whatever you put in front of them. For concept testing or feature prioritization, this is the opposite of what you need.

LLMs reflect the dominant voices in their training data

LLMs are trained predominantly on English-speaking, affluent, tech-literate text. Marginalized perspectives, non-Western users, low-income segments, and edge-case users are systematically underrepresented. A synthetic persona of a "rural farmer in Indonesia" is not actually a rural farmer in Indonesia — it is what the model statistically associates with that label, which is closer to how Western media depicts that persona than how that person would actually answer.

No theory of mind, no preferences, no memory

Large language models convert words into vectors and produce statistically probable next tokens. They do not know, want, remember, or believe anything. There is no stable preference structure underneath the persona — just patterns. Two "interviews" with the same synthetic persona, on the same topic, will often produce contradictory answers because there is no underlying participant who actually exists.

Cannot be validated

This is the crux. A real interview can be wrong but it can also be checked — against analytics, against support tickets, against the participant's actual behavior in your product. A synthetic interview produces a transcript that sounds plausible but has no external referent. You cannot fact-check what a non-existent person said. There is no ground truth.

The ACM Interactions essay summarized it bluntly: synthetic users are assumptions built on assumptions. If you make decisions based on them, you are making decisions based on what your LLM thinks plausible-sounding people would say — which is rarely what your actual customers will do.

---

What Koji Does Differently

Koji is also AI-moderated. The AI conducts the interview, asks follow-up questions, and synthesizes themes. The difference is that Koji interviews real people — the actual customers and prospects you want to understand — at the same speed synthetic platforms claim.

Here is how Koji solves the bottleneck synthetic users tries to solve:

• Always-on AI interviewer. Send a link, the AI conducts the interview whenever the participant is available. Asynchronous, voice or text, no scheduling required. See always-on user interviews.
• Automatic probing. When a participant says something interesting, the AI asks the follow-up. Not a synthetic follow-up — a real one, to a real person. Detailed in the AI probing guide.
• Six structured question types. Open-ended, scale, single choice, multiple choice, ranking, and yes/no — all in the same study. See the structured questions guide.
• Automatic thematic analysis. Across all completed interviews, with quote evidence. Real quotes from real customers. See the AI transcript analysis guide.
• Voice and text both supported. Voice interviews capture nuance that text loses; text interviews work for participants who prefer typing. Configured in setting up voice interviews.
• GDPR-compliant consent collection. Built into every study, see intake forms and consent.

What you get is the speed of synthetic research with the validity of real research.

---

Side-by-Side Comparison

| Capability | Synthetic Users | Koji | |---|---|---| | Who are the participants? | LLM-generated AI personas | Real customers (yours or recruited) | | Can findings be validated? | No external ground truth | Yes — quotes, transcripts, real participants | | Probing follow-up questions | Yes (AI to AI) | Yes (AI to real human) | | Voice interviews | Text only | Voice and text | | Thematic analysis | Yes, on synthetic data | Yes, on real customer data | | Question types | Free-form | 6 structured types + open-ended | | Risk of training-data bias | High — reflects LLM's dominant voices | None — actual respondent voices | | Concept testing reliability | Low — synthetic personas tend to praise everything | High — real users push back | | Pricing | ~€2–€27 per synthetic interview | Free to start; from €29/mo (29 credits) | | Audit trail for stakeholders | Synthetic transcripts | Real transcripts, real participants, real quotes |

---

When Synthetic Users Is the Right Tool

To be fair: there are jobs where synthetic users is the better choice.

Use Synthetic Users when:

• You are stress-testing a discussion guide before recruiting real participants
• You are brainstorming hypotheses that you will later validate with real users
• You need to sketch personas for an internal alignment workshop where the goal is structured disagreement, not data
• You are doing desk research in a domain where real participants are genuinely unreachable in your timeframe
• The decision the research informs is reversible and low-stakes

Synthetic Users honestly has a place in the modern researcher's toolkit — as a complement to real research, not a substitute for it.

When Koji Is the Right Tool

Use Koji when:

• The decision is irreversible or high-stakes (roadmap, pricing, positioning, hiring, fundraising)
• You need to understand why real customers behave the way they do
• Stakeholders need quote evidence from real people in the report
• You are doing churn analysis, win-loss analysis, or concept validation
• The audit trail of the research matters (board, investors, regulators)
• You want the speed of AI moderation without the validity tradeoff

The rule of thumb: if a real decision will be made on the back of the research, the participants need to be real.

---

What This Looks Like in Practice

Scenario: Validating a new pricing tier.

With Synthetic Users, you spin up 30 synthetic "SMB founders" and ask them about willingness to pay for a Pro tier at $99/mo. They respond fluently. Most express interest. The report says SMB founders are receptive to the price point.

You launch the tier. Conversion is 0.4% of trials. The real customers, when interviewed afterward, say the price was the wrong shape entirely — they would have paid annually, but not monthly, and the value was unclear without a feature you had not even highlighted.

With Koji, you import 30 real trial users (or open a public link) and run a pricing research interview with willingness-to-pay questions. The AI probes for the shape of pricing (annual vs monthly, per-seat vs flat) not just the number. Themes surface: real customers ask for an annual option, real customers fixate on a feature you had not foregrounded. You ship the right tier on the first try.

Speed is the same. Validity is not.

---

How Teams Use Both Tools

The sophisticated approach: use synthetic and real research in series, not in parallel.

1. Synthetic phase (1 day). Generate hypotheses about what real customers might say. Stress-test your discussion guide. Identify leading questions.
2. Real research with Koji (3–5 days). Run the refined guide against real customers. AI-moderated, asynchronous, voice or text. Get themes with real quote evidence.
3. Decision (week 2). Make the call on real data, with synthetic data only used to inform what you investigated.

This is how the methodologically rigorous teams at companies that take research seriously are using synthetic tools today: as scaffolding, not as the building.

---

Start a Real-Customer Study with Koji

Koji is free to start. Import your customers from CSV, share a public link, or use Koji's panel. The AI conducts every interview asynchronously, probes follow-ups automatically, and delivers a thematic report with quote evidence — from real people, in days.

Start free →

Related reading: AI-moderated vs human-moderated interviews · How to run AI-powered customer interviews at scale · Why AI interviewers are the future of customer research · Best AI customer interview tools in 2026 · Customer discovery: the ultimate guide for startups