Reference table
Ways to check a synthetic user claim
Koji, AI personas, manual interviews and survey panels on the criteria researchers weigh
A comparison of four ways to test a finding that came from synthetic users: AI-moderated interviews in Koji, AI personas, manual interviews and survey panels. It scores each on who answers, depth, scale, consistency, analysis effort and best use. It is meant for UX researchers deciding how to check AI-generated research before acting on it.
Where this appears
What's in this reference table
| Criterion | Koji | AI personas | Manual interviews | Survey panel |
|---|---|---|---|---|
| Who answers | Real people you invite or recruit | A model playing a role | Real people you schedule | Real people, some may use AI |
| Why behind answers | AI follow-up probes each answer | Plausible, hard to test | Deep, depends on moderator | Mostly short open-ends |
| Scale | Many interviews in parallel | Unlimited | Limited by moderator hours | Large samples |
| Consistency | Same guide for every participant | Shifts with the prompt | Varies by moderator | Fixed questions |
| Can it contradict you? | Yes, from real participants | Rarely, tends to agree | Yes | Yes, but with little detail |
| Analysis effort | Themes with linked quotes | Output is the summary | Hours of coding by hand | Open-ends need coding |
| Setup effort | Brief, questions, screener | Write a persona prompt | Recruit, schedule, moderate | Write and field a survey |
| Best used for | Testing claims that drive decisions | Early hypotheses and vocabulary | Deep, high-stakes topics | Measuring how many |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/synthetic-vs-real-validation-methods-compared"><img src="https://www.koji.so/visuals/synthetic-vs-real-validation-methods-compared/synthetic-vs-real-validation-methods-compared.png" alt="Table comparing Koji, AI personas, manual interviews and survey panels for checking a synthetic user claim, on who answers, depth, scale and analysis effort." width="1200" height="982" /></a>
<p>Source: <a href="https://www.koji.so/visuals/synthetic-vs-real-validation-methods-compared">Ways to check a synthetic user claim</a>, Koji</p>
