How to Screen Participants for Customer Interviews

To screen participants for customer interviews when you recruit from Reddit, Slack groups or LinkedIn, ask three to five multiple-choice questions about what people did recently, mix in answers that sound right but aren't, and never say which answer gets someone in. Let through only people who had the problem in the last few months and did something about it. By the end of this playbook you'll have the questions, a rule for who gets a call, and a way to spot the replies that aren't real before you spend a gift card or an afternoon on them.

When you need this

You posted in a subreddit or a Slack community, and it worked better than you expected. Forty people replied overnight. Now you have a new problem: most of them are probably not the person you need, and you can't tell which ones.

A founder at a laptop reading through replies to a recruiting post before deciding who to interview

Photo by Štefan Štefančík on Unsplash

The signs you need a screener:

Your post says who you're looking for ("freelance bookkeepers who chase receipts"), so anyone who wants the reward knows what to say.
You're offering a gift card, and replies arrive within minutes of posting.
Half the replies say "this is exactly my problem" and nothing else.
You've already had one call where the person clearly didn't do the job they claimed.

The cost is bigger than a wasted call. At idea stage you might run 15 to 20 interviews in total. If five are with people who don't have the problem, your notes say it's more common and more painful than it really is, and you'll build on that.

Why community recruits need screening

Most people who answer research invitations are honest. The trouble is that the few who aren't end up as a large share of the people you select, because they're the ones who make sure they qualify.

Jesse Chandler and Gabriele Paolacci showed this in four studies with online workers [1]. When a prescreen asked whether people were the parent or guardian of a child with autism, 7.8% said yes when the answer could get them into a paid study, against 4.3% in the control group. When the real group is rare, a few points of dishonesty can make up most of the people who get through. That's the paper's title: most prescreening answers are honest, yet most study participants were impostors.

Money is part of it, but more money isn't the cause. In a randomised survey of 2,275 US adults, Holly Fernandez Lynch and colleagues estimated that 10.5% to 22.8% of people lied about their eligibility when a payment depended on it, and higher payments did not lead to more lying [2]. Cutting the reward won't fix a screener that tells people the answer.

Open online samples are where this shows up most. In Pew Research Center's 2020 study, bogus respondents were 7% of a crowdsourced sample, 5% of opt-in panels on average and 1% of address-recruited panels [3]. Reddit and Slack recruits look more like the first two than the last.

The common fixes are weaker than people assume. In that same Pew study, 76% of bogus cases passed both an attention check and a speeding check [3]. Pew's 2026 follow-up, a survey of 11,114 US adults, tested attention checks, an automated prescreening service and matching people to the voter file. The first two helped a little, voter-file matching made the results slightly worse, and no single method reliably removed the fakes [4].

Two newer things make this matter more for founders. First, AI writes convincing sign-up replies. A 2026 study of a UK interview project described about 400 likely false expressions of interest across three university sites, many written or helped by large language models, with templated wording, vague stories and reluctance to do a short video check [5]. Second, bad participants don't just waste time. Comparing impostors with real participants in a 2025 study, Rand and colleagues found the impostors meaningfully changed the qualitative findings [6]. That's your problem statement, your ICP and your pricing, built partly on someone who made it up.

None of this means community recruiting is a bad idea. It's often the best free route an idea-stage founder has (how to find 20 people with no audience). It means you screen before you book.

The method in brief

A screener is a short set of questions people answer before the interview, and each answer decides whether they get in. For a founder recruiting strangers, four rules do most of the work. The method itself is covered in depth in research screener questions; this is how to apply it to community recruits.

Ask about behaviour, not identity. "Which of these did you do in the last 30 days?" is harder to fake than "Are you a bookkeeper?"
Hide the target. Put the answer you want among four to six plausible options, so the right answer isn't obvious.
Add a foil. One option that sounds real but isn't (a tool that doesn't exist, a task no one does), and anyone who picks it is out.
Use multiple choice. Free-text screener answers are exactly what AI now writes well. Save the open questions for the interview.

Then treat what passes as a filter, not proof. Answers are still self-reported. The interview's first five minutes, where you ask about the last time the problem happened, are your second screen. Why a strict screener can cost you good people is covered in why tightening a screener can make findings worse.

Step by step

1. Write down who gets a call, in one sentence

Before writing any question, finish this sentence: "I want to talk to people who [did what] in the last [how long] and who currently [do what today]." For example: "Freelance bookkeepers who chased a client for missing receipts in the last month and who currently use email or spreadsheets to do it."

If you can't finish it, go back to finding your first customer segment. Every screener question below maps to one blank in that sentence.

2. Rewrite your recruiting post so it doesn't give away the answer

Most founders write "Looking for freelance bookkeepers who struggle with receipt collection." That's a script for anyone who wants the gift card. Describe the topic instead: "I'm researching how small finance teams and freelancers handle paperwork from clients. 20 minutes, a $20 gift card if you're a fit." You'll get more replies, and the screener sorts them.

Follow the community's rules. Many subreddits ban paid research posts or ask you to message the moderators first.

3. Write three to five questions

Use the question bank below as a starting point. One question per blank in your sentence, plus a foil, is usually enough. More than five and real people start dropping out.

Question bank of screening questions for interview recruits from online communities: what they did recently, how they handle it today, who they are. — Copy these and swap in your own problem. Every one is multiple choice, with the target hidden among real options · Download or share

A few notes on writing them:

Recency beats frequency. "When did you last..." with options like "This week", "This month", "1 to 6 months ago", "Longer ago", "Never" tells you more than "How often do you...".
Make the right answer one of several. If your target uses spreadsheets, list spreadsheets alongside three other real tools and "Something else".
Put the foil where it's plausible. A made-up tool name in a list of real tools works well. Pick one that doesn't exist; search it first.
Ask a role question with enough options that the right one isn't the obvious one. Include "Student" and "Not working right now" so honest people have somewhere to go.

4. Decide your rules before anyone answers

For each option, decide now whether it qualifies, disqualifies or doesn't matter. Writing the rules after you see the answers is how you talk yourself into letting people through. A typical set:

Did the task in the last month: qualifies. Longer ago or never: out.
Picked the foil: out, whatever else they said.
Wrong role: out.
Tool used: doesn't matter for getting in, but you note it, because it tells you who to compare later.

5. Read the replies for the signs that aren't in the answers

Before you book anyone, look at what the screener can't show. The signs from the research on impostor participants [5] are a good checklist: several replies with the same wording, replies within minutes of the post going live, vague accounts of the problem, details that don't fit (a time zone, a location, a job title that doesn't exist), and pushing back on a two-minute video or phone check. One sign is a reason to look closer. Two or more is a reason to pass.

6. Make the first five minutes of the interview a second screen

Open with "Tell me about the last time you had to chase a client for a receipt. What happened?" A real bookkeeper gives names of tools, the month, the client type and what they did next. Someone faking it stays general. If the story falls apart, end politely, and decide in advance whether you still pay. Most founders do, and treat it as the cost of recruiting.

How many to screen, and how long it takes

Plan to screen two or three times as many people as you want to interview. For 15 interviews, aim for 30 to 45 screener answers. With a post in two or three communities, that usually takes three to five days. Screening itself takes people under two minutes, so it barely affects how many reply.

A worked example

Say you're two people building a tool that helps freelance bookkeepers collect receipts and invoices from clients. You want 15 interviews in two weeks, and you have no audience.

You post in a bookkeeping subreddit (after checking the rules), a Slack group for freelance accountants and a LinkedIn post. The post says you're researching "how freelancers and small finance teams handle paperwork from clients", 20 minutes, a $20 gift card if you're a fit.

Your screener has four questions:

Which best describes your work? Freelance or self-employed bookkeeper (qualifies), in-house accountant, business owner doing my own books, student, other (all out).
When did you last have to ask a client for a missing receipt or invoice? This week or this month (qualifies), 1 to 6 months ago, longer ago, never (out).
What do you mostly use to track what's missing? Email, a spreadsheet, practice software, a made-up tool you call "LedgerLoop" (the foil: out), nothing in particular. Everything except the foil passes.
How many active clients do you have? Under 5, 5 to 20, 21 to 50, over 50. All pass; you note it for comparison.

Sixty-two people answer in four days. Thirty-eight fail question 1 or 2, which is normal; the post was deliberately broad. Three pick the foil. Of the 21 who pass, you read the replies: four share near-identical wording and arrived within ten minutes of the LinkedIn post, so you pass on them. That leaves 17, and you book 15.

In the interviews, two people can't describe a recent case in any detail. You thank them, pay them and leave their notes out of your analysis. You end with 13 solid interviews. Nine of them chase receipts weekly and do it from email, and they're the ones who describe spending real time on it. The four on practice software barely have the problem.

Without the screener, the story would have been "everyone in bookkeeping hates this." The real story is narrower and more useful: freelancers with 5 to 20 clients who track it by email. That's your first segment.

Reading the results and deciding

Three numbers tell you whether the screener and the recruiting channel are working.

Flowchart for deciding who gets a customer interview: four checks on recency, the foil, warning signs and a recent story. — Run every reply through the same four checks, in the same order, before you book · Download or share

Pass rate. If almost everyone passes, your post is giving away the answer or your rules are too loose. If under one in ten passes, either the channel is wrong or your sentence in step 1 is too narrow. Try one more community before you change the sentence.
Foil rate. A handful of foil-pickers is normal. If more than one in ten of your passes would have been foil-pickers, treat the whole channel with suspicion and add a short video or phone check before booking.
Interview failure rate. If more than one in five people who pass can't describe a recent case, your screener is letting the wrong people through. Swap identity questions for behaviour ones and tighten the time window.

What to do next:

Passes look right and stories hold up: keep going, and move on to ranking which problem to solve first.
Too few pass: go to where the problem is discussed, not where the job title gathers. Recruit the people who replied on threads about the problem.
Too many fakes: move to a channel with more accountability (warm intros, a professional community with real names) or add a short video check before booking.

Run it this week with Koji

Everything above works with a free form and a calendar link. Google Forms and Typeform can route people on multiple-choice answers, then send the ones who qualify to a booking page. The catch is that it's manual, it's a separate step people drop out of, and you still have to run every call yourself.

In Koji, screening questions are part of the study. Here's the setup:

Start a study and describe what you want to learn in plain words. The research assistant drafts a brief, and you can ask it to add screening questions for the person you described.
Open the brief's Questions tab and find the Screening Questions section. Add single-choice or multiple-choice questions (those are the two types that can screen).
Set a rule on every option. On single choice, each option qualifies or disqualifies. On multiple choice, each option can be allowed, required or disqualifying, so you can say "must have picked email" or "out if they picked the foil." Koji won't let you publish while any option is missing a rule.
Publish and share one link in your Reddit post, Slack thread or LinkedIn post. Everyone answers the screening questions on the landing page before the interview starts, whatever link they came in on. The AI interviewer never asks them again, and they don't appear as charts in the report.

Koji screener question asking participants which role best describes them, with one option selected. — What a participant sees first: the screening questions, before any interview starts

Watch who's turned away. People who don't qualify are listed under Screened out in your responses, so you can see how many your rules turn away and loosen a rule that's rejecting people you'd have wanted. Someone who was screened out isn't asked again if they come back, and an invitee who failed can't retry on the same personal link. No interview takes place for them.
Run the interviews without booking calls. People who pass go straight into a text or voice interview, at a time that suits them, and the AI asks follow-up questions about their last real case. That gets you your second screen (the story) without scheduling 15 calls.

Screening questions are included on every plan, including the free one, for studies created from September 2026. Answers are still self-reported: Koji filters on what people say, it doesn't verify who they are. For high-value B2B interviews, keep a quick identity check (a LinkedIn profile, a work email) in your process. If you're paying people, how to handle interview incentives covers what to offer and when to pay.

Common mistakes

Naming the target in the post. The single biggest cause of fake qualifiers. Describe the topic, not the person.
Yes/no or open-text questions. "Do you manage client receipts?" has one obvious answer, and AI fills in free text convincingly. Give five options.
Screening on job title alone. Titles are easy to claim. Recent behaviour is harder.
Changing rules after seeing answers. Decide in step 4, before anyone replies.

When research will mislead you

Screening reduces the problem. It doesn't remove it. Even large survey organisations with dedicated tools can't reliably filter out every fake respondent [4], and you won't either.

A screener filters, it doesn't verify. People who pass told you the right things. The interview story is the real test.
Tight screens can bias your sample. If you only let in people who use spreadsheets, you'll conclude spreadsheet users are your market. Keep "doesn't matter" options where you can, and compare groups later.
Community members aren't your whole market. People who hang out in a subreddit about bookkeeping are more engaged than average. Treat them as early adopters and check findings with a second channel.
Interested isn't committed. A real person with a real problem can still not pay. Pair the interviews with a commitment ask, like a pilot or a deposit, before you build.

Where this fits

This sits between two steps on the founder's research playbook:

Before: how to find 20 people to interview with no audience, the outreach that gets you replies to screen, and whether to pay people to talk to you.
After: which problem to solve first, once you have interviews you trust.

For the method behind it, see research screener questions and why most people who pass a screener aren't who you wanted. Koji's page for founders is at /for/founders.

FAQ

How many screening questions should I ask?

Three to five. One per part of your "who gets a call" sentence, plus a foil. More than that and honest people drop off before they finish.

Do people really lie on screeners to get a gift card?

Some do. In a randomised study of 2,275 adults, between 10.5% and 22.8% lied about eligibility when payment depended on it, and bigger payments didn't make it worse [2]. Most people are honest, but the few who aren't end up overrepresented among the people who qualify.

Should I tell people they didn't qualify?

Yes, briefly and kindly. A short thank-you that says the study is looking for a specific group is enough. Don't say which answer ruled them out, or they'll know what to say next time.

Can I screen on Reddit without a tool?

Yes. A Google Form or Typeform with multiple-choice questions and section logic can send qualifiers to a booking link. You'll do more by hand, but the questions and rules in this playbook work the same way.

What do I do if someone passes the screener but clearly isn't a fit?

End the interview politely at the first natural break, pay what you promised, and leave their notes out of your analysis. Then check which screener question let them through and fix it before the next batch.

Your next step

Write your one-sentence "who gets a call" rule, then turn each blank into a multiple-choice question with a foil. If you want the screening and the interviews in one link, start a study in Koji and ask the assistant to draft the screening questions for you.

References

Jesse Chandler and Gabriele Paolacci, "Lie for a Dime: When Most Prescreening Responses Are Honest but Most Study Participants Are Impostors," Social Psychological and Personality Science, 2017. https://journals.sagepub.com/doi/abs/10.1177/1948550617698203
Holly Fernandez Lynch, Steven Joffe, Harsha Thirumurthy, Dawei Xie and Emily A. Largent, "Association Between Financial Incentives and Participant Deception About Study Eligibility," JAMA Network Open, 2019. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2722570
Pew Research Center, "Assessing the Risks to Online Polls From Bogus Respondents," 2020. https://www.pewresearch.org/methods/2020/02/18/assessing-the-risks-to-online-polls-from-bogus-respondents/
Pew Research Center, "No Easy Fix for Bogus Respondents in Online Opt-In Polls," 2026. https://www.pewresearch.org/methods/2026/08/27/no-easy-fix-for-bogus-respondents-in-online-opt-in-polls/
Sam Quinn, Naomi Richards, Joanne Williams, Caroline Mogan, Michelle Myall, Mari Lloyd-Williams and Lynn Calman, "Navigating the Rise of Artificial Intelligence and Imposter Participants in Qualitative Health Research," International Journal of Qualitative Methods, 2026. https://doi.org/10.1177/16094069261431990
Rand and colleagues, "Impostor Syndrome: Fraudulent Participants in Qualitative Research Can Skew Results," Journal of Law, Medicine & Ethics, 2025. https://pubmed.ncbi.nlm.nih.gov/40619588/

How to Screen Participants for Customer Interviews When You Recruit From Reddit