{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-13T12:39:17.028Z"},"content":[{"type":"documentation","id":"9e52e8e5-9502-465a-8b56-963faf9a42a2","slug":"screening-participants-effectively","title":"Screening Research Participants Effectively","url":"https://www.koji.so/docs/screening-participants-effectively","summary":"Effective participant screening uses qualifying and disqualifying questions to ensure research interviews are conducted with relevant people. 36% of researchers cite recruiting the right participants as their biggest challenge. Best practices include hiding desired answers, using behavioral questions, keeping screeners under 5 minutes, and including attention checks.","content":"Recruiting participants who match your research criteria is the foundation of useful qualitative data. Interview the wrong people and no amount of skilled questioning or sophisticated analysis will save your study. According to User Interviews' 2023 Research Benchmark Report, 36% of researchers cite recruiting the right participants as their single biggest challenge — more than budget, time, or analysis complexity.\n\nEffective screening is not just about finding people who are willing to participate. It is about finding people whose experiences are relevant to your research questions.\n\n## What Is Participant Screening?\n\nParticipant screening is the process of evaluating potential research participants against predefined criteria to determine whether they are a good fit for your study. It typically happens through a short questionnaire (a screener) completed before the interview is scheduled.\n\nA good screener does three things:\n1. **Includes** participants who match your target profile\n2. **Excludes** participants who would provide irrelevant data\n3. **Does both without revealing** which answers lead to selection (preventing gaming)\n\n## Defining Your Screening Criteria\n\nBefore writing a single screening question, clearly define who you want to talk to and who you do not.\n\n### Primary Criteria (Must-Have)\n\nThese are non-negotiable qualifications directly tied to your research questions.\n\n**Examples:**\n- \"Currently uses [product category] at least weekly\"\n- \"Made a purchase decision for [product type] in the last 6 months\"\n- \"Manages a team of 5 or more people\"\n- \"Has experience with [specific workflow or tool]\"\n\n### Secondary Criteria (Nice-to-Have)\n\nThese add diversity or specificity but are not dealbreakers.\n\n**Examples:**\n- \"Mix of company sizes (startup, mid-market, enterprise)\"\n- \"Geographic diversity across at least 3 regions\"\n- \"Balance of tenure (new users and power users)\"\n\n### Exclusion Criteria (Must-Not-Have)\n\nThese disqualify participants who would introduce problematic data.\n\n**Examples:**\n- Works for a competitor or in the market research industry\n- Has a personal relationship with anyone on your team\n- Participated in your research within the past 6 months (to avoid \"professional participants\")\n\n## Writing Effective Screening Questions\n\n### Rule 1: Never Reveal the \"Right\" Answer\n\nThe most common screening mistake is asking questions where the desired answer is obvious.\n\n| Revealing (Bad) | Non-Revealing (Good) |\n|-----------------|---------------------|\n| \"Are you an active user of project management tools?\" | \"Which of the following tools do you use regularly? (Select all that apply)\" followed by a list that includes PM tools mixed with other categories |\n| \"Do you make purchasing decisions at your company?\" | \"What is your role in software purchasing decisions?\" with options: Final decision maker / Influencer / Evaluator / No involvement / Other |\n| \"Have you experienced frustration with onboarding?\" | \"How would you describe your most recent onboarding experience?\" with a neutral scale |\n\n### Rule 2: Use Behavioral Questions, Not Attitudinal Ones\n\nWhat people *do* is a better predictor of fit than what people *say* they do.\n\n- **Attitudinal (weaker):** \"How important is data security to you?\" (Nearly everyone says \"very important\")\n- **Behavioral (stronger):** \"In the last 3 months, have you taken any specific actions to review or improve your data security practices?\" with specific action options\n\n### Rule 3: Include Attention Checks\n\nAdd one or two questions designed to catch inattentive respondents:\n- \"Please select 'Strongly Disagree' for this question\" buried in a matrix\n- A question with a clearly correct factual answer\n- A question asking them to type a specific word\n\nResearch by the Pew Research Center found that attention check questions identify approximately 8-12% of survey respondents as inattentive, and removing these respondents significantly improves data quality (Kennedy et al., 2020).\n\n### Rule 4: Keep It Short\n\nYour screener should take 3-5 minutes to complete. Every additional question reduces your completion rate. According to SurveyMonkey's research, screener completion rates drop by approximately 15% for every additional minute beyond 5 minutes.\n\n## Screener Question Types\n\n### Multiple Choice (Single Select)\nBest for: Categorical criteria like role, industry, company size\n\n*\"What best describes your current role?\"*\n- Product Manager\n- Designer\n- Engineer\n- Researcher\n- Marketing\n- Sales\n- Other (please specify)\n\n### Multiple Choice (Multi-Select)\nBest for: Usage patterns, tool inventories\n\n*\"Which of the following tools do you use at least monthly? (Select all that apply)\"*\n- [Tool A]\n- [Tool B]\n- [Tool C]\n- [Unrelated tools as decoys]\n- None of the above\n\n### Numeric / Range\nBest for: Frequency, team size, budget, tenure\n\n*\"Approximately how many user interviews has your team conducted in the past 12 months?\"*\n- 0\n- 1-5\n- 6-15\n- 16-30\n- More than 30\n\n### Open-Ended (Short Answer)\nUse sparingly. Best for: Verifying genuine experience.\n\n*\"In 1-2 sentences, describe the last time you evaluated a new software tool for your team.\"*\n\nOpen-ended responses help you distinguish between people who genuinely have the experience and those who are guessing their way through the screener.\n\n## Building the Screener Flow\n\nA well-structured screener follows this pattern:\n\n1. **Warm-up question** (non-threatening, builds engagement)\n2. **Primary qualifying questions** (determine basic eligibility)\n3. **Secondary qualifying questions** (refine the sample)\n4. **Disqualifying questions** (catch exclusion criteria)\n5. **Attention check** (verify engagement)\n6. **Demographics** (for sample diversity tracking)\n7. **Contact information and availability**\n\n### Early Disqualification\n\nPlace your most important qualifying question early and use branching logic to screen out unqualified respondents immediately. This respects their time and reduces your review workload.\n\n## Screener Scoring\n\nFor studies where you need to select from a pool of qualified respondents, assign point values to responses:\n\n| Criteria | Response | Points |\n|----------|----------|--------|\n| Uses PM tools weekly | Yes | 3 |\n| Uses PM tools monthly | Yes | 1 |\n| Manager of 5+ people | Yes | 2 |\n| Made purchase decision recently | Last 3 months | 3 |\n| Made purchase decision recently | Last 6 months | 2 |\n\nTotal the points per respondent and select those above your threshold. This creates a transparent, defensible selection process.\n\n## How Koji Handles Participant Screening\n\nIf you are using Koji for your research, you can set up intake forms that serve as built-in screeners. Participants who access your interview link first complete your screening questions. Based on their responses, they either proceed to the interview or receive a polite message explaining they are not a fit for this particular study.\n\nThis automates the screening-to-interview pipeline — no manual review needed between the screener and the session. For setup instructions, see [intake forms and consent](/docs/intake-forms-and-consent).\n\n## Common Mistakes to Avoid\n\n1. **Screening too broadly**: If your criteria are too loose, you will waste interviews on people whose experiences are not relevant. Be specific about who you need.\n\n2. **Screening too narrowly**: If your criteria are too tight, you will struggle to recruit and may introduce selection bias. Make sure your criteria serve the research — not your assumptions.\n\n3. **Revealing the target profile**: If participants can guess which answers qualify them, motivated respondents will game the screener.\n\n4. **Skipping the pilot test**: Run your screener past 3-5 colleagues before launching. If they can guess the \"right\" answers, so can participants.\n\n## Key Takeaways\n\n- Define primary, secondary, and exclusion criteria before writing any screening questions\n- Never reveal which answers qualify a participant — use non-obvious question design\n- Behavioral questions (\"what did you do?\") outperform attitudinal questions (\"how important is X to you?\")\n- Keep screeners under 5 minutes to maintain completion rates\n- Include at least one attention check question\n- Use point-based scoring for transparent participant selection\n\nFor strategies on where to find participants in the first place, see [finding research participants](/docs/finding-research-participants). For setting up your screening flow within Koji, explore [intake forms and consent](/docs/intake-forms-and-consent).\n\n## Frequently Asked Questions\n\n**How many screening questions should I include?**\n\n5 to 10 questions is the sweet spot. Fewer than 5 may not give you enough signal to differentiate candidates. More than 10 increases drop-off and fatigue. Focus on questions that directly map to your primary and exclusion criteria.\n\n**Should I compensate people for completing the screener?**\n\nFor short screeners (under 5 minutes), compensation is not expected. For longer screeners or specialized populations, a small incentive (a gift card raffle entry, for example) can improve completion rates.\n\n**How do I screen for a behavior without leading?**\n\nEmbed the target behavior within a list of plausible alternatives. Instead of asking \"Do you use Slack for team communication?\" ask \"Which of the following do you use for team communication?\" with Slack as one of many options.\n\n**What is the ideal qualification rate for a screener?**\n\nA qualification rate between 15% and 40% is typical for targeted research. Below 15% suggests your criteria may be too narrow. Above 40% suggests your screener may not be selective enough.\n\n**Can I re-screen participants from a previous study?**\n\nYes, but verify their information is still current. Roles, tool usage, and behaviors change. A quick re-screening survey confirming key criteria is good practice before scheduling.\n\n## Further reading on the blog\n\n- [How to Recruit User Research Participants: The Complete Guide (2026)](/blog/how-to-recruit-user-research-participants-2026) — Recruiting the wrong participants is more expensive than recruiting none at all. Here's the complete playbook: screeners, channels, incentiv\n- [Best AI Market Research Tools in 2026: The Complete Buyer's Guide](/blog/ai-market-research-tools-2026) — AI has fundamentally changed market research. This guide compares the leading AI market research platforms—from AI-native interview tools li\n- [Best AI Customer Interview Tools in 2026: The Complete Buyer's Guide](/blog/best-ai-customer-interview-tools-2026) — AI has fundamentally changed how product teams conduct customer research. Here are the best AI customer interview tools in 2026 — ranked by \n\n<!-- further-reading:blog -->\n","category":"Participant Recruitment","lastModified":"2026-05-13T00:25:38.788654+00:00","metaTitle":"Screening Research Participants Effectively","metaDescription":"Write effective screening criteria and questions to recruit the right participants for your qualitative research studies.","keywords":["participant screening","research recruitment","screening questions","qualitative research","participant selection","screener design","user research"],"aiSummary":"Effective participant screening uses qualifying and disqualifying questions to ensure research interviews are conducted with relevant people. 36% of researchers cite recruiting the right participants as their biggest challenge. Best practices include hiding desired answers, using behavioral questions, keeping screeners under 5 minutes, and including attention checks.","aiPrerequisites":["finding-research-participants"],"aiLearningOutcomes":["Define primary, secondary, and exclusion screening criteria","Write non-revealing screening questions that prevent gaming","Design screener flows with branching logic and point-based scoring","Avoid common screening mistakes that reduce participant quality"],"aiDifficulty":"beginner","aiEstimatedTime":"9 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}