{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-06-22T20:50:57.001Z"},"content":[{"type":"documentation","id":"f47e3a1e-f2e1-41cb-a9f6-4d58abb9467f","slug":"survey-fraud-respondent-quality","title":"Survey Fraud & Respondent Quality: How to Detect Fake and Low-Effort Responses (2026)","url":"https://www.koji.so/docs/survey-fraud-respondent-quality","summary":"Between 5% and 26% of online survey responses are fraudulent (averaging ~17%), and AI-generated answers now pass standard attention checks. Fraud takes many forms: bots, professional respondents, duplicates, satisficing (straight-lining, gibberish open ends), and LLM-written answers. Warning signs include impossibly fast completion, straight-lining, generic open ends, inconsistent answers, duplicate fingerprints, and failed attention checks. Layer detection: attention items, time thresholds, CAPTCHA/fingerprinting, consistency-checked screeners, open-ended honeypots, and proportional incentives. Koji's structural defense: an AI-moderated conversation cannot be faked by random clicking. Koji scores every transcript 1–5 on relevance, depth, and coverage, and only conversations scoring 3+ count — automatically excluding low-effort and bot responses from your bill and report. Follow-up probing exposes fabricated answers. Koji uses six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no).","content":"## The short answer\n\nSurvey fraud is the silent tax on every research budget. Depending on the panel and region, **5% to 26% of online survey responses contain fraudulent or fabricated data** — averaging around **17% across 1,008 surveys**, and roughly **20% of market research arrives with bogus feedback** ([ResearchShield](https://researchshield.com/resources/blog/real-impact-of-survey-fraud.html)). Worse, **AI can now corrupt opinion surveys at scale — passing every quality check and mimicking real humans without leaving a trace** ([phys.org, 2025](https://phys.org/news/2025-11-fake-survey-ai-quietly-sway.html)). In small target populations, a handful of fraudulent responses can flip a study's conclusion ([NIH/PMC](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11156680/)).\n\nThe defense is twofold: harden how you *collect* data, and switch to a format fraud cannot fake cheaply. A static survey rewards speed-clicking; an **AI-moderated conversation** rewards genuine, in-the-moment reasoning. **Koji** scores every conversation 1–5 on relevance, depth, and coverage, and **only conversations scoring 3 or higher count** — so low-effort and bot responses are filtered out before they ever reach your report.\n\n## What survey fraud actually looks like\n\nFraud is not one thing. The common forms:\n\n- **Bots and scripts** that complete surveys en masse to farm incentives.\n- **Professional respondents** who join many panels and rush through for the reward, often misrepresenting who they are.\n- **Duplicate submissions** — the same person answering repeatedly from different devices or sessions.\n- **Satisficing** — real people giving the least effort that passes: straight-lining scales, copy-pasted gibberish in open ends, contradictory answers.\n- **AI-generated answers** — increasingly, plausible free-text written by an LLM to defeat attention checks.\n\nThe cost is not just wasted spend. Fraudulent data biases your themes, inflates or deflates scores, and — most dangerously — leads confident teams to ship the wrong thing.\n\n## Warning signs of low-quality responses\n\nWatch for these red flags in your data:\n\n1. **Impossibly fast completion** — finishing a 10-minute survey in 90 seconds.\n2. **Straight-lining** — the same option down every scale question.\n3. **Generic or off-topic open ends** — \"good\", \"nice product\", or text that ignores the question.\n4. **Inconsistent answers** — contradicting an earlier response.\n5. **Duplicate fingerprints** — repeated IP, device, or verbatim text across \"different\" respondents.\n6. **Geographic mismatch** — responses from outside your target market or via VPN.\n7. **Failed attention checks** — missing an instructed-response item (\"select Strongly Agree here\").\n\n## Detection tactics that still work\n\nNo single check is enough; layer them:\n\n- **Attention and instructed-response items** — but assume sophisticated bots and AI now pass them.\n- **Time-to-complete thresholds** — flag both impossibly fast and abandoned-then-resumed sessions.\n- **ReCAPTCHA and device/IP fingerprinting** — catch crude bots and duplicates.\n- **Screeners with consistency checks** — ask the same fact two ways; mismatches reveal fakes. See [screening participants effectively](/docs/screening-participants-effectively).\n- **Open-ended honeypots** — a free-text question is the hardest thing for a careless respondent to fake convincingly; gibberish stands out.\n- **Reasonable incentives** — outsized rewards attract professional fraudsters. Calibrate with the [research participant incentives](/docs/research-participant-incentives) guide.\n\n## Why conversation beats the fraudsters\n\nHere is the structural advantage. A multiple-choice form can be defeated by clicking randomly — the data still *looks* complete. A conversation cannot. To pass a Koji interview, a respondent must give answers that are **relevant** to the question, show **depth** of reasoning, and **cover** the topics the research brief defines. Koji's analysis engine scores each transcript on exactly those three dimensions (1–5) and assigns an overall quality score.\n\n**Only conversations scoring 3 or higher consume a credit** — which means low-effort and bot responses are automatically excluded from both your bill and your report. A bot can click \"7\" on an NPS scale; it cannot improvise a credible, on-topic explanation of *why* it churned and then answer a context-aware follow-up. The follow-up probing is the trap: Koji's AI asks \"you mentioned the price — what would have made it worth it?\", and fabricated answers fall apart under that second question.\n\nLearn the mechanics in [how the quality gate works](/docs/how-the-quality-gate-works) and [understanding quality scores](/docs/understanding-quality-scores).\n\n## Designing a fraud-resistant study in Koji\n\n1. **Lead with open-ended depth.** Use the six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no), but anchor the study on open-ended questions where the AI probes follow-ups — the part fraud cannot fake. See the [structured questions guide](/docs/structured-questions-guide).\n2. **Let the quality gate do the filtering.** Conversations below a score of 3 are excluded automatically; you review only credible data.\n3. **Screen before you interview.** Add consistency-checked screener questions to keep off-target respondents out.\n4. **Watch the quality distribution.** A sudden spike of low scores from one source signals an incentive leak or a bot ring — investigate the channel.\n5. **Keep incentives proportional.** Reward completion enough to be fair, not enough to attract fraud rings.\n\n## How fraud distorts results — and why it compounds\n\nBecause fraudulent responses are not random, they do not \"average out.\" Bots and professional respondents cluster on the answers that maximize reward or minimize effort, systematically skewing distributions. In a small or niche population — early adopters, enterprise buyers, a rare medical cohort — even 10% bad data can reverse a finding ([NIH/PMC](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11156680/)). Pair this guide with [survey response bias](/docs/survey-response-bias) and [sampling bias research](/docs/sampling-bias-research) to understand how collection errors and fraud stack on top of each other.\n\n## A response-quality scorecard you can run today\n\nBefore you trust a dataset, audit a sample against these checks and quarantine anything that fails two or more:\n\n| Check | Red flag | Action |\n| --- | --- | --- |\n| Time-to-complete | Below the 10th percentile (e.g. 90s on a 10-min survey) | Flag for review |\n| Straight-lining | Identical option down all scales | Drop |\n| Open-end quality | Blank, generic, off-topic, or AI-sounding | Drop or review |\n| Internal consistency | Contradicts an earlier answer | Drop |\n| Fingerprint | Duplicate IP, device, or verbatim text | Dedupe |\n| Geography | Outside target market / VPN | Review |\n| Attention item | Missed instructed response | Drop |\n\nWith Koji, this scorecard largely runs itself: the quality score already encodes relevance, depth, and coverage, and only conversations scoring 3+ are kept — so you start from clean data rather than auditing your way back to it.\n\n## Fraud risk by collection channel\n\nNot every channel carries equal risk. **Open, incentivized panels** see the highest fraud as professional respondents and bot rings chase rewards. **Public, anonymous links** — including QR codes — are exposed to low-effort scans. **Authenticated, in-product audiences** and **personalized invitations** to known customers are the cleanest, because identity is established before the response. When you must use an open channel, lean harder on open-ended depth and the quality gate, and keep incentives modest. See [survey response bias](/docs/survey-response-bias) and the [QR code survey guide](/docs/qr-code-survey-guide) for channel-specific guidance.\n\n## Building a quality-first research culture\n\nTools catch fraud; habits prevent it. Standardize a pre-analysis quality pass on every study, document your exclusion rules so findings are reproducible, calibrate incentives to be fair rather than tempting, and prefer formats that are expensive to fake. The strongest defense is structural: when your data source is a probing conversation rather than a clickable form, the cheapest path for a fraudster — random clicking — simply stops producing usable submissions.\n\n## Related Resources\n\n- [Structured Questions Guide](/docs/structured-questions-guide) — the six question types behind every Koji study\n- [How the Quality Gate Works](/docs/how-the-quality-gate-works)\n- [Understanding Quality Scores](/docs/understanding-quality-scores)\n- [Survey Data Quality Guide](/docs/survey-data-quality-guide)\n- [Screening Participants Effectively](/docs/screening-participants-effectively)\n- [Survey Response Bias](/docs/survey-response-bias) and [Sampling Bias in Research](/docs/sampling-bias-research)","category":"Research Operations","lastModified":"2026-06-22T03:20:15.705049+00:00","metaTitle":"Survey Fraud & Respondent Quality: Detect Fake Responses (2026) | Koji","metaDescription":"5–26% of survey responses are fraudulent and AI answers now pass standard checks. Learn the warning signs, layered detection tactics, and how Koji's conversational quality gate filters bad data before it reaches your report.","keywords":["survey fraud","fake survey responses","respondent quality","survey data quality","detect bots in surveys","survey fraud detection","fraudulent survey responses","satisficing straight-lining"],"aiSummary":"Between 5% and 26% of online survey responses are fraudulent (averaging ~17%), and AI-generated answers now pass standard attention checks. Fraud takes many forms: bots, professional respondents, duplicates, satisficing (straight-lining, gibberish open ends), and LLM-written answers. Warning signs include impossibly fast completion, straight-lining, generic open ends, inconsistent answers, duplicate fingerprints, and failed attention checks. Layer detection: attention items, time thresholds, CAPTCHA/fingerprinting, consistency-checked screeners, open-ended honeypots, and proportional incentives. Koji's structural defense: an AI-moderated conversation cannot be faked by random clicking. Koji scores every transcript 1–5 on relevance, depth, and coverage, and only conversations scoring 3+ count — automatically excluding low-effort and bot responses from your bill and report. Follow-up probing exposes fabricated answers. Koji uses six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no).","aiPrerequisites":["A survey or study collecting responses online","Basic understanding of survey design and incentives"],"aiLearningOutcomes":["Recognize the main forms of survey fraud and low-quality responses","Spot the seven warning signs of bad data","Layer detection tactics that still work against bots and AI","Design a fraud-resistant study in Koji","Understand how the conversational quality gate filters bad data automatically"],"aiDifficulty":"intermediate","aiEstimatedTime":"12 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}