Survey Fraud & Respondent Quality: How to Detect Fake and Low-Effort Responses (2026)

The short answer

Survey fraud is the silent tax on every research budget. Depending on the panel and region, 5% to 26% of online survey responses contain fraudulent or fabricated data — averaging around 17% across 1,008 surveys, and roughly 20% of market research arrives with bogus feedback (ResearchShield). Worse, AI can now corrupt opinion surveys at scale — passing every quality check and mimicking real humans without leaving a trace (phys.org, 2025). In small target populations, a handful of fraudulent responses can flip a study's conclusion (NIH/PMC).

The defense is twofold: harden how you collect data, and switch to a format fraud cannot fake cheaply. A static survey rewards speed-clicking; an AI-moderated conversation rewards genuine, in-the-moment reasoning. Koji scores every conversation 1–5 on relevance, depth, and coverage, and only conversations scoring 3 or higher count — so low-effort and bot responses are filtered out before they ever reach your report.

What survey fraud actually looks like

Fraud is not one thing. The common forms:

• Bots and scripts that complete surveys en masse to farm incentives.
• Professional respondents who join many panels and rush through for the reward, often misrepresenting who they are.
• Duplicate submissions — the same person answering repeatedly from different devices or sessions.
• Satisficing — real people giving the least effort that passes: straight-lining scales, copy-pasted gibberish in open ends, contradictory answers.
• AI-generated answers — increasingly, plausible free-text written by an LLM to defeat attention checks.

The cost is not just wasted spend. Fraudulent data biases your themes, inflates or deflates scores, and — most dangerously — leads confident teams to ship the wrong thing.

Warning signs of low-quality responses

Watch for these red flags in your data:

1. Impossibly fast completion — finishing a 10-minute survey in 90 seconds.
2. Straight-lining — the same option down every scale question.
3. Generic or off-topic open ends — "good", "nice product", or text that ignores the question.
4. Inconsistent answers — contradicting an earlier response.
5. Duplicate fingerprints — repeated IP, device, or verbatim text across "different" respondents.
6. Geographic mismatch — responses from outside your target market or via VPN.
7. Failed attention checks — missing an instructed-response item ("select Strongly Agree here").

Detection tactics that still work

No single check is enough; layer them:

• Attention and instructed-response items — but assume sophisticated bots and AI now pass them.
• Time-to-complete thresholds — flag both impossibly fast and abandoned-then-resumed sessions.
• ReCAPTCHA and device/IP fingerprinting — catch crude bots and duplicates.
• Screeners with consistency checks — ask the same fact two ways; mismatches reveal fakes. See screening participants effectively.
• Open-ended honeypots — a free-text question is the hardest thing for a careless respondent to fake convincingly; gibberish stands out.
• Reasonable incentives — outsized rewards attract professional fraudsters. Calibrate with the research participant incentives guide.

Why conversation beats the fraudsters

Here is the structural advantage. A multiple-choice form can be defeated by clicking randomly — the data still looks complete. A conversation cannot. To pass a Koji interview, a respondent must give answers that are relevant to the question, show depth of reasoning, and cover the topics the research brief defines. Koji's analysis engine scores each transcript on exactly those three dimensions (1–5) and assigns an overall quality score.

Only conversations scoring 3 or higher consume a credit — which means low-effort and bot responses are automatically excluded from both your bill and your report. A bot can click "7" on an NPS scale; it cannot improvise a credible, on-topic explanation of why it churned and then answer a context-aware follow-up. The follow-up probing is the trap: Koji's AI asks "you mentioned the price — what would have made it worth it?", and fabricated answers fall apart under that second question.

Learn the mechanics in how the quality gate works and understanding quality scores.

Designing a fraud-resistant study in Koji

1. Lead with open-ended depth. Use the six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no), but anchor the study on open-ended questions where the AI probes follow-ups — the part fraud cannot fake. See the structured questions guide.
2. Let the quality gate do the filtering. Conversations below a score of 3 are excluded automatically; you review only credible data.
3. Screen before you interview. Add consistency-checked screener questions to keep off-target respondents out.
4. Watch the quality distribution. A sudden spike of low scores from one source signals an incentive leak or a bot ring — investigate the channel.
5. Keep incentives proportional. Reward completion enough to be fair, not enough to attract fraud rings.

How fraud distorts results — and why it compounds

Because fraudulent responses are not random, they do not "average out." Bots and professional respondents cluster on the answers that maximize reward or minimize effort, systematically skewing distributions. In a small or niche population — early adopters, enterprise buyers, a rare medical cohort — even 10% bad data can reverse a finding (NIH/PMC). Pair this guide with survey response bias and sampling bias research to understand how collection errors and fraud stack on top of each other.

A response-quality scorecard you can run today

Before you trust a dataset, audit a sample against these checks and quarantine anything that fails two or more:

With Koji, this scorecard largely runs itself: the quality score already encodes relevance, depth, and coverage, and only conversations scoring 3+ are kept — so you start from clean data rather than auditing your way back to it.

Fraud risk by collection channel

Not every channel carries equal risk. Open, incentivized panels see the highest fraud as professional respondents and bot rings chase rewards. Public, anonymous links — including QR codes — are exposed to low-effort scans. Authenticated, in-product audiences and personalized invitations to known customers are the cleanest, because identity is established before the response. When you must use an open channel, lean harder on open-ended depth and the quality gate, and keep incentives modest. See survey response bias and the QR code survey guide for channel-specific guidance.

Building a quality-first research culture

Tools catch fraud; habits prevent it. Standardize a pre-analysis quality pass on every study, document your exclusion rules so findings are reproducible, calibrate incentives to be fair rather than tempting, and prefer formats that are expensive to fake. The strongest defense is structural: when your data source is a probing conversation rather than a clickable form, the cheapest path for a fraudster — random clicking — simply stops producing usable submissions.

Related Resources

• Structured Questions Guide — the six question types behind every Koji study
• How the Quality Gate Works
• Understanding Quality Scores
• Survey Data Quality Guide
• Screening Participants Effectively
• Survey Response Bias and Sampling Bias in Research
• The All-or-None Rule for Research QA - what a spot-check actually detects, and why prevention beats screening
• Tightening Your Screener Makes the Sample Purer and the Findings Worse - the measured sensitivity of attention checks and speeder flags