Key figure
An AI respondent passed 99.8% of attention checks
The headline result from Sean Westwood's 2025 PNAS paper on large language models and online surveys. A deliberately simple autonomous agent passed 99.8% of attention checks drawn from published studies, made zero errors on impossible-event probes and cost about five cents per response. Use it to explain why harder attention checks are not a defence against AI-written survey data.
Where this appears
What's in this key figure
99.8% of attention checks passed by an autonomous AI survey respondent, across 20 types of check10 errors in 6,000 trials, perfect on 18 of the 20 check types, at about five cents per response.
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/ai-respondent-passes-99-8-percent-attention-checks"><img src="https://www.koji.so/visuals/ai-respondent-passes-99-8-percent-attention-checks/ai-respondent-passes-99-8-percent-attention-checks.png" alt="Statistic card: an autonomous AI survey respondent passed 99.8% of attention checks across 20 check types, at about five cents a response." width="1200" height="641" /></a>
<p>Source: <a href="https://www.koji.so/visuals/ai-respondent-passes-99-8-percent-attention-checks">An AI respondent passed 99.8% of attention checks</a>, Koji</p>
