All frameworks and cheat sheets

Key figure

Forced-choice questions found 8 points more

Stat card: forced-choice yes or no questions gave estimates 8 points higher on average than a select-all checklist across 12 items in a Pew test.
Eight points is bigger than most primacy effects, so check the question format before the option order. Source: Pew Research Center, forced-choice vs select-all test with 4,581 US adults

In a Pew Research Center test with 4,581 US adults, asking a separate yes or no question for each of twelve items produced estimates 8 percentage points higher on average than one select-all-that-apply checklist, with a largest gap of 16 points. Pew moved to forced-choice in its online surveys where it could. Use the number when someone asks whether a multi-select is good enough.

Where this appears

What's in this key figure

8 pts Average gap across 12 items: a yes or no question per item vs one select-all-that-apply checklistPew tested both formats with 4,581 US adults. The largest gap was 16 points, 52% vs 36% on one item.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/forced-choice-vs-select-all-8-point-gap"><img src="https://www.koji.so/visuals/forced-choice-vs-select-all-8-point-gap/forced-choice-vs-select-all-8-point-gap.png" alt="Stat card: forced-choice yes or no questions gave estimates 8 points higher on average than a select-all checklist across 12 items in a Pew test." width="1200" height="641" /></a>
<p>Source: <a href="https://www.koji.so/visuals/forced-choice-vs-select-all-8-point-gap">Forced-choice questions found 8 points more</a>, Koji</p>