All frameworks and cheat sheets

Reference table

The four types of survey randomization

Table of the four types of survey randomization: option, question, block and randomized assignment, with what each rotates and when to use it.
The first three control order bias. The fourth is experimental design.

The four kinds of survey randomization side by side: shuffling answer options inside a question, shuffling parallel questions within a section, rotating whole blocks so an early topic does not prime a later one, and randomly assigning respondents to versions for concept, message or wording tests. Use it to pick the right kind for each question, and to remember that randomized assignment is about experiment design, not bias control.

Where this appears

What's in this reference table

TypeWhat rotatesUse it when
Option randomizationThe order of answer choices inside one questionAlmost always, for unordered lists of 4 or more options
Question randomizationThe order of questions within a sectionItems are parallel and independent, like rating eight features
Block randomizationThe order of whole sectionsAn early section could prime a later one, like awareness before attributes
Randomized assignmentWhich version of the study a respondent seesMonadic concept tests, message tests, split-sample wording experiments

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/four-types-of-survey-randomization"><img src="https://www.koji.so/visuals/four-types-of-survey-randomization/four-types-of-survey-randomization.png" alt="Table of the four types of survey randomization: option, question, block and randomized assignment, with what each rotates and when to use it." width="1200" height="716" /></a>
<p>Source: <a href="https://www.koji.so/visuals/four-types-of-survey-randomization">The four types of survey randomization</a>, Koji</p>