All frameworks and cheat sheets

Checklist

What never to randomize in a survey

Checklist of what never to randomize in a survey: anchored options, scale points, ordered lists, narrative options, piped questions and running trackers.
If a respondent would notice the list was in a strange order, leave it alone.

Seven things to keep in a fixed position when you randomize a survey: escape options such as None of the above and Prefer not to say, scale points, logically ordered lists like age bands and frequencies, All of the above, options with a natural narrative flow, questions that feed piping or skip logic, and any tracker whose earlier waves were not randomized. Check every list against it before you switch shuffling on.

Where this appears

What's in this checklist

  • None of the above, Other, Don't know, Prefer not to say: keep them last
  • Scale points: a 1 to 10 scale is ordered by definition
  • Ordered lists: age bands, income, frequency, dates, process stages
  • All of the above, if you must use it
  • Options with a natural narrative flow
  • A question whose answer is piped into a later one
  • A tracker whose earlier waves used a fixed order

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/survey-options-never-to-randomize-checklist"><img src="https://www.koji.so/visuals/survey-options-never-to-randomize-checklist/survey-options-never-to-randomize-checklist.png" alt="Checklist of what never to randomize in a survey: anchored options, scale points, ordered lists, narrative options, piped questions and running trackers." width="1200" height="630" /></a>
<p>Source: <a href="https://www.koji.so/visuals/survey-options-never-to-randomize-checklist">What never to randomize in a survey</a>, Koji</p>