Flowchart
When to stop cognitive interview rounds
A decision flow for planning cognitive interview rounds. Flag the questions most likely to fail, run a small round of 5 to 9 interviews, log and fix what you find, then test the revised version. Stop when a round finds no new significant problems, which for most instruments takes 2 or 3 rounds, then move on to a pilot test.
Where this appears
What's in this flowchart
Start: You have a draft survey or guide
- Flag the high-risk questions. Vague terms, long sentences, two questions in one, long recall periods.
- Run a round of 5 to 9 interviews. One person at a time, thinking aloud, with your planned probes.
- Log each problem and revise. Note the question, the problem type, what you saw and the fix.
- Did this round find new significant problems?
- No: go to 5.
- Yes: go back to 2.
- Finish: Pilot test, then launch. Most instruments get there within 2 or 3 rounds.
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/when-to-stop-cognitive-interview-rounds-flowchart"><img src="https://www.koji.so/visuals/when-to-stop-cognitive-interview-rounds-flowchart/when-to-stop-cognitive-interview-rounds-flowchart.png" alt="Flowchart for cognitive interview rounds: flag risky questions, run 5 to 9 interviews, revise, and repeat until a round finds nothing new." width="1200" height="1140" /></a>
<p>Source: <a href="https://www.koji.so/visuals/when-to-stop-cognitive-interview-rounds-flowchart">When to stop cognitive interview rounds</a>, Koji</p>
