Checklist
Before-and-after usability study checklist
Eight rules that keep a redesign before-and-after usability study fair: agree the win condition first, keep the wording, mode and people the same, keep the redesign quiet in round 1 and wait out the launch weeks. For UX researchers and designers measuring whether a redesign worked. In Koji, duplicating the round 1 study keeps the brief and settings identical for round 2.
Where this appears
What's in this checklist
- Write down what counts as a win before round 1
- Use the same question wording in both rounds
- Keep the same mode: text both times or voice both times
- Invite the same people, or the same segment if you can't
- Don't mention the redesign in round 1
- Close round 1 before any beta or announcement
- Wait at least four weeks after launch to re-measure
- Read the reasons behind falling scores, not just the average
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/fair-before-after-usability-comparison-checklist"><img src="https://www.koji.so/visuals/fair-before-after-usability-comparison-checklist/fair-before-after-usability-comparison-checklist.png" alt="Checklist of eight rules for a fair before-and-after usability comparison, from identical wording and matched mode to waiting four weeks after launch." width="1080" height="1080" /></a>
<p>Source: <a href="https://www.koji.so/visuals/fair-before-after-usability-comparison-checklist">Before-and-after usability study checklist</a>, Koji</p>
