All frameworks and cheat sheets

Flowchart

Fix or ship: reading one usability task

A decision path for product teams reading task results from a small usability test

Flowchart for deciding whether to fix or ship a tested task: three questions, one side step, ending in shipping and retesting later.
Run this for each task before you look at the averages

A decision path for reading the results of one usability test task. It starts with whether enough people reached the task, compares self-reported finishing with what the recordings show, sends you to the clips when people took detours, and tells you to fix and retest when a problem repeats. Use it after each round of five to ten sessions. In Koji, the task section of a report gives you the counts, verdicts and clips this path asks for.

Where this appears

What's in this flowchart

Start: Results for one task are in

  1. Did at least five people reach the task?
    • Yes: go to 2.
    • No: Run more sessions first. Below five, read each result as one person's story, not a rate.
  2. Compare Finished with the recordings. Count direct, indirect and unfinished attempts, not only button presses.
  3. Did most people finish by a direct path?
    • Yes: go to 4.
    • No: Watch every stuck and indirect clip. Name the step where each one went wrong. Then go to 4.
  4. Does any problem repeat across people?
    • No: go to 5.
    • Yes: Fix it and test again. Rate its severity, change the design, retest with five new people.
  5. Check time on task against your target. Compare successful attempts only.
  6. Finish: Ship, then retest after the next change.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/usability-task-result-fix-or-ship-flowchart"><img src="https://www.koji.so/visuals/usability-task-result-fix-or-ship-flowchart/usability-task-result-fix-or-ship-flowchart.png" alt="Flowchart for deciding whether to fix or ship a tested task: three questions, one side step, ending in shipping and retesting later." width="1200" height="1520" /></a>
<p>Source: <a href="https://www.koji.so/visuals/usability-task-result-fix-or-ship-flowchart">Fix or ship: reading one usability task</a>, Koji</p>