Checklist
Number needed to treat: a reporting checklist
A seven-point checklist for reporting product experiment results as counts of people rather than percentage lifts. It covers the absolute risk reduction and its reciprocal, the time horizon, exposure, number needed to harm, the baseline and the interviews that explain why most treated users saw no benefit. Koji can run those non-responder interviews across a whole treated cohort.
Where this appears
What's in this checklist
- Compute the absolute risk reduction and its reciprocal for every effect you report
- State the time horizon in the same sentence as the count
- Report the exposure rate, with both the number needed to treat and to offer
- Compute the number needed to harm on your riskiest metric, and name its owner
- Confirm the baseline came from an event-specific estimate, not a blended curve
- Interview non-responders: never at risk, beyond saving or the wrong problem
- Put the relative figure in the appendix
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/number-needed-to-treat-reporting-checklist"><img src="https://www.koji.so/visuals/number-needed-to-treat-reporting-checklist/number-needed-to-treat-reporting-checklist.png" alt="Checklist of seven steps for reporting a product effect as a number needed to treat, from computing the absolute reduction to interviewing non-responders." width="1080" height="1080" /></a>
<p>Source: <a href="https://www.koji.so/visuals/number-needed-to-treat-reporting-checklist">Number needed to treat: a reporting checklist</a>, Koji</p>
