Reference table
The evidence ladder for impact evaluation
Six ways to evaluate a change, ranked by internal validity: randomised experiment, regression discontinuity, difference-in-differences, interrupted time series, an uncontrolled before-and-after, and a before-and-after on a group picked for being extreme. Each row lists what the design needs and what usually blocks it. Use it to say plainly how strong your evidence is in a readout.
Where this appears
What's in this reference table
| Design | Internal validity | Needs | Common blocker |
|---|---|---|---|
| Randomised experiment | Highest | Ability to assign randomly | Impossible for launches, pricing or policy |
| Regression discontinuity | Very high near the cutoff | A threshold rule and data either side | Other things change at the same threshold |
| Difference-in-differences | High if trends were parallel | A credible untreated comparison group | No comparable group, or pre-trends diverge |
| Interrupted time series | Moderate to high | 8 to 12+ periods each side, a clean date | Concurrent events, seasonality, short series |
| Before-and-after, no control | Low | Two data points | Nearly every validity threat |
| Before-and-after on an extreme group | Lowest, actively misleading | Two data points | Guaranteed bias from regression to the mean |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/quasi-experimental-evidence-ladder-table"><img src="https://www.koji.so/visuals/quasi-experimental-evidence-ladder-table/quasi-experimental-evidence-ladder-table.png" alt="Table ranking six evaluation designs by internal validity, from randomised experiment down to before-and-after on an extreme group, with what each needs." width="1200" height="824" /></a>
<p>Source: <a href="https://www.koji.so/visuals/quasi-experimental-evidence-ladder-table">The evidence ladder for impact evaluation</a>, Koji</p>
