Reference table
Equivalence test vs difference test: the four outcomes
Run TOST alongside the usual significance test and every result lands in one of four cells
A two-by-two table of what happens when you run an equivalence test (TOST) next to an ordinary significance test. The four cells are a real effect too small to matter, a meaningful difference, statistical equivalence, and an inconclusive result. Use it to label a readout honestly before a sunset, migration or parity decision. The layout is adapted from Daniel Lakens's 2017 tutorial. Koji studies let you write the equivalence bound into the brief before fielding.
Where this appears
What's in this reference table
| Equivalence significant | Equivalence not significant | |
|---|---|---|
| Difference significant | Real but too small to matter: the effect exists and you can stop caring about it | A meaningful difference: act on it |
| Difference not significant | Statistically equivalent: you may claim no meaningful difference | Inconclusive: the study could not tell, and the honest report says so |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/equivalence-test-and-difference-test-four-outcomes"><img src="https://www.koji.so/visuals/equivalence-test-and-difference-test-four-outcomes/equivalence-test-and-difference-test-four-outcomes.png" alt="Two by two table of results from a difference test and an equivalence test: too small to matter, a meaningful difference, equivalent, or inconclusive." width="1200" height="630" /></a>
<p>Source: <a href="https://www.koji.so/visuals/equivalence-test-and-difference-test-four-outcomes">Equivalence test vs difference test: the four outcomes</a>, Koji</p>
