Reference table
Bonferroni vs Holm vs Benjamini-Hochberg on ten p-values
One segment sweep, four ways of deciding which results to keep
A worked example of three multiple comparisons corrections on the same ten p-values from a segment sweep. With no correction six results pass 0.05. Bonferroni and Holm, which control the family-wise error rate, keep only the smallest. Benjamini-Hochberg, which controls the false discovery rate, keeps three. Use it to choose a correction: family-wise control when one false positive is costly, false discovery rate when you are screening for leads.
Where this appears
What's in this reference table
| Rank and p-value | Uncorrected | Bonferroni | Holm | Benjamini-Hochberg |
|---|---|---|---|---|
| Threshold | 0.05 | 0.05/10 | 0.05/(11-i) | i/10 x 0.05 |
| 1st: 0.002 | keep | keep (vs 0.005) | keep (vs 0.0050) | keep (vs 0.005) |
| 2nd: 0.009 | keep | drop | drop (vs 0.0056) | keep (vs 0.010) |
| 3rd: 0.012 | keep | drop | drop | keep (vs 0.015) |
| 4th: 0.031 | keep | drop | drop | drop (vs 0.020) |
| 5th: 0.038 | keep | drop | drop | drop |
| 6th: 0.049 | keep | drop | drop | drop |
| 7th: 0.061 | drop | drop | drop | drop |
| 8th to 10th: 0.140+ | drop | drop | drop | drop |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/bonferroni-holm-benjamini-hochberg-worked-example"><img src="https://www.koji.so/visuals/bonferroni-holm-benjamini-hochberg-worked-example/bonferroni-holm-benjamini-hochberg-worked-example.png" alt="Table applying no correction, Bonferroni, Holm and Benjamini-Hochberg to ten sorted p-values: six kept uncorrected, one by Bonferroni and Holm, three by BH" width="1200" height="909" /></a>
<p>Source: <a href="https://www.koji.so/visuals/bonferroni-holm-benjamini-hochberg-worked-example">Bonferroni vs Holm vs Benjamini-Hochberg on ten p-values</a>, Koji</p>
