Reference table
GRADE's five reasons to rate evidence down, for product research
The five reasons the GRADE framework gives for lowering confidence in a body of evidence, translated for product and UX research. Each row pairs a GRADE domain with what it looks like in commercial studies, from leading questions and proxy users to nulls that were never tested for equivalence. Use it when rating how much trust a combined finding deserves before a team acts on it.
Where this appears
What's in this reference table
| GRADE domain | The product research version |
|---|---|
| Risk of bias | Leading questions, convenience samples, moderator effects, recruiting your happiest cohort |
| Inconsistency | The studies disagree with each other and their design does not explain why |
| Indirectness | You studied a proxy population, a proxy behaviour or a prototype, not the real thing |
| Imprecision | Small samples, wide intervals, or nulls that were never equivalence-tested |
| Publication bias | You cannot show that the studies you are pooling are all the studies that were run |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/grade-five-downgrade-domains-product-research"><img src="https://www.koji.so/visuals/grade-five-downgrade-domains-product-research/grade-five-downgrade-domains-product-research.png" alt="Table of the five GRADE downgrade domains (risk of bias, inconsistency, indirectness, imprecision, publication bias) applied to product research." width="1200" height="798" /></a>
<p>Source: <a href="https://www.koji.so/visuals/grade-five-downgrade-domains-product-research">GRADE's five reasons to rate evidence down, for product research</a>, Koji</p>
