Reference table
What Cronbach's alpha can't tell you, and what can
A table for anyone reporting a multi-item survey score. It lists six questions people try to answer with Cronbach's alpha, such as whether items measure one thing, whether scores compare across segments, or whether a scale can detect change, and gives the right tool for each: omega, factor analysis, construct validity evidence, measurement invariance, ceiling analysis or significance testing.
Where this appears
What's in this reference table
| Question you're asking | Right tool | Wrong tool |
|---|---|---|
| Do these items hang together? | Omega, average inter-item correlation | Alpha alone |
| Do these items measure one thing? | Factor analysis, confirmatory factor model | Alpha |
| Am I measuring the concept I named? | Construct validity evidence | Alpha |
| Can I compare this score across segments? | Measurement invariance testing | Alpha |
| Can the scale detect the change I care about? | Ceiling and floor analysis, statistical power | Alpha |
| Is the difference I found real? | Statistical significance, equivalence testing | Alpha |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/scale-quality-questions-right-tool-vs-alpha"><img src="https://www.koji.so/visuals/scale-quality-questions-right-tool-vs-alpha/scale-quality-questions-right-tool-vs-alpha.png" alt="Table matching six questions about a survey scale to the right tool, such as omega, factor analysis or invariance testing, with alpha as the wrong tool." width="1200" height="877" /></a>
<p>Source: <a href="https://www.koji.so/visuals/scale-quality-questions-right-tool-vs-alpha">What Cronbach's alpha can't tell you, and what can</a>, Koji</p>
