Comparison
Averaging ratings vs an ordinal model
A comparison for analysts and quality teams who compare course evaluation ratings across modules, years or groups. Averages treat a five-point scale as equally spaced numbers, which can create false differences, miss real ones or reverse them. Ordinal models such as the cumulative logit or ordered probit treat ratings as ordered categories and report shifts and probabilities. Koji reports full answer distributions and exports responses for this kind of analysis.
What's in this comparison
Comparing averages
- Treats 1 to 5 as numbers with equal gaps
- Two very different modules can share one mean
- Can show false differences, miss real ones or reverse them
- Averaging several items does not fix the problem
Ordinal model
- Treats ratings as ordered categories
- Estimates the cut points between categories
- Reports shifts and the chance of each answer
- Extends to multilevel designs and covariates
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/averaging-ratings-vs-ordinal-model-course-evaluation"><img src="https://www.koji.so/visuals/averaging-ratings-vs-ordinal-model-course-evaluation/averaging-ratings-vs-ordinal-model-course-evaluation.png" alt="Comparison of averaging course evaluation ratings with fitting an ordinal model, showing what each assumes about the scale and what each reports." width="1200" height="642" /></a>
<p>Source: <a href="https://www.koji.so/visuals/averaging-ratings-vs-ordinal-model-course-evaluation">Averaging ratings vs an ordinal model</a>, Koji</p>
