Framework
When scoring rubrics make marking more reliable
Four conditions that Jonsson and Svingby's 2007 review of 75 studies linked to more reliable scoring with rubrics: analytic criteria, task-specific wording, exemplars and rater training. It is for programme leaders and assessment leads who review how marking criteria are written and used. The same review warns that rubrics alone do not make judgements valid.
What's in this framework
- Analytic, not only holistic: Separate criteria, scored one at a time, give markers more to agree on.
- Specific to the task: Criteria written for this assignment work better than generic ones.
- Backed by exemplars: Marked examples show what each level looks like in practice.
- Paired with marker training: Markers who practise on the rubric together score more consistently.
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/scoring-rubric-reliability-conditions"><img src="https://www.koji.so/visuals/scoring-rubric-reliability-conditions/scoring-rubric-reliability-conditions.png" alt="Framework of four conditions under which scoring rubrics make marking more reliable: analytic criteria, task-specific wording, exemplars and marker training." width="1200" height="752" /></a>
<p>Source: <a href="https://www.koji.so/visuals/scoring-rubric-reliability-conditions">When scoring rubrics make marking more reliable</a>, Koji</p>
