Comparison
Evaluating supplemental instruction: two designs compared
A side-by-side comparison for learning support managers, programme directors and institutional researchers evaluating supplemental instruction or peer-assisted study sessions. It contrasts the common attendee versus non-attendee comparison with a randomised encouragement design, and shows why the first tends to overstate the effect. Use it when planning an SI evaluation or reading one. Koji can add students' own accounts of why they did or did not attend.
What's in this comparison
Attendees vs non-attendees
- Uses routine attendance and grade data
- Students choose to attend, so motivation and prior ability are mixed in
- Usually shows attendees doing better
- Attendance cut-offs are often set arbitrarily
Randomised encouragement
- A random group of students gets an extra nudge or incentive to attend
- Groups are compared by assignment, not by choice
- Paloyo et al. found about 0.065 SD per extra hour, not significant
- Needs large samples to detect small effects
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/supplemental-instruction-evaluation-designs-compared"><img src="https://www.koji.so/visuals/supplemental-instruction-evaluation-designs-compared/supplemental-instruction-evaluation-designs-compared.png" alt="Comparison of two ways to evaluate supplemental instruction: comparing attendees with non-attendees, and a randomised encouragement design." width="1200" height="695" /></a>
<p>Source: <a href="https://www.koji.so/visuals/supplemental-instruction-evaluation-designs-compared">Evaluating supplemental instruction: two designs compared</a>, Koji</p>
