All frameworks and cheat sheets

Checklist

Reading student comments without confirmation bias

Seven habits for module reviews and programme boards

Checklist of seven steps for reading student comments without confirmation bias, from writing questions first to recording what you expected.
Use it before the comments reach a module review meeting Source: after Nickerson (1998) and Lincoln and Guba (1985)

A checklist for anyone who reads open-text course evaluation comments and reports them to a committee. Readers tend to find the comments that fit what they already believe. These seven steps make that harder: decide the questions first, count every comment, look for the ones that contradict you, and use a second reader. It follows Nickerson's review of confirmation bias and Lincoln and Guba's negative case analysis. Koji counts themes across every interview and links each one to the interviews behind it, which covers the counting step.

What's in this checklist

  • Write down your questions before you open the comments
  • Code every comment, then count how many fall in each group
  • Search on purpose for comments that contradict each finding
  • Quote in proportion to how often a theme came up
  • Ask a second reader with no stake to group them too
  • Keep the ratings hidden until the comments are coded
  • Record what you expected to find before you start

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/confirmation-bias-student-comments-checklist"><img src="https://www.koji.so/visuals/confirmation-bias-student-comments-checklist/confirmation-bias-student-comments-checklist.png" alt="Checklist of seven steps for reading student comments without confirmation bias, from writing questions first to recording what you expected." width="1080" height="1080" /></a>
<p>Source: <a href="https://www.koji.so/visuals/confirmation-bias-student-comments-checklist">Reading student comments without confirmation bias</a>, Koji</p>