All frameworks and cheat sheets

Flowchart

Can you rank courses on in-class evaluations?

Flowchart for deciding whether in-class course evaluation data can rank courses: three questions, two side steps, one exit, ending in an attendance review.
Use the ratings for the course, and check who was absent before you rank anything. Source: After Wolbring 2012, Wolbring and Treischl 2016, Treischl and Wolbring 2017
Download PNG

A decision flowchart for quality teams and programme leads who run course evaluations in class. It asks whether you know who missed the survey session, whether those students differ from the ones present, and whether you plan to rank courses. It follows field studies from German universities that found absence on survey day is linked to course quality and workload, and that it shifts course rankings far more than course averages. Koji helps by sending every enrolled student their own interview link, so absent students can still explain what happened.

What's in this flowchart

Start: Your evaluation ran in class

  1. Do you know who was absent on survey day?
    • Yes: go to 2.
    • No: Compare forms with enrolment. Check returned forms, attendance or log-ins against the class list. Then go to 2.
  2. Do the absent students differ from the rest?
    • No: go to 3.
    • Yes: Survey the no-shows too. Send a short follow-up to everyone who missed the session. Then go to 3.
  3. Are you comparing or ranking courses?
    • No: go to 4.
    • Yes: Report bands, not exact ranks. Rankings shift most when absence differs between courses.
  4. Use the ratings for that course. Averages change little after adjusting for selection.
  5. Finish: Review attendance with each report.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/can-you-rank-courses-on-in-class-evaluations"><img src="https://www.koji.so/visuals/can-you-rank-courses-on-in-class-evaluations/can-you-rank-courses-on-in-class-evaluations.png" alt="Flowchart for deciding whether in-class course evaluation data can rank courses: three questions, two side steps, one exit, ending in an attendance review." width="1200" height="1408" /></a>
<p>Source: <a href="https://www.koji.so/visuals/can-you-rank-courses-on-in-class-evaluations">Can you rank courses on in-class evaluations?</a>, Koji</p>

Related

Flowchart for reporting Likert results: three questions on groups, crossing distributions and audience, ending with the mean shown beside its distribution.

How to report Likert scale results

Flowchart for deciding whether a customer segment is your first customer: three questions on recent episodes, paid workarounds and reach, one side step.

Is this segment your first customer?

Flowchart for choosing between a live A/B test and asking 20 customers about two offers: three questions, one side step, ending in a split send.

Test two offers with customers or with an A/B test?