Flowchart
Is it the teaching change or the novelty?
A decision path for judging a teaching pilot before it is scaled up
A decision path for educational developers and quality leads who have to judge whether a new teaching approach works or only looks good because it is new. It asks whether learning was measured, whether the gain held later in the term and whether it held for a second cohort. Koji can run the same questions at several points in a term and ask students why they gave each rating.
What's in this flowchart
Start: A teaching pilot scored well
- Did you measure learning, not only ratings?
- Yes: go to 2.
- No: Add a direct learning measure. An exam item, a concept test or graded work marked blind. Then go to 2.
- Did the gain hold later in the term?
- Yes: go to 3.
- No: Treat the early gain as novelty. Check fidelity and what students said about why.
- Has it run with more than one cohort?
- Yes: go to 4.
- No: Hold the verdict for a second run. Report it as one cohort, novelty not yet ruled out.
- Compare with a cohort that did not get it. Last year's cohort or a parallel group, measured the same way.
- Finish: Scale it, and keep measuring.
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/teaching-innovation-novelty-effect-check-flowchart"><img src="https://www.koji.so/visuals/teaching-innovation-novelty-effect-check-flowchart/teaching-innovation-novelty-effect-check-flowchart.png" alt="Flowchart for judging a teaching pilot: three questions on learning measures, cohorts and late-term gains, ending in a scale-up with continued measurement." width="1200" height="1313" /></a>
<p>Source: <a href="https://www.koji.so/visuals/teaching-innovation-novelty-effect-check-flowchart">Is it the teaching change or the novelty?</a>, Koji</p>
