All frameworks and cheat sheets

Flowchart

Should a course evaluation item offer don't know or N/A?

A per-item decision rule based on survey experiments

Flowchart for deciding if a course evaluation item needs a not applicable or don't know option: three questions with side steps, ending in separate reporting.
Decide per item, then keep every non-answer out of the average Source: After Krosnick et al. (2002) and Décieux et al. (2015)

A decision flowchart for questionnaire owners deciding, item by item, whether a course evaluation question should offer Not applicable, I don't know, or neither, and whether it should be required. It follows survey experiments on no-opinion options and forced answering. Koji supports the same rule with branching, so students are only asked about what they used.

What's in this flowchart

Start: You are writing an evaluation item

  1. Has every student experienced what this asks about?
    • Yes: go to 2.
    • No: Branch, or add Not applicable. Ask only those who used it, or offer a clearly labelled Not applicable. Then go to 2.
  2. Is it a factual question students may not know?
    • No: go to 3.
    • Yes: Offer I don't know. Facts like how the final mark is weighted can be honestly unknown. Then go to 3.
  3. Leave Don't know off opinion items. Students who do have a view tend to take the easy exit when one is offered.
  4. Is an answer required to submit?
    • No: go to 5.
    • Yes: Make it optional. Forced answers raise drop-out and lower answer quality. Then go to 5.
  5. Finish: Report N/A and skips separately. Show the counts next to each item and never fold them into the mean.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/dont-know-not-applicable-option-decision-flowchart"><img src="https://www.koji.so/visuals/dont-know-not-applicable-option-decision-flowchart/dont-know-not-applicable-option-decision-flowchart.png" alt="Flowchart for deciding if a course evaluation item needs a not applicable or don't know option: three questions with side steps, ending in separate reporting." width="1200" height="1484" /></a>
<p>Source: <a href="https://www.koji.so/visuals/dont-know-not-applicable-option-decision-flowchart">Should a course evaluation item offer don't know or N/A?</a>, Koji</p>