Checklist
Ceiling and floor effect checklist
Eight checks to run before trusting a rating-scale metric: plot the distribution, count top-box and bottom-box shares against the 15% threshold, compute headroom, repeat by segment, track variance across waves, test against a known change, add new question types if saturated, and pilot any replacement. In Koji, ranking and open questions sit in the same study as the scale.
Where this appears
What's in this checklist
- Plot the distribution before you read any mean
- Compute top-box and bottom-box share, and flag anything above 15%
- Compute headroom and the share who can register improvement
- Repeat by segment: unequal headroom invalidates comparisons
- Check variance across waves: falling variance on a flat mean is saturation
- Test the metric against a change you know was real
- If saturated, add a ranking item, a comparative item and an open question
- Pre-test any new instrument on 30 respondents
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/ceiling-floor-effect-checklist"><img src="https://www.koji.so/visuals/ceiling-floor-effect-checklist/ceiling-floor-effect-checklist.png" alt="Checklist of eight checks for ceiling and floor effects, from plotting the distribution and top-box share to pre-testing a new instrument." width="1080" height="1080" /></a>
<p>Source: <a href="https://www.koji.so/visuals/ceiling-floor-effect-checklist">Ceiling and floor effect checklist</a>, Koji</p>
