All frameworks and cheat sheets

Process

How to build a norm bank in 7 steps

Process diagram of seven steps to build a norm bank of past survey results, from freezing the instrument to computing percentile ranks and publishing the rules.
Twelve waves gives you a usable bank. Step 1 is the one stakeholders will push back on.

A seven-step method for building your own benchmark from past survey or interview waves when no published benchmark matches your method. It covers fixing the instrument, logging metadata, a minimum sample per wave, storing full distributions, the percentile rank formula, re-baselining on a schedule and publishing the rules. It suits anyone running a tracking programme who keeps being asked whether a score is good. Koji's fixed scale questions and stored response distributions make steps one and four automatic.

Where this appears

What's in this process

  1. Freeze the instrument: Lock the wording, scale, anchors and mode, and write them down. A wording change makes two short banks.
  2. Log every wave with its metadata: Date, n, invitations, exact wording, mode, sample source, segments and the full response distribution.
  3. Set a minimum n per wave: A wave below it goes in the archive, not the norm bank.
  4. Store the distribution, not just the mean: Keep the count for each scale point so you can work out top-box later and spot a split behind a stable mean.
  5. Compute percentile ranks: Percentile rank = (B + 0.5E) / N x 100, so tied scores are handled fairly.
  6. Re-baseline on a schedule: Annually is normal. Re-baselining after a bad wave is how a norm bank loses credibility.
  7. Publish the rules: Instrument, minimum n, formula and schedule. Otherwise the loudest stakeholder can argue with the number.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/how-to-build-a-norm-bank-seven-steps"><img src="https://www.koji.so/visuals/how-to-build-a-norm-bank-seven-steps/how-to-build-a-norm-bank-seven-steps.png" alt="Process diagram of seven steps to build a norm bank of past survey results, from freezing the instrument to computing percentile ranks and publishing the rules." width="1080" height="1350" /></a>
<p>Source: <a href="https://www.koji.so/visuals/how-to-build-a-norm-bank-seven-steps">How to build a norm bank in 7 steps</a>, Koji</p>