Reference table
What to split on in a split-ballot test

Six kinds of split-ballot test for product surveys, each with two question versions and what the gap between them tells you. Use it to choose the wording test that matters most for your next survey or tracked metric. Give each arm enough sample, and decide in advance which comparison you will report. In Koji you can run each version as its own study and split the invite list at random.
Where this appears
What's in this reference table
| Split | Version A | Version B | What it measures |
|---|---|---|---|
| Loaded vs neutral term | our AI assistant | the automated suggestions feature | How much of your result is brand language |
| Consequence clause | Would you use this? | Would you use this, if it added a step to your current workflow? | The gap between appeal and willingness to pay a cost |
| Frame | keeps 90% of your data clean | leaves 10% of your data unclean | Sensitivity to gain versus loss framing |
| Reference period | in the last week | in a typical month | How much your usage estimate depends on the recall window |
| Scale direction | Best option first | Worst option first | Order and recency effects, which you then randomize for good |
| Attribute label | reliability | it works every time I need it | Whether respondents share your vocabulary |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/split-ballot-what-to-split-on"><img src="https://www.koji.so/visuals/split-ballot-what-to-split-on/split-ballot-what-to-split-on.png" alt="Table of six split-ballot tests with two versions each: loaded term, consequence clause, frame, reference period, scale direction, attribute label." width="1200" height="899" /></a>
<p>Source: <a href="https://www.koji.so/visuals/split-ballot-what-to-split-on">What to split on in a split-ballot test</a>, Koji</p>




