All frameworks and cheat sheets

Reference table

What to split on in a split-ballot test

Table of six split-ballot tests with two versions each: loaded term, consequence clause, frame, reference period, scale direction, attribute label.
Change exactly one thing between the two versions.

Six kinds of split-ballot test for product surveys, each with two question versions and what the gap between them tells you. Use it to choose the wording test that matters most for your next survey or tracked metric. Give each arm enough sample, and decide in advance which comparison you will report. In Koji you can run each version as its own study and split the invite list at random.

Where this appears

What's in this reference table

SplitVersion AVersion BWhat it measures
Loaded vs neutral termour AI assistantthe automated suggestions featureHow much of your result is brand language
Consequence clauseWould you use this?Would you use this, if it added a step to your current workflow?The gap between appeal and willingness to pay a cost
Framekeeps 90% of your data cleanleaves 10% of your data uncleanSensitivity to gain versus loss framing
Reference periodin the last weekin a typical monthHow much your usage estimate depends on the recall window
Scale directionBest option firstWorst option firstOrder and recency effects, which you then randomize for good
Attribute labelreliabilityit works every time I need itWhether respondents share your vocabulary

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/split-ballot-what-to-split-on"><img src="https://www.koji.so/visuals/split-ballot-what-to-split-on/split-ballot-what-to-split-on.png" alt="Table of six split-ballot tests with two versions each: loaded term, consequence clause, frame, reference period, scale direction, attribute label." width="1200" height="899" /></a>
<p>Source: <a href="https://www.koji.so/visuals/split-ballot-what-to-split-on">What to split on in a split-ballot test</a>, Koji</p>