All frameworks and cheat sheets

Flowchart

Is this study worth running?

Flowchart for deciding whether a study is worth running: four yes or no questions on switch point, open decision, reversibility and range, then run it.
Four questions, asked before anyone talks about sample size.
Download PNGRead the full guide

A decision flowchart based on value of information analysis. After writing down the decision, options, switch point and cost of being wrong, it asks whether any result would change the action, whether the decision is still open, whether it is costly to reverse and whether your honest range of answers straddles the switch point. Any no means the study is not worth running. Researchers and product teams can use it to turn down low-value requests with a clear reason.

Where this appears

What's in this flowchart

  1. Write down the four inputs. The decision, the options, the switch point and the cost of being wrong.
  2. Would any realistic result change what you do?
    • Yes: go to 3.
    • No: Skip the study. Its value is zero, however big the decision.
  3. Is the decision still open?
    • Yes: go to 4.
    • No: Skip it: the call is made. A study due after the decision deadline cannot inform it.
  4. Would reversing the decision be costly?
    • Yes: go to 5.
    • No: Ship it and measure. When rollback costs less than the study, shipping is the experiment.
  5. Does your honest range straddle the switch point?
    • Yes: go to 6.
    • No: Act on what you know. No realistic answer crosses the line, so fieldwork adds nothing.
  6. Finish: Run the study. Then decide how much rigour it deserves.

Use this image

Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:

html
<a href="https://www.koji.so/visuals/is-this-study-worth-running-flowchart"><img src="https://www.koji.so/visuals/is-this-study-worth-running-flowchart/is-this-study-worth-running-flowchart.png" alt="Flowchart for deciding whether a study is worth running: four yes or no questions on switch point, open decision, reversibility and range, then run it." width="1200" height="1343" /></a>
<p>Source: <a href="https://www.koji.so/visuals/is-this-study-worth-running-flowchart">Is this study worth running?</a>, Koji</p>

Related

Table of the four inputs for a value of information check: the decision, the options, the switch point and the cost of being wrong, with how each one fails.

Value of information: four inputs to check before a study

Flowchart for choosing a survey or an interview: three yes or no questions, two interview detours and one exit, ending in a report of what and why.

Survey or interview? A decision flowchart

Flowchart for deciding whether a pre-launch price is ready to publish: three questions about the peak, the buyer and a deposit, with one side step.

Is your pre-launch price ready to publish?