Synthetic Control: How to Build a Comparison Group That Does Not Exist (2026)

Synthetic control estimates what would have happened to one treated unit -- a market, a segment, an account tier, a single large customer -- by building a weighted blend of untreated units that closely reproduced its behaviour before the change, then reading the gap that opens afterwards. It is the design to reach for when you changed something for exactly one group, cannot randomise, and cannot find a single comparison unit that looks anything like the one you changed.

Most causal designs assume you can find a control group. Interrupted time series compares a unit to its own past. Difference-in-differences compares a treated group to an untreated group that was moving in parallel. Both are covered in our quasi-experimental design guide, and both fall apart in the situation product teams hit most often: you launched in one country, or repriced one plan, or gave one enterprise account a dedicated success manager, and nothing else in your business looks like the thing you changed.

Synthetic control answers the question that guide deliberately leaves open -- what to do when no comparable control group exists. The answer is that you build one.

The core idea

You have one treated unit and a set of untreated units, called the donor pool. Instead of choosing one donor as the comparison, you assign each donor a weight between zero and one, with the weights summing to one. The weights are chosen so that the weighted average of the donors reproduces the treated unit's pre-intervention outcome path and its key predictors as closely as possible.

That weighted average is the synthetic control. After the intervention, you keep applying the same fixed weights to the donors and compare the resulting synthetic trajectory to what the treated unit actually did. The vertical distance between the two lines is your estimate.

The discipline is that the weights are fixed using only pre-intervention data, and then never touched. You are not fitting the post-period. You are asserting that a blend which tracked the treated unit for years would have kept tracking it, and letting the data show how far apart they drift.

The canonical worked example

The method's defining application is Abadie, Diamond and Hainmueller's 2010 study in the Journal of the American Statistical Association of California's Proposition 99, a large tobacco control programme passed in 1988. It is worth walking through because every design decision maps onto a product problem.

They began with all US states, then discarded from the donor pool every state that had adopted its own large-scale tobacco control programme during the study window, along with several others, leaving a donor pool of 38 states. This is the step teams skip: a donor that received a similar treatment is not an untreated unit, and including it shrinks your estimated effect toward zero.

The optimisation then concentrated the weights on a handful of donors. Pre-Proposition 99 smoking trends in California were best reproduced by a combination of Colorado, Connecticut, Montana, Nevada and Utah, and, in the authors' words, "all other states in the donor pool are assigned zero W-weights."

The estimated effect was substantial: cigarette consumption fell by an average of almost 20 packs per capita across 1989-2000, "a decline of approximately 25 percent," and by 2000 annual per-capita sales were about 26 packs lower than the synthetic California implied.

The comparison with conventional methods is the part practitioners should notice. An earlier regression-based analysis by Fichtenberg and Glantz had estimated a reduction of about 14 packs per year by 1997; the synthetic control estimate for the same year was 24 packs. Same policy, same period, a 71 percent larger effect. The choice of counterfactual is not a technical detail that rounds out in the end. It is most of the answer.

What the method demands from your data

Synthetic control is unusually honest about its own preconditions, because the pre-period fit is visible. If the blend cannot reproduce your treated unit's history, you can see that it failed before you ever look at the effect.

A long, stable pre-period. You need enough pre-intervention periods that a close fit is evidence rather than coincidence. Fitting five weekly points with a five-donor blend will succeed trivially and mean nothing.

A genuinely untreated donor pool. Every donor must have been unaffected by the intervention. In product terms: if you launched in the UK and your donor pool includes Ireland, where the same marketing campaign ran, the estimate is contaminated.

No spillover between treated and donors. If your treated market's change pulls demand out of a donor market, the donor is not a clean counterfactual -- it moved in the opposite direction because of your intervention. This is a serious problem in marketplaces and in any business where users can switch between the units you are comparing.

No anticipation. If the intervention was announced before it took effect, behaviour may shift during the pre-period you are using to fit the weights. Abadie and colleagues note explicitly that outcomes can react to an anticipated intervention. In product work, a pre-announced price rise is the classic case: purchases pull forward, and the pre-period is no longer clean.

The treated unit must be inside the range of the donors. The weights are constrained to be non-negative and sum to one, so the synthetic control is an interpolation, not an extrapolation. If your treated unit is the largest, fastest-growing, or highest-ARPU unit you have, no blend of smaller, slower donors can reproduce it. The authors frame this as a feature: the method "forces the researcher to demonstrate the affinity between the region exposed to the intervention of interest and the regions in the donor pool," which safeguards against estimating what they call "extreme counterfactuals" -- those falling far outside the convex hull of the data.

That constraint is a gift. A regression will happily extrapolate to a counterfactual with no support in your data and report a confident number. Synthetic control refuses, visibly, by failing to fit.

Inference without a p-value

The obvious objection is that you have one treated unit, so conventional standard errors do not apply. The answer is placebo tests, extended from Abadie and Gardeazabal's earlier study of the economic cost of terrorism in the Basque Country, where per capita income in a synthetic Basque Country without terrorism ran up to 12 percent higher than the actual region.

The procedure is mechanical. Take each untreated donor in turn, pretend it received the intervention on the same date, build a synthetic control for it from the remaining donors, and record its gap. Doing this across the pool gives you a distribution of gaps produced by nothing but noise and model error. If the real treated unit's gap is extreme relative to that distribution, the result is unusual.

This is permutation inference, and it has a property that suits product work: it is a direct empirical answer to "how often would we see a gap this big if we picked a unit at random?" You can hand a stakeholder the placebo chart and they can read it without knowing what a standard error is.

Two practical rules. First, discard placebo units whose own pre-period fit is poor, because a bad fit manufactures a large post-period gap that has nothing to do with any treatment. Second, with a donor pool of 38, the smallest achievable one-sided p-value is roughly 1 in 39 -- your pool size caps your resolution, so a small pool cannot produce a small p-value no matter how large the effect.

Where this fits against the alternatives

A note on naming, because it causes genuine confusion: a synthetic control has nothing to do with synthetic users or AI personas. Every number in a synthetic control is a real measurement from a real untreated unit. The method invents no data; it only decides how to weight data you already have. One is a reweighting of observed reality, the other is a simulation of respondents who do not exist.

What the gap cannot tell you

You now have a defensible number: the treated unit ran 8 percent above its synthetic counterpart, and placebo tests say a gap that size is rare. Your stakeholders will immediately ask why, and the method has nothing to say.

This is not a small limitation. A gap of the same size can be produced by more new users arriving, the same users buying more often, fewer users leaving, or a shift in mix toward higher-value segments -- and those four have completely different implications for whether you should roll the change out everywhere. Worse, the gap is silent about whether the effect will persist, because it cannot distinguish a durable behaviour change from a one-off pull-forward that will show up as a hole in next quarter's numbers.

Only people can answer that, and the design tells you exactly which people to talk to. You have a treated unit and a named, weighted list of donor units that behaved just like it until the change. That is a ready-made sampling frame, and defining it carefully is the same discipline as defining a survey universe.

The modern approach: closing the mechanism gap

The historical reason teams stop at the number is cost. Running interviews in a treated market and three donor markets, in the local language, fast enough to matter to the rollout decision, was a multi-week research programme that most teams could not justify against a metric they had already moved.

This is where AI-moderated research changes the economics. With Koji you can field the same interview guide simultaneously in the treated unit and in the donor units that carry the weight, and get to analysis in hours rather than weeks. Traditional survey tools like SurveyMonkey give you closed-ended answers to questions you wrote before you knew what happened; an AI moderator probes the answer it actually receives, which is what you need when you are hunting for a mechanism you have not yet guessed.

The design that fits is a hybrid. Use structured questions for the parts you need to compare cleanly across markets -- Koji supports six types, and a well-built mechanism study typically uses most of them:

• single_choice to classify what changed for each respondent
• scale to size the strength of the shift on a common metric across markets
• yes_no to test one crisp discriminating fact, such as whether they noticed the change at all
• multiple_choice to capture which alternatives they considered
• ranking to force a priority order among candidate drivers
• open_ended, with AI follow-up, to surface the mechanism nobody on the team hypothesised

Then let automatic thematic analysis do the comparison you actually care about: which themes appear in the treated unit and are absent from the donor units. Themes present in both are background. Themes unique to the treated unit are your candidate mechanism, and they arrive attached to quotes you can put in front of a decision-maker.

A useful sequencing rule: decide the donor pool and fit the weights before fielding any interviews. The weights tell you which units are your real comparison, and interviewing the wrong markets is an expensive way to learn nothing.

A working checklist

1. Name the treated unit and the exact intervention date.
2. Build the donor pool and write down why each excluded unit was excluded.
3. Confirm the pre-period is long enough and free of anticipation effects.
4. Fit the weights on pre-period outcomes and predictors only. Report them.
5. Inspect the pre-period fit. A poor fit means stop, not proceed carefully.
6. Read the post-period gap and plot it over time, not as a single number.
7. Run placebo tests on every donor and place your gap in that distribution.
8. Check for spillover between the treated unit and the weighted donors.
9. Interview in the treated unit and the high-weight donors to recover the mechanism.
10. Report the gap, the weights, the placebo chart, and the mechanism evidence together.

Steps 1 through 8 give you a number your analytics team will defend. Step 9 is what makes the number actionable, and it is the step almost everyone skips.

Frequently asked questions

What is the synthetic control method in plain language?

It builds a comparison group by blending several untreated units together in whatever proportions best reproduce the treated unit's history. If no single market resembles yours, a weighted mix of five markets often does. The blend is then projected forward and the gap between it and the real unit is the estimated effect.

How much pre-intervention data do I need?

Enough periods that a close pre-period fit is meaningful rather than coincidental. Abadie, Diamond and Hainmueller used 19 years of annual data before the intervention. For weekly product metrics, aim for a pre-period several times longer than the post-period you plan to evaluate, and treat a short pre-period as a reason to distrust the fit.

How is this different from just picking the most similar market?

A single comparison unit is chosen by judgement and usually differs from the treated unit on level, trend, or both. A synthetic control is chosen by an explicit optimisation over a declared donor pool, reports the weights it used, and can be checked against the pre-period data. It makes the comparison auditable instead of assertable.

Is synthetic control the same as synthetic users or AI personas?

No, and the naming collision causes real confusion. A synthetic control is a weighted average of real, observed, untreated units and their real measured outcomes. Synthetic users are model-generated respondents with no underlying person. Synthetic control invents nothing; it only reweights data you already collected.

How do I know the gap is not just noise?

Use placebo tests. Reassign the intervention to each untreated unit in turn, build a synthetic control for it, and record the gap. If the real treated unit's gap sits far outside the distribution of placebo gaps, the result is unusual in a way chance rarely produces. This is permutation inference, not a t-test.

What can synthetic control never tell me?

Why the gap exists. The method yields a magnitude and a trajectory, not a mechanism. Two rollouts with identical gaps can be driven by entirely different user behaviour, and only talking to people in the treated and donor units distinguishes them.

Related Resources

• Quasi-Experimental Design -- interrupted time series, regression discontinuity, and difference-in-differences, the designs to try before this one
• The Stepped Wedge -- how to randomise a rollout you were told could not be randomised
• Propensity Score Matching and the Pruning Paradox -- why the obvious way to build a comparable group can make things worse
• Instrumental Variables and Encouragement Designs -- what to do when comparability is unreachable
• Structured Questions in AI Interviews -- the six question types and when each one earns its place
• Regression to the Mean -- the reason your intervention looked like it worked before you built any counterfactual