Error Propagation in Research Metrics: What Happens to Uncertainty When You Combine Numbers (2026)

Answer first: when you build a research number out of other numbers — a gap, an index, a rate, a cost-per-insight — the uncertainty of the inputs does not carry over unchanged. It travels through the formula according to a rule, and the rule treats addition, subtraction, multiplication and division very differently. Averaging four sub-scores makes the result more precise than any component. Subtracting two averages can make the result so imprecise that its sign is undetermined. Both outcomes come from the same equation. This guide gives you that equation, three worked examples with real numbers, and a way to run it on the metrics your team argues about every week.

The formal name is the law of propagation of uncertainty. Metrology has treated it as mandatory since 1993; product research has largely never met it. That gap is why a roadmap gets re-prioritised over a 0.13-point difference that a statistician would refuse to sign.

The rule, in one line

Every derived metric is a function: you feed it inputs, it returns an output. The uncertainty of the output is the sum, in quadrature, of each input's uncertainty multiplied by how strongly the output responds to that input.

"In quadrature" means: square them, add the squares, take the square root. It is Pythagoras, not simple addition, because independent errors partly cancel rather than always stacking.

The "how strongly the output responds" part has a name. The Guide to the Expression of Uncertainty in Measurement (JCGM 100:2008), the standard published jointly by the BIPM, ISO, IEC and four other bodies, calls these partial derivatives sensitivity coefficients, and says they "describe how the output estimate y varies with changes in the values of the input estimates." They are the multipliers. A sensitivity coefficient of 1 passes error through untouched. A coefficient of 8 multiplies it eightfold, and no amount of care in collecting the input will undo that.

That is the whole subject. Everything below is the rule applied to shapes you actually report.

The four combinations and what each one does

The two rows that matter most are the first two, and the difference between them is the source of nearly every reporting mistake in this article and the next.

For sums and differences you combine absolute uncertainties. If importance is measured to plus/minus 0.08 and performance to plus/minus 0.08, the gap between them is uncertain by the square root of (0.08 squared + 0.08 squared) = 0.113 — and that 0.113 is attached to a gap that might only be 0.13 points wide.

For products and ratios you combine relative uncertainties. If your research spend is known to 3% and your count of validated insights to 11%, then cost-per-validated-insight is known to the square root of (3% squared + 11% squared) = 11.3%.

The NIST/SEMATECH Engineering Statistics Handbook tabulates these cases explicitly, including the versions with a covariance term, and attaches a warning worth internalising: "Covariance term is to be included only if there is a reliable estimate."

A worked example a statistical agency actually published

The U.S. Census Bureau publishes the propagation rule inside its own user handbook, because ACS users constantly build derived estimates and get the error wrong.

Their worked case: 74,506,512 owner-occupied housing units, multiplied by the estimated proportion that are 1-unit detached, 0.824 with a margin of error of 0.001. The product is 61,393,366 units. Applying the propagation formula for a product gives a margin of error of 202,289, a standard error of 122,972, and a coefficient of variation of 0.2%.

Two things are worth noticing. First, the agency does not eyeball this — it runs the formula. Second, the CV is the payoff number. The handbook defines it plainly: "A coefficient of variation (CV) measures the relative amount of sampling error that is associated with a sample estimate." Relative error is what tells you whether a number is worth a sentence in a readout, and it is the only form of error that survives comparison across metrics on different scales.

The well-conditioned case: your index is safer than its parts

Averaging is the friendliest thing you can do to uncertainty, and most research teams under-use it.

Take a four-dimension experience index — say ease, speed, trust and support — each on a 1-5 scale, equally weighted:

The composite is more precise than every single input that went into it — 1.17% relative error against a component average of 2.35%. The amplification factor here is exactly 1.0: an equal-weight average is the best-conditioned formula in common use. If you have four noisy sub-measures and one decision to make, the average is the number to put on the slide, and the components belong in an appendix.

Note the asymmetry this sets up. The same law of propagation that makes an average safer than its parts makes a difference riskier than its parts. Nothing changes in the mathematics between those two cases except the sign in front of the second term.

Find the dominant term before you spend a dollar

Because the terms add in quadrature, they do not contribute equally. Squaring is merciless to small numbers.

Return to cost-per-validated-insight: 48,000 dollars of research spend, plus/minus 1,500 (3.1% relative), divided by 37 validated insights, plus/minus 4 (10.8% relative).

• Result: 1,297 dollars per validated insight
• Combined relative uncertainty: 11.3%
• Absolute uncertainty: plus/minus 146 dollars

The spend contributes 3.1% and the count contributes 10.8%. Squared, that is 9.6 against 116.8 — the count accounts for 92% of the total variance and the spend for 8%. Tightening your finance reconciliation from plus/minus 1,500 to plus/minus 500 moves the combined uncertainty from 11.3% to 10.8%. Tightening the definition of "validated insight" so the count is reliable to plus/minus 1 moves it to 4.2%.

This is the practical use of the rule. Before anyone invests in better instrumentation, run the propagation and find out which input is carrying the variance. It is almost never the one people are arguing about.

When the inputs are not independent

Everything above assumes the input errors are unrelated. Often they are not, and the GUM devotes a separate clause (5.2, "Correlated input quantities") to the case, because correlation can push the answer either way.

Two situations turn up constantly in research:

• Two figures from the same respondents. If you compute a Net Promoter Score, the promoter share and detractor share come from the same multinomial sample and are negatively correlated. That correlation makes the variance of the difference larger, not smaller. Ignoring it understates the confidence interval by roughly a fifth — worked out in full in Every Input Was Accurate and the Difference Was Not.
• Two waves with an overlapping panel. If half your Q2 respondents also answered in Q1, the wave-over-wave change is more precise than the independent formula suggests, because respondent-level idiosyncrasy cancels. Here the correlation works in your favour, and treating the waves as independent throws away real precision.

The rule: same people, same instrument, or same weighting scheme means check for correlation before you assume quadrature.

What this changes about how you report

Four changes, all cheap:

1. Report the relative uncertainty of derived numbers, not just the levels. A CV above roughly 25-30% is the conventional line at which statistical agencies stop treating an estimate as publishable on its own. Adopt a house threshold and apply it.
2. Never report a derived metric to more digits than its uncertainty supports. If the gap is 0.13 plus/minus 0.22, "0.13" is three characters of false confidence. Related, at the instrument level: measurement system analysis gives the complementary rule for how many digits a single instrument can resolve at all.
3. Publish the sensitivity coefficients alongside the metric definition. One line — "a 1-point move in the count moves the result 3.5 times as much as a 1-point move in the spend" — retires most metric arguments before they start.
4. Decide the formula before the fieldwork. The amplification is fixed by the definition, not by the data. See You Cannot Sample Your Way Out of a Badly Conditioned Metric.

How Koji helps

Propagation analysis stalls in most teams for a mundane reason: nobody has the component-level uncertainty to feed into it. You have one number from one survey, no repeat measurement, and no way to estimate the standard error of a theme frequency at all.

Koji removes that blocker in three ways.

The inputs arrive quantified. Koji's structured questions run inside an AI-moderated interview rather than as a separate survey instrument, across six types — open_ended, scale, single_choice, multiple_choice, ranking and yes_no. A scale or ranking question yields a mean and a dispersion on every run, so the standard error of each component is a by-product of fieldwork rather than a special study. Legacy tools like SurveyMonkey or Typeform will give you the same scale data, but you then need a separate qualitative round to learn why, and the two samples are not the same people — which quietly breaks the correlation assumptions above. Koji collects both in one conversation with one respondent, so the covariance term is estimable.

The sample is large enough to estimate error, because it is cheap. Propagation needs decent component precision, and component precision needs n. A traditional moderated study of 40 people is four weeks of scheduling; Koji runs interviews in parallel, so the quantification phase closes in days and there is no reason to stop at the smallest n you can defend.

The derived metric is recomputed as data lands. Because reporting is real-time, you watch the confidence interval on the derived number contract as interviews complete, and you stop when the number is precise enough to act on rather than when the calendar runs out. That is a different stopping rule from "we booked 40," and it is the one propagation analysis actually implies.

The reasoning is auditable, not a black box. Koji's customisable AI consultant can be told which metrics matter and asked to report the components and their dispersions alongside the headline, so the propagation inputs appear in the readout rather than in a separate analyst's spreadsheet. Teams adopting AI-assisted research consistently report reaching insight materially faster than with manual synthesis, and the reason is mundane: the aggregation step that used to be a person with a spreadsheet is now part of the pipeline, which is also where an uncertainty calculation belongs.

You do not need a metrology background, or a PhD in research methods, to run this. You need the component standard errors and one line of arithmetic per metric — and with Koji the first of those arrives for free.

Frequently asked questions

What is error propagation in simple terms?

It is the rule for working out how uncertain a calculated number is, given how uncertain its ingredients are. Add or subtract two numbers and their absolute uncertainties combine in quadrature. Multiply or divide them and their relative uncertainties combine in quadrature. Average several numbers and the relative uncertainty falls. The formal statement is the law of propagation of uncertainty in JCGM 100:2008.

Why do you add errors in quadrature instead of just adding them?

Because independent errors are as likely to partly cancel as to reinforce. Straight addition assumes every error goes the same way at the same time, which is a worst case rather than an expectation. Quadrature — square, sum, square-root — gives the standard deviation of the combination when the inputs are independent. If the inputs are correlated, quadrature is wrong and you need the covariance term.

How do I know which input is hurting my metric most?

Compute each input's contribution as (sensitivity coefficient × input uncertainty), then square it. Compare the squares, not the raw contributions. Because of the squaring, an input contributing three times more uncertainty than another accounts for nine times more variance, so the largest term usually dominates completely and the small ones can be ignored.

Does a bigger sample always fix a noisy derived metric?

No. Sample size shrinks the uncertainty of each input, roughly with the square root of n. It does nothing to the sensitivity coefficients, which are set by the formula. If your metric multiplies input error by 65, quadrupling the sample halves the input error and the output error is still 65 times whatever remains. Badly conditioned metrics need a new definition, not a bigger sample.

What relative uncertainty is acceptable for a research metric?

There is no universal threshold, and the Census Bureau explicitly declines to set one, noting that data users must evaluate each application. As a working rule: under 10% relative uncertainty is decision-grade, 10-25% is directional, above 25-30% should not be reported as a point estimate at all. Set the threshold before you see the number.

Can I apply this to qualitative findings like theme frequencies?

Yes, with care. A theme frequency is a proportion, so it has a standard error like any proportion, and differences between theme frequencies propagate exactly as described here. What propagation cannot fix is coding error — if two analysts tag the same transcript differently, that is instrument variance and belongs to measurement system analysis, not to this rule.

Related Resources

• Structured Questions in AI Interviews — the six question types that give every component a mean and a dispersion, so propagation has inputs at all.
• Every Input Was Accurate and the Difference Was Not — what happens when the formula is a subtraction.
• You Cannot Sample Your Way Out of a Badly Conditioned Metric — why amplification is a property of the definition.
• Measurement System Analysis — the complementary question: how many digits can one instrument resolve.
• Margin of Error in Surveys — the single-proportion case, which is where all of this starts.
• Statistical Power and Minimum Detectable Effect — sizing a study for the effect you need to see.