Research Decision Lag: Why Acting on Last Quarter Insight Makes the Metric Worse (2026)

Short answer: the delay between measuring something and being able to see the effect of your response to it is usually seven months or more. Control engineering calls that delay dead time, and it has a well-understood consequence: once the delay is a large fraction of the cycle you are trying to correct, your corrections arrive with the wrong sign and amplify the swing instead of damping it. The prescribed fix is counterintuitive and load-bearing - you must respond less aggressively, or explicitly subtract your own previous action from the reading before you respond at all.

This is not the question of whether your research has gone stale. A finding can be completely fresh and completely true and still produce this failure. The problem is not the age of the insight; it is the length of the loop.

Dead time, and why it is the enemy

The definition, from Douglas Cooper of Control Guru: "Dead time is the delay from when a controller output (CO) signal is issued until when the measured process variable (PV) first begins to respond." The verdict on it is unambiguous - dead time is "never a good thing in a control loop", and as it grows, "the control challenge becomes greater and tight performance becomes more difficult to achieve."

Dead time is worse than simple sluggishness. A system that merely responds slowly still responds in the right direction immediately. Dead time means that for a period after you act, nothing happens at all - and during that window the natural instinct is to conclude the action was too small and do more.

Every research-driven product decision is a control loop with dead time. You measure a customer problem, you act, and the measurement of whether you were right arrives much later. The loop exists whether or not anyone has drawn it.

Add up your own loop

Most teams have never summed their own dead time. The numbers are unwelcome but not controversial.

A realistic research-to-evidence loop

Roughly seven months from "this was true of our customers" to "we can see whether what we did about it helped." And that is the optimistic reading, because two further lags stack on top:

• The study describes the state of the world at the start of fielding, not the end.
• If the metric you watch is a rolling average, it lags by about half its window on top of everything else - a 12-week trailing average adds five and a half weeks.

A team can comfortably be steering on evidence nine months old while believing the loop is quarterly.

The band where your correction has the wrong sign

Here is why length alone is not the whole story. What determines whether a delayed correction helps or hurts is the delay relative to the period of the thing you are correcting.

Express the loop delay as a fraction of the cycle length. If that fraction is small, your correction lands while the situation still resembles what you measured, and it helps. But once the fraction passes about a quarter of a cycle, your correction starts arriving after the metric has already begun moving on its own - and between roughly 0.25 and 0.75 of a cycle, your correction pushes in the direction the metric is already going more often than it opposes it. At exactly half a cycle the correction is perfectly out of phase: you brake precisely when you should accelerate.

Now put the numbers together. Annual seasonality has a period of 52 weeks. A 31-week loop is 31/52 = 0.60 of a cycle - squarely inside the amplifying band, and not far from the worst possible value of 0.5. A conventional enterprise research loop is close to optimally tuned to fight last year season.

This is the mechanism behind a pattern every experienced product person recognises without having a name for it: the initiative that lands just as the problem it addressed had already resolved itself, and whose apparent effect is to make the next quarter look worse.

What the measured experiments show

This is not a theoretical worry. It has been measured in controlled experiments for decades, and the results are dramatic.

The beer game

John Sterman of MIT built the Beer Distribution Game to study exactly this: how people manage a system with delays between ordering and receiving. The findings from his MIT materials:

• "Average costs are ten times greater than optimal!"
• "Orders and inventories are dominated by large amplitude fluctuations, with an average period of about 20 weeks."
• "The amplitude and variance of orders increases steadily from customer to retailer to factory."
• "The peak order rate at the factory is on average more than double the peak order rate at retail."

The diagnosis is the part that transfers directly: "Most people do not account well for the impact of their own decisions on their teammates - on the system as a whole. In particular, people have great difficulty appreciating the multiple feedback loops, time delays and nonlinearities in the system, using instead a very simple heuristic to place orders."

Note that the players are not short of data. They can see their own inventory and their own orders precisely. Intelligent, motivated, fully-informed participants produce ten times the optimal cost because of the delay structure alone. Better information does not fix a delay problem.

The bullwhip effect

The same structure was documented in industry by Hau Lee, V. Padmanabhan and Seungjin Whang in MIT Sloan Management Review. Procter and Gamble found that "While the consumers, in this case, the babies, consumed diapers at a steady rate, the demand order variabilities in the supply chain were amplified as they moved up the supply chain." The general law: "the variabilities of an upstream site are always greater than those of the downstream site."

And the clincher for our purposes: "Because the amount of safety stock contributes to the bullwhip effect, it is intuitive that, when the lead times between the resupply of the items along the supply chain are longer, the fluctuation is even more significant." Longer delay, bigger swings. That is dead time causing oscillation, observed in a business rather than a chemical plant.

Read the P&G finding as a statement about product research and it is uncomfortably familiar: customer needs change slowly and steadily; roadmaps thrash. The amplification is not in the customers. It is in the loop between them and you.

The attribution trap: you are measuring your own last move

There is a subtler consequence, and it is the one that makes this more than a speed problem.

Two explanations, one number

By the time your next measurement arrives, the thing you measure is not the world. It is the world plus your own previous intervention, and the two arrive inseparably in a single number. If the metric improved, two complete explanations fit equally well:

1. Your fix worked.
2. The cycle turned on its own, and your fix did nothing (or did harm that the upswing masked).

Nothing in the measurement distinguishes them, because you changed the system and the system moved, once, at the same time. This is not a sampling problem that a larger study would resolve - it is an identification problem created by the structure of the loop. And it compounds: the more frequently you intervene, the more thoroughly each measurement is a mixture of your interventions, and the less any of them can be attributed.

Which produces the genuinely perverse result: a team that acts on research every quarter can end up knowing less about what works than a team that acts twice a year and leaves a clean window in between. The activity destroys the evidence that would justify it.

What control engineers do about dead time

The engineering literature is clear that dead time cannot be removed by trying harder. There are exactly three responses, and only one of them is the instinct most teams have.

1. Detune - respond less aggressively

The standard remedy is to reduce the gain of the controller: make smaller corrections than the error appears to justify. This is the direct opposite of "the research was clear, so let us commit hard." With a long loop, a confident large move is the thing that produces oscillation. Smaller moves, more often, beat one decisive move per cycle - not because conviction is bad, but because you will not find out you were wrong for seven months.

2. Predict - subtract your own last action first

The more sophisticated remedy is a predictor: a model of what your outstanding actions will have done, so the controller corrects against the predicted state rather than the measured one. The research equivalent is concrete and cheap. Before reacting to a new number, write down explicitly what you expected your last shipped change to do to it, and by how much. Then compare the measurement to that prediction rather than to the previous measurement. If you expected +3 and got +4, the news is +1, not +4. Almost nobody does this, and it is the single highest-value habit in this article.

3. Shorten - attack the largest term

Detuning and predicting manage the delay. Only shortening removes it. And here the audit matters, because the instinct is to compress the build, which is rarely the largest term. In the table above, fielding plus analysis plus waiting for the next wave is 18 of the 31 weeks - well over half the loop, and the part that has historically been hardest to compress and is now the easiest. That is the term an AI-native platform like Koji removes outright rather than merely compressing.

A loop-delay audit you can run this week

Six steps

1. Pick one decision your team made from research in the last year.
2. Date every stage: when fielding started, when the report landed, when the decision was taken, when it shipped, when you next measured. Use real dates.
3. Sum the weeks. Most teams are surprised by 30 to 50 percent.
4. Identify the largest single term. It is usually not build. In the table above it is the wait for the next measurement wave.
5. Compute the ratio of your total delay to the period of the cycle you care about. If it falls between 0.25 and 0.75, your corrections are currently amplifying.
6. Write down one prediction for the next change you ship, with a number and a date, before you ship it. That single sentence converts your next measurement from ambiguous into informative.

How Koji handles this

Since fielding, analysis and waiting for the next wave make up the majority of the delay, that is where the leverage is - and it happens to be the part that AI-native research actually removes rather than merely accelerating.

• Fielding in days instead of weeks. AI-moderated interviews run in parallel and on the participant schedule, so a study that took three weeks to field does not need a moderator calendar at all. Twenty interviews can complete in a day or two.
• Analysis that does not queue. Automatic thematic analysis produces themes, prevalence and supporting quotes as interviews complete, which deletes most of the two-week analysis-and-reporting stage. Real-time reporting means there is no separate step where a report is assembled.
• No next wave to wait for. This is the big one. The 13-week wait exists because studies are discrete, expensive events. When research runs continuously, the post-change measurement starts the moment the change ships, which removes the single largest term in the table.
• Structured questions make the prediction test possible. Comparing a measurement against what you predicted requires the same question asked the same way. The six structured question types - open_ended, scale, single_choice, multiple_choice, ranking, and yes_no - give you a stable numeric series to state predictions against, rather than prose you have to re-interpret each time.
• Customisable AI consultants keep the instrument constant. Because the interviewer is configured rather than improvised, the loop does not acquire new measurement variation every time you close it.
• Voice and text interviews reach people faster, which is what makes a days-long fielding window realistic rather than aspirational.

The contrast with legacy tooling is not about polish, it is about loop length. While traditional panel and survey platforms are built around discrete waves - and therefore quietly impose the 13-week wait that dominates the delay budget - continuous AI-moderated collection turns a seven-month loop into a several-week one. Below about a quarter of a cycle, delayed corrections start helping instead of amplifying, which means shortening the loop does not just speed up learning; it changes the sign of your interventions. That is why Koji treats continuous collection as the default rather than a premium feature, and why teams do not need a background in control theory to benefit from it.

Common mistakes

• Treating a slow loop as an urgency problem. Telling the team to move faster on a loop whose largest term is waiting for the next study does not shorten it.
• Responding harder when nothing happens. During dead time, nothing happening is the expected behaviour, not evidence that the intervention was too small.
• Comparing the new number to the old number. Compare it to what you predicted. Otherwise you are reading your own last move and calling it news.
• Shipping several changes into one measurement window. Every additional simultaneous change makes the eventual number less attributable, permanently.
• Confusing this with insight decay. Decay asks whether a finding is still true. This asks whether your response to a true finding arrives in phase. A perfectly fresh finding, acted on through a long loop, still oscillates.
• Assuming more data fixes it. The beer game participants had complete information about their own state and still hit ten times optimal cost. Delay is a structural problem, not an information problem.
• Optimising the build while the waiting dominates. Compressing a six-week build to four weeks removes about 6 percent of a 31-week loop. Removing the wait for the next wave, which is what continuous collection in Koji does, removes about 42 percent of it. Teams routinely spend a quarter arguing about the first and never name the second.

The bottom line

Add up your loop before you tune your conviction. If the total delay from measurement to re-measurement is a substantial fraction of the cycle you are steering, then acting decisively on each reading is what produces the thrash, and the swings will grow with the length of the delay rather than with the size of your mistakes. Do the three things engineers do: make smaller moves, state a numeric prediction so the next reading carries information rather than your own echo, and attack the largest term in the delay budget - which is almost never the build, and almost always the waiting.

Frequently asked questions

What exactly is dead time in a research context?

It is the total elapsed time from the moment a measurement describes the world to the moment you can observe the effect of your response to it. That includes fielding, analysis, decision-making, build, adoption ramp, and critically the wait for the next measurement. In control engineering it is the interval after you act during which nothing observable happens, which is precisely the interval in which teams wrongly conclude their action was too small.

How do I know whether my loop is amplifying rather than correcting?

Divide your total loop delay by the period of the cycle you are trying to influence. If the ratio falls roughly between 0.25 and 0.75, your corrections arrive while the metric has already moved on and will tend to push in the direction it is already going. A 31-week loop against annual seasonality gives 0.60, which is inside the amplifying band and close to the worst case of 0.5.

Is not the answer simply to act faster and more decisively?

No, and this is the central counterintuitive point. With a long delay, acting more decisively is exactly what generates oscillation, because you will not learn you overshot for months. The engineering remedies are to detune - make smaller corrections than the error seems to warrant - or to predict what your outstanding actions will do and correct against that. Decisiveness is only safe once the loop is short.

How is this different from insight decay or stale research?

Decay is about whether a finding is still true, and its remedy is to re-run the study. This is about whether your response arrives in phase with the thing you measured, and it bites even when the finding is perfectly fresh and perfectly accurate. Two teams with identical, equally current research can get opposite outcomes purely because one has a shorter loop.

Why does acting on research more often sometimes make learning worse?

Because each measurement captures the world combined with all your outstanding interventions, and they arrive as a single inseparable number. The more often you intervene, the more thoroughly mixed each measurement becomes, and the less any individual change can be attributed. Leaving a clean window with no changes in it is sometimes worth more than another initiative, which is why stating an explicit numeric prediction before shipping is so valuable.

How does Koji shorten the loop?

Koji attacks the two largest terms directly. AI-moderated interviews field in days rather than weeks because they need no moderator scheduling, and automatic thematic analysis with real-time reporting removes most of the analysis-and-reporting stage. Most importantly, continuous collection eliminates the wait for the next discrete wave, which is typically the single biggest component of the delay - so post-change measurement begins the moment a change ships.

Related Resources

• How Long Is User Research Valid? - the companion question of whether a finding is still true, which is distinct from whether your response arrives in phase.
• Why Your Quarterly Metric Shows a Trend That Is Not There - how the interval between measurements can manufacture the very cycle you are reacting to.
• The Moving Average Is Hiding the Week That Mattered - the extra lag a smoothed metric adds on top of the loop delay.
• Closed-Loop Feedback - closing the loop with customers, the communication counterpart to the measurement loop described here.
• Customer Interview Cadence - how often to talk to users, which sets the floor on how short your loop can be.
• Structured Questions Guide - the six question types that give you a stable series to state predictions against.