Changing a Study While It Is Running: Pre-Planned Adaptations vs Protocol Amendments

Answer first: a change made to a running study is either an adaptation you planned before you saw any comparative results, or an amendment you made because of what you saw. The first is a design feature with known statistical properties. The second is a new study wearing the old study's sample. The FDA draws exactly this line. Its 2019 guidance Adaptive Designs for Clinical Trials of Drugs and Biologics defines an adaptive design as "a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial," and specifies that prospective "means that the adaptation is planned and details specified before any comparative analyses of accumulating trial data are conducted."

This is the capstone of the running-study cluster, because every discipline in the other three guides assumes the instrument holds still. Interim analysis prices your looks against a fixed primary outcome. Futility rules assume the thing being judged futile is the thing you started. Analysis populations assume everyone answered the same questions. Change the study mid-field and all three quietly stop applying - and unlike stopping early, nobody flags it, because changing a question feels like an improvement rather than a decision.

The three kinds of mid-study change

The distinction that does all the work is not what changed. It is what you knew when you changed it.

Changes you can make freely, because they are blind to the answer

These depend on information that does not involve comparing groups on the outcome, so they cannot be motivated by the result:

• Fixing a typo or a broken option in a single_choice list.
• Adding a screener criterion because you discovered the recruitment source is contaminated with the wrong population.
• Extending fielding because response pace is below target.
• Adjusting an incentive because completion rate is low.
• Re-estimating sample size from pooled variance without splitting by group - the FDA guidance treats this as an adaptation based on non-comparative data, which is why it is the least controversial adaptation there is.

The test to apply: could the change have gone the other way if the results had been reversed? If a stranger who has seen no outcome data would make the same change, it is blind. Log it and continue.

Changes that need a pre-planned rule

These are legitimate adaptations, but only if the rule was written before any comparative look. In clinical trials the standard examples are sample size re-estimation, dropping a treatment arm, adaptive population selection and adaptive randomization. In research the equivalents are:

• Adding a segment when a pre-specified subgroup fills below target.
• Dropping a concept from a multi-concept test when it fails a pre-declared screening threshold.
• Extending or truncating fielding against a pre-declared futility or efficacy rule.
• Escalating probing depth on a question when a pre-declared proportion of responses come back uninformative.

The FDA guidance explains precisely why the rule has to be written in advance rather than merely applied honestly: complete prespecification "helps increase confidence that adaptation decisions were not based on accumulating knowledge in an unplanned way," and specifying the exact rule "reduces concern that the adaptation could have been influenced by knowledge of comparative results." Good faith is not the mechanism. The written rule is.

Changes that split the study in two

Rewording the primary question. Changing the scale from 1-5 to 1-7. Swapping a yes_no item for a multiple_choice one. Adding a leading follow-up because early responses hinted at something. Restricting analysis to a subgroup that looked promising at the halfway mark.

The FDA guidance describes this scenario exactly: when data are examined in a comparative interim analysis, analyses that were not prospectively planned "may unexpectedly appear to indicate that some specific design change (e.g., restricting analyses to some population subset, dropping a treatment arm, adjusting sample size, modifying the primary endpoint, or changing analysis methods) is ethically important or might increase the potential for a statistically significant final trial result." The consequence: "Such revisions based on non-prospectively planned analyses can create difficulty in controlling the Type I error probability and in interpreting the trial results. Sponsors are strongly discouraged from implementing such changes without first meeting with FDA."

You do not have a regulator to meet with. What you have instead is a reporting obligation. If the instrument changed on the basis of comparative results, the honest analysis treats pre-change and post-change respondents as two studies. You can report both. You cannot pool them and describe the pooled number as one measurement, because the two halves were not measuring the same thing on the same population under the same conditions.

The integrity control that prevents most of this

Reactive amendments are almost always downstream of one root cause: the people who could change the study could also see the comparative results.

The FDA states the control directly: "it is strongly recommended that access to comparative interim results be limited to individuals with relevant expertise who are independent of the personnel involved in conducting or managing the trial and have a need to know." The reason given is worth quoting because it is not about honesty: knowledge of comparative interim results by trial management personnel "may make it difficult for regulators to determine whether a protocol amendment seemingly well-motivated by information external to" the trial actually was. In other words, once you have seen the results, you can no longer prove your reasonable change was not motivated by them - and neither can anyone else, including you.

The practical version for a product team is three rules:

1. The person who can edit a live study does not watch the comparative results.
2. Anyone who does watch comparative results has no edit rights while fielding is open.
3. If the roles must be combined - and in a small team they often must - every edit made after the first look is logged with a timestamp and a stated reason, and the readout reports it.

That third rule is the realistic one for most companies, and it is worth more than it looks. An amendment log turns an invisible problem into a visible one, and visible problems get argued about on the merits.

The amendment log

Six columns. Keep it with the brief, not in a chat thread.

A study with an empty amendment log and a study with no amendment log look identical in the readout and are completely different objects. Publish the empty one.

The AI-moderated case: adaptation in the probe is not adaptation in the design

This is the point where an honest guide has to turn the argument on its own tooling. An AI moderator that generates follow-up questions is, by any plain reading, changing the instrument during the study. Every participant gets a different conversation. So does the whole framework collapse?

No, but only because of a distinction that has to be maintained deliberately.

Adaptation in the probe means the follow-up varies with what the participant just said, under a fixed instruction set that was written before fielding. Participant 3 and participant 30 face the same questions, the same probing rules and the same depth limits; what differs is their own answers. This is what a good human interviewer does, and it is the reason interviews outperform static surveys.

Adaptation in the design means the question set, the primary outcome, the probing instructions or the eligibility rules change between participants because of what earlier participants said. That is a protocol amendment, and it is the thing the FDA definition rules out unless it was pre-planned.

The line between them is whether the rule changed or only the realisation of the rule. And the practical consequence is reassuring: an AI moderator applying a fixed instruction set is more protocol-compliant than a human moderator, not less, because the human drifts. A human running twenty interviews across three weeks probes harder on whatever seemed promising in week one - an unlogged, unnoticed protocol amendment that happens in every manual study and appears in none of their limitations sections. See model version drift for the one genuine AI-specific version of this problem, which is the underlying model changing beneath a study that has not changed at all.

How Koji keeps the design fixed while the conversation adapts

Structured questions are the fixed frame. The six question types - open_ended, scale, single_choice, multiple_choice, ranking and yes_no - define the instrument. A scale question asked at position 4 with the same anchors is the same measurement for participant 3 and participant 300, whatever the AI probed around it. That is what makes cross-participant aggregation legitimate at all. See the structured questions guide.

Probing instructions are set at design time, per question. Follow-up depth and probing guidance are configured before fielding rather than adjusted live, so the adaptive behaviour is a property of the design, not a running intervention.

Branching is pre-declared logic, not a mid-study edit. Conditional paths through an interview are a pre-planned adaptation in the exact FDA sense: the rule, the triggering data and the consequence are all specified before anyone is fielded. See adaptive AI interview branching.

Every conversation is retained with its status and transcript, so if an amendment does happen you can split the analysis cleanly at the change point rather than discovering that the pre-change responses were overwritten.

Traditional survey tools such as SurveyMonkey, Typeform and Qualtrics let you edit a live survey with no record, no warning and no way to reconstruct which respondents saw which version. Platforms like Koji do not make the underlying judgment for you, but a fixed structured frame plus a full conversation record is the difference between an amendment you can analyse around and one you can only apologise for.

Frequently asked questions

Can I fix a typo in a live study?

Yes. A typo fix is blind to the outcome - you would make it whichever way the results were going - so it costs nothing statistically. Log it with a timestamp anyway, because the log is what lets you prove it was that kind of change rather than the other kind.

What if a question is clearly confusing people mid-field?

If you noticed from operational signals - drop-off at that question, response times, participants saying they do not understand - that is a non-comparative observation and you can act on it. Fix the question, log it, and treat responses before and after as two versions of that item. Do not pool them into a single distribution. If you noticed because the answers were not supporting your hypothesis, that is a reactive amendment and a much bigger problem.

Does adding a new segment mid-study invalidate the results?

Not for the original segment, whose data is unaffected. The new segment is a later, smaller, separately recruited sample and should be reported as such, with its own fielding dates. The failure mode is pooling them and reporting one overall number, because the two groups were fielded under different conditions at different times.

How is this different from an adaptive AI interview?

An AI moderator varies the follow-up questions under a fixed instruction set written before fielding, which is adaptation in the probe. A protocol amendment changes the question set, the primary outcome or the probing rules themselves. The test is whether the rule changed or only its realisation for a particular participant.

Do I need to disclose mid-study changes in the readout?

Yes, in one line: what changed, when, and whether comparative results had been seen first. Reviewers now assume undisclosed changes happened, so disclosure is a credibility gain rather than a cost - and a study that reports an empty amendment log is making a stronger claim than one that says nothing.

What about changes forced by something outside the study?

The FDA guidance handles this case explicitly: unpredictable external events - new safety information, a competitor launch, a pricing change - may justify revising a running study rather than terminating it and starting over. The same rules apply. Log the external trigger, note the date, and split the analysis if the instrument changed. An external motivation is easier to defend precisely because it is documentable independently of your results.

Related Resources

• Interim Analysis and Sequential Testing - what a look costs, and why seeing results changes what you may do next
• Futility Analysis - stopping as the alternative to amending
• Intention to Treat vs Per Protocol - what an amendment does to your analysis population
• Model Version Drift in Research - when the instrument changes without anyone editing it
• Adaptive AI Interview Branching - pre-declared conditional logic done properly
• Research Peer Review: The Pre-Launch QA Gate - the cheapest place to catch a change you would otherwise make mid-field
• Structured Questions Guide - the fixed frame that lets conversations vary safely