Partial Interviews: Should You Analyse Someone Who Answered Half Your Questions?

Usually yes - analyse the questions they actually answered, and exclude them from anything that depends on finishing. A partial interview is not a failed interview. It is breakoff, which is a distinct third category that is neither unit nonresponse nor item nonresponse, and it carries information the completes cannot give you.

In Koji a conversation that stops early is explicitly recorded as partial rather than being silently discarded or quietly counted as complete, so this is a decision you get to make deliberately.

Three different kinds of missing

Treating all missing data as one problem is the mistake that makes this hard. The survey methodology literature separates the cases, and the distinction is load-bearing.

Unit nonresponse

The person never started. You have no data about them at all, and the risk is that the people who never showed up differ systematically from those who did. This is the classic sampling problem covered in Nonresponse Bias.

Breakoff: the partial

The person started, answered some questions, and left. You have real data from a real participant - just not all of it. This is the case this article is about, and it is the one most teams handle worst.

Item nonresponse

The person finished but skipped or dodged particular questions. The gap is scattered rather than terminal. The asymmetry of what a blank is worth is covered in Missing Answers vs Wrong Answers.

The reason to keep these apart is that the diagnosis differs. Unit nonresponse indicts your recruiting. Breakoff indicts your instrument. Item nonresponse indicts a specific question.

Andy Peytchev makes exactly this structural argument in Survey Breakoff (Public Opinion Quarterly, 2009, volume 73, issue 1, pages 74 to 97). The paper presents "a framework within which to study different response behaviors, unit nonresponse, breakoff, item nonresponse, and measurement properties", and reports that "Features within the survey that are only seen after starting are predictive of breakoff rate, distinguishing this behavior from unit nonresponse."

That last clause is the crux. Breakoff is driven by what is inside your study, which is why it is the most actionable of the three.

The people who break off are not the people you think

The most common reason teams discard partials is an assumption that anyone who quits was not taking it seriously. Peytchev tested that and found the opposite: "respondents who broke off did not seem inattentive, supporting further efforts in their retention." He also found that "Education, a proxy for respondent cognitive sophistication, was significantly related to breakoff rates."

So discarding partials wholesale is not a neutral hygiene step. It is a decision to delete engaged participants, with a demonstrated relationship to a respondent characteristic - which is precisely the shape of a bias you did not intend to introduce.

This is different from genuine low-effort responding, which has its own detection methods - see Attention Check Questions and Survey Data Quality. Breakoff and carelessness are not the same phenomenon and should not share a remedy.

Where breakoff actually happens

Breakoff is not evenly distributed, and its shape tells you where your instrument hurts.

In Breakoffs in an hour-long, online survey (Emery, Cabaco, Fadel, Lugtig, Toepoel, Schumann, Lueck and Bujard, Survey Practice, 2023), the authors instrumented a long multi-country survey. Overall, "The final breakoff rate at the end of the survey was 17.23%, which means that 82.77% finished the survey." The country spread was wide: "In Croatia, 89.16% of respondents who started the survey, finished it", while "In Germany and Portugal, the rates were much higher at 20.33% and 21.75%, respectively."

Two findings from that paper are worth more than the headline rate.

The first is that attrition accumulates steadily rather than all at once: "Breakoffs were relatively linear across nearly 300 questions." After the first 100 items, breakoffs stood at 3%, 6%, and 9% for Croatia, Germany, and Portugal respectively, with a further 4%, 8%, and 7% between items 100 and 200. Length itself is a tax you pay continuously.

The second is that specific content causes specific cliffs: "The large drop at item 116 in Portugal and Germany corresponds with the start of the social network module." A concentrated spike is a diagnosis. It names the module that cost you those participants.

This is the same logic as reading paradata: a signal concentrated at one question indicts the question, while a signal spread evenly indicts the design. If your breakoffs cluster at question four, you have a question four problem, not a participant problem.

Worth noting how much of this is a survey-era problem. An hour-long instrument with nearly 300 fixed items is the format breakoff punishes hardest. A conversational interview that adapts and closes in a fraction of that time removes much of the burden being measured here, which is a large part of why completion rates differ between the two formats.

How Koji records a partial

Koji distinguishes the states rather than collapsing them. A conversation is active while it is running, completed when it finishes properly, partial when it produced real content but stopped short, and abandoned when it did not get far enough to be worth anything.

That distinction is the one that matters for your analysis, because partial means there is something here and abandoned means there is not.

What a partial costs you

Generally nothing, and this is worth understanding because it removes the financial incentive to fudge the classification.

Koji's quality gate means only conversations scoring 3 or above consume a credit. An interview that stopped early and produced little is scored accordingly and is marked as skipped for low quality rather than billed, and a conversation already charged can be refunded on quality grounds. See How the Quality Gate Works and Understanding Usage and Credits.

The consequence is that you can afford to be honest about what a partial is. You are not paying for the half-interview, so there is no pressure to either count it as a complete or pretend it never happened.

The decision rule

Here is the rule worth adopting, and it is narrower than include or exclude.

Include partials for question-level analysis

For any question the participant actually answered, their answer is as valid as anyone else's. They were engaged, they were eligible, and they responded. Excluding them costs you statistical power and, per Peytchev, introduces a slant toward the respondent profile least likely to break off.

Report the denominator per question rather than per study. If 52 people started, 40 finished, and 47 answered question three, then question three has n = 47. Say so. This is ordinary practice and it is more honest than a single study-level n that is wrong for most questions.

Exclude partials from anything completion-dependent

Some measures require the whole interview and must not be computed on a partial: any score summed across all questions, any ranking of themes by prevalence across the full guide, any per-participant journey that needs the end, and any comparison of early versus late answers within a person.

Never mix the two silently

The failure mode is a report where some numbers include partials and others do not, with no note saying which. State your rule once, apply it consistently, and record it. This is the same discipline as research quality inspection: the documented rule is worth more than the individually clever call.

How Koji handles this

• Koji records partial as a first-class conversation state, distinct from both completed and abandoned, so a half-finished interview is never silently promoted or dropped.
• The quality gate means only conversations scoring 3 or above consume a credit, so partials that produced little are skipped for billing rather than charged.
• Every structured answer a partial did produce is analysed normally, across all six question types - open_ended, scale, single_choice, multiple_choice, ranking, and yes_no - so the answers you did get remain usable.
• Because Koji conducts interviews conversationally and adaptively rather than marching through a fixed item list, the accumulating length burden that drives breakoff in long surveys is far smaller to begin with.
• Reports show you where interviews stopped, so a concentration of breakoffs at one question is visible rather than buried.
• Koji's AI asks follow-up questions in the moment, which means a participant who leaves after four questions has usually still given you depth on those four - not four one-word answers. See the AI probing guide.

Common mistakes

Deleting partials by default

The expensive one, for the reasons above. It looks like hygiene and behaves like a bias.

Treating a partial as a complete

The mirror error. Summing a score across questions the participant never saw produces a number that is simply wrong, and it will usually be low in a way that looks like a real finding.

Reading one study-level n

If your questions have different denominators, one n is a fiction. Report per question.

Blaming the participant for a question problem

When breakoffs concentrate at one point, the participants are not the variable. The question is.

Frequently asked questions

Does a partial interview count against my credits?

Usually not. Koji's quality gate means only conversations scoring 3 or above consume a credit, so an interview that stopped early and produced little is skipped for billing rather than charged, and a conversation already charged can be refunded on quality grounds. This is deliberate: you should not have a financial reason to misclassify a partial.

Should I include partial interviews in my analysis?

Include them for the questions they actually answered, and exclude them from anything that needs the whole interview, such as a score summed across all questions. Report the denominator per question rather than one study-level n, and state your rule in the write-up.

How is breakoff different from nonresponse bias?

Nonresponse is about people who never started, so you have no data from them at all. Breakoff is about people who started and left partway, so you have real answers from a real participant. Peytchev showed the two are driven by different things - breakoff is predicted by features only visible after starting, which means it points at your instrument rather than your recruiting.

What counts as partial rather than abandoned in Koji?

Koji records a conversation as partial when it produced substantive content but stopped before finishing, and abandoned when it did not get far enough to be useful. The distinction is what tells you whether there is anything worth analysing.

Where do most breakoffs happen in an interview?

It depends on whether length or content is the cause. In a long fixed-item survey, attrition accumulates steadily - Emery and colleagues found breakoffs relatively linear across nearly 300 questions. But a sharp spike at one point names a culprit: in that study a large drop at item 116 lined up with the start of the social network module. Look for the cliff before blaming general fatigue.

Can I follow up with someone who broke off?

Yes, and Peytchev's finding that people who broke off did not seem inattentive is a direct argument for doing so rather than writing them off. They were engaged participants who ran into something - time, a confusing question, or a topic they did not want to answer.

Related Resources

• Structured Questions Guide - the six question types and how each is captured
• Nonresponse Bias - the people who never started, which is a different problem
• Missing Answers vs Wrong Answers - item-level gaps and what a blank is worth
• Paradata: Process Signals - reading timing and drop-off as a signal about your questions
• How the Quality Gate Works - why a weak interview does not consume a credit
• Answer Confidence Flags - what to do when an extracted answer is uncertain