Capture-Recapture for Research: How to Estimate the Themes Your Study Never Found (2026)

Bottom line up front: "We stopped because no new themes came up" is a stopping rule, not a measurement. Capture-recapture turns it into a number. Split your study into two independently gathered passes, count the themes each pass found and the themes both found, and the overlap estimates how many themes exist that neither pass reached: N-hat = (themes in pass A x themes in pass B) / (themes in both). Two coders who find 24 and 21 themes with 16 in common have observed 29 distinct themes and are looking at a population of about 31.5 -- roughly two and a half themes they have never seen. The method is 130 years old in ecology, has been used on software inspections since 1992, and needs nothing you are not already collecting.

The problem with "no new themes"

Every qualitative sample-size guide, including our own guide to data saturation, ends at the same place: keep interviewing until additional interviews stop producing new codes. That advice is sound and it is also unfalsifiable in the direction that matters. A flat theme curve is consistent with two completely different worlds:

1. You have found nearly everything there is to find.
2. Your sampling process is systematically incapable of reaching the rest.

A saturation check cannot separate them, because both look identical from inside the sample. What you need is an estimate of the themes outside the sample, and for that you need something a single pass cannot give you: a second, independent look at the same population.

Hennink, Kaiser and Marconi made the ambiguity concrete in Qualitative Health Research in 2017. Examining 25 in-depth interviews, they found that code saturation was reached at nine interviews, while 16 to 24 interviews were needed to reach meaning saturation. Their conclusion is the sentence to keep: code saturation "may indicate when researchers have 'heard it all,' but meaning saturation is needed to 'understand it all.'" Even the point at which the code list stops growing arrives long before the study is actually finished.

The estimator, in one line

Capture-recapture was built for counting animals nobody can count directly. Mark and release a first sample, take a second sample later, and the fraction of the second sample that is already marked tells you what fraction of the population your samples reach. The classical Lincoln-Petersen form (C.G.J. Petersen, 1896; F.C. Lincoln, 1930) is:

N-hat = (n1 x n2) / m

where n1 is the number of distinct items found in pass 1, n2 the number found in pass 2, and m the number found in both. Because that ratio is unstable when the overlap is small, most practitioners use Chapman's 1951 bias-corrected version:

N-hat = ((n1 + 1)(n2 + 1) / (m + 1)) - 1

Replace "animals" with "themes" and nothing else changes. Here is the two-coder case worked end to end:

Two teams looking at the same corpus will report "29 themes." Only one of them can also report that the corpus probably contains about 31 or 32, and that the study is missing something on the order of two or three distinct issues.

The same arithmetic on interview batches

You do not need two coders. You can split the fieldwork instead. Run the first eight interviews, code them, then run eight more recruited through a different channel and code those independently before comparing. The comparison of the two theme lists is the recapture step.

The two rows differ only in how many themes recurred. The first says keep going; the second says the theme space is close to enumerated. Notice that the second row also produced fewer observed themes -- 19 against 22 -- which is the tell that this measure is not simply "more is better." It is a ratio, and the ratio carries information the raw count does not.

A serious caveat lives in that second row, and it is the subject of its own article: high overlap is exactly what you get when both passes were drawn through the same recruiting channel. See why a flat theme-discovery curve usually measures your recruiting before you trust a coverage number above 95 percent.

What software engineering learned in ten years of trying this

Research is not the first discipline to import this method from ecology. Software inspection has been doing it since 1992: two reviewers read the same document, a defect found by both is a "recapture," and the overlap estimates the defects still in the file. Petersson, Thelin, Runeson and Wohlin surveyed that decade of work in the Journal of Systems and Software in 2004, and their summary of accumulated findings is directly transferable:

• "Most estimators underestimate." Treat any capture-recapture number as a floor, never a ceiling.
• Accuracy depends on the amount of data, which means the number of distinct items found and the number of passes. Studies cited in that review found that at least four to five reviewers are needed before estimates become comfortably accurate; with only two passes, estimators are less robust.
• "To get any overlap there must be data from at least two reviewers." Two is the minimum, not the target.

The assumptions they flag are the ones that break in research too: whether the reviewers are equal in ability, whether "there is a risk that the reviewers co-operate, which violates the assumption of independence", and whether the defects are equally difficult to discover. Translate: coders who calibrate together produce inflated overlap, and themes are not equally easy to elicit.

The four assumptions, in research language

Only the last of these is fully under your control, and it is the one most teams skip. Before you compute anything, write the matching rule down: two codes count as the same theme when they describe the same user problem in the same context, not when they share a label.

Why estimating the tail beats collecting it

The instinct after seeing a coverage figure below 100 percent is to go collect the rest. It is worth knowing what that costs. Gotelli and Chao, writing in the Encyclopedia of Biodiversity (2013), give a worked case from a real dataset: a reference sample of 20,225 individual ant records containing 127 observed species, with an asymptotic estimate of about 135 species. To actually collect the last eight by sampling, they note, "the extrapolation to reach the Chao1 estimator extends to over 70,000 museum records."

That is 3.5 times the collection effort for 6.3 percent more species. The same shape governs interviews: the themes you have not heard are, by definition, the rare ones, and rare things arrive at a rate proportional to how rare they are. This is the strongest practical argument for estimation. You will never buy your way to complete coverage with sample size. You can, cheaply, know how much you are missing and decide whether it matters for the decision in front of you.

How Koji helps

The obstacle to doing this by hand has never been the arithmetic; it is producing two genuinely independent passes without doubling the budget and the calendar. That is exactly what an AI-native research platform changes.

• Two coding passes at no extra fieldwork cost. Koji's automatic thematic analysis produces a full theme set from every transcript. Run it, then have a human code an independent pass, and you have n1, n2 and m within an afternoon. In a traditional workflow, a second independent coder is a budget line that gets cut first.
• Two fieldwork passes in parallel, not in sequence. AI-moderated interviews run concurrently rather than one calendar slot at a time, so pass 1 and pass 2 can be fielded through different channels in the same week. That is what makes independence affordable; see in-product recruiting and community and event recruiting for two channels that rarely overlap.
• Structured questions give you a clean denominator. Koji supports six question types -- open_ended, scale, single_choice, multiple_choice, ranking, and yes_no -- and the closed types fix the answer space so the open_ended follow-ups are the only place new themes can appear. See the structured questions guide. When every respondent answers the same ranking and scale items, the overlap you measure on the open-ended text is not contaminated by differences in what was asked.
• Voice interviews widen catchability. Themes that people will not type, they will often say. Adding a voice modality is one of the few practical ways to attack the equal-catchability assumption rather than just noting it.
• Real-time reporting makes the estimate a live number. Because analysis runs as interviews land, the coverage estimate updates during fieldwork, which is when it can still change your recruiting. A traditional flow -- field for three weeks with SurveyMonkey or a scheduling tool, then analyse -- produces the number after every decision it could have informed.
• Customizable AI consultants keep the passes distinct. Two differently briefed consultants probing the same population are closer to independent than one moderator running the same guide twice.

A one-page workflow

1. Decide the unit before fieldwork: distinct themes, distinct user problems, or distinct feature requests. Mixing units invalidates the count.
2. Write the matching rule, one paragraph, and give it to whoever compares the two lists.
3. Build two passes that differ on purpose -- different recruiting channel, different moderator or consultant brief, different week.
4. Code them blind to each other. If pass 2 is coded against pass 1's codebook, m is inflated and N-hat collapses toward the observed count.
5. Compute both estimators. If Lincoln-Petersen and Chapman disagree by more than a theme or two, your overlap is too small to trust.
6. Report the triple, never the point estimate alone: observed themes, estimated total, and the assumption you are least confident in.
7. Decide, do not chase. If the estimate says two unseen themes and the decision is a roadmap sequencing call, ship. If it says fifteen, you have a recruiting problem, not a sample-size problem.

Common mistakes

• Using the same coder for both passes. The overlap then measures memory, not the population.
• Reporting N-hat as a ceiling. The software-inspection literature is unambiguous that these estimators run low. It is a floor.
• Computing it on merged themes. If the two lists were reconciled before counting, m is 100 percent by construction and the estimator returns the observed count.
• Ignoring m = 0. No overlap at all does not mean infinite themes; it means the two passes sampled different populations and the estimator does not apply.
• Treating a coverage percentage as a quality score. High coverage of a narrow, badly recruited population is worse than moderate coverage of the right one. Start with who counts.

Frequently asked questions

How many interviews do I need before capture-recapture works?

Enough that each pass produces a stable theme list on its own, which in practice means at least six to eight interviews per pass. The estimator needs overlap to divide by, so the useful precondition is not total sample size but that m is comfortably above zero -- five or more shared themes is a reasonable floor.

Can I use two AI passes instead of a human and an AI?

You can, but treat them as dependent unless the prompts, models, or consultant briefs genuinely differ. Two runs of the same analysis on the same transcripts share most of their errors, which inflates overlap and makes coverage look better than it is. A human pass against an AI pass is the more defensible pairing.

What if my two passes find wildly different numbers of themes?

That is informative rather than fatal. Very unequal n1 and n2 usually mean unequal catchability -- one coder splits fine-grained, one lumps -- so fix the matching rule and recount before computing. If the imbalance survives a shared matching rule, it is a real difference in the passes and the Chapman estimator is the safer of the two formulas.

Is this the same thing as inter-rater reliability?

No, and they answer opposite questions. Inter-rater reliability asks whether two coders agree on what they both saw; capture-recapture uses their disagreement to estimate what neither one saw. A study can have excellent agreement and terrible coverage, which is precisely the failure mode this method exists to catch.

Does this work for survey open text as well as interviews?

Yes, and it is often easier there because the volume is higher. Split the responses into two random halves, code them independently, and the overlap of theme lists gives the same estimate. With surveys the closed-population assumption is usually safest, since all responses arrive in a short window.

What coverage number is good enough?

There is no universal threshold, because the answer depends on the decision. For a directional roadmap call, 85 to 90 percent estimated theme coverage is usually plenty. For a pricing change, a compliance question, or anything where the rare case is the expensive case, the unseen themes are exactly the ones you needed, and a 90 percent figure is a reason to keep recruiting from a new channel rather than to stop.

Related Resources

• Why Your Theme Discovery Curve Flattens -- why overlap this method depends on is the first thing standardised recruiting destroys.
• The One-Off Comments Are the Only Estimate You Have -- the single-pass version of this measurement, computed from singletons.
• Data Saturation in Qualitative Research -- the heuristic this article puts a number on.
• Inter-Rater Reliability in Qualitative Research -- how to make "the same theme" mean the same thing across two passes.
• The Complete Guide to Thematic Analysis -- how the theme lists themselves get built.
• Structured Questions in AI Interviews -- the six question types that fix the answer space so overlap is measurable.