Corroboration in Research: Why Three Sources Saying the Same Thing Can Be One Source

Answer first: the rule that consistent evidence from several sources is stronger than evidence from one carries a condition that almost everyone drops. ISA 500 states that more assurance is ordinarily obtained from consistent audit evidence obtained from different sources, and the worked example is corroborating information obtained from a source independent of the entity. Independent is the operative word. When sales, support, and an executive all report that customers are demanding SSO, and all three heard it from the same three accounts, you do not have three sources. You have one source and three transmissions of it, and stacking them raises your confidence without raising the evidence at all.

This is the failure mode that makes a research programme feel well-corroborated right up until the moment it is wrong in an expensive way.

The rule, and the condition attached to it

The auditing standard on evidence is careful in a way that folk versions of triangulation are not. It says that more assurance is ordinarily obtained from consistent audit evidence obtained from different sources or of a different nature than from items of audit evidence considered individually - and then immediately grounds it in a specific example: corroborating information obtained from a source independent of the entity may increase the assurance the auditor obtains from audit evidence that is generated internally, such as evidence existing within the accounting records, minutes of meetings, or a management representation.

The mechanism is not quantity. It is the possibility of contradiction. A second source raises confidence only to the extent that it could have disagreed. A source that was always going to say the same thing, because it drew on the same underlying observation, adds volume and no information.

This is a different claim from the one made in triangulation in research, which covers combining methods, analysts, and theoretical lenses, and correctly insists on analysing each method cleanly before pooling. That article is about the independence of your methods and your analysts. This one is about the independence of your sources - the people and channels the raw observations came from - which can collapse even when your methods are impeccably separated.

The reliability ladder, translated

ISA 500 sets out five generalisations about when evidence is more reliable. They translate into research almost without modification.

The fourth row deserves a moment. A contemporaneously written record beats a later oral account of the same events - which means that in most organisations, the single most-cited category of customer evidence (what someone remembers a customer saying) sits at the bottom of the ladder, and it is usually the input that travels furthest.

Inquiry alone is not sufficient, and research is almost entirely inquiry

The standard is blunt about the limits of asking. Although inquiry may provide important audit evidence, and may even produce evidence of a misstatement, inquiry alone ordinarily does not provide sufficient audit evidence of the absence of a material misstatement at the assertion level, nor of the operating effectiveness of controls.

Product research is, structurally, an inquiry-dominant discipline. That is not a reason to abandon it - inquiry is the only way to reach reasons, motivations, and mental models, which behavioural data cannot supply. It is a reason to be precise about which claims inquiry can carry alone and which it cannot. It carries "here is how this person describes the problem and why they say they made that choice" comfortably. It does not carry "this feature will lift retention" on its own, ever, at any sample size. That boundary is the substance of attitudinal versus behavioural research and stated versus revealed preferences.

The management-intent problem, and the procedure that fixes it

There is one passage in ISA 500 that reads as though it were written for product discovery. On inquiries about what management intends to do, the standard concedes that the information available to support management's intent may be limited, and then prescribes a remedy: understanding management's past history of carrying out its stated intentions, management's stated reasons for choosing a particular course of action, and management's ability to pursue a specific course of action may provide relevant information to corroborate the evidence obtained through inquiry.

"Would you use this?" is an inquiry about intent. So is "would you pay for this?", "would you switch?", and every purchase-intent scale ever fielded. The auditing remedy converts directly into three interview moves that most guides do not include:

1. Past history of carrying out stated intentions. Not "would you switch" but "walk me through the last time you switched a tool in this category - what actually triggered it, and how long did it take?" A person who has never completed the behaviour is giving you a forecast, not a report.
2. Stated reasons for choosing a course of action. Ask for the reasoning, not the verdict. Reasons can be checked against the person's circumstances; verdicts cannot.
3. Ability to pursue the course of action. Does this person control the budget, the integration, the rollout? An enthusiastic intent from someone with no authority is a real attitude and a worthless forecast.

Koji's AI moderator does move 1 and 2 by default, because open_ended questions carry follow-up probing that pushes from claim to episode rather than accepting the first answer, and it does so identically for every participant rather than only when the moderator remembers. Move 3 is a screener question, and the single_choice and yes_no structured types make it a filterable field rather than a line buried in a transcript.

Four ways research sources stop being independent

Common origin. The most frequent and the least visible. Three channels report the same demand because all three are downstream of the same handful of loud accounts. Test: name the accounts behind each channel and look for overlap.

Shared instrument. Two studies agree because they asked the same leading question. Question wording travels between studies through copy-paste far more than anyone admits, and a flaw in the wording is inherited along with it. Two studies sharing a question are one instrument tested twice. See survey question wording.

Shared analyst. The same person coded both datasets, and the second coding inherited the first one's theme structure. This is the case triangulation in research addresses with investigator triangulation, and it is a genuine remedy for this failure mode specifically.

Sequential contamination. The second wave was recruited from people who had seen the output of the first, or moderated by someone who had. Once the interviewer knows what the study is expected to find, probing asymmetry does the rest - the mechanism described in observer bias and interviewer bias.

A ten-minute independence test

Before you let a claim inherit confidence from multiple sources, run each source through four questions. Any claim scoring badly should be reported as one source, however many channels carried it.

Absence of information is also evidence

One more line from the same standard is worth carrying into research practice, because it inverts the usual reading of what counts as data. Audit evidence, ISA 500 says, comprises both information that supports and corroborates management's assertions, and any information that contradicts such assertions. And then: in some cases the absence of information - for example, management's refusal to provide a requested representation - is used by the auditor, and therefore also constitutes audit evidence.

In research, the equivalents are concrete and routinely discarded. The segment that declined to be interviewed. The question nobody could answer. The workflow no participant mentioned unprompted even though the roadmap assumes it is central. Silence from a channel that would have spoken if the problem were real is evidence, and it belongs in the report next to the quotes. Who is missing from the sample in the first place is handled in nonresponse bias and survivorship bias in customer research.

How Koji keeps sources genuinely separate

One instrument, many independent participants. Every participant meets an identical structured instrument, so the questions are a controlled constant rather than an uncontrolled variable drifting across a three-week fieldwork period. Where a human moderator's phrasing evolves as their hypothesis firms up - creating shared-instrument dependence between the first five interviews and the last five - the AI interviewer asks the same thing of everyone, and the follow-up probing adapts to the participant rather than to the researcher's expectations.

Direct evidence rather than relayed evidence. Every claim in a Koji report traces to a timestamped transcript you can open. That moves the evidence up two rungs of the ISA 500 ladder at once - obtained directly rather than by inference, and documentary rather than oral - and it removes the relay step where an account manager's summary of a customer becomes the customer's view.

Independent analysis you can actually afford. Investigator independence used to mean funding a second senior analyst. Comparing your own themes against the platform's independent pass over the same transcripts gives the same structural benefit at no additional headcount, which is the practical version of the point made in triangulation in research.

Six structured question types that make agreement countable. open_ended, scale, single_choice, multiple_choice, ranking, and yes_no. When two studies agree on a scale distribution you can quantify the agreement; when they agree on a vibe, you cannot tell corroboration from an echo. The structured questions guide covers the mapping from type to analysis.

Honest objections

This makes corroboration sound impossible. It makes cheap corroboration impossible, which is the point. Genuinely independent sources exist - behavioural logs, support tickets, and interviews with a disjoint set of accounts are three real ones - and the test is there to help you find them, not to argue nothing counts.

Sometimes the same three accounts really are the market. In a concentrated enterprise segment that can be true. Then say so: the finding is well-evidenced for those accounts and untested elsewhere. That is a scope statement, not a corroboration claim, and it is exactly the distinction drawn in levels of assurance in research.

We do not have time to trace every claim to its origin. You do not need to. Trace the two or three claims the roadmap is about to be built on. The rest can carry their uncertainty.

Frequently asked questions

What makes two research sources independent?

They are independent when either could have contradicted the other. In practice that means they do not share underlying accounts or events, do not share an instrument or coding frame, and neither saw the other's output before producing its own. Sources that fail any of those tests should be counted once.

Does this contradict triangulation?

No, it states triangulation's precondition. Triangulation works because different methods have different weaknesses, so the errors do not line up. When the sources share an origin, an instrument, or an analyst, the errors do line up, and combining them concentrates the error instead of cancelling it.

Why is inquiry considered weak evidence when interviews are the core of research?

Auditing standards say inquiry alone does not establish the absence of a material misstatement or the effectiveness of a control - not that inquiry is worthless. Inquiry is the only route to reasons, motives, and mental models. The limit is that it cannot carry a behavioural forecast on its own, which is why intent questions need corroboration from past behaviour.

How should we handle purchase-intent or would-you-use-this questions?

Treat them as inquiries about intent and apply the standard remedy: ask about the last time the person actually performed the behaviour, ask for their reasoning rather than their verdict, and confirm they have the authority and ability to act. An intent answer with none of those three is a forecast from someone with no track record.

Is a lack of evidence really evidence?

It can be. A segment that declines to participate, a question nobody can answer, or a workflow no participant mentions unprompted are all informative, and auditing standards explicitly treat the absence of expected information as evidence. Record it in the report rather than leaving it out because it produced no quotes.

How many independent sources are enough?

There is no fixed number, because a second genuinely independent source moves confidence far more than a fifth dependent one. The useful question is not how many sources agree but how many of them could have disagreed - and if the answer is one, report it as one.

Related Resources

• Structured Questions Guide - the six question types that make agreement countable rather than impressionistic
• Levels of Assurance in Research - what a single study is entitled to conclude before corroboration enters
• Triangulation in Research - independence of methods and analysts, the companion to source independence
• Stated vs. Revealed Preferences - why intent answers need behavioural corroboration
• Nonresponse Bias - the sources that never reached you at all
• Research Independence - independence of the people grading the evidence