No Summary Preserves Everything: What Map Projections Teach About Research Reporting (2026)

Every research summary distorts. This is not a comment on your writing, your analyst or your AI tool - it is the same structural fact that stops anyone from making a flat map of the Earth that is correct in every respect. The useful question is never whether a summary is faithful. It is which fidelity it chose to keep, and whether that choice matches the decision the summary is about to be used for.

The cartographic version of this problem is solved, in the sense that it is fully understood and openly declared. The research version is not, mostly because research summaries ship without the one artefact every serious map carries: a statement of its own distortion.

The short answer

• A summary that preserves what people said most cannot simultaneously preserve what mattered most. The trade is structural, not a quality issue.
• Every element of a distorted summary can be individually accurate while the proportions are badly wrong. Spot-checking quotes cannot detect this, by construction.
• A distortion is only an error when the summary is used for a decision it was not projected for.
• The fix is to declare the projection: say what the summary preserves and what it sacrifices, before anyone acts on it.

The impossibility is a theorem

Carl Friedrich Gauss proved that the curvature of a surface is an intrinsic property: "The Gaussian curvature of a surface is invariant under local isometry." The consequence is physical and familiar. "A piece of paper cannot be bent onto a sphere without crumpling. Conversely, the surface of a sphere cannot be unfolded onto a flat plane without distorting the distances." From which: "No planar (flat) map of Earth can be perfect, even for a portion of the Earth's surface. Thus every cartographic projection necessarily distorts at least some distances."

Cartography states the general case plainly - "All projections of a sphere to a plane necessarily distort the surface in some way" - and then names the specific trade that matters most: "Because the sphere is not a developable surface, it is impossible to construct a map projection that is both equal-area and conformal."

That sentence is worth holding onto, because its research analogue is requested constantly. A stakeholder who asks for a summary that is faithful to the proportions of what was said and faithful to the texture and nuance of how it was said is asking for an object that cannot exist. Proportion and shape are the two properties you cannot have at once. The honest response is not to work harder. It is to choose, and to say which you chose.

Mercator is not wrong

The most-mocked map in the world is an excellent map. Its reputation suffers because it is famous for a use it was never designed for.

On a Mercator projection, Greenland appears about the same size as Africa, although Africa's area is 14 times as large. Alaska looks much the same size as Australia, which is in fact about 4.5 times larger. The inflation is not arbitrary - the area scale factor is sec^2 of the latitude, which is about 1.2 at 25 degrees, exactly 4 at 60 degrees, and roughly 11.7 at 73 degrees, the median latitude of Greenland.

But Mercator was built so that a line of constant compass bearing is a straight line on the paper. For a navigator holding a course, that property is worth more than area fidelity, and the projection is close to optimal. It becomes a catastrophe only when someone uses it to answer "which country is bigger" - a question it was explicitly designed not to answer.

Research summaries fail the same way and for the same reason. A summary built to answer "what did customers talk about most" is a legitimate, well-constructed instrument. Used to answer "what should we build next", it will inflate the loud and shrink the important with roughly the confidence of a Mercator map sizing Greenland. The distortion did not appear when the question changed. It was always there; only now does it matter.

Why checking the quotes will not save you

This is the part that makes the analogy genuinely useful rather than merely decorative.

Tissot's indicatrix is the cartographer's device for making distortion visible: a circle of infinitesimal radius on the globe, drawn as it lands on the map. And here is the crucial detail. "In conformal maps, where each point preserves angles projected from the geometric model, the Tissot's indicatrices are all circles of size varying by location." In an equal-area map, the indicatrices "all have the same area, though their shapes and orientations vary with location."

Look at what that means for a conformal map like Mercator. Locally, at every single point, nothing is distorted at all - angles are preserved, and a small circle is still a small circle. Zoom in anywhere and the map is flawless. The entire distortion lives in the relationship between places, in how the sizes of those circles vary across the sheet.

A research summary behaves identically. Pull any quote from it and check the transcript: accurate. Pull another: accurate. Check a third: accurate. Every local reading passes, and the reviewer concludes the summary is sound. But the error was never in any individual element. It is in the relative weight given to each - the proportions, which no amount of element-level verification can see. Spot-checking a summary is a conformal test. It is structurally blind to the exact failure mode that matters, and the confidence it produces is unearned.

This is the same trap as validating an analysis against a reference that cannot detect the error class you care about, and it is worth reading alongside the imperfect gold standard problem.

Publish the distortion

Cartographers do not solve this by finding a better projection. They solve it by disclosure. The projection is named in the margin. Serious maps draw the indicatrices. Nobody is expected to guess.

Research reporting has no equivalent convention, and it should. A distortion note costs three lines and prevents the most expensive category of misreading:

• Name the projection. State what this summary was built to preserve - frequency of mention, severity as rated, revenue weighting, chronology, or the texture of individual accounts. Pick one primary property.
• Name the sacrifice. Say explicitly what was compressed to achieve it. If themes are ranked by mention count, say that magnitude and severity were not preserved.
• Name the intended decision. A summary projected for prioritisation should say so, so that a reader using it for sizing knows they are holding a Mercator.
• Give the scale factor where you can. "Three of 24 participants" is a scale bar. "Several participants" is a map with no legend.
• Keep the globe reachable. The summary is the projection; the transcripts are the sphere. Any claim should be traceable back to source so a reader can re-project rather than argue about the flat version.

That last point is the structural safeguard. A map is dangerous when the territory is unavailable. A research summary is dangerous when the underlying interviews are archived somewhere nobody can query.

How Koji handles this

Koji is designed so that the projection is explicit and the globe stays reachable.

• Structured questions supply a second, undistorted projection. Koji supports six question types - open_ended, scale, single_choice, multiple_choice, ranking and yes_no. A scale distribution and a ranking are equal-area instruments: they preserve proportion across every participant, regardless of who talked longest. Read them next to the narrative summary and you have two projections of the same territory. See the structured questions guide.
• Themes stay linked to their source moments. Koji thematic analysis keeps each theme addressable back to the interview excerpts behind it, so a reader can descend from the map to the territory instead of taking the projection on trust.
• Real-time reporting shows distributions, not adjectives. Counts and distributions are scale bars. Koji reports the numbers rather than leaving "several participants" to do the work.
• Consistent AI moderation keeps the grid regular. Because Koji AI-moderated and voice interviews ask the agreed questions the same way every time, differences between transcripts reflect participants rather than interviewer variation - which is what makes proportions across interviews comparable at all.
• Customisable AI consultants let you project deliberately. You can ask for a summary weighted for severity rather than frequency, which is a different projection of the same data, chosen on purpose.

The goal is not an undistorted summary. That object does not exist. The goal is knowing which one you are holding.

Frequently asked questions

Why can a research summary not be fully accurate?

Because summarising requires compressing many accounts into fewer statements, and no compression preserves every property at once. It is the same constraint that prevents a flat map from preserving area, shape and distance simultaneously. The limitation is structural rather than a failure of effort or skill.

What does it mean to say a summary chose a projection?

It means the summary was optimised to preserve one property - most often how frequently something was mentioned - at the cost of others such as severity, revenue impact or the nuance of individual accounts. Naming that choice lets readers judge whether the summary suits their decision.

If every quote checks out, is the summary accurate?

Not necessarily. Element-level accuracy and proportional accuracy are different properties. A conformal map is locally perfect at every point while badly misrepresenting relative sizes. In the same way, a summary can contain only true quotes and still give a seriously wrong impression of how much weight each theme deserves.

Should I stop using frequency to rank themes?

No, but you should label it. Ranking by frequency is a legitimate projection and useful for some decisions. It becomes misleading when it is read as a ranking by importance. State which one the summary provides, and supply a severity or scale measure alongside it where the decision needs one.

What is a distortion note?

A short statement attached to a research summary saying what it preserves, what it sacrifices, and which decision it was built for. It is the reporting equivalent of naming the projection in a map margin, and it typically takes three lines.

How does Koji reduce summary distortion?

Koji pairs the narrative summary with structured question results - scale, ranking and choice distributions that preserve proportion across all participants - and keeps every theme traceable to the interview excerpts behind it. That gives readers a second projection of the same data plus a route back to the source material.

Related Resources

• Structured Questions in AI Interviews - the six question types, and which of them preserve proportion
• When the Human Baseline Is Wrong - the companion failure, where the check cannot detect the error class
• AI Interview Summary Generator - how automated summarisation works in practice
• The Complete Guide to Thematic Analysis - the compression step this article is describing
• Most of Your Research Should Not Be Kept - deciding what survives the compression
• Every Input Was Accurate and the Difference Was Not - another case where correct parts produce a wrong whole