{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-10-01T10:27:52.484Z"},"content":[{"type":"documentation","id":"3498d9b5-5ead-409a-8249-e3fb105c5454","slug":"mixed-evidence-belief-polarization-research","title":"When a Readout Divides the Room: Mixed Evidence and Belief Polarization (2026)","url":"https://www.koji.so/docs/mixed-evidence-belief-polarization-research","summary":"Presenting mixed evidence to a divided team can make both sides report that their view was strengthened. Greitemeyer (2014, N=662) found reported polarization alongside measured depolarization in the same session: reported change was +0.12 for believers versus -0.20 for skeptics, F(1,660)=14.26, p<.001, while measured attitudes converged, F(1,396)=62.26, p<.001. Lord, Ross and Lepper (1979) showed that methodological detail is what let participants rebuild their original belief. The fix is to measure each position before and after with the same scale, keep responses paired per person, and report the change in spread rather than the mean.","content":"**Answer first:** Present mixed evidence to a team that already disagrees and both sides can report that the evidence strengthened their position. In a controlled study of 662 people, those self-reports showed polarization while the measured attitudes actually moved closer together. The two instruments pointed in opposite directions, and the unreliable one is the one every readout collects informally. Never ask whether a readout changed someone's mind. Measure the position itself, before and after, per person.\n\n## Biased assimilation, and what the original study actually showed\n\nThe founding result is Charles Lord, Lee Ross and Mark Lepper, Biased Assimilation and Attitude Polarization: The Effects of Prior Theories on Subsequently Considered Evidence, in the Journal of Personality and Social Psychology, volume 37, issue 11, pages 2098-2109, 1979.\n\nThe design: participants who felt strongly about capital punishment, half in favour and half against, read descriptions of two studies. One compared states with and without the death penalty. The other compared murder rates in a single state before and after the death penalty was introduced. The evidence set was deliberately mixed, with one study supporting deterrence and one undermining it.\n\nThe usual summary of what happened is that both groups became more extreme. The actual sequence is more specific and more useful. Participants, whether supporters or opponents, reported shifting their attitudes slightly toward whichever study they happened to read first. Then they read the detailed descriptions of the two studies, and almost all of them returned to their original belief regardless of the evidence, pointing to the details that supported their position and disregarding anything contrary.\n\nSit with that sequence, because it carries an uncomfortable implication for research practice. The belief was briefly movable. On that published sequence, what restored it was the arrival of **the methodological detail**. Read the ordering carefully before leaning on it: that the detail did the restoring is an inference from the order of events rather than a contrast the authors isolated experimentally. The practical consequence survives either way. Participants used the fine-grained description of each study as raw material for finding reasons to reject the study they did not like.\n\nMethodological detail is exactly what we add to a readout to make it more rigorous: sample sizes, segment breakdowns, the limitations slide, the note about who was excluded. The honest reading of Lord, Ross and Lepper is that the same detail that makes a finding defensible also supplies a motivated reader with the tools to dismantle it. That is not an argument for hiding your method. It is an argument for sequencing it, which the protocol later in this article handles.\n\n## The study where reported and measured belief change pointed opposite ways\n\nThe most decisive result for a research team comes from Tobias Greitemeyer, writing in PLoS ONE, volume 9, issue 4, article e93440, in 2014, under the title I am right, you are wrong: how biased assimilation increases the perceived gap between believers and skeptics of violent video game effects.\n\nParticipants, 662 of them, first indicated whether they believed violent video games increase aggression. They then read summaries of two studies, one showing an effect and one showing none. Believers evaluated the study showing an effect more favourably than the study showing none, and the opposite was observed for skeptics. That is standard biased assimilation.\n\nThe important part is what happened next, because the study measured belief change two different ways.\n\n**Asked whether their view had changed,** participants reported polarizing. Believers reported becoming more convinced that violent video games increase aggression, averaging +0.12 with a standard deviation of 0.92, while skeptics reported becoming less convinced, averaging -0.20 with a standard deviation of 1.20. The difference was significant, F(1, 660) = 14.26, p below .001.\n\n**Measured directly, before and after,** the same participants depolarized. Believers were less convinced that violent video games increase aggression after reading the summaries, with a mean of 2.02 and a standard deviation of 1.47, than they had been before, with a mean of 2.42 and a standard deviation of 1.21. That shift was large and significant, F(1, 396) = 62.26, p below .001.\n\nSame people, same session, two instruments, opposite conclusions. The abstract states it plainly: both believers and skeptics reported becoming more convinced of their initial view, and in contrast, for actual attitude change, a depolarization effect was found.\n\n| Instrument | What it reported | Reliability |\n| --- | --- | --- |\n| Did this evidence change your mind? (self-report of change) | Both camps hardened | Wrong. This is the one readouts collect |\n| Position measured before, position measured after (difference computed by you) | Both camps softened and converged | Correct, and almost never collected |\n\nGreitemeyer also found that the perceived gap between the two sides increased even when the evidence was mixed and inconclusive. So the readout can leave a team believing it is further apart than it is, while the members have quietly moved closer. Those are two different failures and they compound: the disagreement feels worse and the actual convergence goes uncredited.\n\n## How often does this happen\n\nHonesty requires the caveat, and the caveat is interesting in its own right.\n\nDeanna Kuhn and Joseph Lao, in Psychological Science, volume 7, issue 2, pages 115-120, 1996, concluded that polarization is a real phenomenon but far from inevitable, occurring only in a small minority of cases. Their work also found something more awkward: polarization was prompted not only by considering mixed evidence, but by **merely thinking about the topic**.\n\nThat last clause should change how research operations plans a workshop. If simply turning a question over in your mind can move beliefs apart, then an alignment session in which a divided team discusses a decision without any data at all can polarize it. The readout is not required. Your evidence is not necessarily the cause of the division that follows it, which also means you should not conclude your study backfired on the basis of a tense meeting.\n\nLee Ross revisited the original finding and its reception decades later in Critical Review, volume 24, issue 2, pages 233-245, 2012. Treat polarization as a risk to detect rather than a law to expect, in the same spirit as the independence failures catalogued in [evidence corroboration from independent sources](/docs/evidence-corroboration-independent-sources).\n\n## Measure per person, and look at the spread\n\nThe core measurement error is reporting a mean. A mean can be perfectly flat while every individual in the room has moved, and polarization is precisely the pattern that hides inside a stable average.\n\nTake a team of eight, rating their support for a decision on a 1 to 7 scale. Four sit at 2 and four sit at 6. The mean is 4.\n\nRun the readout. The four at 2 move to 1, and the four at 6 move to 7. The mean is still 4. The standard deviation has gone from 2.0 to 3.0, and the range from 4 points to 6. A mean-only report says the readout changed nothing. What actually happened is the most consequential thing that can happen in an alignment meeting.\n\nSo report three numbers, not one:\n\n- **Mean change,** which answers whether the team moved on average.\n- **The distribution of per-person change,** which answers whether anybody moved.\n- **The change in spread,** which answers whether they moved together or apart.\n\nThe third is the one that is always missing, and it is the only one that detects the failure this article is about. The general lesson, that a summary statistic can conceal the mechanism that produced it, is the same one described in [measurement invariance when comparing groups](/docs/measurement-invariance-comparing-groups).\n\n## A readout protocol that resists polarization\n\nThis sequence is built directly from the two studies above. The ordering is the active ingredient.\n\n1. **Collect positions before the readout,** on the actual decision, per person, on a numeric scale. Attach the decision explicitly: support for shipping feature X, not a general attitude.\n2. **State the decision rule before the data.** What result would favour which choice? Agreeing this in advance is the one step that removes the room's freedom to reinterpret the evidence afterwards, and it is the same pre-commitment described in [falsifiable research questions and severe tests](/docs/falsifiable-research-questions-severe-tests).\n3. **Present each camp its disconfirming evidence first, and separately.** Lord, Ross and Lepper found that detailed description lets people rebuild their prior belief. Presenting the mixed set all at once invites exactly that sorting. Taking each side through the evidence that cuts against it, before the evidence that supports it, denies the sorting step its raw material.\n4. **Collect positions again afterwards,** with the identical instrument. Never substitute a question about whether minds were changed, because that is the measure Greitemeyer showed points the wrong way.\n5. **Report the divergence as a property of the readout,** not of the people in it. If the spread widened, the meeting design is the first suspect, not the stakeholders.\n\n## How Koji helps\n\nThis protocol needs a numeric position from every stakeholder twice, minutes apart, with the per-person link preserved. Doing that by hand means a spreadsheet, a chase for responses and a reconciliation step, and it is the first thing dropped when the readout runs late. Koji makes it a configuration rather than a project.\n\nKoji supports six structured question types, and the protocol uses all six:\n\n| What you collect | Koji question type | Why |\n| --- | --- | --- |\n| Support for the decision, before and after | `scale` | The only instrument that lets you compute per-person change and a change in spread |\n| Which option would you choose today? | `single_choice` | A committed choice, not a sentiment |\n| Would you act on this result? | `yes_no` | Converts a position into a decision |\n| Which evidence did you find most persuasive? | `multiple_choice` | Reveals the assimilation pattern directly |\n| Order these four options by preference | `ranking` | Returns an average position per option, exposing a split the mean hides |\n| What would change your mind? | `open_ended` | Collected before the readout, this is the pre-commitment in the stakeholder's own words |\n\nBecause Koji runs the same instrument twice against the same named panel, the before and after values stay linked per person, which is the requirement the arithmetic above depends on. A mean computed from two anonymous rounds cannot produce a per-person change distribution. Koji also attaches a quality score from 1 to 5 to each conversation, broken down across relevance, depth and coverage, so a hurried response is visible rather than quietly weighted the same as a careful one.\n\nTwo of Koji's built-in frameworks already encode the instinct behind this. The Jobs to be Done framework probes for anxieties, meaning what almost stopped someone from switching, and the Open Exploration framework lists as a probe point *Contradictions - gently explore the tension*. Both are built on the premise that the interesting material is where a person's position is unstable, which is the same premise as measuring a position twice. The [structured questions guide](/docs/structured-questions-guide) explains how each of the six types is aggregated into a report.\n\n## Common mistakes\n\n- **Asking whether the readout changed anyone's mind.** This is the measure that pointed the wrong way in a study of 662 people. Measure the position, compute the change yourself.\n- **Reporting only the mean.** A flat mean is consistent with the entire room splitting to the extremes.\n- **Running the before and after rounds anonymously.** Anonymity is right for predictions and wrong here, because you need per-person pairing. Pair the responses and report only aggregates.\n- **Presenting the full mixed evidence set at once.** It invites each camp to sort the evidence in its own favour.\n- **Adding more methodological detail to overcome resistance.** Detail is what let participants rebuild their original belief. Sequence it rather than piling it on.\n- **Concluding your study caused the division.** Kuhn and Lao found that merely thinking about a topic can polarize, so a tense meeting is not proof the evidence did it.\n- **Treating a widened spread as a people problem.** It is usually a sequencing problem in the meeting.\n\n## Frequently asked questions\n\n### What is biased assimilation?\n\nBiased assimilation is the tendency to evaluate evidence that supports your existing position as stronger and better conducted than evidence that contradicts it. Lord, Ross and Lepper demonstrated it in 1979 with participants who held strong views on capital punishment: shown one study supporting deterrence and one undermining it, each group rated the congenial study as more convincing and found methodological reasons to reject the other.\n\n### Why should I not ask whether a readout changed someone's mind?\n\nBecause self-reported belief change and measured belief change can point in opposite directions. In Greitemeyer's 2014 study of 662 participants, both believers and skeptics reported becoming more convinced of their initial view, while their directly measured attitudes converged. The self-report is the unreliable instrument, and it is the one most readouts collect informally.\n\n### How do I detect polarization in a stakeholder group?\n\nMeasure each person's position on the decision before and after the readout using the same numeric scale, keep the responses paired per person, and then report three things: the mean change, the distribution of per-person changes, and the change in spread. Only the third detects polarization, because a team splitting to the extremes can leave the mean completely unchanged.\n\n### Does presenting more evidence reduce disagreement?\n\nNot reliably. Lord, Ross and Lepper found that participants briefly moved toward whichever study they read first and then returned to their original belief once they read the detailed descriptions, using the detail to justify rejecting the uncongenial study. More methodological detail can supply more material for that rejection, which is why sequencing the evidence matters more than the volume of it.\n\n### Is belief polarization common?\n\nKuhn and Lao concluded in 1996 that polarization is real but far from inevitable, occurring in only a small minority of cases. They also found it could be prompted by merely thinking about a topic rather than by considering mixed evidence, so a division that appears after a readout may not have been caused by the readout at all. Treat it as a risk worth measuring rather than an outcome to expect.\n\n### Should I avoid presenting mixed or inconclusive findings?\n\nNo. Suppressing inconvenient results is a far worse failure than an awkward meeting. Present the mixed evidence, but agree the decision rule before showing the data, give each camp its disconfirming evidence first and separately, and measure positions before and after so you can report what actually moved instead of what the room says moved.\n\n## The bottom line\n\nA divided team can leave a readout convinced the evidence strengthened both positions, while the measurements show it brought them together. One study of 662 people found exactly that, with the self-reports and the measured attitudes significant in opposite directions.\n\nCollect a numeric position from each stakeholder before the readout, agree what result would favour which decision, lead each camp with the evidence that cuts against it, and collect the position again afterwards with the same instrument. Then report the spread, not just the mean. The spread is the only number that tells you whether your readout aligned the room or split it.\n\n## Related Resources\n\n- [Structured Questions Guide](/docs/structured-questions-guide) - the six question types and how each is aggregated\n- [We Already Knew That: Obvious Research Findings](/docs/we-already-knew-that-obvious-findings) - the companion measurement, taken before the same readout\n- [The Barnum Test for Research Findings](/docs/barnum-test-generic-research-findings) - when a finding is accepted because it fits everyone\n- [Falsifiable Research Questions and Severe Tests](/docs/falsifiable-research-questions-severe-tests) - pre-committing to what a result would mean\n- [Evidence Corroboration from Independent Sources](/docs/evidence-corroboration-independent-sources) - when agreement is not confirmation\n- [Measurement Invariance When Comparing Groups](/docs/measurement-invariance-comparing-groups) - how a summary statistic conceals its mechanism\n- [Presenting Research Findings to Stakeholders](/docs/presenting-research-findings) - the readout this protocol restructures\n","category":"Research Operations","lastModified":"2026-10-01T03:39:36.932461+00:00","metaTitle":"Mixed Evidence and Belief Polarization in Readouts (2026)","metaDescription":"Both camps can report mixed evidence strengthened their view while measured attitudes converge. Measure positions per person, before and after.","keywords":["belief polarization","biased assimilation","mixed evidence readout","does research change minds","stakeholder alignment research","pre post belief measurement","research readout protocol"],"aiSummary":"Presenting mixed evidence to a divided team can make both sides report that their view was strengthened. Greitemeyer (2014, N=662) found reported polarization alongside measured depolarization in the same session: reported change was +0.12 for believers versus -0.20 for skeptics, F(1,660)=14.26, p<.001, while measured attitudes converged, F(1,396)=62.26, p<.001. Lord, Ross and Lepper (1979) showed that methodological detail is what let participants rebuild their original belief. The fix is to measure each position before and after with the same scale, keep responses paired per person, and report the change in spread rather than the mean.","aiPrerequisites":["Experience running a research readout with stakeholders","Comfort with means and standard deviations"],"aiLearningOutcomes":["Explain the difference between biased assimilation and attitude polarization","Recognise why self-reported belief change is an unreliable instrument","Measure stakeholder positions per person before and after a readout","Report the change in spread rather than only the mean","Sequence a readout so each camp meets its disconfirming evidence first"],"aiDifficulty":"advanced","aiEstimatedTime":"12 min"}],"pagination":{"total":1,"returned":1,"offset":0}}