Anchoring Bias in Research and Surveys: How the First Number Skews Every Answer

TL;DR

Anchoring bias is the tendency to rely too heavily on the first piece of information offered — the "anchor" — when making judgments, even when that anchor is arbitrary or irrelevant. In research it silently corrupts pricing studies, willingness-to-pay questions, scale ranges, and any survey where a number, example, or option appears before the respondent answers.

The effect is strong enough that a random number from a spinning wheel changed people's estimates of world facts, and the last two digits of a Social Security number moved how much people would pay for wine by over 200%. The defense is to avoid planting anchors, randomize order, use neutral open-ended elicitation before showing options, and probe reasoning — all of which AI-moderated, structured research automates.

What Is Anchoring Bias?

Anchoring bias was first demonstrated by Amos Tversky and Daniel Kahneman in their 1974 Science paper Judgment Under Uncertainty: Heuristics and Biases. In the classic experiment, participants spun a wheel of fortune that landed on a number between 0 and 100, then estimated the percentage of African nations in the United Nations. The wheel was rigged. Participants who saw 10 gave a median estimate of 25%; those who saw 65 gave a median estimate of 45% (Nielsen Norman Group, The Anchoring Principle).

The number was visibly random and had nothing to do with Africa or the UN — yet it pulled estimates toward it. We anchor on whatever number is in front of us and adjust insufficiently from there.

The Most Striking Anchoring Study for Researchers

If a wheel of fortune sounds too artificial, consider the pricing experiment by Dan Ariely, George Loewenstein, and Drazen Prelec, published as Coherent Arbitrariness. MIT students wrote down the last two digits of their Social Security number, then bid on products — wine, chocolates, a keyboard — in a real auction with real money.

Students whose SSN ended in the top quintile (80-99) bid between 216% and 346% more than students in the bottom quintile (00-19). For a 1998 Cotes du Rhone, high-number students bid an average of $27.90 versus $8.64 for low-number students (Ariely, Loewenstein & Prelec). A government ID number — containing exactly zero information about wine — set their willingness to pay. Ariely called this "coherent arbitrariness": once an arbitrary anchor sets the initial level, every later valuation lines up consistently beneath it.

For anyone running pricing research, that is the whole warning in one experiment.

Where Anchoring Corrupts Research

• Pricing and willingness-to-pay studies. Show a price first and you anchor every subsequent answer. "Would you pay $99/month?" produces different demand curves than "Would you pay $19/month?" — even among identical buyers.
• Scale ranges. A satisfaction scale that runs 1-10 collects different answers than one labeled 1-5, because the endpoints anchor where "good" sits.
• Numeric estimates. Asking "How many hours per week do you spend on this — more or less than 20?" anchors the estimate near 20.
• Leading examples. Listing "features like A, B, and C" before asking what users want anchors their wishes to your list.
• Option order. The first option shown in a single_choice or ranking question becomes a reference point for the rest.
• Negotiation and sales feedback. The first figure mentioned in a buyer interview frames the entire conversation about value.

How to Design Anchors Out of Your Research

• Elicit before you reveal. Ask open-ended "What would you expect to pay?" before showing any price. Capture the unanchored number first.
• Use unanchored pricing methods. Van Westendorp and Gabor-Granger are structured precisely to measure price sensitivity without planting a single anchor; pair them with open elicitation.
• Randomize order. Rotate option order and question sequence across participants so no single item is always first.
• Avoid numeric primes in question wording. Drop "more or less than X" framings; ask for the estimate cold.
• Keep scales consistent and labeled. Use the same anchored scale across studies so ranges are comparable rather than re-anchored each time.
• Probe the reasoning. A number with no rationale behind it is often just an anchor echoed back. Always ask why.

The Modern Approach: AI-Moderated Research

Anchoring is hard to avoid in static surveys because every respondent sees the same fixed wording, the same first option, the same price — so every respondent is anchored identically and the bias is baked into the data. Legacy tools like SurveyMonkey or Google Forms present a frozen form; they can not ask an open question first and then adapt.

An AI-native platform like Koji runs the unanchored sequence by default. The AI moderator can ask "What would you expect a tool like this to cost, and why?" capture the genuine, unanchored figure, and only then explore reactions to specific price points — preserving the clean number that a static survey would have destroyed.

Koji's six structured question types — open_ended, scale, single_choice, multiple_choice, ranking, and yes_no — let you sequence elicitation deliberately: open_ended first to capture the unanchored response, then scale or single_choice to test specific levels. Option order can be randomized across participants, and every numeric answer is probed for the reasoning behind it. See the structured questions guide for how to order them.

How Koji Helps

• Elicits before it anchors. The AI moderator asks open expectations first, capturing willingness-to-pay before any price is shown.
• Probes every number. It follows up on each figure to separate a reasoned valuation from an anchor echoed back.
• Randomizes and standardizes. Option order can vary across participants while scales stay consistent, removing fixed-position anchors.
• Runs unanchored pricing research at scale. Combine open elicitation with Van Westendorp-style follow-ups across hundreds of AI-moderated interviews in the time a manual study would take to recruit.
• Surfaces anchoring in analysis. Automatic thematic analysis flags when respondents simply repeat a number from the question, so you can discount it.

Anchoring vs. Priming and Order Effects

These biases overlap, but naming the right one tells you how to fix it:

• Anchoring vs. priming. Priming is when exposure to one concept influences responses to a later, related concept; anchoring is specifically a numeric or magnitude reference point that pulls quantitative estimates toward it.
• Anchoring vs. question-order bias. Order bias is about how earlier questions reshape the context for later ones; anchoring is about a specific value becoming the starting point a respondent adjusts from. A price shown in question two can anchor an estimate in question five.

A worked example: a B2B pricing interview

You are testing pricing for a new analytics tool. The lazy script asks, "Our Pro plan is $499 per month — does that feel reasonable?" Every buyer now anchors on $499. Those who would have volunteered $700 round down; those who privately thought $200 feel the gap, say "a bit high," and let their stated number creep toward $499. Your demand curve is quietly compressed around the anchor you planted.

The unanchored script instead opens: "Before I show you anything, what would you expect a tool that does this to cost per month — and how did you arrive at that figure?" You capture the real distribution first, then probe reactions to specific tiers later. The two scripts can produce willingness-to-pay estimates that differ by hundreds of dollars per seat — which is the difference between a viable price and a broken one. The anchor is not a detail; it is the experiment.

A Field Checklist for Defusing Anchoring Bias

Before fielding a pricing or estimation study, confirm that:

• You ask an open_ended expectation question ("What would you expect this to cost?") before showing any number.
• No question wording contains a numeric prime such as "more or less than X."
• Option and tier order is randomized across respondents.
• Scales use consistent, pre-set ranges so results stay comparable across studies.
• Pricing questions use an unanchored method (Van Westendorp or Gabor-Granger) rather than a single "is $X reasonable?" prompt.
• Every numeric answer is followed by a "how did you arrive at that?" probe.
• The first figure mentioned in any buyer interview is the participant's, not yours.

Bottom line: in pricing and estimation research, the first number on the table is rarely neutral — it becomes the gravity well every later answer falls toward. Tversky and Kahneman moved world-fact estimates with a roulette wheel; Ariely moved willingness-to-pay with Social Security digits. If an arbitrary number can do that, the price you casually mention in question one will absolutely reshape your demand curve. Capture the unanchored answer first, probe the reasoning, and randomize whatever you can. AI-moderated interviews run that disciplined sequence on every participant, turning anchor-free elicitation from a best practice you forget under deadline into the default path.

One more trap worth naming: do not let a competitor's price set the anchor either. If you open with "Acme charges $50 — what would you pay us?", you have simply borrowed someone else's anchor instead of planting your own. Establish the buyer's independent reference point first, then introduce competitive context only once their unanchored expectation is safely recorded.

Related Resources

• Research Bias: The Complete Guide
• Pricing Research Interviews
• Van Westendorp Price Sensitivity Meter
• Question Order Bias
• Avoiding Leading Questions
• Structured Questions Guide