{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-08-25T08:47:06.514Z"},"content":[{"type":"documentation","id":"2b626b25-a3be-4526-8f37-90f3402d134a","slug":"dumping-effect-attribute-scales-research","title":"The Dumping Effect: Why the Attributes You Leave Out Change the Scores of the Ones You Keep (2026)","url":"https://www.koji.so/docs/dumping-effect-attribute-scales-research","summary":"The dumping effect is the displacement of a perception onto the nearest available rating scale when no matching scale exists. Clark and Lawless demonstrated in 1994 that measured intensity of a fixed stimulus changed with the number of scales offered, that practice did not reduce it, and that adding a scale could depress an unrelated rating. It cannot be detected by inspecting responses, only by changing the instrument.","content":"A respondent notices something about your product. Your questionnaire offers no place to record it. They do not discard the perception - they put it on the nearest scale you did give them. The score you get back is therefore partly a measurement of the product and partly a measurement of your attribute list, and nothing in the response itself tells you which is which.\n\nThis is the dumping effect. It has been documented experimentally since the early 1990s in sensory science, it is routinely designed against in that field, and it is almost never considered in product research - where fixed-form surveys with five to ten attributes are the default instrument.\n\n## The answer, up front\n\n**When respondents perceive something your scales do not cover, the perception is displaced onto whichever scale is closest, inflating or deflating it.** The effect is real, it is measurable, and rehearsing the task does not remove it. The only structural defense is to make sure there is always somewhere for an unanticipated perception to go: an open, unbounded response channel alongside the closed items, and an interviewer who follows up on what appears there. That is a design property of conversational research and an inherent limitation of fixed forms.\n\n## The experiment that pinned it down\n\nClark and Lawless published the definitive demonstration in *Chemical Senses* (volume 19, issue 6, 1994, pages 583 to 594), under the title *Limiting response alternatives in time-intensity scaling: an examination of the halo-dumping effect*. The design is simple enough to restate in a sentence: panelists rated a beverage containing a sweetener plus an aromatic flavouring, and a second beverage containing the sweetener alone, while the experimenters varied **how many scales they were allowed to use**.\n\nThe results, in the authors' own words:\n\n- \"The aromatic flavor caused an increase in sweetness intensity and especially so when the panelists were limited to sweetness responses only.\"\n- \"The odor-induced enhancement of sweetness was smaller when panelists were given both flavor and sweetness response options than when the panelists were given only a sweetness scale.\"\n\nSo the measured sweetness of a fixed physical stimulus changed depending on what else the rating form allowed people to say. Nothing about the product moved. The instrument moved.\n\nTwo further findings make this harder to dismiss than a typical order effect.\n\n**Practice does not fix it.** \"Prior use of both scales in a previous experimental session did not lessen the halo-dumping enhancement effect.\" Respondents who had already used the full set of scales in an earlier session still dumped when the set was narrowed again. You cannot train it away, which rules out the comfortable explanation that it is a novice artifact.\n\n**It runs in both directions.** \"In one study, sweetness ratings of sucrose alone were depressed when the additional scale for flavoring was provided, perhaps due to inappropriate partitioning of responses.\" Adding a scale moved a score that had nothing to do with the added attribute. The distortion is not simply *missing attributes inflate their neighbours*; changing the attribute list at all can shift scores in either direction.\n\nA recent review by Spence and Di Stefano in *Psychonomic Bulletin and Review* (volume 33, issue 4, 2026) summarizes the mechanism as \"the tendency of participants to dump their feelings and experience onto whatever response scale they have been presented with, no matter whether those scales capture their experience or not.\"\n\nThe effect is also actively managed in current practice rather than treated as a historical curiosity. Jeong, Kwak and Lim, comparing two sensory profiling methods in *Foods* (volume 13, issue 17, 2024, article 2853), attribute a weak correlation between methods partly to scale coarseness, noting that a disparity in scale granularity \"may lead to a dumping effect, potentially limiting the discriminatory power\" of the coarser instrument. And Weir and colleagues, in a 2023 study in *Physiology and Behavior* (volume 271, article 114331), explicitly presented all of their intensity scales in every condition rather than only the relevant ones, stating that they did so to minimize dumping artifacts - a design decision taken purely to protect the measurement.\n\n## What this looks like in product research\n\nNothing about the mechanism is specific to taste. It requires only a perception, a set of scales, and no matching place to put it.\n\n- You ask about **ease of use** and **visual design**. You do not ask about **speed**. A user who found the feature sluggish has one usable channel for that irritation, and ease-of-use absorbs it. Your redesign then targets the interface, and the interface was never the problem.\n- You ask about **the product**. You do not ask about **support**. A customer who waited nine days for a ticket response rates the product lower. Product quality is now carrying a service-desk metric.\n- You ask a battery of **feature satisfaction** items with no item for **price**. Value perceptions distribute themselves across the battery, and the whole battery shifts down together in a way that reads like a broad quality problem.\n- You run a concept test with scales for **relevance** and **clarity** but none for **trust**. A concept that felt intrusive scores low on clarity, and you rewrite copy that was already clear.\n\nIn every case the arithmetic is untroubled and the conclusion is wrong. The averages are correct, the confidence intervals are honest, and the number is answering a question you did not ask.\n\n## This is not the halo effect\n\nThe two are easy to conflate and the confusion leads to the wrong fix.\n\nThe **halo effect** is a global impression bleeding into specific judgements: someone who likes the brand rates every attribute higher, including attributes they have never encountered. The cause is an overall evaluation, the direction is uniform, and the countermeasures are separation, order and forcing discrimination - covered in [the halo effect in customer research](/docs/halo-effect-customer-research).\n\nThe **dumping effect** is a specific, real perception with no matching scale being displaced onto an adjacent one. The cause is instrument incompleteness, the direction depends on which scale is nearest, and no amount of randomizing, separating or reordering helps, because the missing channel is still missing.\n\nOne useful diagnostic: halo predicts that attributes move *together*; dumping predicts that a *particular* attribute moves when an *unrelated* attribute is added to or removed from the form. That second pattern is what Clark and Lawless observed.\n\n## You cannot detect it by reading the responses\n\nThis is the part worth sitting with, and it is why the effect survives in mature research programs.\n\nAn inflated rating is not malformed. It is a plausible number from an attentive respondent who answered the question you asked, in good faith, on the scale you supplied. There is no data-quality signal to catch - no straightlining, no speeding, no contradiction. Reviewing individual responses more carefully cannot surface it, and neither can a larger sample, because the distortion is a property of the **instrument**, and every respondent is being measured with the same distorted instrument. More responses give you a more precise estimate of the wrong quantity.\n\nThe only way to see it is to **change the instrument and watch the scores move**. Two practical protocols:\n\n**The added-attribute test.** Field two versions of the same questionnaire on matched samples, identical except that version B adds one attribute. If the shared attributes score differently across versions, the added attribute was being dumped into them. This is a split-ballot design applied to the attribute list rather than to question wording.\n\n**The open-channel audit.** Add an unstructured *anything else you noticed?* question and code what comes back. Any theme that appears there with real frequency and has no matching closed item is a dumping candidate, and its likely landing zone is the nearest semantic neighbour on your form.\n\nThe second is cheaper, runs on every study, and is the one to institutionalize.\n\n## Why this is a structural argument for conversational research\n\nHere is the uncomfortable implication for the standard survey stack. The fix for dumping is a complete attribute list. A complete attribute list requires knowing in advance everything a respondent might notice. If you knew that, you would not be doing exploratory research.\n\nSo completeness is unreachable, and the practical target shifts: not *enumerate every attribute*, but *guarantee that an unanticipated perception has somewhere to land other than your nearest scale*.\n\nThat is exactly what an unbounded response channel provides, and it is where Koji's approach differs structurally rather than cosmetically from a form builder:\n\n- **Every closed question can carry an open follow-up.** Pair `scale`, `single_choice`, `multiple_choice`, `ranking` and `yes_no` items with `open_ended` probes so the respondent always has a non-numeric outlet. The six question types and how to combine them are covered in the [structured questions guide](/docs/structured-questions-guide).\n- **The AI interviewer probes what it did not expect.** When a participant volunteers something outside the attribute list, Koji follows the thread rather than discarding it. A static form cannot do this - not because of a missing feature, but because there is no one there to notice.\n- **Themes surface without being pre-declared.** Because Koji analyzes every transcript rather than a sample, a perception with no matching scale shows up as a named theme in the report. That is the open-channel audit above, running automatically on every study.\n- **The closed data stays intact.** You still get the distributions and the crosstabs from Koji's structured questions. You just also get the thing the distributions were quietly absorbing.\n- **It stays affordable enough to do routinely.** A text interview costs 1 credit and a voice interview 3, so the open channel is not a luxury reserved for a handful of moderated sessions.\n\nSurveyMonkey, Typeform and Qualtrics all let you bolt an open text box onto the end of a form. The difference is what happens to it: a text box collects an answer nobody follows up on and most teams never code. Koji treats it as the beginning of a conversation, which is the only version of this that actually recovers the dumped perception.\n\n## The general principle\n\nEvery closed instrument imposes a vocabulary, and every vocabulary is incomplete. The dumping effect is what incompleteness looks like in the data: **the answer to a question you did not ask, recorded in the answer to one you did.** It is invisible in the response, invisible in the analysis, and visible only when you change the instrument - which means the discipline it demands is not sharper scrutiny of your data, but a standing habit of leaving a door open.\n\n## Frequently asked questions\n\n### How large is the dumping effect in practice?\n\nIt varies with how closely the missing perception resembles the available scales, and no single number generalizes across domains. What the Clark and Lawless work establishes is that it is large enough to change a conclusion: the same physical stimulus produced systematically different intensity ratings depending only on which scales were offered. Treat it as a threat to validity rather than as a correction factor to subtract.\n\n### Does adding more attributes solve it?\n\nPartly, and it introduces its own costs. More scales means longer questionnaires, more fatigue and more straightlining, and the same research showed that adding a scale can itself shift an unrelated rating. The better strategy is a short, well-chosen closed set - built the way an [attribute lexicon](/docs/attribute-lexicon-reference-anchors-research) is built - plus a genuine open channel, rather than an ever-expanding grid.\n\n### Is an open text box at the end enough?\n\nIt is better than nothing and much weaker than a follow-up. A single end-of-survey box gets short, low-effort answers, is frequently skipped, and typically goes uncoded. The value comes from probing at the moment the perception is live, which is what an AI-moderated interview does by default and what [open-ended questions in AI interviews](/docs/open-ended-questions-ai-interviews) covers in detail.\n\n### Can dumping affect just-about-right scales too?\n\nYes, and the consequences are more expensive because JAR data feeds directly into a fix list. If an attribute has no JAR item, dissatisfaction with it lands on a neighbouring item and shows up as an off-target reading you will then try to fix. Because [penalty analysis](/docs/penalty-analysis-jar-fix-list) ranks attributes by the liking they cost, a dumped perception can promote the wrong attribute to the top of a roadmap.\n\n### How do I check an existing study retrospectively?\n\nCode the open-ended responses you already have and compare the resulting theme list against the closed attribute list. Themes with meaningful frequency and no matching closed item are your dumping candidates. Then check whether the closed attribute nearest each candidate scored unusually low relative to your benchmarks - that is where the perception most likely landed.\n\n### Does this apply to voice interviews as well as text?\n\nThe mechanism applies to any closed response format in either mode. The protection is the same in both: conversational modes have an unbounded channel by construction, so a perception with no scale still gets spoken aloud, recorded and analyzed rather than silently redistributed into a number.\n\n## Related Resources\n\n- [Structured Questions in AI Interviews](/docs/structured-questions-guide) - pairing closed items with open probes across the six question types\n- [The Halo Effect in Customer Research](/docs/halo-effect-customer-research) - the related but distinct distortion, and why the fixes differ\n- [Attribute Lexicons and Reference Anchors](/docs/attribute-lexicon-reference-anchors-research) - choosing and defining the closed set in the first place\n- [Just-About-Right Scales](/docs/just-about-right-scale-product-research) - directional attribute items and how dumping distorts them\n- [Penalty Analysis](/docs/penalty-analysis-jar-fix-list) - what happens downstream when a dumped perception reaches a fix list\n- [Open-Ended Questions in AI Interviews](/docs/open-ended-questions-ai-interviews) - how Koji probes free-form answers for depth","category":"Research Methods","lastModified":"2026-08-25T03:29:32.906017+00:00","metaTitle":"The Dumping Effect: How a Missing Attribute Distorts Your Ratings | Koji","metaDescription":"Respondents dump perceptions your questionnaire has no scale for onto the nearest scale you offered. Why it is invisible in the data, and the only real fix.","keywords":["dumping effect","halo dumping","attribute list bias","missing attribute survey","questionnaire artifact","rating scale distortion","survey instrument bias"],"aiSummary":"The dumping effect is the displacement of a perception onto the nearest available rating scale when no matching scale exists. Clark and Lawless demonstrated in 1994 that measured intensity of a fixed stimulus changed with the number of scales offered, that practice did not reduce it, and that adding a scale could depress an unrelated rating. It cannot be detected by inspecting responses, only by changing the instrument.","aiPrerequisites":["Experience designing closed-form attribute batteries","Familiarity with common response biases such as the halo effect"],"aiLearningOutcomes":["Distinguish the dumping effect from the halo effect by cause and by fix","Explain why larger samples cannot correct an instrument artifact","Run an added-attribute split test and an open-channel audit","Design studies with an unbounded response channel alongside closed items"],"aiDifficulty":"advanced","aiEstimatedTime":"12 min"}],"pagination":{"total":1,"returned":1,"offset":0}}