{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-07-30T17:59:21.367Z"},"content":[{"type":"documentation","id":"5bec4544-f763-4956-9113-2038bc1c5f3e","slug":"content-testing-guide","title":"Content Testing: How to Test Microcopy, Labels, and UX Writing With Real Users (2026)","url":"https://www.koji.so/docs/content-testing-guide","summary":"Content testing measures whether users understand and act correctly on interface copy, and differs from A/B testing by explaining why a variant confused people rather than only which one converted. Six methods cover most cases: cloze tests (delete every nth word, roughly 60% restoration indicates comprehensible prose), highlighter tests (mark confusing and reassuring phrases), comprehension checks (restatement in the users own words, never a yes/no did-you-understand), term-choice or nomenclature tests (single_choice plus ranking with probing on the mental model evoked), expectation tests (what will this button do, with a confidence scale), and label first-click for findability. Readability scores are a drafting aid, not evidence, because they cannot detect unfamiliar vocabulary or correct-sounding copy that describes the wrong behaviour. Never test on colleagues or power users. Sample 20-30 for clearly different term candidates and 50+ for close calls or segment comparisons. AI-moderated conversational studies remove the scheduling constraint: text mode for reading tasks, all six Koji structured question types in one pass, and automatic follow-up probing to capture the reasoning behind each choice.","content":"# Content Testing: How to Test Microcopy, Labels, and UX Writing With Real Users (2026)\n\n**The bottom line:** Content testing measures whether people understand and act correctly on your words — button labels, error messages, form hints, empty states, pricing copy, onboarding instructions. It is not proofreading, and it is not A/B testing: A/B tells you which variant converted, content testing tells you *why* one variant confused people, which is the part you can generalise to the next hundred strings. Six methods cover almost every case, and with an AI-moderated platform like Koji you can run them as a single conversational study across a hundred participants instead of scheduling twelve sessions.\n\n---\n\n## Why words deserve their own test\n\nCopy is the interface. A user does not experience your information architecture; they experience the eleven words on the screen that tell them what will happen if they click. Yet copy is usually the least-tested layer of a product: it changes late, it changes often, and it is written by people who have spent months internalising the exact vocabulary the user has never heard.\n\nThat last point is the core problem. Content testing exists because **the author cannot un-know the meaning.** You cannot self-assess comprehension of your own terminology, and neither can your colleagues — insiders share the company-specific assumptions that make unclear copy feel obvious. Testing with representative users is not a nice-to-have here; it is the only way to obtain the signal at all.\n\nThree symptoms suggest you have a content problem rather than a design problem:\n\n- Support tickets quote your own UI text back at you as the source of confusion.\n- Users complete a task but describe what they did incorrectly afterwards.\n- Two internal teams argue about a label and both cite \"clarity\" as the reason.\n\n---\n\n## The six methods\n\n### 1. Cloze test — does the copy hold together?\n\nThe classic comprehension measure. Take a passage, delete every *n*th word, and ask participants to fill in the blanks from context. For long-form prose, *n* = 6 is conventional; for microcopy you need a much smaller interval, and you want roughly 25 to 50 blanks in total to get a stable score.\n\n**Scoring:** count exact and near-synonym matches. A commonly used rule of thumb is that copy is comprehensible when participants restore around 60% or more of the missing words. Below that, the text is not predictable enough — usually because sentence structure is convoluted or terminology is unfamiliar.\n\n**Use it for:** legal-adjacent explanations, policy text, onboarding paragraphs, security or billing copy where misunderstanding is costly.\n\n**Do not use it for:** three-word button labels. There is nothing to restore.\n\n### 2. Highlighter test — where does confidence break?\n\nShow the copy and ask participants to mark anything confusing, and separately anything reassuring. Digitally this is a two-pass exercise: first the friction, then the trust.\n\nThe output is a heat map of specific strings rather than an overall score, which makes it the most directly actionable method for a writer. In a conversational study you replicate it by asking participants to quote back the exact phrase that felt unclear, then probing why. Koji's follow-up probing does the \"why\" automatically, which is where the rewrite instruction actually comes from.\n\n**Use it for:** pricing pages, plan comparison tables, consent and permission dialogs, anything where hesitation kills conversion.\n\n### 3. Comprehension check — did the meaning survive?\n\nShow the copy, remove it, then ask what it meant and what would happen next. The rule is to ask for a restatement in the participant's own words, never a yes/no \"did you understand?\" — everyone says yes.\n\nStrong variants:\n- \"In your own words, what does this setting do?\"\n- \"If you click this, what happens to your existing data?\"\n- \"Who is this plan for?\"\n\n**Use it for:** destructive actions, permissions, data-sharing explanations, plan selection.\n\n### 4. Term-choice test — which word wins?\n\nPresent two to five candidate terms for the same concept and ask which one participants would expect to lead where. This is nomenclature research and it is the highest-leverage content test in most products, because the term propagates across navigation, docs, support macros, and sales collateral.\n\nRun it as a `single_choice` question for the winner plus a `ranking` question when you need the full preference order — then probe the choice: \"what did you expect *Workspace* to contain that *Project* would not?\" The reasoning matters more than the vote count, because it tells you what mental model the term evoked.\n\n### 5. Expectation test — what do they think the button does?\n\nShow the control in isolation, before the click. Ask what the participant expects to happen, how confident they are, and what would make them hesitate. A `scale` question captures confidence; the probed open-ended answer captures the reason.\n\nThis catches the most expensive class of copy failure: labels that are perfectly clear and describe the wrong thing.\n\n### 6. Label first-click — can they find it at all?\n\nPresent the navigation or menu labels and ask where they would go to accomplish a specific task. Findability failures are frequently vocabulary failures in disguise; if 40% pick the wrong label, the problem is rarely the layout.\n\nPair it with the term-choice test: first learn what people call the thing, then verify that the label you chose is where they look.\n\n---\n\n## Readability scores are not content testing\n\nFlesch-Kincaid, grade-level scores, and their cousins measure sentence and word length. They do not know whether *reconciliation*, *entitlement*, or *seat* means anything to your audience, and they cannot detect a perfectly readable sentence that describes the wrong behaviour. Use readability tools as a drafting aid and a floor, never as evidence. Comprehension is a property of the reader, not of the text — which is why it has to be measured with readers.\n\n---\n\n## Running content tests at conversational scale\n\nThe traditional constraint on content testing is throughput. Each method above is cheap to run once and painful to run twenty times: you schedule sessions, read copy aloud, take notes, and hand-code the answers. In practice teams test the redesign and skip the two hundred strings that ship every quarter.\n\nAn AI-moderated study removes the scheduling layer. A few specifics that matter for content work:\n\n- **Use text mode for reading tasks.** Copy has to be *seen*. Text interviews let participants read the string and respond to it; voice is better suited to the expectation and reasoning parts of the study. Koji supports both from a single link, so a mixed design works.\n- **Mix structured and open in one pass.** Term choice as `single_choice`, preference order as `ranking`, confidence as `scale`, comprehension restatement as `open_ended` with probing, and a quick `yes_no` on whether the participant would take the action. The six [structured question types](/docs/structured-questions-guide) mean one study returns both the vote counts and the reasoning — no second round.\n- **Let the AI ask the \"why\".** The value in content testing sits entirely in the follow-up: not \"which label did you pick\" but \"what did you expect that label to contain?\" Koji probes each answer up to three times automatically, which is exactly the interviewer behaviour a form cannot replicate and a busy team rarely sustains by hand.\n- **Recruit outsiders.** Never test copy on colleagues, and be careful with power users, who have already learned your vocabulary. Screen for the segment whose comprehension you actually care about — often new or prospective users.\n- **Watch for order effects.** Showing variant A before variant B primes the comparison. Randomise where you can and keep the sequence identical across participants where you cannot, so the bias is at least constant. See [question order bias](/docs/question-order-bias-guide).\n\n### How many participants?\n\nMore than qualitative usability work, less than a survey. Comprehension is a proportion, and proportions need a bit of sample: 20 to 30 participants gives a usable read on a term-choice test between two clearly different candidates; 50 or more if the options are close or you need to compare segments. Cloze tests are more forgiving because each participant contributes dozens of data points. For the reasoning layer, the usual qualitative saturation logic applies — see [how many user interviews you need](/docs/how-many-user-interviews).\n\n---\n\n## A ready-made study structure\n\nA single 8-to-12 minute conversation can cover an entire feature's copy:\n\n1. **Warm-up** (`open_ended`) — What do you use a tool like this for today?\n2. **Expectation** (`open_ended` + `scale`) — Here is the button. What happens if you click it? How confident are you, 1 to 7?\n3. **Comprehension** (`open_ended`, probed) — Here is the explanation text. In your own words, what does it mean for your data?\n4. **Highlight** (`open_ended`, probed) — Which exact phrase, if any, made you pause? Why that one?\n5. **Term choice** (`single_choice` + probe) — Which of these would you click to find your saved work?\n6. **Preference order** (`ranking`) — Rank these four headings by how clearly they describe the page.\n7. **Action** (`yes_no`, probed) — Based only on this copy, would you turn the setting on?\n\nKoji aggregates the structured items into distributions automatically and themes the open-ended answers with supporting quotes, so the writer receives \"62% expected *Archive* to delete the file, and here are the eleven quotes explaining why\" rather than a folder of recordings.\n\n---\n\n## What to do with the results\n\n- **Rewrite against the failure, not the score.** A 45% cloze score tells you the passage failed; the highlighter quotes tell you which clause did it.\n- **Fix the term everywhere at once.** A nomenclature change that lands in the UI but not in docs, emails, and support macros creates a new comprehension problem.\n- **Keep a decision log.** Record the tested alternatives and the winning rationale. It ends the recurring internal argument and it is the fastest onboarding artefact a new writer can get.\n- **Re-test after the rewrite.** Content changes are cheap enough that a second wave is realistic, and comprehension gains are the easiest research win to demonstrate to stakeholders.\n\n---\n\n## Related Resources\n\n- [Structured Questions in AI Interviews](/docs/structured-questions-guide) — the six question types used throughout this guide\n- [Cognitive Interviews](/docs/cognitive-interview-guide) — the sibling method for testing whether your *questions* are understood\n- [Open-Ended vs Closed-Ended Questions](/docs/open-ended-vs-closed-ended-questions) — when to count and when to probe\n- [Choice and Ranking Questions in AI Interviews](/docs/choice-ranking-questions-guide) — running term-choice and preference-order tests\n- [Question Order Bias](/docs/question-order-bias-guide) — avoiding priming when comparing copy variants\n- [AI Usability Testing](/docs/ai-usability-testing-guide) — where content testing fits inside a broader usability study\n- [How Many User Interviews Do You Need?](/docs/how-many-user-interviews) — sample size for the qualitative layer\n","category":"Research Methods","lastModified":"2026-07-30T03:25:36.132045+00:00","metaTitle":"Content Testing: Test Microcopy & UX Writing (2026 Guide)","metaDescription":"Six ways to test whether your UX copy works — cloze tests, highlighter tests, comprehension checks, term-choice tests, expectation tests, and label first-click — plus how to run them conversationally at scale.","keywords":["content testing","ux writing research","microcopy testing","copy testing method","cloze test ux","comprehension testing","terminology testing","label testing","ux content research","test button labels"],"aiSummary":"Content testing measures whether users understand and act correctly on interface copy, and differs from A/B testing by explaining why a variant confused people rather than only which one converted. Six methods cover most cases: cloze tests (delete every nth word, roughly 60% restoration indicates comprehensible prose), highlighter tests (mark confusing and reassuring phrases), comprehension checks (restatement in the users own words, never a yes/no did-you-understand), term-choice or nomenclature tests (single_choice plus ranking with probing on the mental model evoked), expectation tests (what will this button do, with a confidence scale), and label first-click for findability. Readability scores are a drafting aid, not evidence, because they cannot detect unfamiliar vocabulary or correct-sounding copy that describes the wrong behaviour. Never test on colleagues or power users. Sample 20-30 for clearly different term candidates and 50+ for close calls or segment comparisons. AI-moderated conversational studies remove the scheduling constraint: text mode for reading tasks, all six Koji structured question types in one pass, and automatic follow-up probing to capture the reasoning behind each choice.","aiPrerequisites":["Draft copy or candidate labels ready to test","Access to participants outside your company"],"aiLearningOutcomes":["Choose between cloze, highlighter, comprehension, term-choice, expectation, and first-click tests","Score a cloze test and interpret the result","Write comprehension questions that avoid false yes answers","Run a nomenclature test that produces reasoning, not just votes","Size a content test sample","Structure a single conversational study covering a whole features copy"],"aiDifficulty":"beginner","aiEstimatedTime":"10 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}