{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-11-10T01:06:56.301Z"},"content":[{"type":"blog","id":"181d999a-f60f-4ecf-8744-0cde7d41cf90","slug":"translate-rating-scales-multilingual-study","title":"Translating Rating Scales and Answer Options: Keeping a Multilingual Study Comparable","url":"https://www.koji.so/blog/translate-rating-scales-multilingual-study","summary":"How to translate rating scales and answer options so a multilingual survey or interview study stays comparable. Scales drift in four ways: intensity words (the ESS Round 11 translation checklist says extremely and completely must be translated as true end points), the midpoint, order and polarity, and response style by language (Harzing 2006, 26 countries: English-language questionnaires drew more middle responses, native-language ones more extreme responses). Harzing et al. 2009 (1,965 undergraduates and 1,714 MBA students, 16 countries) found 7-point scales and especially ranking reduced response and language bias versus 5-point scales. A GESIS experiment found no systematic differences between four German agree-disagree versions. Steps: translate labels as a set, keep point count and order, localise options, store answers against a master version, review with a fluent speaker, pilot, and read the reasons behind numbers before comparing markets. Koji translates questions, options, scale labels and ranking items on publish, stores answers against the original options, and has a Viewing in toggle to review and edit each language.","content":"> **In short:** A rating scale can lose comparability in translation even when every word is \"correct\". The labels shift in intensity (\"strongly\", \"completely\"), the midpoint changes meaning, and people answer differently in different languages: in a 26-country study, English-language questionnaires drew more middle answers and native-language questionnaires more extreme ones (Harzing, 2006). To keep a multilingual study comparable, translate the labels as a set rather than word by word, keep the same number of points and the same order, have a fluent reviewer check every label and option, store answers against one master version, and compare languages with care. Read the reasons behind the numbers before you call a gap between markets real.\n\n**Running a study in several languages this week?** Jump to [the per-language review checklist](#a-review-checklist-for-every-language). The rest of this guide explains why each step matters.\n\nYou run the same satisfaction question in English, German and Spanish. Germany comes out lower. Is the product worse there, or did \"very satisfied\" and \"sehr zufrieden\" set a different bar? Unless you designed for comparability, you cannot tell. Open questions survive translation reasonably well, because you read what people meant. Scales and answer options are harder: each label carries an intensity, each option draws a boundary, and the numbers that come out look precise whether or not they mean the same thing.\n\n## Four ways a translated scale drifts\n\nSurvey methodologists have studied this for decades. Most problems fall into four groups.\n\n![Framework of four ways a translated rating scale drifts: intensity words, the midpoint, order and polarity, and response style by language.](/visuals/four-ways-a-translated-rating-scale-drifts/four-ways-a-translated-rating-scale-drifts.png \"Check each one in every language before you compare markets. · Source: Koji synthesis, after Harzing (2006) and ESS translation guidelines\")\n\n### 1. Intensity words\n\n\"Strongly agree\", \"extremely likely\" and \"completely satisfied\" do not have exact equivalents in every language. A translator may pick a word that is a little stronger or a little weaker, and the scale's end point moves. The European Social Survey's translation quality checklist for Round 11 tells translators that intensifiers such as \"extremely\" and \"completely\" in answer scales must be translated as extreme endpoints, so that \"nothing can go beyond it\", and that identical scales used throughout the questionnaire are translated consistently (ESS R11 Translation Quality Checklist, 2024). If one of Europe's largest cross-national surveys writes that down as a rule, it is worth following in a product study.\n\n### 2. The midpoint\n\nAn English midpoint like \"Neither agree nor disagree\" can become \"partly agree, partly disagree\" in another language. Those are different statements: one is indifference, the other is a mixed view. Decide what your midpoint means, write that meaning down for the translator, and check that every language says the same thing.\n\n### 3. Order and polarity\n\nKeep the same number of points, the same direction (negative to positive, or the reverse) and the same kind of scale (one that runs from \"not at all\" to \"extremely\", or one that runs from \"very dissatisfied\" to \"very satisfied\") in every language. Do not let a translator turn a 5-point scale into 4 points because one label is awkward, or flip the order to match a local convention. The Dolnicar and Grün study on \"translating\" between survey answer formats (Journal of Business Research, 2013) starts from the problem that the sheer variety of answer formats makes results nearly impossible to compare across studies. Changing the format between languages creates the same problem inside one study.\n\n### 4. Response style by language\n\nEven a perfect translation does not guarantee that people use the scale the same way. In a study across 26 countries, Harzing (2006, International Journal of Cross Cultural Management) found large national differences in acquiescence (agreeing regardless of content) and extreme responding, linked to cultural traits such as power distance and collectivism. The language mattered too: English-language questionnaires drew more middle responses, and questionnaires in the respondent's own language drew more extreme ones. So if you run one market in English as a shortcut, you can create a gap that is only about the language.\n\n## What helps, according to the research\n\n- **Translate in the respondent's own language.** The Harzing result is a reason not to make non-native speakers answer in English to \"keep it simple\".\n- **Consider ranking for comparisons.** In two studies with 1,965 undergraduate and 1,714 MBA students in 16 countries, Harzing and colleagues (International Business Review, 2009) found that both a 7-point scale and a ranking format reduced response and language bias compared with a 5-point scale, and that ranking generally did better. If what you really need is \"which of these matters most?\", a ranking question may travel better than five separate rating scales.\n- **Use a team process, not back-translation alone.** The TRAPD approach (translation, review, adjudication, pretesting and documentation), used by the European Social Survey, puts a reviewer and a pretest after the first translation. We explain why back-translation alone misses problems in [translating research questions with TRAPD](/docs/translating-research-questions-trapd).\n- **Do not assume every difference is a problem.** A GESIS survey experiment (Schick, Behr, Neuert and Lechner, Survey Research Methods) tested four German versions of a five-point agree-disagree scale, including the versions used by two large international survey programmes, which differ in polarity and in the intensity of their end labels. It found no systematic differences in the answers or in response quality. The authors call for replication with other samples and other language pairs. The lesson is to check, not to panic.\n\n## Answer options: the other half of the problem\n\nChoice questions have their own traps, separate from the scale labels:\n\n- **Categories that do not exist in every market.** Job titles, education levels, company types and payment methods differ by country. \"Community college\" or \"GmbH\" needs a local equivalent, not a literal translation.\n- **Bands that cross a different line.** Income or price bands in one currency may cut through a meaningful local threshold in another. Set bands per market where it matters, and keep the analysis plan in mind.\n- **Options that get longer.** German and Finnish labels can run much longer than English ones. Check that options still fit and still read as parallel.\n- **Order effects.** If option order matters, shuffle it for everyone, in every language, so no market sees a different first option.\n- **One master answer.** Whatever the participant reads, store the answer against one master option, so \"Sehr zufrieden\" and \"Very satisfied\" count as the same answer in your chart.\n\n## A review checklist for every language\n\nRun this with a fluent speaker, ideally someone who knows the product category, before you launch.\n\n![Checklist for reviewing a translated rating scale or answer option set before launch, from point count and order to a pilot in each language.](/visuals/translated-rating-scale-review-checklist/translated-rating-scale-review-checklist.png \"A fluent reviewer and a small pilot catch most problems before they reach your data.\")\n\n1. Same number of points, in the same order, in every language.\n2. End labels are equally strong: \"extremely\" and \"completely\" are true end points.\n3. The midpoint means the same thing (indifference or a mixed view, not one in one language and the other elsewhere).\n4. Identical scales are worded identically everywhere they appear.\n5. Every answer option has a real local equivalent, or a local replacement.\n6. Bands (income, price, company size) make sense in each market.\n7. Options read as parallel and none is noticeably longer or more attractive.\n8. Answers are stored against one master version of each option.\n9. A short pilot in each language reads as natural, and people explain their answers the way you expected.\n\n## Comparing results across languages\n\nEven after a careful translation, treat a gap between markets as a question, not a finding.\n\n![Two colleagues comparing printed results side by side at a table, checking whether a gap between markets is real](https://images.unsplash.com/photo-1713947506934-c0b6519e069d?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3w5NDUzMDN8MHwxfHNlYXJjaHw2fHxjb2xsZWFndWVzJTIwcmV2aWV3aW5nJTIwZG9jdW1lbnRzJTIwdGFibGV8ZW58MXwwfHx8MTc5NDIwNzAxNHww&ixlib=rb-4.1.0&q=80&w=1080)\n\n_Photo by [Vitaly Gariev](https://unsplash.com/@silverkblack?utm_source=koji&utm_medium=referral) on [Unsplash](https://unsplash.com/?utm_source=koji&utm_medium=referral)_\n\n- **Look at the distribution, not just the mean.** If one market uses the end points far more, the averages can differ while the order of items is the same.\n- **Compare within a market first.** \"In Germany, onboarding scores lower than support\" is safer than \"Germany scores lower than Spain\".\n- **Read the reasons behind the numbers.** If German participants who chose 3 say the same things as Spanish participants who chose 4, the gap is in the scale, not the product.\n- **Test formally when the decision is big.** For high-stakes comparisons, check measurement invariance before comparing group means. Our guide to [measurement invariance](/docs/measurement-invariance-comparing-groups) explains when it matters.\n\n## How Koji handles translated scales and options\n\nIn Koji, you choose the interview languages for a study, and when you publish, Koji translates every question into each language: the question text, the answer options, the scale labels and the ranking items. Participants see the whole study in their own language, by text or voice.\n\n![Koji interview welcome page with the language menu open, showing English, Spanish, French, German and more](/email/shots/shot-language-picker.jpg \"Participants choose their language, and every question follows.\")\n\nTwo details matter for comparability:\n\n- **Answers are stored against the original options.** A participant reads the translated label, but the answer is recorded against the master option, and a scale answer is stored as its number. Charts, exports and screener results are comparable across languages without any recoding.\n- **You review every language.** The machine translation is a first draft, like the first T in TRAPD. A **Viewing in** toggle in the Research Brief shows the whole study in any enabled language, so a fluent reviewer can check each label and option against the checklist above and edit the wording. Hand edits are kept. If you later change or reorder a source question, its translations are marked out of date and the original wording shows until the next publish translates it again.\n\nVoice and phone interviews use the reviewed wording. And because every structured answer can come with a follow-up in the participant's own words, you can read why people in each market chose what they chose, which is the most direct check that a gap is real. This works on every plan, including Free. For the full setup, see [running research in multiple languages](/docs/multilingual-research-guide).\n\nMachine translation is not certified translation. For regulated, legal or clinical wording, use a professional translator and keep the Koji review step for the final check.\n\n## The short version\n\n- Translate the scale as a set, with the intensity of each label pinned down.\n- Keep the point count, order and kind of scale identical in every language.\n- Localise answer options where categories differ, and store answers against one master version.\n- Ask people in their own language, and consider ranking when the comparison is the point.\n- Have a fluent reviewer check every language, then pilot.\n- Before you call a market gap real, read the reasons behind the numbers.\n\n**Try it:** [create a free Koji account](/), add a second interview language to a study, publish, and use **Viewing in** to review every scale label and option with a fluent colleague before you launch. Planning a new market? Read [research a new country before you translate](/blog/research-a-new-country-before-you-translate) next.\n","category":"Research","lastModified":"2026-11-09T07:20:14.187353+00:00","metaTitle":"How to Translate Rating Scales and Keep Results Comparable","metaDescription":"Translating Likert scales and answer options for a multilingual study: four ways scales drift, what cross-national research shows, and a review checklist for every language.","keywords":["translate survey rating scale into other languages","translate likert scale","multilingual survey answer options","cross-language survey comparability","survey translation checklist"],"aiSummary":"How to translate rating scales and answer options so a multilingual survey or interview study stays comparable. Scales drift in four ways: intensity words (the ESS Round 11 translation checklist says extremely and completely must be translated as true end points), the midpoint, order and polarity, and response style by language (Harzing 2006, 26 countries: English-language questionnaires drew more middle responses, native-language ones more extreme responses). Harzing et al. 2009 (1,965 undergraduates and 1,714 MBA students, 16 countries) found 7-point scales and especially ranking reduced response and language bias versus 5-point scales. A GESIS experiment found no systematic differences between four German agree-disagree versions. Steps: translate labels as a set, keep point count and order, localise options, store answers against a master version, review with a fluent speaker, pilot, and read the reasons behind numbers before comparing markets. Koji translates questions, options, scale labels and ranking items on publish, stores answers against the original options, and has a Viewing in toggle to review and edit each language.","aiKeywords":["survey translation","Likert scale translation","cross-cultural response styles","TRAPD","measurement invariance","multilingual research","answer option localisation"],"aiContentType":"guide","faqItems":[{"answer":"Only with care. Translated labels can shift in intensity, midpoints can change meaning, and people answer differently by language. Keep the scale identical in form, review every label with a fluent speaker, compare distributions as well as means, and read the reasons behind the numbers. For high-stakes comparisons, test measurement invariance.","question":"Can you compare Likert scale results across languages?"},{"answer":"Usually not. In Harzing's 26-country study, English-language questionnaires drew more middle responses and native-language questionnaires more extreme ones, so answering in English can create a gap that is only about the language. Ask people in their own language and translate carefully.","question":"Should non-native speakers answer in English to keep a study consistent?"},{"answer":"No. Back-translation can return the right English words while the target-language label is unnatural or has a different strength. A team approach such as TRAPD (translation, review, adjudication, pretesting, documentation) with a fluent reviewer and a pilot catches more.","question":"Is back-translation enough to check a translated scale?"},{"answer":"Often. Harzing and colleagues (2009), with 3,679 students in 16 countries, found that a 7-point scale and a ranking format both reduced response and language bias compared with a 5-point scale, and ranking generally did better. Use ranking when the question is which item matters most.","question":"Is ranking better than rating for cross-country comparisons?"},{"answer":"Koji translates question text, answer options, scale labels and ranking items into each interview language when you publish, but stores every answer against the original option, and scale answers as numbers. A Viewing in toggle lets a fluent reviewer check and edit each language. It works on every plan, including Free.","question":"How does Koji keep translated answers comparable?"}],"relatedTopics":["survey translation","cross-cultural research","Likert scales","multilingual research","response styles"]}],"pagination":{"total":1,"returned":1,"offset":0}}