Plain Language Survey and Interview Questions: How to Write Them

Short answer: write research questions in plain language: common words, short sentences, one idea per question, and the key condition stated before the question rather than buried after it. Then test them with real participants before you field the study. Plain wording is not dumbing down. It reduces the share of answers that respond to a question you did not mean to ask, and it improves data from experts as well as from everyone else.

Every research question passes through a participant's comprehension before it produces an answer. If people read the question differently, they are answering different questions, and the numbers you add up are not measuring one thing. That makes question readability a validity problem, not a style preference.

Why reading level matters more than you think

It is easy to write for the people in your team: highly educated, fluent in your product's vocabulary, comfortable with long sentences. Your participants are a wider group.

The OECD's Programme for the International Assessment of Adult Competencies (PIAAC) measures adult literacy directly. In the 2023 round, released by the US National Center for Education Statistics in December 2024, 28% of US adults scored at the lowest levels of literacy, up from 19% in 2017. Average literacy scores were lower in 2023 than in 2017, and the US was tied for 14th of the 31 countries and economies that took part. Adults at the lowest level can typically read short texts well enough to perform simple tasks, such as filling out a basic form. A dense, multi-clause question is not a simple task.

Even in a B2B study of educated professionals, comprehension is fragile. People answer quickly, on a phone, between other tasks, and often in their second language. The question that seemed clear in a document review is read once, at speed.

What makes a question hard to understand

Timo Lenzner, Lars Kaczmirek and Alwine Lenzner tested this directly in a 2010 experiment published in Applied Cognitive Psychology. They compared well-written survey questions with versions containing specific problem features and measured response times as a sign of cognitive burden. At least six of the seven text features they tested made questions harder to process. In later work, Lenzner reported that respondents who received less comprehensible questions gave lower-quality answers than those who received easier versions.

The features to watch for include:

Low-frequency words. "Utilise", "facilitate", "remuneration". Use "use", "help", "pay".
Vague or abstract noun phrases. "Your experience with onboarding-related functionality" instead of "setting up your account".
Complex sentence structure. Long clauses placed before the main point ("Thinking about the last three months, and excluding any time spent on training, how...") force people to hold information in memory before they know what it is for.
Ambiguous terms. "Regularly", "team", "use" all mean different things to different people unless you define them.
Hidden double questions. "How easy and fast was it?" asks two things.
Unnecessary negatives. "Do you disagree that it is not difficult?" defeats most readers.

The survey methodologist Floyd J. Fowler Jr. put the goal simply in Improving Survey Questions: "Word questions so that every respondent is answering the same question." Each feature above works against that goal.

Plain language is not just for low literacy

The most common objection is that expert audiences will find plain wording patronising. The evidence runs the other way. Joseph Kimble, a long-standing researcher of legal drafting, summarised studies in which nearly 1,200 judges and lawyers in three US states were shown paired passages from legal documents, one traditional and one in plain language. In every state, at least 80% preferred the plain version. If people whose profession is reading complex text prefer plain text, your product managers and engineers will too.

Experts are also the group most exposed to jargon traps. They may know your industry term, but use it differently from you. Plain wording forces you to say what you mean.

How to write plain research questions

Lead with the context, keep it short

Put the time frame or condition first and keep it to a short phrase: "In the past month, how often did you export a report?" is easier than a question that ends with three qualifying clauses.

One question, one idea

If you find "and" or "or" joining two concepts, split the question. In interviews, a double question gets an answer to whichever half the participant remembers.

Use the participant's words, not yours

Your team says "workspace", "seat" or "integration". Customers may say "account", "login" or "connecting it to Slack". Early interviews and support tickets show you the words people actually use. Ask in those words.

Define anything that could vary

If "team" matters, say what you mean: "the people you work with day to day". Put the definition before the question, not after it.

Prefer concrete over abstract

"What happened the last time you tried to share a report?" is easier to answer accurately than "How would you describe your report-sharing experience?" Concrete questions are also less likely to invite generic answers; see Why a Vague Question Gets a Short Answer.

Keep answer options as plain as the question

Scale labels and choice options need the same care. "Somewhat" and "moderately" mean little in isolation. Short, distinct options are easier to tell apart.

Read it aloud

If you run interviews by voice, participants hear the question once and cannot re-read it. A question that needs a second reading on screen will fail out loud. Reading every question aloud before fielding is the quickest test there is.

Testing comprehension before you field

Readability formulas give a rough signal but miss ambiguity and jargon. The reliable test is people.

Paraphrase test. Ask a few people who match your audience to say the question back in their own words. Any paraphrase that changes the meaning is a rewrite.
Cognitive interviewing. Ask participants to think aloud as they answer, then probe specific words: "What did 'regularly' mean to you here?" See the Cognitive Interview Guide.
Pilot the whole study. A short pilot shows where people hesitate, ask for clarification or give answers that do not fit. See the Pilot Study Guide.
Check the answers, not just the questions. After fielding, look for questions with unusually short, off-topic or "it depends" answers. These are often comprehension failures, not disengaged participants.

Before and after

Before	After
To what extent has the implementation of the new permissions functionality facilitated your team's collaborative workflows?	Since we changed who can edit reports, has it become easier or harder for your team to work on reports together?
How satisfied and confident are you with our onboarding?	How easy was it to set up your account? (Then ask about confidence separately.)
Do you not find the export feature difficult to use?	How easy or hard is it to export a report?
Please indicate the frequency with which you utilise dashboards.	How often do you look at a dashboard?

How Koji helps

Koji runs AI-moderated interviews by text or voice. Plain wording matters more in this format, not less, because the question is delivered once, conversationally, often by voice. Koji supports this in several practical ways.

Test before anyone else sees it. The Preview tab on a study's Interviews page lets you take your own interview, by text or voice, exactly as a participant would. Previews are free, never use credits and never appear in the report. Run each question by voice to hear whether it holds up when spoken.
Follow-up probing catches confusion. Because the AI interviewer asks follow-up questions on the participant's answer, a reply that shows a misunderstanding can be explored instead of being recorded as a final answer. A static form records the wrong answer and moves on. You set how many follow-ups each question gets.
Edit wording with the brief assistant. The assistant that builds your study brief can change any question on request, and it tells you exactly what it changed. Ask it to shorten a question, remove jargon or split a double question.
Signals when an answer is uncertain. When Koji extracts answers to structured questions from the conversation, each extracted answer carries a high, medium or low confidence flag. A question that collects many low-confidence answers is a candidate for rewording. See Answer Confidence in Koji Reports.
Quality scores that reflect what people said. Every interview gets a quality score, capped by how much the participant actually said. A cluster of low-scoring interviews around one part of a study can point to a question participants could not engage with.
Interviews in the participant's language. Koji interviews in 28 languages, so participants who would struggle with a second language can answer in their first.

Compared with a traditional survey, where a confusing question produces silent, unusable data, an AI-moderated interview gives you a transcript of exactly where comprehension broke down. It does not fix a badly written question for you. The writing still matters most.

Key takeaways

Question comprehension is a validity issue: people who read a question differently answer different questions.
Over a quarter of US adults scored at the lowest literacy levels in the 2023 PIAAC round; write for your real audience, not your team.
Low-frequency words, abstract phrasing, complex structure, ambiguity, double questions and negatives all make questions harder and answers worse.
Experts prefer plain language too: in studies of nearly 1,200 judges and lawyers, at least 80% preferred plain versions.
Test comprehension with paraphrase tests, cognitive interviews and pilots before fielding.

Related Resources

Sources

National Center for Education Statistics (2024). Highlights of the 2023 U.S. PIAAC Results. Press release and report, December 10, 2024.
Lenzner, T., Kaczmirek, L., & Lenzner, A. (2010). Cognitive burden of survey questions and response times: A psycholinguistic experiment. Applied Cognitive Psychology, 24(7), 1003–1020.
Lenzner, T. (2011). A psycholinguistic look at survey question design and response quality. Doctoral dissertation, University of Mannheim.
Fowler, F. J., Jr. (1995). Improving Survey Questions: Design and Evaluation. Sage.
Kimble, J. (2012). Writing for Dollars, Writing to Please: The Case for Plain Language in Business, Government, and Law. Carolina Academic Press. See also Kimble's Plain Language columns in the Michigan Bar Journal.

Plain Language for Research Questions: Writing Questions Every Participant Can Understand