The Hawthorne Effect: How Being Observed Changes Research Results

The Hawthorne effect is the tendency of people to change their behavior because they know they are being observed. In research, it means participants try harder, behave more "correctly," or perform tasks more carefully than they ever would in real life — producing results that describe the study, not the world. It is one of the most important biases to understand in any observational method, from usability testing to diary studies.

The practical takeaway: the more visible and intense the observation, the more behavior distorts. You reduce the Hawthorne effect by lowering the felt presence of the observer, using natural tasks, observing over longer periods, and — wherever possible — collecting behavior in the participant's own environment and on their own time. Modern AI-moderated and unmoderated formats, like those on Koji, are powerful tools for exactly this.

What Is the Hawthorne Effect?

According to Statistics By Jim, "the Hawthorne effect occurs when experimental participants change their behavior because they know researchers are watching them." Typically the change is a short-term improvement in performance that vanishes once observation stops.

The name comes from a series of studies in the 1920s at Western Electric's Hawthorne Plant outside Chicago. Researchers wanted to know whether workplace lighting affected worker productivity. The strange finding: productivity rose when they brightened the lights — and rose when they dimmed them. Workers were responding not to the lighting, but to the knowledge that they were being studied. The act of observation was the real intervention.

Why It Matters in User Research

The Nielsen Norman Group — the most authoritative voice in UX research — treats the Hawthorne effect as a core threat to validity in user studies. The distortions are everywhere:

• Usability testing: Observed participants try harder, persist through friction they would normally abandon, and read instructions they would normally skip. Your success rates look better than reality.
• Moderated interviews: People give more thoughtful, more socially acceptable answers when a researcher is visibly present and listening.
• Field studies: A researcher in the room changes the very behavior they came to observe.
• Diary studies: Knowing entries will be read can make participants more diligent — or more performative — than they are day-to-day.

The Hawthorne effect is closely related to, and often discussed alongside, social desirability bias. The difference: social desirability bias distorts the answers people give, while the Hawthorne effect distorts the behavior people perform. Both are driven by the felt presence of a human evaluator.

The Hawthorne Effect vs. Observer Bias

These two are easy to confuse:

• The Hawthorne effect is about the participant changing their behavior because they know they are watched.
• Observer bias is about the researcher's own expectations unconsciously shaping how they record, interpret, or steer the session.

Well-designed research has to defend against both — and notably, automated and standardized data collection helps with each, because there is no human observer whose expectations leak into the moderation.

How to Reduce the Hawthorne Effect

Methodology sources, including Scribbr and NN/G, recommend a consistent set of techniques:

1. Reduce the felt presence of the observer. Conducting research online or asynchronously minimizes the researcher's presence and the performance pressure that comes with it.
2. Use blinding where appropriate. When participants don't know exactly which behavior or outcome you're measuring, they can't optimize toward it.
3. Observe over a longer period. The Hawthorne effect is largely a short-term spike; longitudinal designs let novelty wear off so natural behavior re-emerges. See our diary study guide.
4. Design natural, realistic tasks. The closer the task is to real life, the less room there is for performance.
5. Build rapport and lower the stakes. Reassure participants there are no right answers and you are testing the product, not them.
6. Collect behavior in the natural environment. Studying users in their own context, on their own devices, on their own schedule, reduces the "lab effect."

The Modern Approach: Designing Around the Observer

Here is the structural insight: the Hawthorne effect is strongest when a human is visibly watching and intensely present — a moderator on a video call, a researcher with a clipboard. The two most effective levers are therefore (1) removing the live human observer and (2) extending observation over time so the effect decays.

AI-moderated and unmoderated research are uniquely good at both:

• Lower felt observation. An AI moderator running an asynchronous interview does not create the same "someone important is watching me" pressure as a live human researcher. Participants relax into more natural responses.
• Natural setting and timing. Self-serve, always-on interviews happen wherever and whenever the participant actually is — not in the artificial frame of a scheduled, observed session.
• Longitudinal at scale. Because AI-moderated studies are cheap to run repeatedly, you can observe the same users over weeks, letting the short-term Hawthorne spike fade and revealing steady-state behavior.
• No observer bias on the moderation side. A consistent AI moderator asks every participant the same neutral questions, with none of the unconscious expectation-steering a tired human interviewer introduces.

How Koji Helps

Koji's research model is, by design, lighter on the observer than traditional moderated research:

• AI-moderated interviews run asynchronously and self-serve, so participants are not performing for a live human audience — reducing both the Hawthorne effect and observer bias in one move.
• Voice and text interviews at scale let you study far more participants over far longer periods, so the short-lived novelty spike decays and you see real, stable behavior.
• Six structured question types — open_ended, scale, single_choice, multiple_choice, ranking, yes_no — let you blend natural open-ended narrative with neutral, standardized measures that every participant experiences identically. See the structured questions guide.
• Consistent, neutral moderation means no researcher micro-reactions nudging behavior, and no expectation bias creeping into how sessions are run.
• Real-time reporting across many sessions makes it practical to compare early ("observed novelty") responses against later, settled behavior.

The point is not to eliminate observation — that is impossible — but to design research where the act of observing distorts behavior as little as possible. That is how you collect data about how people actually behave, not how they perform when they know you are looking.

Related Resources

• Social Desirability Bias — the answer-side counterpart to the Hawthorne effect
• Avoiding Bias in Interviews — a complete bias-resistant interviewing playbook
• Cognitive Biases in User Interviews — the wider family of research biases
• AI vs Human Moderators — when removing the live observer improves data
• Diary Study Guide — longitudinal observation that lets novelty fade
• Structured Questions Guide — standardized question types for consistent measurement