{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-10-18T10:44:15.788Z"},"content":[{"type":"documentation","id":"47f93c38-60be-4d95-ae22-91af646324f3","slug":"observing-user-research-sessions","title":"How to Observe User Research Sessions: Rules for Observers and Note-Takers","url":"https://www.koji.so/docs/observing-user-research-sessions","summary":"Observing a user research session means watching a usability test or interview live without taking part, so the team sees users' problems first-hand. Jared Spool's exposure-hours research (2011) linked at least two hours of watching users every six weeks per team member to much better design outcomes, a rule GOV.UK's Government Digital Service adopted in 2014. The guide sets out four session roles, eight ground rules for observers based on Steve Krug's observer handout, how to protect participants (NN/g), why observers' expectations matter (Rosenthal and Rubin's review of 345 studies) and what Levitt and List's reanalysis says about the Hawthorne effect, remote observation, the debrief, and how Koji gives teams full transcripts and quote-backed findings from AI-moderated interviews.","content":"Observing a user research session means watching a usability test or interview live, without taking part in it, so the team sees users struggle and decide with their own eyes. It's one of the most effective things a team can do with research, and also the easiest to get wrong. Done well, observers stay silent and invisible to the participant, write down what they see rather than what they think, pass questions to the moderator instead of asking them, and meet straight after the session to compare notes. Done badly, they laugh, whisper, interrupt, or leave with a single vivid anecdote they treat as the finding.\n\nThis guide covers why observation matters, the roles in a session, the ground rules to give observers, how to run remote observation, and how to turn what observers saw into decisions.\n\n## Why getting the team to watch matters\n\nThe strongest argument for observation comes from Jared Spool's research at UIE. In a 2011 article on what he called \"exposure hours\", Spool reported that teams where each member spent at least two hours every six weeks watching real users saw far greater improvements to their designs than teams that fell short of that. Teams that watched more often did better still. Spool presented it as a correlation, and a striking one: the gain appeared when the whole team watched, not when only the researchers did.\n\nThe UK Government Digital Service took up the same rule in 2014, recommending that everyone in a product team observe user research for at least two hours every six weeks.\n\nThe reason is simple. A report tells people what happened. Watching someone fail at a task you built makes the problem real in a way a slide never does, and it settles arguments that would otherwise go round in circles. Steve Krug built his whole low-cost testing method in _Rocket Surgery Made Easy_ around this: test with three participants one morning a month, get as many people as possible to watch, then debrief over lunch.\n\n## The roles in a session\n\n![Table of the four roles in an observed research session: moderator, note-taker, observers and backchannel lead, with what each does and avoids.](/visuals/user-research-session-roles-observers/user-research-session-roles-observers.png \"Give everyone a job, and nobody ends up running the session from the back of the room\")\n\n**Moderator.** Runs the session, talks to the participant and is the only person who does. Decides whether to follow up on observers' questions.\n\n**Note-taker.** Captures what happens as close to verbatim as possible, with timestamps, so the moderator can concentrate on the participant. One dedicated note-taker per session is the minimum; see [note-taking in user research](/docs/note-taking-user-research).\n\n**Observers.** Watch, note what they see and pass questions through the agreed channel. They don't speak to the participant or the moderator during the session.\n\n**Backchannel lead (optional).** In larger sessions, one person gathers observers' questions and passes the best ones to the moderator at a natural break. This keeps the moderator from being flooded with messages.\n\n## Ground rules to give observers\n\nBrief observers before every round, ideally in writing. Krug publishes a one-page handout for exactly this. Among its instructions: \"During the break between sessions, use the attached sheet to jot down the three most serious usability problems you noticed in that session.\" He also asks observers to stay as long as they can, leave quietly if they must, take calls and side conversations outside, and write their questions down rather than ask them.\n\n![Checklist of eight ground rules for observers of a user research session, from staying silent to writing down the top three problems after each session.](/visuals/observer-ground-rules-user-research-sessions/observer-ground-rules-user-research-sessions.png \"Share these with every observer before the first session · Source: Steve Krug, Rocket Surgery Made Easy\")\n\nA fuller set of rules that works for most teams:\n\n1. **Stay silent and still.** No reactions, no laughing, no sighs. In a shared room, participants notice, and they change what they do.\n2. **Arrive on time and stay for the whole session if you can.** Coming and going is distracting, and the ending is often where the explanation comes.\n3. **Write down what you see, not what you conclude.** \"Scrolled past the Save button twice\" is useful. \"The navigation is broken\" is a guess.\n4. **Note quotes word for word.** A participant's exact phrase is evidence; your paraphrase is an interpretation.\n5. **Send questions through the backchannel.** Never directly to the participant, and never in a way that interrupts the moderator mid-question.\n6. **Don't design solutions during the session.** You'll stop watching. Save ideas for the debrief.\n7. **Don't judge the participant.** The test is of the product, not the person. If they struggle, that's the finding.\n8. **Write down your top three problems after each session.** Before you talk to anyone else, so the group's view doesn't overwrite yours.\n\n## Protect the participant\n\nObservers change the situation, and the participant has a right to know about them. The Nielsen Norman Group's advice is to be honest: tell participants at the start that colleagues are watching and that the session is recorded, even if the recruitment emails already said so, and remind them they can stop at any time. Never tell a participant that nobody is watching when someone is.\n\nNN/g also gives the moderator authority over observers: \"When observers make unethical requests or act in ways that violate proper usability practices, researchers should act to protect the participant.\" That covers requests to push a participant who is upset, to ask for personal information, or to keep going after they've asked to stop. Agree before the session that the moderator has the final word.\n\nBe realistic about how much observation itself changes behaviour. The famous Hawthorne studies are often cited as proof that being watched transforms results, but when Steven Levitt and John List (2011) reanalysed the original illumination data, they found that the most famous patterns were not supported by it, though they did find more subtle signs of an effect. The lesson for practitioners is moderate: observation can shift behaviour, so keep observers out of sight where you can, and don't treat a nervous first five minutes as typical. See [the Hawthorne effect](/docs/hawthorne-effect-research).\n\n## Observers bring expectations too\n\nObservers aren't neutral. People see what they expect to see, especially when they built the thing being tested. Robert Rosenthal and Donald Rubin's 1978 review of 345 studies on interpersonal expectancy effects is the classic summary of how an observer's expectations shape what they record and how others behave. In a research session this shows up as the engineer who notices every moment the participant succeeds with their feature, and misses the five minutes of confusion before it.\n\nThree habits reduce it. Ask observers to write observations, not conclusions. Have each person record their top problems independently before the group talks. And look for patterns across sessions rather than acting on a single dramatic moment. More on this in [observer bias](/docs/observer-bias).\n\n## Observing remotely\n\nMost sessions are now remote, which makes observation easier to scale and easier to spoil.\n\n- **Keep observers hidden.** Cameras off and microphones muted, or better, watching through a separate stream so the participant sees only the moderator.\n- **Use a separate channel for questions.** NN/g suggests a messaging tool such as Slack so observers can talk among themselves and with the facilitator without disturbing the session.\n- **Tell the participant how many people are watching.** Being watched by a row of black squares is less comfortable than being told \"two colleagues are listening in\".\n- **Give observers a shared notes space.** One document or spreadsheet per study, so notes are in one place when the debrief starts.\n\n## Debrief straight after\n\nThe debrief is where observation pays off. Hold it right after the last session of the day, while memories are fresh, and keep it short.\n\n![A product team gathered around a table with sticky notes, comparing what each person noticed during the research sessions](https://images.unsplash.com/photo-1620326079720-500ba364af6a?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3w5NDUzMDN8MHwxfHNlYXJjaHwyfHx0ZWFtJTIwbWVldGluZyUyMHN0aWNreSUyMG5vdGVzJTIwZGlzY3Vzc2lvbnxlbnwxfDB8fHwxNzkyMzAyNjExfDA&ixlib=rb-4.1.0&q=80&w=1080)\n\n_Photo by [Brands&People](https://unsplash.com/@brandsandpeople?utm_source=koji&utm_medium=referral) on [Unsplash](https://unsplash.com/?utm_source=koji&utm_medium=referral)_\n\n1. Everyone reads out their top three problems from each session.\n2. Group the problems that came up more than once and agree how serious each one is.\n3. Pick the few that matter most to fix next. Krug's rule is to focus ruthlessly on the most serious problems rather than the long list.\n4. Record what you agreed and who owns each fix.\n\nA [rainbow spreadsheet](/docs/rainbow-spreadsheet), Tomer Sharon's method of logging observations in one shared sheet with a colour per participant, works well here: observers fill it in during sessions and the patterns are visible before the debrief begins. For more on running the meeting, see the [research debrief guide](/docs/research-debrief-guide).\n\n## How Koji helps\n\nKoji runs AI-moderated interviews by voice, text or phone. Participants take them in their own time, so there is no live session to sit in on, and no observers in the room to change the conversation. Observation moves from the session to the evidence.\n\n**Every transcript is open to the team.** Members of a study's workspace can read each interview's full transcript and its individual analysis: key findings, pain points, sentiment and a quality rating. The exposure Spool describes came from watching, and reading isn't the same thing. Still, asking each person on the team to read a few full transcripts every week puts them in direct contact with what customers actually say.\n\n**Findings link back to quotes.** Koji's report groups what participants said into themes across all interviews, and each finding links to the supporting responses. That's the same discipline the debrief asks of observers: a claim, and the evidence behind it.\n\n![Koji report summary with a side panel of supporting participant quotes behind one finding](/email/shots/shot-evidence-panel.jpg \"Each finding opens the quotes that support it, so nobody has to take a summary on trust\")\n\n**Share with the people who should be watching.** Studies can be shared with specific teams in the workspace, and reports can be published as a public page for stakeholders who don't use Koji.\n\n**Run it yourself first.** The Preview tab lets anyone on the team take the interview exactly as a participant would, by voice or text, without using credits. It's a quick way for a stakeholder to experience the study before reading the results.\n\n## Related Resources\n\n- [Note-taking in user research](/docs/note-taking-user-research)\n- [Observer bias](/docs/observer-bias)\n- [The Hawthorne effect](/docs/hawthorne-effect-research)\n- [Rainbow spreadsheet](/docs/rainbow-spreadsheet)\n- [Research debrief guide](/docs/research-debrief-guide)\n- [Structured questions guide](/docs/structured-questions-guide)\n\n## Sources\n\n- Government Digital Service (2014). Have you had your recommended dose of research? GOV.UK User Research blog.\n- Krug, S. (2010). _Rocket Surgery Made Easy_. New Riders. Including the handout \"Instructions for usability test observers\".\n- Levitt, S. D., & List, J. A. (2011). Was there really a Hawthorne effect at the Hawthorne plant? An analysis of the original illumination experiments. _American Economic Journal: Applied Economics_, 3(1), 224–238.\n- Nielsen Norman Group (2016). Team members behaving badly during usability tests.\n- Rosenthal, R., & Rubin, D. B. (1978). Interpersonal expectancy effects: The first 345 studies. _Behavioral and Brain Sciences_, 1(3), 377–415.\n- Sharon, T. (2013). The rainbow spreadsheet: A collaborative lean UX research tool. Smashing Magazine.\n- Spool, J. (2011). Fast path to a great UX: Increased exposure hours. UIE.\n","category":"Research Methods","lastModified":"2026-10-18T05:50:11.648255+00:00","metaTitle":"How to Observe User Research Sessions: Observer Rules (2026)","metaDescription":"Ground rules for observers of usability tests and user interviews: session roles, what to note, remote observation, protecting participants and debriefing.","keywords":["observing user research sessions","user research observers","usability test observers","observer guidelines","stakeholders observing usability tests","exposure hours","research debrief"],"aiSummary":"Observing a user research session means watching a usability test or interview live without taking part, so the team sees users' problems first-hand. Jared Spool's exposure-hours research (2011) linked at least two hours of watching users every six weeks per team member to much better design outcomes, a rule GOV.UK's Government Digital Service adopted in 2014. The guide sets out four session roles, eight ground rules for observers based on Steve Krug's observer handout, how to protect participants (NN/g), why observers' expectations matter (Rosenthal and Rubin's review of 345 studies) and what Levitt and List's reanalysis says about the Hawthorne effect, remote observation, the debrief, and how Koji gives teams full transcripts and quote-backed findings from AI-moderated interviews.","aiPrerequisites":["A planned usability test or round of user interviews"],"aiLearningOutcomes":["Explain why team observation of research improves product decisions","Assign the moderator, note-taker, observer and backchannel roles","Brief observers with clear ground rules","Run remote observation without disturbing participants","Turn observers' notes into decisions in a short debrief"],"aiDifficulty":"beginner","aiEstimatedTime":"10 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}