{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-10-27T18:01:24.558Z"},"content":[{"type":"blog","id":"83c77615-a766-4e37-a7bb-21c7617f4769","slug":"ai-drafted-requirements-validation-business-analyst-playbook","title":"AI Drafted the Requirements. Now Who Checks Them? A Business Analyst Playbook for Stakeholder Validation Interviews","url":"https://www.koji.so/blog/ai-drafted-requirements-validation-business-analyst-playbook","summary":"Read-back validation is a method for business analysts to check AI-drafted requirements against stakeholders. Each person rules on up to six specific requirements with four answers, a decoy is planted to catch rubber-stamping, and a requirement counts as validated at three verdicts from two roles with 80% right as written. Koji runs the interviews as short AI-moderated text or voice conversations and reports counts and quotes per requirement.","content":"If an AI assistant drafted your requirements, someone still has to check them against the people who will live with them. This playbook gives you a way to do that in a week: run short AI-moderated read-back interviews, where each stakeholder rules on a handful of specific requirements in their own words. By the end you can walk into sign-off with a verdict for every requirement, the people behind it and the quotes that explain it.\n\n## Why validating AI-drafted requirements matters now\n\nDrafting got cheap. Checking did not. IIBA's 2026 Global State of Business Analysis report draws on [2,120 professionals in 122 countries](https://vancouver.iiba.org/stories/the-iiba-2026-global-state-of-business-analysis-report), and IIBA's summary reports that 69% say AI is having a positive effect on their careers. IIBA's own practitioner writing on AI in validation is just as clear that the analyst [stays central, using judgment, context and stakeholder understanding to guide and validate AI output](https://www.iiba.org/business-analysis-blogs/enrich-business-analysis-experience-for-your-stakeholders-using-ai-tools/).\n\n![A business analyst reading a printed requirements draft at a desk with a laptop beside it, checking it before sign-off](https://images.unsplash.com/photo-1758876203474-4425ac96181b?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3w5NDUzMDN8MHwxfHNlYXJjaHw3fHxhbmFseXN0JTIwcmVhZGluZyUyMGRvY3VtZW50JTIwbGFwdG9wJTIwZGVza3xlbnwxfDB8fHwxNzkzMDg3NDEwfDA&ixlib=rb-4.1.0&q=80&w=1080)\n\n_Photo by [Vitaly Gariev](https://unsplash.com/@silverkblack?utm_source=koji&utm_medium=referral) on [Unsplash](https://unsplash.com/?utm_source=koji&utm_medium=referral)_\n\nThe draft is fast. The evidence that it is right is not. A 2025 study by Quattrocchi and colleagues had ten language models play the customer in interviews and write user stories. The stories matched human ones on coverage and style, but [showed lower diversity and met the acceptance quality criteria less often, whatever the model size](https://arxiv.org/abs/2507.15157). A 2026 survey of AI test generation from requirements found that [hallucination control is weakly addressed, with four of eight representative studies rated Low](https://arxiv.org/pdf/2606.06563).\n\nThere is a second trap, and it is a human one. In a 2026 REFSQ study, 26 participants rated [LLM revisions of the 130 requirement statements they had written](https://arxiv.org/abs/2601.16699) higher on alignment, readability, reasoning and unambiguity. The authors keep stakeholders in the validation loop. My reading of it: polished text is easy to approve. A clean draft invites a quick \"looks fine\", and \"looks fine\" is not the same as \"right\".\n\nRequirements were already where projects bleed. PMI's 2014 Pulse of the Profession found that [37% of organizations named inaccurate requirements as the primary reason for project failure](https://www.pmi.org/learning/library/requirements-management-survey-13449). PMI's 2026 Pulse says [about one in three complex projects fails, roughly twice the 13% rate for projects overall](https://www.pmi.org/about/press-media/2026/pulse-why-complex-projects-fail-best-practices-are-not-enough-system-thinking).\n\nThe cost of skipping validation: the draft is signed on the strength of a meeting where two people spoke, and the first honest review is the user acceptance test. By then the wrong requirement is code.\n\n## What good looks like\n\nValidation is not elicitation with a document attached. In elicitation you ask people what they need. In validation you show people a specific claim and ask whether it is true for them. I call the method **read-back validation**: each person reads back a small set of requirements, rules on each one, and says what is missing.\n\n**Package requirements in sets of six.** Group the draft by who it affects: approvals, data entry, reporting, exceptions. Give each interviewee one set of at most six requirements, plus one open question about what is missing. Past six, people start agreeing to finish. A set takes 10 to 12 minutes in text.\n\n**Four answers per requirement.** Each requirement gets one question with four options: right as written, right but needs changing, wrong, or not something I can judge. \"Not something I can judge\" matters. Without it, people guess to be polite, and the guess goes in your tally as a confirmation.\n\n**Cover every requirement three times, from two roles.** Whoever commissioned the work, plus at least two people who do it daily. A requirement checked only by its sponsor is checked by the person least likely to hit the exception.\n\n**Plant one decoy per set.** Alter one requirement so it is subtly wrong, for example a threshold of 5,000 where the draft's source says 500. Tell everyone in the invitation that some statements may be wrong and that you want to hear it. If someone confirms the decoy, discount their other confirmations. This is a practitioner's check, not a published method, and it works because it costs one question.\n\n**Decision rule for sign-off.** Treat a requirement as validated when at least three people from at least two roles rule on it and at least 80% say \"right as written\". Take it to a short workshop when two or more people say \"wrong\" or \"needs changing\", or when two roles disagree. Anything with fewer than three verdicts is not validated yet. These thresholds are rules of thumb from running this kind of review, not a standard.\n\n![Checklist of seven rules for validating AI-drafted requirements with stakeholders, from sets of six to a workshop for disputes.](/visuals/requirements-validation-sign-off-rules-checklist/requirements-validation-sign-off-rules-checklist.png \"Pin these before you send the first invitation\")\n\n**Common mistakes:**\n\n- Sending the whole specification. Nobody reads 90 requirements, and \"looks fine\" comes back from all of them.\n- Asking \"do you agree?\" People agree. Ask \"is this how you do it today?\" and anchor on the last time they did.\n- Showing the AI's reasoning or source. It anchors people. Show the requirement, not where it came from.\n- Validating only with managers. They confirm policy. The exception lives with the person who covers the desk.\n- Counting a confirmation without a role next to it.\n- Treating one dissent as noise. One \"wrong\" from the person who runs month end is worth more than five confirmations from people who never see it.\n\n## How to set it up in Koji, step by step\n\nText interviews work on every plan, including Free. Voice interviews need the Interviews plan or Enterprise. For read-back, text is the better default: people look at the exact wording of a requirement while they answer.\n\n![Process showing six steps to run read-back validation of AI-drafted requirements in Koji, from the template to the verdict table.](/visuals/ai-drafted-requirements-validation-setup-steps/ai-drafted-requirements-validation-setup-steps.png \"The six steps from draft to verdict table\")\n\n1. **Start from the template.** In the dashboard's template gallery, open the Business Analysts templates and pick \"Stakeholder Requirements Gathering\". Keep its brief structure and change the question, or type your question on the home screen and let Koji draft a brief.\n2. **Rewrite the brief.** The problem statement becomes: \"Which of these requirements are right as written, which need changing, and what is missing, according to the people who use the process?\" Name the decision: what goes to baseline sign-off.\n3. **Upload the draft as a context file.** Add the full requirements draft and any source documents (PDF, TXT, JSON or MD, up to 25 MB each) so the interviewer understands the vocabulary and asks sharper follow-ups. The participant does not see the file, so the requirement wording itself goes in each question.\n4. **Build one question per requirement.** Use single choice with your four answers and paste the requirement text into the question. Set follow-up probing to 2 on the questions where you most need a reason. The interviewer can ask zero to three follow-ups per question; see [how AI follow-up probing works](/docs/ai-probing-guide).\n5. **Add one open question about gaps, then preview.** Run a preview as yourself. It is saved with its own analysis but not counted as a response.\n6. **Invite and read.** Share the interview link per stakeholder group, or import a CSV for personalised links on the Interviews plan or higher. Set a response limit if you want a cap.\n\n![Koji question editor with open ended, 1 to 10 scale and single choice questions, each with a follow-up setting.](/email/shots/shot-question-types.jpg \"Single choice questions carry the requirement text and the four answers\")\n\n### Copy this interview guide\n\nPaste this brief into the \"What do you want to research?\" box:\n\n> Show stakeholders six specific requirements from a draft written with AI help. For each one, find out whether it is right as written, needs changing or is wrong, why, and what is missing, so the team can sign off a baseline with evidence from the people who do the work.\n\nOpening line: \"I'm checking a draft of requirements for [project]. Some statements may be wrong, and I want to hear which. Your answers go to the project team, and this is not an assessment of you.\"\n\nQuestions 1 to 6 follow the same pattern, one per requirement:\n\n1. **(Single choice)** \"Requirement 1: [paste requirement]. Is this how it works for you today? Right as written / Right but needs changing / Wrong / Not something I can judge.\" Probe: \"Tell me about the last time this came up.\"\n2. **(Single choice)** Same format for requirement 2. Probe: \"What would you change in the wording?\"\n3. **(Single choice)** Requirement 3, plus the decoy in one of the six. Probe: \"What makes you say that?\"\n4. **(Single choice)** Requirement 4. Probe: \"Who else would see it differently?\"\n5. **(Single choice)** Requirement 5. Probe: \"What happens if this is not in place on day one?\"\n6. **(Single choice)** Requirement 6. Probe: \"How would you know it was working?\"\n7. **(Open)** \"What is missing from these six that you would expect to see?\" Probe: \"When did that last cause trouble?\"\n8. **(1 to 10 scale)** \"How confident are you that these six cover what your team needs on day one? 1 is not at all, 10 is completely.\" Probe: \"What would move your score up two points?\"\n\nClosing question: \"Is there anything about how you work that this draft would get wrong?\"\n\nAdd this as a checklist for the team: freeze the draft, tag each requirement with its source, split into sets of six, plant one decoy per set, invite three people per requirement from two roles, close in five working days.\n\n## What you get\n\nEach interview is analysed on its own, with a quality rating, key findings and pain points. The report aggregates your single choice questions into a distribution per requirement and pulls the open answers into themes with the share of participants behind them, and each finding links to supporting responses. See [how to analyse AI-moderated interview results](/docs/analyzing-ai-moderated-interview-results).\n\n![Koji report summary with a highlighted claim and a side panel of supporting participant quotes.](/email/shots/shot-evidence-panel.jpg \"Every claim in the report opens onto the quotes behind it\")\n\nThe next day you build the verdict table: one row per requirement, the answers by role, the decision (validated, amend, workshop), and the quote that explains it. In the sign-off meeting you stop arguing about opinions. You say \"14 of 18 are validated, three are in a 30-minute workshop on Thursday, and one was wrong in a way that would have broken month end.\" You also get the missing-requirements list straight from the gap question, with the participant's words.\n\n## The same playbook elsewhere\n\nThe same read-back works whenever a plan was written without the people it affects. Agencies use it to test a proposal against each client team; see [new-client stakeholder discovery](/blog/new-client-stakeholder-discovery-agency-playbook). Founders use it to hear a pilot's end users before the sponsor decides; see [enterprise pilot interviews](/blog/enterprise-pilot-user-interviews-founders-playbook). For the step before the draft, [AI agent workflow triage](/blog/ai-agent-workflow-triage-business-analyst-playbook) shows how to find what is worth drafting requirements for. HR can run it on a draft policy before it ships.\n\n## Why Koji for requirements validation\n\nPricing for these alternatives varies and is not always public, so the table compares what each method gives you.\n\n![Table comparing email sign-off, workshops, manual interviews and Koji for validating requirements on candour, follow-up, reach and analysis effort.](/visuals/requirements-validation-methods-comparison-table/requirements-validation-methods-comparison-table.png \"How four ways of validating requirements compare\")\n\n- **People answer alone.** No sponsor in the room and no loudest voice. A \"wrong\" is easier to say to an interviewer who does not report to anyone.\n- **Every \"wrong\" gets a \"why\".** The interviewer follows up on each answer, up to three times, so you get the reason and the last real case, not a bare disagreement.\n- **Same wording for everyone.** One guide means a difference in verdicts is a difference in the work, not in the interviewer.\n- **It fits calendars.** Participants answer when they are free, so twenty stakeholders do not need twenty slots. See [asynchronous interviews](/docs/async-user-interviews).\n- **A tally you can defend.** Single choice answers become a count per requirement, with the quotes behind them, which is what a sponsor asks for when you change a requirement they liked.\n\n## Questions you might have about AI interviews\n\n**Will stakeholders tell an AI that a requirement is wrong?** Often more readily than they would tell the person who wrote it. Say in the invitation that some statements may be wrong, who reads the results and that this is not an assessment. For evidence on how people respond, read [will customers open up to an AI interviewer](/blog/will-customers-open-up-to-ai-interviewer-2026).\n\n**Can it handle a precise requirement without paraphrasing it?** The requirement text is in the question you wrote, so every participant sees the same wording. Check that in a preview before launch.\n\n**What happens to what people say?** Tell participants in the invitation how their answers are used, and read the [AI interview data privacy and security guide](/docs/ai-interview-data-privacy-security) before you invite anyone. Use guardrails to exclude topics you do not want covered. Only conversations that pass the [quality gate](/docs/how-the-quality-gate-works) consume credits.\n\n## When Koji may not be the right fit\n\nIf the draft has ten requirements and four stakeholders who sit in one room, a 45-minute walkthrough does the job and is faster. If a requirement is a legal or regulatory obligation, it is not a matter of opinion and interviews cannot validate it. If you need to see people use a prototype, put the prototype in front of them. Koji is for the middle: dozens of requirements, a dozen or more stakeholders across roles, and a baseline that needs evidence.\n\n## Metrics to track\n\n- Share of requirements validated against the three-people, two-roles rule\n- Requirements amended or rejected after read-back, and the share of the draft they represent\n- Gaps found per set of six, and how many became requirements\n- Decoys confirmed (a rubber-stamping signal)\n- Change requests raised after sign-off, compared with your last project\n\n## FAQ\n\n**How many requirements per interview?** Six at most, plus one gap question. Beyond that, people agree to finish.\n\n**How many people do I need per requirement?** At least three, from at least two roles.\n\n**Is it fair to plant a decoy?** Yes, if you tell everyone in the invitation that some statements may be wrong. It measures rubber-stamping, not trickery.\n\n**Can I run this on the Free plan?** Yes, with text interviews and the template. Voice and CSV import of participants need a higher plan.\n\n**Does this replace a requirements workshop?** No. It tells you which requirements need one. Take the disputed ones there.\n\n**What does it cost in credits?** A text interview is 1 credit and a voice interview is 3 credits. Only conversations that pass the quality check consume credits.\n\n## Start your read-back\n\nOpen the Business Analysts templates, pick Stakeholder Requirements Gathering and paste the brief above. More about the role is on the [Koji page for business analysts](/for/business-analysts), every playbook for analysts is in the [use case library](/use-cases/business-analysts), and the general method is in the [requirements gathering interviews guide](/docs/requirements-gathering-interviews-guide).\n","category":"Use Cases","lastModified":"2026-10-27T07:50:11.009229+00:00","metaTitle":"Validate AI-Drafted Requirements With Stakeholders","metaDescription":"A business analyst playbook for checking AI-drafted requirements: sets of six, four answers, a decoy check and a sign-off rule, run as short AI interviews.","keywords":["validate AI-drafted requirements","requirements validation","business analyst playbook","stakeholder validation interviews","requirements sign-off","AI-moderated interviews"],"aiSummary":"Read-back validation is a method for business analysts to check AI-drafted requirements against stakeholders. Each person rules on up to six specific requirements with four answers, a decoy is planted to catch rubber-stamping, and a requirement counts as validated at three verdicts from two roles with 80% right as written. Koji runs the interviews as short AI-moderated text or voice conversations and reports counts and quotes per requirement.","aiKeywords":["validate AI-drafted requirements","requirements validation","business analyst","stakeholder interviews","read-back validation","AI-moderated interviews","requirements sign-off"],"aiContentType":"guide","faqItems":[{"answer":"Six at most, plus one open question about what is missing. Beyond six, people start agreeing to finish.","question":"How many requirements per interview?"},{"answer":"At least three, from at least two roles: whoever commissioned the work and people who do it daily.","question":"How many people do I need per requirement?"},{"answer":"Yes, if the invitation says some statements may be wrong. The decoy measures rubber-stamping, not trickery.","question":"Is it fair to plant a decoy requirement?"},{"answer":"Yes, with text interviews and the Stakeholder Requirements Gathering template. Voice interviews and CSV import of participants need a higher plan.","question":"Can I run this on the Free plan?"},{"answer":"No. It tells you which requirements need one. Take the disputed requirements there.","question":"Does this replace a requirements workshop?"},{"answer":"A text interview is 1 credit and a voice interview is 3 credits. Only conversations that pass the quality check consume credits.","question":"What does it cost in credits?"}],"relatedTopics":["validating AI-drafted requirements","requirements validation interviews","stakeholder sign-off evidence"]}],"pagination":{"total":1,"returned":1,"offset":0}}