{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-09-20T06:03:05.589Z"},"content":[{"type":"documentation","id":"8d44007f-ac69-4207-bd56-fcc6285347c0","slug":"full-credibility-standard-sample-size-per-segment","title":"How Much Data a Segment Needs Before Its Own Number Is Enough (2026)","url":"https://www.koji.so/docs/full-credibility-standard-sample-size-per-segment","summary":"Full credibility is the sample size at which a segment needs no borrowing from the overall average, given by n = (z/k)squared. The classical 1,082 standard means within 5 percent of expected 90 percent of the time, and counts events rather than people. For a mean on a bounded rating scale the relevant form is n = (z x SD / m)squared, which yields 25 to 200 for typical research precision targets. Below full credibility, assign partial weight Z.","content":"*How many people do I need per segment?* has an exact answer, it is older than modern market research, and it is not 30. The answer depends on two things you choose and one thing you measure, and the formula fits on a line:\n\n**n = (z / k)^2**\n\nwhere **k** is how close you need the estimate to be, expressed as a fraction, and **z** is the normal quantile for how often you need to be that close. That is the **full credibility standard** - the sample size at which a segment's own number is good enough to use on its own, with no borrowing from the overall average.\n\n## The answer, stated first\n\nPick your tolerance and your confidence, then read the number off:\n\n| Maximum departure from expected (k) | 90% of the time | 95% of the time | 99% of the time |\n| --- | --- | --- | --- |\n| 2.5% | 4,326 | 6,147 | 10,623 |\n| 5% | 1,082 | 1,537 | 2,656 |\n| 7.5% | 481 | 683 | 1,180 |\n| 10% | 271 | 384 | 664 |\n\nThis table is from L. H. Longley-Cook's 1962 introduction to credibility theory for the Casualty Actuarial Society, and the cell that became the industry convention is **1,082** - the number required to be within 5 per cent of the expected value 90 per cent of the time. Longley-Cook notes that this figure \"is frequently used as an accepted standard of credibility.\"\n\nEvery cell reproduces from (z/k)^2. At 90 per cent confidence z is 1.6449, so (1.6449/0.05)^2 = 1,082.3. At 95 per cent and 2.5 per cent, (1.96/0.025)^2 = 6,146.6 against a published 6,147. Eight of the twelve cells match exactly; the worst discrepancy in the table is 7 in 10,623, which is 0.07 per cent and consistent with the rounding of normal tables in 1962. The whole table is one closed form.\n\n## What the numbers are actually counting\n\nHere is the part that makes the table usable rather than terrifying. **The unit is events, not people, and it is the unit in which the randomness lives.**\n\nThe classical standard counts *claims*, not policyholders, because a claim is the thing that either happens or does not. Longley-Cook is explicit about the interpretation: when the standard for the number of claims is 1082, \"the actual number will be within 5% of 1082 90% of the time.\"\n\nFor research metrics this translates directly, and the translation is where most of the fear evaporates:\n\n- If your metric is a **rate** - the proportion who churned, the proportion hitting a bug, the proportion who said yes - then the relevant count is the number of *events*, and you do need large numbers, because rare events carry enormous relative noise.\n- If your metric is a **mean of a bounded rating** - a 1-5 satisfaction score, a 0-10 scale - the randomness per respondent is much smaller relative to the quantity, and the sample sizes required are far more modest.\n\nFor a mean, the usable form is the one built from the standard error. To be within a margin **m** of the true mean with confidence z:\n\n**n = (z x SD / m)^2**\n\nOn a 1-5 scale with a within-segment standard deviation of 1.2, to be within 0.15 of the truth 90 per cent of the time: (1.6449 x 1.2 / 0.15)^2 = **173**. To be within 0.25: **62**. To be within 0.40: **25**.\n\nThat last line is the honest version of the *30 per segment* folk rule. Thirty respondents on a 1-5 scale buys you roughly plus or minus 0.36 at 90 per cent confidence. That is a perfectly reasonable precision for a directional read and completely inadequate for declaring one segment better than another when the segments are 0.2 apart. **The rule of thumb is not wrong; it is a precision claim with the precision left out.**\n\n## Partial credibility: what to do with the sample you actually have\n\nAlmost no research segment reaches full credibility, and that is fine, because the standard was never meant as a gate. Longley-Cook frames it carefully: the standard of full credibility is not, he writes, especially important in itself, but is important \"as a means of introducing consistency in the rate making procedure and establishing proper relationships as respects reliability between different volumes of experience.\"\n\nIn other words, its job is to make different-sized segments comparable to each other, not to disqualify anyone.\n\nBelow full credibility you assign a partial weight Z and blend the segment with the overall average - the mechanics are in [credibility weighting for small segment estimates](/docs/credibility-weighting-small-segment-estimates). The two standard ways to set Z:\n\n**The variance-ratio form.** Z = n / (n + k), where k is within-segment variance divided by between-segment variance. This is the principled version, because k is estimated from your own data and therefore knows how much your segments genuinely differ.\n\n**The square-root form.** Z = sqrt(n / n_full), where n_full is the full credibility standard from the table. This is the traditional actuarial approximation, and it is useful when you cannot estimate a between-segment variance - with only three or four segments, for example. With n_full = 1,082, a segment of 100 gets Z = 0.30 and a segment of 400 gets Z = 0.61.\n\nThe square-root rule is more conservative than the variance-ratio rule in most research settings, because it implicitly assumes segments differ more than they usually do. If you have the data to estimate k, prefer the variance-ratio form.\n\n## Choosing k and z honestly\n\nThe two parameters are choices, and Longley-Cook is refreshingly direct that \"the choice of P and k are in any case arbitrary\". That is a licence to choose them for your decision rather than by convention - but it obliges you to state them.\n\nThe right question is not *what is the standard?* but ***how wrong can this number be before I would make a different decision?*** Work backwards from that:\n\n- If you would ship the same roadmap whether the segment is at 4.1 or 4.4, your tolerance is at least 0.3 and you need far less data than you think.\n- If a 0.1 difference triggers a pricing change, you need a great deal more, and you should find that out before you run the study rather than after.\n\nThere is a practical warning in the same passage that transfers perfectly to research operations. Longley-Cook observes that setting too high a standard \"would considerably delay the response of premiums to changed accident conditions and might well lead to overall inadequacy of premium levels.\" The research equivalent is a team that sets an unreachable evidence bar, ships nothing on the evidence it has, and ends up making decisions on anecdote instead - which is strictly worse than acting on a partially credible number whose credibility you have written down.\n\n## A worked target for a real study\n\nSuppose you are comparing satisfaction across six segments on a 1-5 scale, and the decision is which segment gets dedicated onboarding work next quarter. You judge that a 0.3 difference would change the decision and a 0.15 difference would not.\n\n1. Set m = 0.15, confidence 90 per cent, SD 1.2. Full credibility for a segment mean is 173.\n2. You have budget for 600 interviews.\n3. Split evenly: 100 per segment. Z under the square-root rule against n_full = 173 is sqrt(100/173) = **0.76**.\n4. That is a good place to be. Each segment carries three quarters of its own weight, and the quarter it borrows from the average protects you from exactly the ranking artefact described in [why the top and bottom segments in your report are both the smallest ones](/docs/segment-ranking-sample-size-artifact).\n5. Report each segment as a blended estimate with its Z and its interval.\n\nNote what the even split did: it made the segments comparable. Unequal sizes are the thing that corrupts segment comparisons most reliably, and splitting evenly is free.\n\n## How Koji changes the arithmetic\n\nThe sample sizes above are the reason segment-level research has historically been the preserve of large quantitative teams. Getting 100 genuine conversations in each of six segments through moderated interviews is a quarter of work and a five-figure budget. Through a survey tool it is cheap but yields no *why* - you get the number and nothing that explains it.\n\nKoji collapses that trade-off, and the mechanism is specific:\n\n**AI-moderated interviews remove the per-interview cost driver.** There is no moderator to schedule, no session to attend, and no transcript to send out for processing. The marginal cost of the 100th interview in a segment is close to the marginal cost of the 10th, so hitting a credibility target is a matter of leaving the study open rather than expanding a budget.\n\n**Asynchronous voice and text collection removes the calendar.** Participants respond when they are free, which is what makes it realistic to fill a hard-to-reach segment to parity with an easy one - the requirement that actually determines whether your segment comparison is trustworthy.\n\n**Structured questions give you the numbers the formula needs.** Koji's six question types - open_ended, scale, single_choice, multiple_choice, ranking, and yes_no - produce per-respondent structured values, not just aggregate charts. A scale question yields the individual ratings you need to estimate the within-segment standard deviation, which is the one empirical input to every formula on this page. A yes_no question yields a clean event count, which is the correct unit when your metric is a rate. single_choice and multiple_choice screeners assign segment membership inside the interview, so your denominators are trustworthy. A ranking question captures relative priority, which frequently separates segments that look identical on a mean.\n\n**Real-time reports make the credibility check continuous.** You can watch each segment's n and its interval while collection is live, and top up the thin ones. The traditional alternative is to discover in analysis that one segment came in at 11 and write a caveat.\n\n**And the open_ended responses make a credible number actionable.** Reaching Z = 0.76 tells you the segment really is different. Koji's AI follow-up questions, which probe each answer automatically rather than accepting the first response, tell you why - and Koji's automatic analysis links the structured score to the verbatim explanations behind it. A survey platform gets you to the first half of that and stops.\n\n## Frequently asked questions\n\n### Is 1,082 really the number I need per segment?\n\nOnly if your metric is an event count and you want to be within 5 per cent of the true rate 90 per cent of the time. For a mean rating on a bounded scale, the corresponding number is usually between 25 and 200 depending on the precision you need. Use n = (z x SD / m)^2 for means and reserve the 1,082 standard for rates and counts.\n\n### Where does the *30 per group* rule come from?\n\nIt comes from the point at which the t-distribution becomes close to the normal distribution, which is a statement about which formula to use, not about whether you have enough data. It has been repeated so often that it acquired authority it never had. Thirty gives you roughly plus or minus 0.36 on a 1-5 scale at 90 per cent confidence - fine for direction, insufficient for ranking.\n\n### What if I cannot reach full credibility for any segment?\n\nThat is the normal case, and it is what partial credibility is for. Assign each segment the weight its sample size earns, report that weight alongside the estimate, and blend the remainder with the overall average. A segment at Z = 0.4 is genuinely informative; you simply have to say so rather than present it as a standalone fact.\n\n### Should I use the square-root rule or the variance-ratio rule?\n\nUse the variance-ratio rule, Z = n/(n+k), whenever you have enough segments - roughly eight or more - to estimate the between-segment variance. Use the square-root rule when you do not. The variance-ratio version adapts to how much your segments actually differ, which the square-root version cannot see.\n\n### Does a larger sample fix the bias in my sample?\n\nNo, and this is the most expensive misreading of everything on this page. These formulas govern *precision*, not *representativeness*. A segment of 2,000 recruited from your most engaged users estimates the wrong quantity with great confidence. Sampling design determines what you are measuring; sample size determines how sharply.\n\n### How do I set the precision target before I know the standard deviation?\n\nRun a small pilot - 20 to 30 responses is enough for a usable SD estimate on a bounded scale - and compute the required n from it. Because Koji studies can be left open and extended rather than re-fielded, the pilot is usually just the first days of the real study rather than a separate exercise.\n\n## Related Resources\n\n- [Structured questions guide](/docs/structured-questions-guide) - the six question types and when to use each\n- [Credibility weighting for small segment estimates](/docs/credibility-weighting-small-segment-estimates) - what to do below full credibility\n- [Why the top and bottom segments in your report are both the smallest ones](/docs/segment-ranking-sample-size-artifact) - why unequal n corrupts rankings\n- [Survey sample size guide](/docs/survey-sample-size-guide) - setting overall n for a study\n- [Statistical power and minimum detectable effect](/docs/statistical-power-minimum-detectable-effect) - the detection counterpart to precision\n- [Is 4.1 good? Internal benchmarks and percentile norms](/docs/internal-benchmarks-percentile-norms) - what to compare the result against\n","category":"Analysis & Synthesis","lastModified":"2026-09-18T12:21:00.336047+00:00","metaTitle":"Sample Size Per Segment: The Full Credibility Standard","metaDescription":"The exact sample size a segment needs to stand on its own, from n = (z/k)squared, plus what to do with the smaller sample you actually have.","keywords":["sample size per segment","full credibility standard","how many respondents per segment","minimum segment sample size","credibility theory sample size","segment precision","research sample size formula"],"aiSummary":"Full credibility is the sample size at which a segment needs no borrowing from the overall average, given by n = (z/k)squared. The classical 1,082 standard means within 5 percent of expected 90 percent of the time, and counts events rather than people. For a mean on a bounded rating scale the relevant form is n = (z x SD / m)squared, which yields 25 to 200 for typical research precision targets. Below full credibility, assign partial weight Z.","aiPrerequisites":["Basic understanding of confidence and margin of error","A metric you report broken out by segment"],"aiLearningOutcomes":["Compute full credibility for rates and for means","Translate the classical 1,082 standard to research metrics","Choose precision and confidence from the decision at stake","Assign partial credibility to the sample you actually have"],"aiDifficulty":"intermediate","aiEstimatedTime":"12 min"}],"pagination":{"total":1,"returned":1,"offset":0}}