Somewhere in your organisation there is a slide that says a benchmarking survey is fine as long as a third party runs it, the data is at least three months old, at least five companies contribute, and nothing is attributable. That rule is quoted in vendor decks, association charters and compliance training. It has one problem.
The document it comes from was withdrawn three years ago.
Answer first
The "third party, three months old, five participants, aggregated" formula comes from Statement 6 of the 1996 DOJ and FTC health care policy statements. The DOJ rescinded those statements in February 2023 and the FTC withdrew them in July 2023. The 2016 HR guidance that pointed people to them was itself replaced in January 2025, by a document that contains no safe harbour at all. The conditions may still describe a sensible design. They are no longer a promise that anyone will decline to challenge it.
That matters less than the second half of this article, which is the part nobody puts on the slide: every one of those four conditions works by destroying a property the research needs. The safeguards are not a tax on a useful study. They are a description of a study that can no longer answer the question you commissioned it for.
The rule everyone still quotes, in full
Statement 6 was titled "Provider Participation In Exchanges Of Price And Cost Information." Its safety zone said the agencies would not challenge participation in written surveys of prices, or of wages, salaries and benefits, if three conditions held:
First, "the survey is managed by a third-party (e.g., a purchaser, government agency, health care consultant, academic institution, or trade association)". Second, "the information provided by survey participants is based on data more than 3 months old". Third, "there are at least five providers reporting data upon which each disseminated statistic is based, no individual provider's data represents more than 25 percent on a weighted basis of that statistic, and any information disseminated is sufficiently aggregated such that it would not allow recipients to identify the prices charged or compensation paid by any particular provider."
It was written for health care. It was adopted everywhere, because it was the only numeric guidance anyone had.
The 2016 Antitrust Guidance for Human Resource Professionals generalised it into four bullets: an exchange may be lawful if "a neutral third party manages the exchange," "the exchange involves information that is relatively old," "the information is aggregated to protect the identity of the underlying sources," and "enough sources are aggregated to prevent competitors from linking particular data to an individual source." It then told readers where to go for detail: "For more information on information exchanges, you can review the DOJ's and FTC's specific guidance to the healthcare industry on when written surveys of wages, salaries, or benefits are less likely to raise antitrust concerns (see Statement 6)."
So the general rule pointed at the specific rule. Then the specific rule was withdrawn, and later the general one.
Withdrawn, twice, and never replaced
The FTC announced its withdrawal on 14 July 2023, under a subheading that leaves little room for interpretation: "Outdated statements no longer serve as useful guidance or reflect market realities." The release notes it was not acting first: "The Commission's withdrawal follows the Department of Justice's decision to rescind the same statements in February 2023." The vote was 3-0.
Eighteen months later, on 16 January 2025, the two agencies jointly issued Antitrust Guidelines for Business Activities Affecting Workers, which states that "This document replaces the Antitrust Guidance for Human Resource Professionals (2016)."
Read the replacement looking for the numbers and you will not find them. The 2025 guidelines contain no safety zone, no participant minimum and no data-age threshold. What they contain instead is the observation that an exchange may be unlawful "whether or not that effect was intended," and that it may be illegal "even if companies use a third party or intermediary - including a third party using an algorithm - to share such information."
That last point deserves emphasis, because it inverts the first item on the old checklist. Third-party administration was safeguard number one in both 1996 and 2016. In 2025 it is named as something that does not cure the problem.
The inversion: each safeguard removes what the study was for
Here is the part that should change how you plan benchmarking work. Take the four conditions one at a time and ask what each one costs the analysis.
A neutral third party manages it. You lose the ability to ask a follow-up. The administrator collects a fixed schedule of fields; nobody can probe an anomaly, and no respondent can explain why their number looks strange. You get figures without reasons, which is the failure mode described in why price is never the real churn reason.
The information is more than three months old. You lose the decision. Pricing decisions are made against current conditions. A number describing last quarter is a description of a market that has already moved, and the older the safe threshold gets, the less it describes anything you can act on.
Aggregated so no source is identifiable. You lose the comparison you actually wanted. Nobody commissions a benchmark to learn the industry mean. They commission it to learn where they sit against specific rivals of similar size in similar segments. Aggregation is precisely the removal of that.
Enough sources that no data links to an individual. You lose granularity, and with it the only cuts that matter. A benchmark that cannot be broken out by segment, region or company size is a single number, and a single number is not a decision input.
Notice the symmetry with the leading case. In United States v. Container Corp. of America (1969) the Supreme Court distinguished the unlawful exchange from the lawful one exactly this way: there was "an exchange of information concerning specific sales to identified customers, not a statistical report on the average cost to all members, without identifying the parties to specific transactions."
Specific and identified is unlawful. Averaged and anonymous is lawful. And specific and identified is what makes a benchmark useful. The safe version cannot answer the question. The version that answers the question is the one the cases are about. That is not a loophole to engineer around; it is the design intent.
There is a further trap in the arithmetic. Statement 6 required that no single contributor exceed 25 percent of a statistic on a weighted basis. In a concentrated market this is not a formality - it is a bar the leader cannot clear. If the largest firm holds 40 percent of the volume in the participating pool, no volume-weighted statistic that includes it can put it under 25 percent, whatever the participant count. The only way to publish the statistic is to leave the market leader out of it, and a benchmark that excludes the largest player is not a benchmark of that market. The old safety zone was therefore hardest to satisfy exactly where price coordination is most plausible, which is the point the Court made in United States v. United States Gypsum Co. (1978) when it identified "the structure of the industry involved and the nature of the information exchanged" as the two dominant factors.
What a court thinks safe data looks like in 2026
If you want a current answer to "how old is old enough," the government has published one, and it is nothing like three months.
In United States v. RealPage, Inc., Civil Action No. 1:24-cv-00710 in the Middle District of North Carolina, the proposed Final Judgment and Competitive Impact Statement were published in the Federal Register on 5 December 2025. The terms restrict what nonpublic competitor data the pricing software may use at all: "Subject to limited exceptions, RealPage will not be allowed to use nonpublic data from competing properties in runtime operation." For model training, the permitted data is "historical or backward-looking Unaffiliated Property Data that is at least 12 months old and not from Active Leases."
The government then explains the practical effect of combining those two conditions, using a public statistic: "According to the U.S. Bureau of Labor Statistics, 12 months is the most common lease length with only about 7% of leases being over 12-months." Therefore "Aging the data for at least 16 months will exclude virtually all active leases to be used in training the model."
Sixteen months. The 1996 safety zone said three. The lag a regulator now treats as safe is more than five times the one your compliance deck still quotes - 16 divided by 3 is 5.3. And the litigation is not finished: a proposed Final Judgment for landlord defendant Willow Bridge Property Company was filed on 6 July 2026.
Ask honestly what a 16-month-old, state-level, competitor-anonymised figure would change about a pricing decision you have to make this quarter. That is the inversion made concrete.
What to do instead
The conclusion is not that benchmarking is forbidden. It is that competitor-fed benchmarking is a weak instrument that carries legal weight, and that two better instruments are available.
Build internal benchmarks. Your own history, cut by segment and cohort, answers "are we improving" without any competitor data at all. That is the method in internal benchmarks and percentile norms, and it is more decision-relevant than an industry mean because it is measured on your actual population.
Ask buyers directly. Customers and lost prospects evaluated you and your competitors. They will tell you what they compared, what they thought was fair, and what made the difference. That is first-party research with no competitor on the other side of the table, and it produces reasons, not just levels. The framing is in our customer experience benchmarking guide and the pricing research survey guide.
There is also a reporting discipline worth importing. If you publish segment-level results, the same aggregation questions arise for your own respondents' privacy: small cells identify people. The techniques are the same ones the disclosure-control literature uses, covered in k-anonymity and minimum base sizes and quasi-identifiers in research data. Published category benchmarks like our NPS benchmarks by industry exist precisely so that the coarse comparison is free and you can spend your research budget on the specific one.
Get the reasons, not the averages
A benchmarking survey tells you that you are eleven points behind on some metric. It cannot tell you why, because the design that makes it lawful is the design that strips out every explanation.
Koji is built for the other half. Run AI-moderated voice interviews with your own customers and lost deals, at panel scale and without scheduling anyone. Every conversation probes for the reason behind the number, and the six structured question types - open_ended, scale, single_choice, multiple_choice, ranking and yes_no - give you countable data alongside the verbatims, so you get a level and a cause from the same study.
Legacy platforms make you choose. Qualtrics and SurveyMonkey deliver scale without depth; UserTesting and dscout deliver depth at a per-session cost that caps your sample; Dovetail organises conversations you still had to run yourself. Koji is AI-native: no moderator bias, no research expertise required, one-click reports, and 10x faster insights from question to answer.
Run your first study with Koji and stop benchmarking against a number nobody can explain.
Frequently asked questions
Is the "five participants, three months old" rule still valid?
Not as a safe harbour. It came from Statement 6 of the 1996 health care policy statements, which the DOJ rescinded in February 2023 and the FTC withdrew in July 2023. The conditions may still be sensible design choices, and many practitioners still follow them, but no agency has committed to declining challenge on that basis since 2023.
What replaced it?
For employment-related exchanges, the DOJ and FTC Antitrust Guidelines for Business Activities Affecting Workers of January 2025, which state that they replace the 2016 HR guidance. They contain no safe harbour, no participant minimum and no data-age threshold, and they emphasise that an exchange can be unlawful "whether or not that effect was intended."
Does using a third-party survey vendor make a benchmarking study safe?
It helps, but it is not decisive. Third-party administration was the first condition in both the 1996 and 2016 guidance; the 2025 guidelines say an exchange may be illegal "even if companies use a third party or intermediary - including a third party using an algorithm." Structure matters more than the presence of an intermediary.
Why does aggregation make the data less useful?
Because the question a benchmark is meant to answer is comparative and specific: how do we compare with rivals of our size in our segment, now. Aggregation removes attribution, ageing removes currency, and minimum-cell rules remove the segment cuts. Each safeguard subtracts one of the properties that made the comparison decision-relevant.
How old does data have to be to be considered historical?
There is no general rule any longer. The most recent concrete figure in a US enforcement context comes from the proposed RealPage Final Judgment published in December 2025, which permits training data "at least 12 months old and not from Active Leases" and explains that ageing "at least 16 months will exclude virtually all active leases." That is specific to those facts, not a general standard.
What is the alternative to a competitor benchmarking survey?
Internal benchmarks plus direct buyer research. Your own trend data answers whether you are improving; interviews with customers and lost prospects tell you how you were compared and why you lost, with no competitor involved. Together they cover almost everything a benchmarking survey is bought to do, with better causal content and no information-exchange exposure.
