Reference table
How often product ideas work: published hit rates
A table of the success rates that teams running experiments have published for their own product ideas, from Microsoft's one in three to QualPro's 150,000 tested business ideas. Use it as the outside view when your team scores confidence on a roadmap, since it is the prior a new idea starts from. Koji helps teams test ideas with customer interviews before they are built.
Where this appears
What's in this reference table
| Source | Domain | Reported rate |
|---|---|---|
| Microsoft (Kohavi et al.) | Software features, controlled experiments | About 1/3 improve the target metric |
| Bing (Kohavi et al.) | Search | Lower than 1/3 |
| Netflix, cited in Moran (2007) | Web product | Considers 90% of what they try to be wrong |
| QualPro (Holland and Cochran, 2005) | 150,000 business improvement ideas over 22 years | 75% had no impact on performance or hurt it |
| Avinash Kaushik (2006) | Web analytics | 80% of the time wrong about what a customer wants |
| Regis Hadiaris, Quicken Loans, cited in Moran (2008) | Financial services testing | Right about a test outcome about 33% of the time after five years |
Use this image
Free to use in your docs, decks and posts under CC BY 4.0, as long as you credit Koji with a link to this page. To embed it, paste this into your page:
html
<a href="https://www.koji.so/visuals/product-idea-success-rates-published-sources"><img src="https://www.koji.so/visuals/product-idea-success-rates-published-sources/product-idea-success-rates-published-sources.png" alt="Table of published hit rates for product ideas from Microsoft, Bing, Netflix, QualPro, Avinash Kaushik and Quicken Loans, most near one in three or lower." width="1200" height="852" /></a>
<p>Source: <a href="https://www.koji.so/visuals/product-idea-success-rates-published-sources">How often product ideas work: published hit rates</a>, Koji</p>
