{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-09-22T18:00:26.055Z"},"content":[{"type":"documentation","id":"09f0e3ff-0e55-4931-a49e-7a4eb9f75e56","slug":"feature-sunset-research","title":"Feature Sunset Research: How to Deprecate a Feature Without Losing Customers","url":"https://www.koji.so/docs/feature-sunset-research","summary":"Low feature usage justifies investigation, not deletion. Pendo found around 80% of features in the average software product are rarely or never used and about 12% of features drive 80% of daily usage, so low usage alone cannot be the deprecation criterion. Analytics-only decisions fail three ways: confusing undiscovered with unwanted, missing downstream workaround dependencies, and averaging away an intense minority that is often concentrated in high-revenue accounts. Research must answer four questions before deprecation: why usage is low, who the remaining users are and what revenue they represent, what job the feature performs and what breaks downstream, and what users will actually do if it is removed. Run three studies: a pre-decision study across active, lapsed, and never-users; a migration study testing the replacement path before announcing; and a post-sunset check at 30-60 days. Notify 90 days ahead minimum, ship the replacement before removing the original, and give high-dependency accounts a human conversation. Koji fits because sunset populations are small and scattered: asynchronous AI-moderated interviews reach everyone, automatic follow-up questions surface the why at survey scale, and six structured question types produce both a leadership-ready distribution and the reasoning behind it.","content":"## The short answer\n\n**Low usage is a reason to investigate a feature, not a reason to remove it.** Pendo's analysis of feature usage across 615 subscriptions found that roughly **80% of features in the average software product are rarely or never used**, and that about **12% of features drive 80% of daily usage volume**. If low usage alone justified deprecation, you would delete four fifths of your product.\n\nThe number that should actually drive the decision is different: **how much revenue sits with the people who do use it, and what happens to them if it disappears.** A feature used by 2% of accounts can be load-bearing for 40% of revenue — and it is almost always the enterprise segment, the one with a renewal date and a procurement team.\n\nAnalytics can tell you *that* usage is low. Only research can tell you *why* it is low, *who* the remaining users are, and *what they will do* when you remove it. This guide is the research playbook for that decision.\n\n## Why analytics-only sunset decisions go wrong\n\nThree failure modes account for most deprecations that backfire:\n\n**1. Confusing \"undiscovered\" with \"unwanted.\"** Low usage frequently signals a discoverability or usability problem, not absent value. A feature buried three menus deep, or one that failed at first use and was never retried, produces the same dashboard as a feature nobody wants. Removing it destroys value you never delivered; fixing the entry point releases it. These are opposite responses to identical data.\n\n**2. Missing the workaround dependency.** Features often sit inside a workflow they were never designed for. A CSV export intended for reporting turns out to be how three enterprise accounts feed their billing system. Usage looks trivial — a handful of clicks a month — while the dependency is existential. Nothing in your analytics reveals this, because the significance lives in what happens *after* the data leaves your product.\n\n**3. Averaging away the intense minority.** Aggregate usage hides concentration. Ten accounts using a feature forty times a day and ten thousand ignoring it produces an unremarkable average. That intense minority is exactly who churns, and they are disproportionately your largest customers, because complex needs correlate with contract size.\n\n## The four questions research must answer\n\nBefore any deprecation decision, get evidence on all four. Skipping any one is where sunsets go wrong.\n\n| # | Question | What it decides |\n|---|---|---|\n| 1 | **Why is usage low?** Never discovered, tried and failed, or genuinely not needed? | Fix vs remove |\n| 2 | **Who are the remaining users, and what revenue do they represent?** | Whether this is a deprecation or a strategic risk |\n| 3 | **What job is the feature doing for them?** What breaks downstream if it disappears? | Whether a migration path is even possible |\n| 4 | **What will they actually do if it goes?** Adopt the alternative, build a workaround, or leave? | Timeline, comms, and churn exposure |\n\nQuestion 3 is the one teams underestimate. The job the feature does is rarely the job it was designed for — and the migration path you plan is only viable if it serves the *actual* job.\n\n## A three-study sequence\n\n**Study 1 — the pre-decision study (before you commit).**\nTalk to three segments separately, because they answer different questions:\n- **Active users** — the job, the workflow, the downstream dependency, the switching cost.\n- **Lapsed users** — people who tried it and stopped. They tell you whether it failed or was simply unnecessary. This is the segment that distinguishes fix-from-remove, and it is the one most often skipped.\n- **Never-users who fit the profile** — people who should have used it and did not. Pure discoverability signal.\n\n**Study 2 — the migration study (after deciding, before announcing).**\nTest the replacement path with the people who will be forced onto it. The failure mode here is a migration designed around the feature's *documented* purpose rather than its *actual* use. If your export replacement drops a column three customers' billing depends on, you will discover it either in a two-week research study or in a support escalation during the cutover.\n\n**Study 3 — the post-sunset check (30-60 days after).**\nDid people land safely? What broke? This is also your early-warning system for renewal risk, while there is still time to intervene.\n\n## The interview guide\n\nStructure the conversation around the workflow, not the feature. People describe workflows accurately and features vaguely.\n\n**Opening — context, not opinion**\n- \"Walk me through the last time you used [feature]. Start from what prompted it.\"\n- \"What happened right before? What did you do with the result afterwards?\"\n\n**The job**\n- \"If [feature] vanished tomorrow, what would you do instead?\" — the single most predictive question in the guide.\n- \"How long would that take you?\" — quantifies switching cost.\n- \"Does anything else in your business depend on the output?\"\n\n**Alternatives and severity**\n- Scale: \"How disruptive would losing this be, 1-10?\"\n- Single choice: \"If it were removed, would you most likely — use the alternative, build a workaround, use another tool, or reconsider your subscription?\"\n- Yes/no: \"Have you looked for another way to do this in the last six months?\"\n\n**Lapsed users**\n- \"You used it a few times and stopped. What happened?\"\n- \"Was there a moment it did not do what you expected?\"\n\nMixing question types here is deliberate. The scale and single-choice answers give you a distribution you can put in front of a leadership team — *\"31% say they would reconsider their subscription\"* — while the open-ended answers explain what sits behind it. Koji supports six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, and yes_no), so one study produces both the quantitative distribution and the qualitative reasoning. See the [structured questions guide](/docs/structured-questions-guide).\n\nDo **not** ask \"would you be upset if we removed this?\" It invites a defensive yes from everyone and measures nothing.\n\n## Where Koji fits this problem specifically\n\nSunset research has an awkward shape that traditional methods handle badly, and it is worth being concrete about why.\n\n**The population is small, scattered, and unenthusiastic.** You may have 60 remaining users of a feature across nine countries and six time zones. Scheduling 15 moderated calls with that group takes three weeks and gets you a biased sample — the people who accept a research invitation are the engaged ones, not the quiet enterprise admin whose billing job depends on the export. Koji's [AI-moderated interviews](/docs/how-ai-interviewers-work) run asynchronously from a link, so you can invite **all 60** and get responses from the ones who would never book a call.\n\n**You need the why, at survey scale.** A survey gets you volume but stops at the first answer — and \"I use it for reporting\" is where the useful part begins, not ends. Koji's AI asks follow-up questions automatically, so when someone says they would build a workaround, the AI probes what the workaround is and how long it would take. That is the difference between a percentage and a decision.\n\n**Deprecation decisions are time-boxed.** They surface in roadmap planning and need an answer in days, not the weeks a traditional study takes. Because analysis is automatic, [reports arrive as responses come in](/docs/how-to-automate-user-research) rather than after a manual coding pass — fast enough to inform the decision rather than ratify it.\n\n**Segmentation is the whole game.** Screen participants by usage intensity and account value so you can report on active, lapsed, and never-users separately. An aggregate finding across all three is worse than useless here, because it averages exactly the signal you need.\n\n## The sunset timeline\n\nOnce the research supports removal:\n\n1. **Notify well ahead** — 90 days minimum for anything workflow-critical, 6-12 months where enterprise contracts or integrations are involved.\n2. **Tell affected users directly and specifically.** A changelog entry is not notice. Name the feature, the date, and the path forward.\n3. **Ship the replacement before you remove the original**, with an overlap period where both work.\n4. **Give the intense minority a human conversation.** For the handful of accounts with deep dependencies, an email is not enough — and this is where saved renewals happen.\n5. **Run the post-sunset check** at 30-60 days.\n\nThe pattern in successful sunsets is unglamorous: the research was done early enough that the migration path was designed around real jobs, and the people most affected heard about it first and personally.\n\n## Five common mistakes\n\n1. **Deciding from the dashboard alone.** Usage tells you what, never why.\n2. **Skipping lapsed users.** They hold the fix-vs-remove answer.\n3. **Announcing before testing the migration path.** You cannot un-announce a date.\n4. **Treating all users as one population.** The intense minority is the churn risk.\n5. **Researching only after the decision.** Post-hoc research becomes justification, not evidence — and everyone involved can tell.\n\n## Related Resources\n\n- [Feature Validation Guide](/docs/feature-validation-guide) — the mirror-image decision before you build\n- [Feature Adoption Research](/docs/feature-adoption-research) — diagnosing low usage before concluding it is unwanted\n- [Structured Questions Guide](/docs/structured-questions-guide) — the six question types behind the mixed-method guide above\n- [Churn Interview Questions](/docs/churn-interview-questions) — probing departure risk directly\n- [How to Prioritize Customer Feedback](/docs/how-to-prioritize-customer-feedback) — weighing the intense minority against the majority\n- [Customer Segmentation Research Interviews](/docs/customer-segmentation-research-interviews) — building the segments this study depends on\n- [Research-Driven Roadmap Prioritization](/docs/research-driven-roadmap-prioritization) — feeding the result back into planning\n- [Product Feedback Triage Guide](/docs/product-feedback-triage-guide) — handling the inbound during a sunset","category":"product-management","lastModified":"2026-07-27T03:19:07.223197+00:00","metaTitle":"Feature Sunset Research: How to Deprecate Without Losing Customers","metaDescription":"Low usage is a reason to investigate, not to delete. The research playbook for feature deprecation: the four questions, a three-study sequence, the interview guide, and the sunset timeline.","keywords":["feature sunset research","how to deprecate a feature","feature deprecation process","sunsetting a product feature","feature removal customer research","end of life feature communication","low feature usage analysis","product deprecation playbook","feature migration research","deprecation churn risk"],"aiSummary":"Low feature usage justifies investigation, not deletion. Pendo found around 80% of features in the average software product are rarely or never used and about 12% of features drive 80% of daily usage, so low usage alone cannot be the deprecation criterion. Analytics-only decisions fail three ways: confusing undiscovered with unwanted, missing downstream workaround dependencies, and averaging away an intense minority that is often concentrated in high-revenue accounts. Research must answer four questions before deprecation: why usage is low, who the remaining users are and what revenue they represent, what job the feature performs and what breaks downstream, and what users will actually do if it is removed. Run three studies: a pre-decision study across active, lapsed, and never-users; a migration study testing the replacement path before announcing; and a post-sunset check at 30-60 days. Notify 90 days ahead minimum, ship the replacement before removing the original, and give high-dependency accounts a human conversation. Koji fits because sunset populations are small and scattered: asynchronous AI-moderated interviews reach everyone, automatic follow-up questions surface the why at survey scale, and six structured question types produce both a leadership-ready distribution and the reasoning behind it.","aiPrerequisites":["Access to feature-level usage analytics","A defined candidate feature for deprecation"],"aiLearningOutcomes":["Distinguish undiscovered features from genuinely unwanted ones","Answer the four questions required before any deprecation decision","Run the three-study sequence across active, lapsed, and never-users","Write a sunset interview guide that mixes qualitative and structured questions","Plan a deprecation timeline that protects high-dependency accounts"],"aiDifficulty":"intermediate","aiEstimatedTime":"12 min read"}],"pagination":{"total":1,"returned":1,"offset":0}}