Feature Sunset Research: How to Deprecate a Feature Without Losing Customers

The short answer

Low usage is a reason to investigate a feature, not a reason to remove it. Pendo's analysis of feature usage across 615 subscriptions found that roughly 80% of features in the average software product are rarely or never used, and that about 12% of features drive 80% of daily usage volume. If low usage alone justified deprecation, you would delete four fifths of your product.

The number that should actually drive the decision is different: how much revenue sits with the people who do use it, and what happens to them if it disappears. A feature used by 2% of accounts can be load-bearing for 40% of revenue — and it is almost always the enterprise segment, the one with a renewal date and a procurement team.

Analytics can tell you that usage is low. Only research can tell you why it is low, who the remaining users are, and what they will do when you remove it. This guide is the research playbook for that decision.

Why analytics-only sunset decisions go wrong

Three failure modes account for most deprecations that backfire:

1. Confusing "undiscovered" with "unwanted." Low usage frequently signals a discoverability or usability problem, not absent value. A feature buried three menus deep, or one that failed at first use and was never retried, produces the same dashboard as a feature nobody wants. Removing it destroys value you never delivered; fixing the entry point releases it. These are opposite responses to identical data.

2. Missing the workaround dependency. Features often sit inside a workflow they were never designed for. A CSV export intended for reporting turns out to be how three enterprise accounts feed their billing system. Usage looks trivial — a handful of clicks a month — while the dependency is existential. Nothing in your analytics reveals this, because the significance lives in what happens after the data leaves your product.

3. Averaging away the intense minority. Aggregate usage hides concentration. Ten accounts using a feature forty times a day and ten thousand ignoring it produces an unremarkable average. That intense minority is exactly who churns, and they are disproportionately your largest customers, because complex needs correlate with contract size.

The four questions research must answer

Before any deprecation decision, get evidence on all four. Skipping any one is where sunsets go wrong.

Question 3 is the one teams underestimate. The job the feature does is rarely the job it was designed for — and the migration path you plan is only viable if it serves the actual job.

A three-study sequence

Study 1 — the pre-decision study (before you commit). Talk to three segments separately, because they answer different questions:

• Active users — the job, the workflow, the downstream dependency, the switching cost.
• Lapsed users — people who tried it and stopped. They tell you whether it failed or was simply unnecessary. This is the segment that distinguishes fix-from-remove, and it is the one most often skipped.
• Never-users who fit the profile — people who should have used it and did not. Pure discoverability signal.

Study 2 — the migration study (after deciding, before announcing). Test the replacement path with the people who will be forced onto it. The failure mode here is a migration designed around the feature's documented purpose rather than its actual use. If your export replacement drops a column three customers' billing depends on, you will discover it either in a two-week research study or in a support escalation during the cutover.

Study 3 — the post-sunset check (30-60 days after). Did people land safely? What broke? This is also your early-warning system for renewal risk, while there is still time to intervene.

The interview guide

Structure the conversation around the workflow, not the feature. People describe workflows accurately and features vaguely.

Opening — context, not opinion

• "Walk me through the last time you used [feature]. Start from what prompted it."
• "What happened right before? What did you do with the result afterwards?"

The job

• "If [feature] vanished tomorrow, what would you do instead?" — the single most predictive question in the guide.
• "How long would that take you?" — quantifies switching cost.
• "Does anything else in your business depend on the output?"

Alternatives and severity

• Scale: "How disruptive would losing this be, 1-10?"
• Single choice: "If it were removed, would you most likely — use the alternative, build a workaround, use another tool, or reconsider your subscription?"
• Yes/no: "Have you looked for another way to do this in the last six months?"

Lapsed users

• "You used it a few times and stopped. What happened?"
• "Was there a moment it did not do what you expected?"

Mixing question types here is deliberate. The scale and single-choice answers give you a distribution you can put in front of a leadership team — "31% say they would reconsider their subscription" — while the open-ended answers explain what sits behind it. Koji supports six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, and yes_no), so one study produces both the quantitative distribution and the qualitative reasoning. See the structured questions guide.

Do not ask "would you be upset if we removed this?" It invites a defensive yes from everyone and measures nothing.

Where Koji fits this problem specifically

Sunset research has an awkward shape that traditional methods handle badly, and it is worth being concrete about why.

The population is small, scattered, and unenthusiastic. You may have 60 remaining users of a feature across nine countries and six time zones. Scheduling 15 moderated calls with that group takes three weeks and gets you a biased sample — the people who accept a research invitation are the engaged ones, not the quiet enterprise admin whose billing job depends on the export. Koji's AI-moderated interviews run asynchronously from a link, so you can invite all 60 and get responses from the ones who would never book a call.

You need the why, at survey scale. A survey gets you volume but stops at the first answer — and "I use it for reporting" is where the useful part begins, not ends. Koji's AI asks follow-up questions automatically, so when someone says they would build a workaround, the AI probes what the workaround is and how long it would take. That is the difference between a percentage and a decision.

Deprecation decisions are time-boxed. They surface in roadmap planning and need an answer in days, not the weeks a traditional study takes. Because analysis is automatic, reports arrive as responses come in rather than after a manual coding pass — fast enough to inform the decision rather than ratify it.

Segmentation is the whole game. Screen participants by usage intensity and account value so you can report on active, lapsed, and never-users separately. An aggregate finding across all three is worse than useless here, because it averages exactly the signal you need.

The sunset timeline

Once the research supports removal:

1. Notify well ahead — 90 days minimum for anything workflow-critical, 6-12 months where enterprise contracts or integrations are involved.
2. Tell affected users directly and specifically. A changelog entry is not notice. Name the feature, the date, and the path forward.
3. Ship the replacement before you remove the original, with an overlap period where both work.
4. Give the intense minority a human conversation. For the handful of accounts with deep dependencies, an email is not enough — and this is where saved renewals happen.
5. Run the post-sunset check at 30-60 days.

The pattern in successful sunsets is unglamorous: the research was done early enough that the migration path was designed around real jobs, and the people most affected heard about it first and personally.

Five common mistakes

1. Deciding from the dashboard alone. Usage tells you what, never why.
2. Skipping lapsed users. They hold the fix-vs-remove answer.
3. Announcing before testing the migration path. You cannot un-announce a date.
4. Treating all users as one population. The intense minority is the churn risk.
5. Researching only after the decision. Post-hoc research becomes justification, not evidence — and everyone involved can tell.

Related Resources

• Feature Validation Guide — the mirror-image decision before you build
• Feature Adoption Research — diagnosing low usage before concluding it is unwanted
• Structured Questions Guide — the six question types behind the mixed-method guide above
• Churn Interview Questions — probing departure risk directly
• How to Prioritize Customer Feedback — weighing the intense minority against the majority
• Customer Segmentation Research Interviews — building the segments this study depends on
• Research-Driven Roadmap Prioritization — feeding the result back into planning
• Product Feedback Triage Guide — handling the inbound during a sunset