Back to blog
Founders

How to Automate Churn Interviews So Every Cancellation Gets a Conversation

A founder playbook for always-on churn interviews: invite every voluntary leaver within 48 hours, file the reason on the account, keep the reason list fixed and read a sample yourself each month.

Koji

Koji Team

Product · · 16 min read

To automate churn interviews, automate the invitation and the filing, not the reading. Every voluntary cancellation gets a short interview link within 48 hours, tied to the account's ID. When the interview finishes, the reason code and a link to the conversation land on the account in your CRM. Then a person still reads a sample every month. By the end of this playbook you'll have the pipeline, the questions, a reason list that stays the same quarter to quarter, and the rules for what to read yourself.

This is for founders and first PMs somewhere after Series A, when there are too many cancellations to call each one but too few to ignore any of them. If you are still at the stage where one batch of interviews will tell you what's wrong, start with churn interviews for leavers and matched stayers first. Come back here once you want that study to run every week without anyone remembering to run it.

When you need this

Automation starts to pay when you see some of these signs:

  • You get more voluntary cancellations a month than one person can interview, say more than ten.
  • You ran one round of churn interviews, learned a lot, and then nobody ran the next one.
  • The cancellation dropdown says "too expensive" or "not using it" every quarter, and the board has stopped believing it.
  • Account owners hear about a cancellation weeks later, and the reason they write down is their own guess.
  • Each quarter's churn slide uses different categories, so you can't tell whether anything changed.

The cost of not knowing is easy to underrate. SaaS Capital's 2025 survey of private B2B companies puts median gross revenue retention at 91% and median net retention at 101%, with a median of 95% for companies whose average contract value is above $250,000 and about 91% below it [1]. Aleph's 2026 report with Benchmarkit, covering 342 companies, puts the 2025 median lower, at 84%, down from 88%, with a 75th percentile of 91% [2]. Either way, a typical scaleup loses somewhere between a tenth and a sixth of its revenue base a year to churn, and has to replace it before growing at all.

Range chart of B2B SaaS gross revenue retention from two reports: a median of 91% and 84%, and 91% below and 95% above a 250,000 dollar contract value.
Two reports, seven points apart on the median. Benchmark against the sample that looks like your company. · Source: SaaS Capital, 2025 B2B SaaS Retention Benchmarks, 2025; Aleph with Benchmarkit, gross revenue retention in SaaS, 2026 · Download or share

Guessing at the reason costs more than the churn itself. You rebuild onboarding, cut a price or hire another customer success manager, and next quarter looks the same because the real reason was something else.

The method in brief: an always-on exit interview

The method is a standing exit interview. It's the same study, open all the time, that every new leaver is invited to. It differs from a one-off churn study in three ways:

  1. The invitation is triggered, not scheduled. A cancellation event sends the link. Nobody has to remember.
  2. Each answer is filed against the account. The reason sits next to plan, size, tenure and owner, so you can count it by revenue, not just by heads.
  3. The reason list stays fixed. Every interview ends with the same coded question, so this quarter's count compares with last quarter's.

What you keep from the one-off study is the interview itself: open questions about what happened, follow-ups on every vague answer, and a question bank built around events, not opinions. A dropdown can't ask "what happened that week?". An interview can. If you want the interview to run inside your cancel page instead of after it, with a save offer at the end, that's a different job; see cancel-flow exit interviews. This playbook is for the conversation that happens after the decision, when people are calmer and more honest about what went wrong.

One thing to take out first: failed payments. Recurly estimates that involuntary churn, mostly failed cards, makes up 20% to 40% of total churn across subscription businesses [3]. Those customers didn't decide to leave. Send them to dunning and a card update, not to an interview about what disappointed them.

Step by step

1. Decide who gets invited

Invite everyone who cancels voluntarily. Sampling is tempting, but when volumes are this small every account is worth a link, and an automated invite costs you nothing. Two exceptions:

  • Accounts that churned through an acquisition or a shutdown. Log the reason without an interview.
  • Your largest accounts. Invite them too, but also have a founder or the account owner ask for a live call. A big account's reason is worth the hour.

In B2B, the person who clicked cancel is often not the person who decided. Send the link to the admin who cancelled and, where you know them, to the main day-to-day user.

2. Invite within 48 hours, from a person

Send the invitation within two days, while the decision is fresh and before the story gets tidier. Write it as a short note from a founder or the account owner, not from a no-reply address. Something like: "You cancelled on Tuesday. I'd like to understand what we got wrong. Ten minutes, by text or voice, whenever suits you. No sales pitch, and we won't try to talk you out of it." Then keep that promise.

Use a personal link per person that carries your account ID. That ID is what lets the answer find its way back to the right record.

3. Ask about the last 90 days

Ten minutes is enough. Ask about events: when they first thought about leaving, what was happening in their business, what they tried, who else had a say, what they're using now. Keep opinions about the product for the end. The final question is the coded one: "Which of these comes closest to the main reason you left?", with the same options every time.

Question bank of eight churn interview questions in three groups: what happened, what comes next, and one reason-code question asked the same way every time.
Eight questions for a churn study that runs all year. The last one never changes, so the counts compare quarter to quarter. · Download or share

Keep the reason list to about eight options, and don't change it mid-quarter. A list that works for most B2B products:

CodeWhat it covers
Never got set upBought, didn't reach first value
Missing capabilityNeeded something the product can't do
Switched to a competitorMoved to a named alternative
Budget or company changeCuts, merger, project ended
Owner or champion leftThe person who chose you moved on
Service or reliabilityBugs, outages, support
Price for the valuePaid more than it was worth to them
OtherAnything else, always read by a person

The interview, not the code, tells you what "price for the value" meant for that account. The code is there so you can count.

4. Send the result back to the account

When the interview finishes, the result should flow to where the account lives: the CRM record, a Slack channel for the account owner, a sheet for the monthly review. Send three things: the reason code, whether the interview was good enough to count, and a link to the conversation. Leave the transcript where it is; the link is enough, and it keeps customer words out of systems that don't need them.

5. Read a sample yourself, every month

This is the step that keeps the pipeline honest. Every month, a founder or the PM reads:

  • every interview coded "Other",
  • every interview from an account above your large-account line,
  • five more picked at random.

Read the "Other" pile first. A new reason almost always shows up there before it shows up anywhere else. Hennink and Kaiser's review of saturation studies found that 9 to 17 interviews were usually enough to find most themes, when the group was fairly alike and the question narrow [4]. Your leavers in a given month are rarely that alike, which is why the random five matter: they're how you check that the codes still mean what you think.

Process in six steps for automated churn interviews, from separating failed payments to reporting the same reason codes every quarter.
The whole pipeline on one page. Steps one and five are the ones teams skip. · Download or share

6. Report the same codes every quarter

Count each code two ways: by accounts and by lost annual revenue. They tell different stories. Ten small accounts that never got set up are an onboarding problem. Two large accounts whose champion left are a relationship problem, and they may cost more. Put both counts on one page for the board, with two or three quotes that explain the top reason, and the same codes as last quarter. How to show customer evidence to your board covers that page in detail.

A worked example

Say you're a 45-person company selling scheduling software to physiotherapy clinics. You raised a Series A last year, and you now get around 30 cancellations a month. Eight of them are failed cards.

A physiotherapy clinic reception desk, the kind of small business customer whose cancellation reasons a churn study collects
A physiotherapy clinic reception desk, the kind of small business customer whose cancellation reasons a churn study collects

Photo by CDC on Unsplash

You route those eight to dunning. The other 22 get a personal link two days after cancelling, from the head of customer success, by email. About a third answer in the first month, mostly by text, a few by voice.

The coded answers come back as "price for the value" more than anything else, which is what your dropdown always said. Then you read the interviews behind them. Most of the clinics that said price had one physio using the product and two who never logged in. They were paying for three seats and getting value from one. The "Other" pile has two clinics that moved to a practice-management suite that bundles scheduling for free.

So you decide two things. First, seat activation in the first 30 days becomes the onboarding goal, and you check next quarter whether "price for the value" falls among clinics that activated every seat. Second, you add "Moved to an all-in-one suite" as a code, because two in a month is a pattern you want to count from now on, and you note on the board page that the list changed and why.

Nobody had to schedule anything to get there. Somebody did have to read.

Reading the results and deciding

Flowchart for each finished churn interview: check quality and fraud risk, write the reason to the account, read new reasons yourself.
Run every finished interview through these checks before it counts. Only the side branch needs a person. · Download or share

Some rules that keep you from over-reading a small pile:

  • Act on a reason when it holds for two months running, or when one month brings it from a large account and the interview is specific. One vivid quote isn't a trend.
  • Compare against your stayers before you fix anything big. If customers who stay give the same complaint just as often, it isn't what drives churn. The case-control approach explains why, and the leaver and stayer playbook shows how to run it.
  • Watch the "Other" share. If more than about one in five interviews land there, your code list is out of date. Add a code at the start of the next quarter, never in the middle.
  • Treat "price" as a question, not an answer. Read what price was being compared with. Usually it's value they didn't get. Sometimes it's a cheaper rival or a budget cut, and those are different fixes.
  • Count by revenue before you prioritise. A reason that costs 3% of ARR beats one that costs 30 small accounts worth half that.

Run it this week with Koji

Here's the exact setup in Koji. It takes an afternoon, and most of it needs no code.

Set up the study

Start from the Churn Exit Interviews template, or describe the study to the research assistant ("an always-on churn interview for customers who cancelled in the last week"). Add the open questions above, then end with a single-choice question for the reason code, with your eight options. Participants answer by text or voice, and Koji asks a follow-up whenever an answer is vague.

Turn on Remove personal details under Branding & Settings, then Privacy. Names, emails, phone numbers, addresses and employers are taken out of each transcript when it finishes, before analysis. It can miss things, so still check before sharing quotes widely.

Two ways, depending on whether you want to write code.

No code. Once a week, export last week's voluntary cancellations from your billing tool or CRM as a CSV, with a column called ID holding your account ID, plus email and name. In the study's share sheet, choose "A link per person" and upload the file (Import CSV on the Self-recruit tab does the same thing). Download the list back with an interview-link column and send it from your own email tool or CRM, using that column as a merge field. Re-uploading the same people reuses their links instead of making duplicates. Or add the leavers to Participants and send the invitation and a reminder from Koji itself, and see who opened and who took part.

Koji CSV import dialog creating personalised interview links for three respondents
Each person in the file gets their own link, tied to the ID column you uploaded.

With code. Your cancel event can start the interview through Koji's API, passing your account ID, and you can show it in your own page with the embed. Research automation with webhooks walks through the full loop. The API needs a plan that includes API access; webhooks work on any paid plan.

Send each finished interview back to the account

In the study, open Integrate, then Webhooks, and add the URL that should receive results. When an interview finishes and has been scored, Koji sends a signed message with your account ID, whether the interview meets the same quality bar the report uses, a fraud-risk level, and a link to the interview. The transcript stays in Koji, so the message carries IDs, scores and links only.

Koji webhooks screen with a delivery log of interview completed events, their status and attempts, and a Give this to your AI brief
The delivery log shows every message sent, and you can resend any that failed.

If you don't have an engineer free, point the webhook at a Zapier, Make or n8n webhook trigger and map the fields to your CRM (Zapier setup). If you do, the Give this to your AI button copies a full brief that a coding assistant can use to build the receiver, including the signature check. Use Send test event before you go live. Failed deliveries are retried, and the log lets you resend by hand. Webhooks need a paid plan; webhook setup has the details.

In your receiver, write the reason only when the interview meets the quality bar and the fraud risk isn't high. Everything else gets logged with a "check" flag for a person.

Read it and take it to the board

The study's report groups the reasons into themes, with the quotes behind each one, and you can refresh it as new interviews come in. For the quarterly board meeting, download it as an editable PowerPoint deck or as a PDF, cut it to your one page, and put the same codes next to last quarter's.

Common mistakes

  • Automating the reading too. A pipeline that only produces a chart turns into the dropdown you were trying to replace. Keep the monthly read.
  • Changing the codes every quarter. Then nobody can tell whether churn reasons moved or just the labels.
  • Interviewing failed payments. You learn nothing, and you annoy people who never meant to leave.
  • Sending the link three weeks later. By then, people have settled on a simple story, and many won't answer at all.
  • Counting accounts and not revenue. The biggest pile is rarely the most expensive one.
  • Pushing whole transcripts into the CRM. Send the code and a link. Fewer copies of what customers said is safer and easier to keep tidy.

When research will mislead you

People are poor witnesses to their own reasons. Nisbett and Wilson's classic review found that when people explain why they did something, they often report a plausible theory of why someone would do it, not what actually drove them [5]. That's why the interview asks about events and the code comes last, and why you read the interviews behind any code you plan to act on.

The people who answer aren't everyone. Leavers who were angry, or who liked you, reply more than leavers who just drifted away. Compare the reasons with what usage data says about the ones who didn't answer: if silent leavers stopped logging in months earlier, "never got set up" is probably undercounted.

An AI interviewer helps with depth, not with representativeness. Geiecke and Jaravel's study at the London School of Economics found that trained sociologists rated AI-led interviews as approaching the quality of human experts, and that respondents wrote 142% more words than they did in conventional open text fields [6]. That's a good reason to replace the dropdown with a conversation. It isn't a reason to believe whoever answered speaks for everyone who left.

And a churn interview tells you why people left, not what would have kept them. Test the fix: change onboarding for one cohort and watch whether that reason shrinks among the next quarter's leavers.

Where this fits

This sits on the scaleup stage of the founder's research playbook, under "why are customers leaving?".

More playbooks for founders are at /for/founders.

FAQ

How many churn interviews do I need each month?

There's no fixed number. Invite every voluntary leaver and read every "Other" answer, every large account and five at random. Over a quarter that usually adds up to enough to see the main reasons. Saturation research suggests 9 to 17 interviews are enough to find most themes in a fairly similar group [4], so a quarter of answers across several segments is a reasonable base.

Should I automate the interview or only the invitation?

Automate the invitation, the filing and the counting. The interview can be AI-moderated, and it runs whenever the customer has time. Keep a person reading the sample, and keep live calls for your largest accounts.

Will customers who just cancelled actually answer?

Many won't, and that's fine as long as you know who didn't. A short, honest note from a person, sent within two days, with a promise not to pitch, gets more answers than a no-reply survey. Text or voice on their own time helps too.

Can an AI interviewer handle a cancellation conversation?

Yes, for most accounts. It asks the same questions every time and follows up on vague answers, and an LSE study found AI-led interviews approaching the quality of human experts [6]. For your largest accounts, still ask for a live call with a founder.

What about customers whose card failed?

Take them out before you invite anyone. Involuntary churn is estimated at 20% to 40% of total churn [3], and those people need a working card, not an interview.

What should the board see?

One page a quarter: the same reason codes as last quarter, counted by accounts and by lost revenue, two or three quotes explaining the top reason, and what you changed because of it.

Your next step

Before you build anything, export last month's voluntary cancellations and send each one a personal link to a ten-minute interview. If a third of them answer and the reasons surprise you, build the rest of the pipeline next week.

References

  1. SaaS Capital. "2025 B2B SaaS Retention Benchmarks." SaaS Capital Research, 2025. https://www.saas-capital.com/research/saas-retention-benchmarks-for-private-b2b-companies/
  2. Aleph with Benchmarkit. "Gross revenue retention in SaaS: 2026 benchmarks" (CY2025 data, 342 companies). Aleph, 2026. https://www.getaleph.com/answers/gross-revenue-retention-saas-2026
  3. Recurly. "Failed payment recovery: a data-based strategy." Recurly blog, 30 April 2026. https://recurly.com/blog/failed-payment-recovery-data-based-strategy/
  4. Hennink, M. and Kaiser, B. N. "Sample sizes for saturation in qualitative research: A systematic review of empirical tests." Social Science & Medicine, 292, 114523, 2022. https://pubmed.ncbi.nlm.nih.gov/34785096/
  5. Nisbett, R. E. and Wilson, T. D. "Telling more than we can know: Verbal reports on mental processes." Psychological Review, 84(3), 231 to 259, 1977. https://deepblue.lib.umich.edu/handle/2027.42/92167
  6. Geiecke, F. and Jaravel, X. "Conversations at Scale: Robust AI-led Interviews." Working paper, London School of Economics, revised 2026. https://papers.ssrn.com/abstract=4974382

Further reading

Run your first AI-moderated study in 10 minutes

10 free credits on signup. No credit card required.

GDPR compliantEU or US data residencyNo AI training on your data
Koji

Koji Team

Product

Share this article

Keep reading