{"site":{"name":"Koji","description":"AI-native customer research platform that helps teams conduct, analyze, and synthesize customer interviews at scale.","url":"https://www.koji.so","contentTypes":["blog","documentation"],"lastUpdated":"2026-10-31T14:31:25.741Z"},"content":[{"type":"blog","id":"661a8883-a076-49aa-a376-9ffac4a40427","slug":"ai-substitution-customer-interviews-pe-playbook","title":"Will Your Customers Switch to an AI-Native Rival? A PE Operating Partner Playbook for Mid-Hold Customer Interviews","url":"https://www.koji.so/blog/ai-substitution-customer-interviews-pe-playbook","summary":"A playbook for PE operating partners on interviewing portfolio company customers about AI substitution before an operating review. It sets out a revenue-weighted sampling rule, the Four Moats Test for sorting accounts as anchored, drifting or exposed, a four-week timeline, a nine-question interview guide, and how to run it in Koji with AI-moderated interviews.","content":"If you own a software or tech-enabled services company that was underwritten before generative AI, the question for your next operating review is simple: which customers would leave if an AI-native rival offered the same job for less, and which would not? Management will tell you the moat is deep. Four weeks of AI-moderated customer interviews will tell you whether the customers agree, account by account and weighted by revenue. By the end of this playbook you will have the sampling rule, a day-by-day plan, a full interview guide to paste, and the setup in Koji.\n\n## Why this review needs customer evidence now\n\nAn operating review used to test the plan against the budget. Since early 2026 it also has to test the equity story against a new kind of competitor, and the numbers you already see (churn, NRR, pipeline) are lagging indicators for that risk. A customer decides to replace a tool long before the contract shows it.\n\nFour facts frame the problem.\n\n**Your peers are already changing how they diligence this.** In EY's Private Equity Pulse for Q1 2026 (April 2026), 64% of general partners said they had become more selective and 60% had increased diligence on AI disruption risk. 44% were focusing on AI-enabled software ([EY, PE Pulse, April 2026](https://www.ey.com/en_gl/newsroom/2026/04/pe-navigates-a-more-complex-geopolitical-and-macroeconomic-environment-ey-analysis)). A buyer at your exit will ask the same question about your company, so you want your answer before theirs.\n\n![Bar chart of how general partners responded to AI risk in software: 64% more selective, 60% more diligence on AI disruption risk, 44% focus on AI-enabled software](/visuals/pe-gp-response-to-ai-software-risk/pe-gp-response-to-ai-software-risk.png \"Most GPs are already testing for AI disruption risk · Source: EY, Private Equity Pulse Q1 2026, April 2026\")\n\n**The exposure is large and concentrated in software.** The Bank for International Settlements reported in March 2026 that loans to SaaS firms grew from about $8 billion in 2015 to more than $500 billion by the end of 2025, which is 19% of direct loans ([BIS Quarterly Review, March 2026](https://www.bis.org/publ/qtrpdf/r_qt2603v.htm)). Your lenders care about this as much as your LPs do.\n\n**The vintage matters.** A TPG co-managing partner told S&P Global in February 2026 that \"more risk\" sits in companies underwritten in 2018, 2019 and 2020, before generative AI, and that those fund vintages are \"more susceptible to risk and disruption\" ([S&P Global Market Intelligence, February 2026](https://spglobal.com/market-intelligence/en/news-insights/articles/2026/2/private-equity-s-volume-of-software-deals-slowed-as-ai-risks-grew-98507665)). If you bought in that window, you are holding the asset the quote describes.\n\n**The threat may come from your customers, not a startup.** Cruxy surveyed 300 B2B SaaS CEOs in the UK and US in 2026: 54% said customer-built solutions (\"vibe coding\") were the bigger threat, and 45% said AI-first startups taking market share ([SecurityBrief UK, 2026](https://securitybrief.co.uk/story/saas-chiefs-expect-pricing-shift-as-ai-threatens-sales)). Cruxy is a vendor and these are CEO expectations, not observed behaviour. Retool's 2026 Build vs. Buy report, from 817 respondents, found 35% of enterprises had already replaced a SaaS tool with custom software ([Retool, 2026](https://retool.com/newsroom/build-vs-buy-report-2026)). Retool sells a build platform and its sample skews to engineers, so read it as a signal, not a market rate.\n\nAdd the hold period. Private Equity Info put the median hold at 6.0 years in February 2026, the longest it has tracked ([Private Equity Info, 2026](https://privateequityinfo.com/blog/private-equity-holding-periods-continue-to-climb)). The longer you hold, the more AI-native products reach maturity inside your ownership. The cost of not asking is a plan that assumes a moat nobody has tested, and a buyer who tests it for you.\n\n## What good looks like: the Four Moats Test\n\nManagement's claim is usually one sentence: \"our customers can't leave, we're the system of record.\" The Four Moats Test turns that into something you can count. It is Koji's working method, built from standard switching-cost thinking, not an industry standard.\n\nFor each customer, listen for four moats, and note whether the customer **named it unprompted** or only agreed when asked:\n\n1. **System of record.** Their data and history live in the product, and exporting it would hurt.\n2. **Embedded workflow.** Other systems, approvals or teams depend on it daily.\n3. **Compliance or trust.** An audit, a certification or a regulator makes replacing it slow.\n4. **Service and relationship.** A named person at your company is the reason they stay.\n\nThen sort every account:\n\n- **Anchored:** names two or more moats unprompted.\n- **Drifting:** names one, or names moats only when prompted.\n- **Exposed:** names none, or has already built or bought something that does part of the job.\n\nThe unprompted/prompted split is the whole point. A customer who says \"we'd never leave\" after you list the benefits has told you nothing. A customer who volunteers \"we'd have to re-do our audit trail\" has.\n\nRules that make the result trustworthy:\n\n**1. Sample by revenue, then report by revenue.** Fewer than 25 customers: invite all. 25 to 250: aim for 25 to 50 completed interviews, with every top-10 account invited and at least two contacts at each. More than 250: 40 to 80 completes, stratified by revenue band and product line. Always add customers who churned or cut seats in the last 12 months. Report each theme twice: by count (\"9 of 38\") and by share of revenue. The second number is the one the board cares about.\n\n**2. Ask unaided first, aided second.** Ask what they use AI for in this job before you mention any alternative. Never name a competitor in the invitation or the opening. You are measuring what is already in their head, not putting it there.\n\n**3. Ask about behaviour in the last 12 months.** \"What have you built or bought to do part of this?\" gets a fact. \"Would you consider an AI alternative?\" gets politeness.\n\n**4. Interview the user and the buyer.** The sponsor often loves the product while the team that uses it is quietly working around it. Two contacts per top account.\n\n**5. Let the CEO send it.** The invitation comes from the portfolio company, from the CEO's address, with one honest line: \"We are reviewing how the product fits your work as tools change, and want to hear it from you.\"\n\n**6. Decide the call before you read the data.** Write the decision the review will make (invest in the roadmap, protect the base with multi-year terms, reprice, prepare a sale story) and tie each question to one.\n\n**Common mistakes**\n\n- Asking \"are you worried about AI?\" It produces headlines, not behaviour.\n- Letting management pick the accounts. You get the loyal ones.\n- Reporting only counts. Nine loud small accounts can hide one exposed top-five account.\n- Mixing in a price test. Keep willingness to pay for a separate study.\n- Treating \"anchored\" as permanent. Rerun the same guide in six months.\n\n## The four-week timeline\n\nCounted back from the day of the operating review (T-0):\n\n![Process showing the AI substitution interview timeline for a PE portfolio company, from four weeks before the operating review to the one-page readout](/visuals/pe-ai-substitution-interview-timeline/pe-ai-substitution-interview-timeline.png \"Four weeks back from the operating review\")\n\n- **T-28 to T-26:** Write the decision. Pull the customer list with revenue, tenure, product and the last renewal date, plus accounts that churned or cut seats.\n- **T-25 to T-22:** Build the guide, set guardrails, preview it yourself, and run 5 pilot interviews.\n- **T-21:** CEO signs off. Send wave 1 to the top 10 accounts and the churned customers, with a personal link each.\n- **T-14:** Wave 2 to the remaining accounts by revenue band, one reminder each.\n- **T-7:** Close fieldwork. Top up thin bands with personal follow-ups.\n- **T-6 to T-3:** Read the report. Cut every theme by count and by share of revenue.\n- **T-2:** One-page readout: anchored, drifting and exposed accounts, with the quotes behind each.\n\n## How to set it up in Koji, step by step\n\nKoji has PE and VC templates in the template gallery. The closest ones are Portfolio Company NPS Interviews and Value Creation Monitoring. Start from either and replace the brief with the one in the guide below, or type your own research question into \"What do you want to research?\".\n\n1. **Start a study.** Open the dashboard and choose a template from the gallery, or start from your research question.\n2. **Shape the research brief.** State the decision, the audience (current customers by revenue band, plus churned and downsized accounts) and what to avoid. Edit the brief by hand or ask the AI consultant to change it.\n3. **Add structured questions where you need a number or a choice.** Koji supports open-ended, scale, single choice, multiple choice, ranking and yes/no questions. Use a 1 to 10 scale for \"how hard would this be to replace in 12 months\" and a single choice for who owns the decision, so you can compare accounts and rerun it.\n4. **Set guardrails.** List what the interviewer must not do: name competitors, mention a sale process, discuss pricing plans or refer to other customers.\n5. **Choose voice or text, then preview.** Run the preview yourself. Voice interviews suit executives who would rather talk for ten minutes than type. Text works for everyone else.\n6. **Invite the customers.** Import your customer list from a CSV so each person gets a personal interview link, and set a response limit so one segment does not fill the sample. Send it from the CEO's mailbox.\n7. **Read the report.** Findings show how many participants raised each theme, with the supporting quotes.\n\n![Koji question editor showing open ended, 1 to 10 scale and single choice questions with follow-up settings](/email/shots/shot-question-types.jpg \"Mix a replaceability scale with open follow-ups in the question editor\")\n\n**Plan and cost.** A text interview costs 1 credit and a voice interview 3. The Free plan includes 10 credits at signup, enough for a pilot of text interviews with personal links. Voice interviews and the CSV import need the Researcher plan (€99 a month, 70 credits) or higher. Forty text interviews use 40 credits; the same forty by voice use 120. Check the pricing page before you commit, since plans change.\n\n## Copy this interview guide\n\nPaste this into any tool. In Koji, paste the brief into \"What do you want to research?\" and edit.\n\n**Brief (one line):** _We own [Company]. We want to understand which parts of a customer's work the product does today, what they already do with AI tools or in-house builds, and what would make them move or stay, so we can plan the next two years without guessing._\n\n**Opening line:** \"Thanks for making the time. We are looking at how the product fits your work as tools change, and your honest answers help us decide where to invest. We will report themes across customers, not individual answers by name. It takes about 10 minutes.\"\n\n| # | Question | Type | Follow-up probe |\n| --- | --- | --- | --- |\n| 1 | What work does the product do for your team in a normal week? | Open | Who else depends on that output? |\n| 2 | If it disappeared on Monday, what would you do on Tuesday? | Open | How long until you were back to normal? |\n| 3 | Which parts of that work do you already do with an AI tool? | Open | What made you try it for that part? |\n| 4 | In the past year, what have you built or bought to do part of this job? | Open | Who made that call, and is it still in use? |\n| 5 | Tell me about the last time someone pitched you an alternative. | Open | What happened next? |\n| 6 | How hard would it be to replace the product within 12 months? | 1 to 10 scale | What would move that number down by two points? |\n| 7 | What would an alternative have to do before you moved? | Open | Who would have to agree? |\n| 8 | Who decides whether you keep the product? | Single choice: me, my team, procurement, finance, someone else | Does that person use it? |\n| 9 | What do we do that you could not replace in a year? | Open | What would be hardest to copy? |\n\n**Closing question:** \"Is there anything about how you work today that we have not asked and should have?\"\n\n![Question bank of nine AI substitution interview questions for portfolio company customers, grouped into the work today, what you have tried and what holds you](/visuals/pe-ai-substitution-interview-question-bank/pe-ai-substitution-interview-question-bank.png \"Nine questions to reuse, in order\")\n\n## What you get\n\nA report with themes ranked by how many participants raised them, the quotes behind each theme, and the scale and choice answers as distributions. Findings link back to the interviews that support them, so a board member who asks \"who said that?\" gets an answer.\n\n![A leadership team around a table with laptops reviewing customer findings before an operating review.](https://images.unsplash.com/photo-1503423571797-2d2bb372094a?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3w5NDUzMDN8MHwxfHNlYXJjaHw0fHxib2FyZCUyMG1lZXRpbmclMjB0YWJsZSUyMGxhcHRvcHN8ZW58MXwwfHx8MTc5MzQ0MDIxMHww&ixlib=rb-4.1.0&q=80&w=1080)\n\n_Photo by [Pawel Chu](https://unsplash.com/@pawell?utm_source=koji&utm_medium=referral) on [Unsplash](https://unsplash.com/?utm_source=koji&utm_medium=referral)_\n\nHow to use it at the review:\n\n- **The sort:** list every account as anchored, drifting or exposed, with revenue next to each. This is the page the review spends its time on.\n- **The roadmap:** the jobs customers already hand to AI tools show where your product must absorb the work or stop charging for it.\n- **The commercial plan:** multi-year renewals for drifting accounts; save-plays for exposed top accounts before they decide.\n- **The exit story:** a dated, documented record of what customers said, and what you did about it.\n\n## The same playbook elsewhere\n\nThe logic is not specific to PE. A [customer success team](/blog/renewal-risk-interviews-customer-success-playbook) runs renewal-risk interviews on one account at a time. A deal team runs the outside-in version before signing, in [change-of-control interviews](/blog/change-of-control-customer-interviews-due-diligence-playbook) and [confirmatory customer calls](/blog/confirmatory-customer-calls-due-diligence-playbook). A GTM team asks [buyers what they asked AI before they called](/blog/ai-buyer-research-interviews-gtm-playbook). On the sell side, founders prepare for [investors calling their customers](/blog/investor-customer-reference-calls-founders). The first-weeks version for a new acquisition is the [100-day customer baseline](/blog/100-day-customer-baseline-pe-playbook); if you ran one, this is the follow-up using the same sample frame.\n\n## Why Koji for this review\n\n![Table comparing Koji with consultant-led interviews and surveys for testing AI substitution risk among portfolio company customers](/visuals/pe-ai-substitution-methods-compared/pe-ai-substitution-methods-compared.png \"How three ways of testing customer stickiness compare\")\n\n| Criterion | Koji | Consultant-led interviews | Survey |\n| --- | --- | --- | --- |\n| Depth | Open conversation with follow-ups | Deep, trained interviewer | Shallow, fixed answers |\n| Sample control | You set quotas and invite by segment | Agreed with the firm | Self-selected responders |\n| Time to results | Days to a few weeks | Weeks | Days |\n| Scale | Dozens to hundreds | Tens | Hundreds |\n| Probing \"we'd never leave\" | Follow-ups on every answer | Skilled probing | None |\n| Consistency | Same guide for every account | Varies by interviewer | Identical |\n| Rerun in six months | Same guide, same questions | New engagement | Easy |\n| Analysis effort | Themes and quotes generated | Done by the firm | Manual for open text |\n\nA consultant-led study is the right choice for a handful of C-suite buyers at your biggest accounts, where a trained human adds judgement a script cannot. A survey is fine for a quick pulse. For the middle ground, which is dozens of accounts that each need follow-up questions, Koji is built for the job. Reasons it works:\n\n1. **Follow-ups on every answer.** \"We'd never switch\" gets a \"why not?\" and a \"what would change that?\" every time, which is where moats are confirmed or exposed.\n2. **Numbers and stories in one interview.** The replaceability scale and the decision-owner choice become distributions you can rerun; the open answers carry the story.\n3. **Personal links from a CSV.** You know who answered, so you can weight the findings by revenue and chase thin segments.\n4. **Guardrails.** The interviewer will not name competitors or mention a sale, which keeps the aided questions clean.\n5. **Evidence behind every finding.** Each theme shows how many participants raised it, with the quotes. See [tracing report findings to evidence](/docs/tracing-report-findings-to-evidence).\n\n## Questions you might have about AI interviews\n\n**Will customers talk to an AI sent by an investor?** The invitation comes from the portfolio company's CEO, not from the fund, and says how long it takes. Customers answer a vendor they know. Offer voice and text so executives can choose. See [how AI interviewers work](/docs/how-ai-interviewers-work) and [researching hard-to-reach audiences](/docs/hard-to-reach-participants-research).\n\n**Is the sample representative if only willing customers answer?** Not by default, which is why you set quotas by revenue band, invite every top-10 account with a personal link, add churned customers, and report both counts and revenue share. Say in the readout how many of each band you reached.\n\n**Where does the data go, and who sees the quotes?** Read [AI interview data privacy and security](/docs/ai-interview-data-privacy-security) and [the GDPR and LLMs guide](/blog/can-i-paste-user-interviews-into-chatgpt-a-guide-to-gdpr-and-llms), and agree the data-handling terms with the portfolio company's counsel before you launch. Teams that need their own model keys can use bring-your-own-key on Enterprise plans ([docs](/docs/bring-your-own-key)). Interview quality is checked, and only conversations that pass the check use credits ([how the quality gate works](/docs/how-the-quality-gate-works)).\n\n## When Koji may not be the right fit\n\nIf the company has fewer than about 15 customers that matter, call them yourself: each account deserves a personal conversation and a quota makes no sense. If the customers are public bodies or regulated entities that require a named human interviewer, or the real question is a technical proof of concept with a rival product, interviews will not settle it. And no interview replaces cohort data. Use usage and renewal data to find who to ask, and interviews to learn why.\n\n## Metrics to track\n\n- Completion rate by revenue band, against quota\n- Share of top-10 accounts reached, with two contacts each\n- Share of revenue classed as anchored, drifting and exposed\n- Share of customers who volunteer an AI tool or in-house build unprompted\n- Median replaceability score (1 to 10) by segment, and the change at six months\n- Days from first invitation to the one-page readout\n\n## FAQ\n\n**How many customers should I interview before an operating review?** Fewer than 25 customers: invite all. 25 to 250: 25 to 50 completes with every top-10 account. More than 250: 40 to 80, stratified by revenue band. Always add churned and downsized accounts.\n\n**Should I name specific AI competitors in the interview?** Not at first. Ask what they use AI for and what they have built or bought, then, if the guide calls for it, ask about specific alternatives after the unaided answers are recorded.\n\n**How long does it take?** Four weeks counted back from the review: one week to prepare and pilot, about two weeks of fieldwork in two waves, and one week to read and write up.\n\n**What does it cost?** A text interview is 1 credit and a voice interview 3. Voice and CSV import need the Researcher plan or higher; a pilot of text interviews works on the Free plan.\n\n**Can I run it across several portfolio companies?** Yes. Use the same guide so the replaceability scale compares across companies, and keep each company's study separate.\n\n## Run it before the next review\n\nStart from a PE and VC template in the dashboard, or paste the brief above into \"What do you want to research?\". See [Koji for PE and VC](/for/pe-vc) or browse [every PE and VC playbook](/use-cases/pe-vc).\n","category":"Use Cases","lastModified":"2026-10-31T09:50:10.707591+00:00","metaTitle":"AI Substitution Interviews for PE Operating Partners","metaDescription":"Test whether portfolio customers would switch to an AI-native rival. Sampling rule, four-week plan, interview guide and Koji setup for PE operating partners.","keywords":["AI disruption portfolio company","customer interviews private equity","operating partner playbook","AI substitution risk","mid-hold customer research","software portfolio customer retention"],"aiSummary":"A playbook for PE operating partners on interviewing portfolio company customers about AI substitution before an operating review. It sets out a revenue-weighted sampling rule, the Four Moats Test for sorting accounts as anchored, drifting or exposed, a four-week timeline, a nine-question interview guide, and how to run it in Koji with AI-moderated interviews.","aiKeywords":["AI substitution","portfolio customer interviews","private equity operating partner","Four Moats Test","mid-hold review","AI-native competitors"],"aiContentType":"guide","faqItems":[{"answer":"Fewer than 25 customers: invite all. 25 to 250: aim for 25 to 50 completed interviews with every top-10 account invited. More than 250: 40 to 80, stratified by revenue band. Always add churned and downsized accounts.","question":"How many customers should I interview before an operating review?"},{"answer":"Not at first. Ask what customers use AI for and what they have built or bought, then ask about specific alternatives only after the unaided answers are recorded.","question":"Should I name specific AI competitors in the interview?"},{"answer":"Four weeks counted back from the review: one week to prepare and pilot, about two weeks of fieldwork in two waves, and one week to read and write up.","question":"How long does it take?"},{"answer":"A text interview costs 1 credit and a voice interview 3. Voice interviews and CSV import need the Researcher plan or higher. A pilot of text interviews works on the Free plan with its 10 signup credits.","question":"What does it cost in Koji?"},{"answer":"Yes. Use the same guide so the replaceability scale compares across companies, and keep each company's study separate.","question":"Can I run it across several portfolio companies?"}],"relatedTopics":["AI substitution interviews","PE operating partner","portfolio company customer research"]}],"pagination":{"total":1,"returned":1,"offset":0}}