You launched, people signed up, and almost nobody did the thing your product is for. The decision in front of you is narrow: is this a problem with who you attract, or a problem with what happens in the first session? The short answer is that you can find out in a week by talking to about ten people who stalled and ten who got going, and you should do it before you rebuild onboarding or buy more traffic. By the end of this guide you will have a question bank, a rule for when to stop, and a decision for each thing you might hear.
When you need this
You need this when you have somewhere between 15 and 200 signups, a funnel too small to read, and a nagging feeling that the numbers are bad. Typical signs: sign-ups come in and the activation event (the one action that proves they got value) rarely follows; people "like the idea" in a reply but never log in again; you keep adding tooltips and an email sequence without a clue whether they help.
Photo by Michał Parzuchowski on Unsplash
The cost of guessing is real. If you are attracting the wrong people, a better onboarding flow polishes a door nobody should be walking through. If the flow is the problem, more traffic just fills a leaky bucket. Amplitude's 2025 Product Benchmark Report, which looked at over 2,600 companies, found no meaningful correlation between how well a product acquires users and how well it keeps them [1]. Good acquisition numbers tell you very little about whether the first session works.
There is also a trap in the benchmarks. Userpilot's 2024 benchmark of 547 SaaS companies put median B2B activation near 37% [2], and other published figures run from about 30% to 38%, depending on how each source defines "activated". Your 12% might be a disaster or might be a stricter definition. A benchmark can't tell you why. People can.
The method in brief
For this decision, use short customer discovery interviews about the first session, in the style of the Mom Test: ask about what people did, not whether they liked it. The closest fit among Koji's reference guides is user onboarding research, and onboarding drop-off research covers the diagnosis at larger scale. Aha moment research helps you pick the activation event, which you need before you start.
Two choices matter more than the method itself.
Compare stalled and activated users. Interviewing only the people who stalled tells you what they said went wrong. Interviewing the people who did activate shows you what the working path looks like, and often shows that your "bad" signups and "good" signups arrived wanting different things. This is the same logic as case-control research, applied to week one.
Interview, don't survey, at this size. With 40 signups, a survey gets you six answers of one sentence each. A ten-minute conversation lets you follow the thing they said that you did not expect.
Step by step
1. Pick one activation event
Write down the single action that means "they got value". Not "logged in" and not "finished the tour". For a scheduling tool it might be "sent a booking link to a real client". If you can't name it, interview three happy users first and ask what they did the first time it clicked.
2. Split the last 30 or so signups
Make two lists: those who did the event, and those who did not. Within the second list, separate people who never came back from people who came back and got stuck. These are different problems, and the second group is gold because they tried.
3. Decide how many
You do not need a statistical sample. In a published study of 132 respondents across four sites, Hagaman and Wutich found that 16 or fewer interviews were enough to identify the common themes in relatively homogeneous groups, while themes that ran across very different sites needed 20 to 40 [3]. That echoes Guest, Bunce and Johnson's earlier finding that about a dozen interviews can be enough for a homogeneous group [3]. Your signups are one product and, with luck, one kind of person, so aim for about 10 stalled and 10 activated. Stop early if the last three calls told you nothing new. If you can see two different kinds of people in your signup list (say, freelancers and agencies), treat each as its own group and plan about ten for each.
4. Invite within a week
Memory fades quickly, and a person who signed up three weeks ago will tell you about the tool they wish they had signed up for. Send a short, honest message: who you are, that you are not selling anything, that you would like ten minutes to hear what they were trying to do. Offering a small thank-you for the stalled group is fair, because they owe you nothing. Expect to send three or four invitations for each call you get.
5. Ask about what happened, in this order
Start before the product. "What were you trying to get done when you found us?" tells you whether this was your customer at all. Then go to the first session and ask them to walk you through it, screen by screen if they remember. Then ask what happened next, because that is where the real reason lives: a missing teammate, a missing integration, a boss who has to say yes, or simply a busy week.
Avoid "Why didn't you use it?" People answer it with the neat reason ("no time") rather than the true one. Ask what they did instead, that same week, for the same job. Also resist pitching the fix. If you say "what if we added templates?" they will say templates sound great.
6. Tag the first five minutes
For every call, write down the first moment they hesitated, in their words. A person who says "I wasn't sure what to put in the first box" has told you something no funnel chart can. If you have product analytics, line the story up with the event log for that person. The two together are far more convincing than either one.
A worked example
Say you're a two-person team selling scheduling software to dental clinics. You have 46 signups in six weeks and three of them sent a booking link to a patient. Activation is 6.5%.
You invite 16 of the stuck signups and all 3 of the activated ones, and get 9 and 3 to talk. Among the nine stuck:
- Five were office managers who signed up from a blog post about no-shows. They wanted to cut cancellations, and the first screen asked them to set up chairs and providers. They closed the tab.
- Two were dentists who signed up for themselves and could not send a link because the front desk owns the booking system.
- Two said they simply forgot.
Among the three who activated, all were office managers at single-chair practices, and all skipped setup entirely and used the one-off booking link on the home page.
What do you decide? It is both a first-screen problem and an audience problem. The fix with the best odds is to put the booking link on the first screen and move setup behind it. You also notice that the dentists bought into the idea but cannot act without the front desk, which is a hint about who the real buyer is. You write down both, make one change, and run another ten signups through the same questions two weeks later.
Reading the results and deciding
Count, don't impress yourself with a single vivid quote. After ten calls, tally each reason once per person.
- Most stalled people wanted something else. Fix who you attract: the page, the channel or the promise. Don't touch onboarding yet.
- They wanted the right thing but could not tell what to do first. Rewrite the first screen. You can test the new one with a few people before you ship it by showing a screenshot for ten seconds and asking what they think they can do there.
- They knew what to do but a step asked for something they did not have (data, a teammate, a credit card). Defer or remove that step.
- The reasons are all over the place. You probably have more than one kind of user. Split the list and do ten more for the largest group.
- Activated users are all from one narrow group. That is your ICP signal. Write it down and read the founder's research playbook for the next steps.
A rule of thumb to apply to the benchmark above: Amplitude's analysis suggests that if about 7% of a signup cohort is back on day seven, that is the line for the top quarter of products for activation [4]. It is a vendor's framing of one large dataset, not a law. If you are well below it, the interviews will tell you why. If you are above it and still unhappy, check whether your activation event is the right one.
Run it this week with Koji
You can do all of this with a notes app and a calendar. Koji makes the invitation, the interviewing and the tally faster, and is worth using if you do not want to run twenty calls yourself.
- Start the study. On the home screen describe the question in your own words ("Why do new signups never send a booking link?") and Koji's research brief assistant drafts the brief. Paste the question bank above and ask it to adjust the questions to your product. Koji's interviewer follows up on each answer, so you don't need to script every probe.
- Add a first-impression check (optional). Attach a screenshot of your first screen to a question and show it once, for 3, 5, 10 or 15 seconds, then ask what they remember and what they think they can do there. This works in text and voice interviews, not phone calls.
- Add a screener. Add one screening question so only people who signed up and meet your definition get in.
Invite the people on your list
Import your signup list into Participants (a CSV with column mapping works), group the stalled and activated people into two audiences, and email each one from the study's Self-recruit tab. Each person gets their own interview link, and you see who received the email, opened it and took part, plus reminders for the people who did not. Recruitment email is included on the paid plans (Insights, Interviews and Enterprise). If you want to thank people, you can turn on an incentive of €10 to €500 per participant and approve each one after you see the interview. Text interviews are on every plan; voice interviews need the Interviews plan or above.
Read what comes back
Koji analyses each interview and builds a report with themes, each backed by the participants who said it and their quotes. Check the quotes, not just the summary, and then count the reasons yourself using the rules above. When you need to show a co-founder or an investor, download the report as an editable slide deck or a PDF.
Common mistakes
- Interviewing only the people who stalled. You will hear a list of complaints with nothing to compare it to.
- Asking whether the onboarding was confusing. It leads them to say yes. Ask them to walk you through what they did.
- Treating "no time" as the answer. It is almost always a stand-in for "it wasn't worth the effort yet".
- Changing five things at once. You will not know what worked. Change one and re-run the same questions on the next ten signups.
- Counting friends. If your first 20 signups include people who signed up to be kind, remove them before you calculate anything.
When this research will mislead you
Interviews with people who signed up tell you about people who signed up. They say nothing about the ones who saw your page and left, so pair this with a look at your landing page (see test your landing page before you launch). Stated reasons are also softer than behaviour: people explain after the fact, and some of what they say will be a story. Where you can, check what they said against what the log shows. With very few signups, a handful of loud voices can look like a pattern, which is why you count per person. And if your product is complex enough that weeks of learning are normal, "stalled" at day five may just be "still deciding". Look at what happens over a longer window before you conclude anything.
Where this fits
Before this, you needed people to talk to: how to find 20 people to interview when you have no audience. After it, you probably need to know what to charge: price your first product with 30 customer conversations. The whole stage is laid out in the founder's research playbook, and Koji for founders shows how the pieces connect.
FAQ
How many signups do I need before this is worth doing? About 15 to 20 is enough to find people to talk to. With fewer, interview everyone who will talk to you.
What if nobody replies to my invitation? Expect a low reply rate and send more invitations rather than longer ones. A short message, a specific ask of ten minutes and a small thank-you help. Remind once.
Should I interview people who haven't signed up? That is a different question (why they didn't convert), and it needs a different sample. Start with the signups, because you already have their emails and they have already shown interest.
Is an in-app survey enough? A one-question survey is a good addition for volume, but it will not tell you what the person was trying to do or what they did instead. Use it to find people for the interview, not to replace it.
What counts as activation? The first action that delivers the value you promised, such as sending a booking link, importing a list or finishing a first report. Pick one and use it for both lists.
Can I run the interviews with AI? Yes, and it is a good fit when you cannot book twenty calls. A text or voice interview follows up on what the person says. Read a few transcripts yourself, so you know what your users sound like.
Next step
Pick your activation event today, split your last 30 signups into two lists, and invite ten from each. If you want the invitations, interviews and report in one place, start a study in Koji from a free account and paste in the question bank above.
References
- Amplitude. "The leaky bucket problem: More users don't mean more long-term growth." Amplitude blog, 2 October 2025, drawing on the 2025 Product Benchmark Report (over 2,600 companies). https://www.amplitude.com/blog/leaky-bucket-problem
- Userpilot. "SaaS product metrics benchmark report" (2024 edition, 547 SaaS companies). Userpilot blog, 2024. https://userpilot.com/blog/saas-benchmarks-report/
- Hagaman, A. K., and Wutich, A. "How Many Interviews Are Enough to Identify Metathemes in Multisited and Cross-Cultural Research? Another Perspective on Guest, Bunce, and Johnson's (2006) Landmark Study." Field Methods, 29(1), 2017. https://asu.elsevierpure.com/en/publications/how-many-interviews-are-enough-to-identify-metathemes-in-multisit
- Amplitude. "The 7 percent retention rule." Amplitude blog, 30 September 2025, drawing on the 2025 Product Benchmark Report. https://www.amplitude.com/blog/7-percent-retention-rule
Further reading
- User onboarding research: Koji's method guide for interviewing new users, with more detail on larger studies.
- Aha moment research: how to find the activation event this playbook asks you to pick.
- The Mom Test for your next 10 customer calls: the question discipline behind every script here.
- How many customer interviews does a founder need?: when to stop, and when to keep going.
