How to determine your activation metric

https://www.lennysnewsletter.com/p/how-to-determine-your-activation

👋 Hey, I’m Lenny and welcome to a 🔒 subscriber-only edition 🔒 of my weekly newsletter. Each week I tackle reader questions about product, growth, working with humans, and anything else that’s stressing you out about work.

Q: Learned a lot from your post on . But can you dive deeper into how to quantitatively determine your activation milestone? I’m building a multi-player B2B SaaS product.

In our post on activation rate (which is currently the fourth most popular post of all time 😮), Yuriy and I shared examples of activation milestones across hundreds of companies. But we didn’t get very concrete about how to go about nailing down your actual activation metrics. So I thought I’d do a follow-up post on this.

Below, I’ll share a three-step process for finding your activation metric, with real-life examples from a dozen companies. As a bonus, I’ll expand on activation milestones for multi-player SaaS products with help from Merci Grace (former Head of Growth at Slack), Lauryn Isford (Head of Growth at Airtable), Karri Saarinen (CEO of Linear), Ben Williams (VP of Product at Snyk), Badrul Farooqi (first PM at Figma), Jackie Bavaro (first PM at Asana), and Rachel Wang (Product Lead at Sprig). As always, if you have any feedback, additional suggestions, or questions, just leave a comment 👇

How to determine your activation metric

Follow this three-step process:

Brainstorm, and explore your product usage data, for some potential “aha” moments in the user journey.
Run a regression analysis to see if there’s an inflection in retention when someone hits any of those moments, to establish a correlative relationship between some potential activation milestones and product retention.
Run some experiments to see if increasing the percentage of users hitting that moment increases their retention rate, to see if any of those correlative relationships translate into true causality. A good activation metric is causal for your retention, not just correlative.

Here are six real stories from readers putting this process into action:

“For our new products with little or no usage data:

We conducted in-depth interviews with our target segment and asked them, ‘What are the signals that this product is solving your problem?’

We brainstormed a list of actions that reflect those signals.

We ran a quantitative survey targeting the target segment, showing the list of actions and asking respondents, ‘Which of the following is the first moment that you feel that this product solves your problem?’

We then picked the milestone that had been selected by the majority.

For existing products with a lot of historical data, we started with steps one and two above, but after that, we ran regression analyses to find which action completion correlated with long-term retention (30 days in our case).”

“We have a lot of qualitative data from our users because we do interviews on a weekly basis with them. So when we asked this question, we already knew about some candidates for the activation milestone.

We focused on when the users were getting value from our product. Then, once we had some candidates, we used data to see which of them were correlated with long-term retention. Mark Roberge has a really good process on how to do this.

After we saw some results, we just went with the one that ‘felt’ the best. That’s because we don’t have too much data. Around 300 customers. So we know there could be a lot of errors in the numbers, so we thought we shouldn’t do a decision based solely on data if the results were similar.”

“Building [a top-five e-commerce site’s] self-checkout product, we looked at retention rates across customer segments that completed various tasks successfully, for example:

Scanning an item

Completing a checkout

Placing 1, 2, 3, n orders per month

We noticed that completing a transaction was the best indicator of long-term retention. Particularly, customers who placed at least two orders retained at a 2x higher rate than with one order. So that became our activation milestone.”

“I work in B2B enterprise SaaS with a multi-use-case complex product. We don’t have enough data to conduct a causal analysis with conversion. Therefore, we broke down our product into three main JTBDs that prospects come to us for, as well as required behaviors that need to be set up in order to use the workspace successfully (e.g. import your data). For each of these areas, we conducted an analysis to see what events correlated most with trial retention (two weeks). We evaluated what the strongest correlations were and chose activation metrics for each area.”

“We initially focused mostly on qualitative insights to determine this milestone for our customer survey product—interviewing participants, doing guided onboardings and product demos, reviewing Hotjar recordings.

We learned that our ‘aha!’ moment occurs when they discover the analysis and automations that are possible with their results—and they can only find that once they’ve got at least one survey participant. So that became our activation milestone: getting you first survey responses. This gave us an easy-to-measure metric that allows us to focus on building and optimizing rather than tweaking our data forever.”