🧠 Deep dive — how to choose the right metrics (without wasting time on irrelevant ones)
Let's talk about what's available to measure, how you pick the right measurement, and where to place your focus. It's a deep dive into Pirate Metrics and my Flow Razor.
Hey friend 👋
Not all metrics are worth your time — and most can lead you astray.
I often see founders drowning in a sea of metrics, because a startup never has a shortage of things they can measure. The challenge is finding the right things to measure.
The single best tools for creating measurement focus are the Pirate Metrics framework and a weird little maxim I call the Flow Razor.
In this issue, I’ll walk you through both, and show how employing Pirate Metrics can lead to more effective and focused decision-making, leading to capital efficiency and rapid growth.
Let’s dive deep 👇
Effective measurement is the key to unlocking rapid startup growth.
Not to mention using time and resources wisely!
In fact, the build-measure-learn loop is just that:
What should you measure and when?
How should you measure it?
How do you optimise it?
But before we dive into how to choose a metric, let's talk about available metrics.
You can describe any business — from Walmart to your startup — with the same six categories: awareness, acquisition, activation, revenue, retention, & referral.
AAARRR! The pirate metrics. 🏴☠
It feels like I make this comical noise often…
While the specifics will look different from org-to-org, they are still measuring the same core things: the “flow” of people through a business model.
People need to know about you (awareness);
Encounter you (acquisition);
Get value out of your product (activation);
Pay for it (revenue);
Stay with it (retention); and
Help you acquire others (referral).
If you’re old school, it's a kind of funnel:
These metrics are the “engine” (or funnel, or factory, or whatever metaphor you prefer) of your company.
In fact, a successful startup is an engine that just hums along, consuming tons of customers and losing none along the way.
In other words: traction.
A problem in early-stage startups is that we can’t just measure “traction”.
That’s not a thing — yet.
Traditional metrics — revenue, market share, etc — are zero on day 1, and functionally zero on day 300. They aren't big enough to be meaningful yet, and they aren't true indicators of progress.
Again: yet.
But if dollars and customers aren’t the right measurement, what is?
We’re looking for leading indicators for what we can’t measure directly yet. What are the signals that point toward the eventual dollars and customers?
And then… what are the signals that we’re heading toward those signals that point toward dollars and customers?
And then… what are the signals that point toward those signals that point toward … you get the idea. It's turtles all the way down.
It’s chaining indicators. Innovation accounting.
At one end of the chain, we measure a successful, profitable business.
Hopefully. 🤞
And at the other end of the chain is today, with the things we can measure meaningfully right now, which tell us we're on the right track.
So, what are the right signals for each pirate metric for your startup, and how do you optimise it?
Let’s break ‘em down! 👇
Awareness: how do customers discover you before you know who they are?
This is highly unlikely to be relevant to an early-stage startup, because you’re creating something new.
There’s nothing to be aware of yet, and once there is, you can’t afford the extraordinary cost of awareness until you already have a bunch of customers.
This is because, without existing customers or risky guerrilla campaigns, awareness is bought with ads, and an early-stage startup can’t afford to buy it. It’s customer discovery 101:
Find demand before you can create demand.
Find demand by finding early adopters.
So, for now, you shouldn’t care about awareness, so I don’t either.
Moving on!
Acquisition: how do they begin to engage with your company?
In sales language, this is turning from prospect into lead. They haven’t bought yet, but they’ve signalled they might.
It’s the top of the funnel, as far as an early-stage startup is concerned, and it’s my favourite one, because it’s what trips founders up most often.
This is the hilarious mistake where you add new features to the app no one is using hoping “just one more feature” will solve everything. 😂
New features will not get you customers — ever.
But a new offer might.
Because acquisition isn't based on delivery. It's based on the promise of delivery. It's the promise to the customer that you understand their pain, and that their life will be substantially better after using your product.
If they don’t want what you’re promising, who cares if you can deliver on it? Who cares about optimising retention? Who cares about the right pricing?
None of it matters until they want what you’re selling.
Common measures of acquisition include:
Percent of site visitors who becomes leads
Engagement rate on social media posts
Email click rates
Tweaking those metrics is always tweaking one or more of three things:
Customer (who)
Channel (where)
Copy (what)
When you’re trying to optimise acquisition — when you’re trying to get customers to enter your ecosystem and begin to engage with you — then you can tweak each of these levers.
Put another way, if your message isn’t converting:
Do you have the wrong audience?
Do you have the wrong value proposition?
Did you post that message in the wrong place?
Run an experiment to tweak one of those at a time to slowly home in. Once you can reliably get customers interested, you can move move on.
Activation: how do customers begin to get value?
Ok, you’ve got people interested on the surface. How do they get more interested?
Activation metrics are signals that users may become paying customers, because they have started to receive some value.
Something like… they haven’t agreed to marry you yet, but they’ll meet your parents.
The classic example is freemium. But there’s more:
Percent of site visitors who create an account
Time spent browsing the site
Percent of users that complete onboarding
Lead magnet opt-in rate
The goal of optimising this step is increasing the value to customers who are not paying, so that you increase the likelihood that they’ll find sufficient value to start paying.
How you tackle activation depends on how you measure it. For example…
Percent of site visitors who create an account?
Tweak the call-to-action
Make it easier to buy
Split test the hero messaging
Percent of users who complete onboarding?
Tweak user experience
Ensure clear connection to the sign up call-to-action
Add empty states
Activation is a “connective tissue” step — the segue between hmm, that’s interesting and oh, I get it now!
Until your activation rates are honed, you should continuously run experiments to range yourself to the target.
Once they get it, you can start optimising for:
Revenue: how do customers become paying customers?
Users are interested — fantastic! Let's get the money.
What it means to become a customer depends on your business:
SaaS: freemium → premium
Service business: lead → signed contract
Content creator: newsletter → course purchase
While the measurement of revenue should include a conversion rate from activation to revenue, it should also include broader metrics:
Freemium conversion rate
Monthly or annualised recurring revenue (MRR/ARR)
Customer acquisition cost (CAC)
Average deal size
etc.
Remember: the relevant comparator (the denominator) to revenue isn’t “leads” or “prospects” — it’s the users that have activated. That’s great, because it reduces the number of levers you have to play with drive revenue:
Are you charging the right amount?
Are you charging the right way?
Have you optimally placed the offer?
Have you presented it with optimal frequency?
etc.
In other words:
Is it the right offer?
In the right place?
At the right time?
And is it easy to buy?
The goal is to find the combination of the above that makes your offer irresistible.
Retention: how long do customers stay customers?
This doesn’t only apply to SaaS with recurring monthly revenue.
Consumer products have retention, too! There are whole business model patterns designed to capture longer-term retention. For example, “razors and blades” is premised on selling the main unit up front (sometimes at a discount) in order to capture subsequent, and probably recurring, revenue.
But even if it’s a one-and-done product, you still want to capture a larger share of their wallet:
Do they need to buy another unit at some point?
Are there related products that you can sell as add-ons?
etc.
SaaS or not, retention is just how long you can keep a customer in your ecosystem before they switch to something else — or abandon entirely.
Some common retention metrics:
Lifetime value (LTV) — the dollars per customer you can capture over the entire relationship.
Churn rate — the percent of customers who leave in a time period, e.g. monthly.
Engagement rate — how often do customers come back?
Net promoter score — while not a direct measure of retention, it’s a powerful signal of what’s to come.
Retention is often a tough nut to crack, because it’s a lagging indicator of another, often unclear problem. Therefore, the things you try may not have an immediate effect on retention.
Sometimes, a solution to try is obvious:
If users don’t come back to check the app as often as they should, you can add push notifications or email reminders.
If the reminders are clicked, that’s a signal that you’ll see an uptick in retention.
If the reminders aren’t clicked, customers aren’t seeing the value you promised.
Unfortunately, most of the time, a solution to try is not obvious. In those cases, the tendency of founders is to add features to try increase value — blindly.
That’s a mistake.
To increase retention, run an experiment:
Form an hypothesis of what you can do that would increase retention.
Find a signal that you can measure which implies that retention will increase of you're right.
Pick a target value for that signal. Never just “see what happens”.
Run the smallest possible experiment to test that hypothesis.
Assimilate and repeat.
Do this iteratively, and you’ll either see the increase, or clarify why you’re not.
And, lastly:
Referral: how does serving customers help you acquire more customers?
It’s easy to oversimplify this one as a “viral loop” — customers literally referring other customers to use your product.
Viral loops are fantastic… when your product supports a viral loop.
Outside true viral loops, it’s still great if your product is so amazing that people just can’t help but tell their friends about it. That should always be your goal.
But referral is so much more than the literal interpretation.
Referral is when the dollars you spend on product get you more customers, rather than dollars you spend on sales or marketing.
This is best explained by example.
Amazon is a retailer; they sell stuff. But they have a sneaky referral loop built in: customer reviews.
Even standing in an aisle at Best Buy, customers will pull out their phones to see how a product rated on Amazon. Those reviews are an example of co-creating value with your customers.
It’s akin to content creators: the YouTube product becomes more valuable when a creator posts a video, because someone might want to watch it.
Amazon product reviews are similar. They bring in new customers. And they only spent product dollars to get them.
That’s referral!
Here are some example referral metrics:
Net promoter score (likelihood to recommend)
Rate of customer reviews (e.g. on Yelp)
Number of shares on social media
You should probably measure something like that, but don’t stop there!
If you’re at the referral stage, ask yourself how you can help your customers co-create value in your product. What kinds of network effects can you create?
And then test them.
But enough theory!
Put it in practice with the flow Razor.
The best way to get immediate impact from pirate metrics is through what I call the “flow razor”:
The most important task at any moment is that which will have the greatest immediate impact on the flow of customers through the business model at that moment.
All businesses look different, but we can describe their flow of customers the same way, using pirate metrics:
Awareness
Acquisition
Activation
Revenue
Retention
Referral
From one step to the next, customers “flow” through the business.
So you can think of each of these metrics as levers you can pull to try to increase the flow of customers from one stage to the next. In other words, you can increase revenue by optimising at that step, but you can also increase revenue by driving up activation.
It’s largely linear.
This is often where we find “procrastivity” — e.g. founders try to optimise retention before they have customers to retain, because it’s much more painful to our egos to face customers in the market and get rejected!
So: which lever can you pull that will move the most customers right now? Which is the right pirate metric to focus on today?
This is always the earliest in the flow that you have meaningful unknowns. It is also, not coincidentally, where you usually find your riskiest assumption — and a founder’s job is always to test their riskiest assumption, right? 😅
Find the metrics within that, and then follow the standard experiment process:
Step 1: create an hypothesis.
Ask what if…?
Remember: an hypothesis always answers three questions:
What’s the one thing you’re testing?
What are the details: who, what, where, how, and when?
What does failure look like specifically?
In other words, an hypothesis is discrete, specific, and falsifiable.
Step 2: run an experiment.
Design a way to answer the question your hypothesis is asking.
Honestly, this is the easy part, because the hard part is already done. And, if you’ve designed your hypothesis to be precise, discrete, and falsifiable, your experiment produces unambiguous results — it’s crystal clear if you were right or wrong. There’s no room for grey.
Step 3: learn from the data.
In addition to the result of your experiment, you get to ask my favourite question:
Then you have decisions to make:
Do you pivot or persevere?
Do you try something new to achieve the goal?
Do you set a different goal?
Do you change the path we’re on entirely?
And then… rinse, and repeat until you have optimal flow, and can scale.
Because a startup is just an organisation in search of flow.
See you next week.
—jdm