What They Are
Vanity metrics are numbers that look impressive but reveal nothing about whether your decisions are working. Total registered users, total page views, cumulative revenue — these grow as long as you keep spending on acquisition, regardless of whether your product is improving or your customers are actually getting value. They create the illusion of progress.
Actionable metrics are causally connected to decisions. When an actionable metric changes, you can draw a clear line to a specific action you took. They answer the question: Did that thing we did actually work?
Eric Ries frames the distinction sharply in The Lean Startup - Ries - 2011: vanity metrics flatter, actionable metrics inform.
The Three Tests for Actionable Metrics
A metric passes only if it satisfies all three conditions:
- Actionable — There is a clear cause-and-effect relationship. A change in the metric traces back to a specific decision, not seasonal variation or a press mention.
- Accessible — Simple enough that the team understands and uses it. A metric that requires a data scientist to interpret every week will not drive behavior.
- Auditable — Generated from real customer behavior, not proxy data. The numbers can be verified independently; they don’t come from a “black box.”
Concrete Examples
| Metric | Type | Why |
|---|---|---|
| Total registered users | Vanity | Grows with ad spend; tells nothing about retention or value |
| Weekly active users / total users (engagement rate) | Actionable | Measures whether users find ongoing value |
| Total page views | Vanity | Inflated by crawlers, accidental clicks, one-time spikes |
| Signup-to-activation rate (cohort) | Actionable | Tied directly to onboarding changes |
| Cumulative revenue | Vanity | Hides churn; a declining business can have rising cumulative revenue |
| Revenue per cohort by acquisition month | Actionable | Shows whether the business is improving over time |
Why This Matters: Metric Corruption
Goodhart’s Law (economist Charles Goodhart, 1975): “When a measure becomes a target, it ceases to be a good measure.” Optimizing for a vanity metric incentivizes gaming it — buying users who never activate, inflating page views through pop-ups — while the underlying business deteriorates.
Campbell’s Law (sociologist Donald T. Campbell, 1979) extends this: the more any quantitative social indicator is used for social decision-making, the more it corrupts the processes it was intended to monitor. The same dynamic applies to startup KPIs.
Dave McClure’s AARRR framework (Pirate Metrics) — Acquisition, Activation, Retention, Referral, Revenue — offers a practical structure for choosing actionable metrics across the customer lifecycle. Each stage has metrics that respond to specific interventions. Croll and Yoskovitz’s One Metric That Matters concept from Lean Analytics operationalizes this further: at each stage of a startup, there should be one metric that matters most to optimize now.
This distinction underpins Validated-Learning: you can only validate a hypothesis if the metric you measure is genuinely actionable. Measuring vanity metrics produces the illusion of Build-Measure-Learn-Loop without the substance.
Future Connections
Will connect to Innovation-Accounting, Cohort-Analysis, Split-Testing when created.
Related Concepts
Sources
-
Ries, Eric (2011). The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Publishing. ISBN: 978-0-307-88791-7.
- Chapter 7 (Measure) provides the primary framework: the three tests for actionable metrics (actionable, accessible, auditable) and the critique of vanity metrics.
-
Goodhart, Charles A.E. (1975). “Problems of Monetary Management: The U.K. Experience.” In Papers in Monetary Economics, Vol. I. Reserve Bank of Australia.
- Origin of Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. Foundational to understanding why vanity metrics corrupt decision-making.
-
Campbell, Donald T. (1979). “Asshole Criteria and the Ethics of Social Indicators.” American Behavioral Scientist, Vol. 23, No. 6, pp. 811–819.
- Campbell’s Law: the more a quantitative indicator is used for social decision-making, the more it becomes subject to corruption pressures and distorts the social processes it was intended to monitor.
-
Croll, Alistair and Benjamin Yoskovitz (2013). Lean Analytics: Use Data to Build a Better Startup Faster. O’Reilly Media. ISBN: 978-1-449-33489-0.
- “One Metric That Matters” concept; practical framework for selecting stage-appropriate actionable metrics across the startup lifecycle. Available: https://www.oreilly.com/library/view/lean-analytics/9781449335687/
-
McClure, Dave (2007). “Startup Metrics for Pirates: AARRR!” 500 Startups Blog.
- AARRR (Acquisition, Activation, Retention, Referral, Revenue) framework: a structured approach to selecting actionable metrics at each stage of the customer lifecycle. Available: https://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
Note
This content was drafted with assistance from AI tools for research, organization, and initial content generation. All final content has been reviewed, fact-checked, and edited by the author to ensure accuracy and alignment with the author’s intentions and perspective.