lean-analytics

# Lean Analytics

A data discipline for startups distilled from Alistair Croll and Benjamin Yoskovitz's *Lean Analytics*: separate metrics that change decisions from numbers that merely flatter, then point the whole company at the One Metric That Matters for your business model and stage. Use it to choose metrics, audit dashboards, set targets, and plan instrumentation.

## Core Principle

**Focus on the one metric that matters right now — everything else is noise that feels like progress.** Startups die from lack of focus more often than lack of data. The discipline is knowing your business model, knowing your stage, and tracking the single number that tells you whether the riskiest part of the business is working. A metric earns attention only if it changes what you do next.

## Scoring

**Goal: 10/10.** Rate metric choices, dashboards, and instrumentation plans 0-10 against these principles. Report the current score and the specific changes needed to reach 10/10.

- **9-10:** One OMTM matched to model and stage, paired counter-metric, a line in the sand with a pre-committed miss response, cohorted and segmented data
- **7-8:** Mostly actionable ratios and a plausible OMTM, but no explicit target, weak cohorting, or too many "key" metrics
- **5-6:** Actionable and vanity metrics mixed; dashboard exists but rarely changes a decision; model and stage never named
- **3-4:** Vanity metrics dominate — totals, cumulative charts, blended averages; metrics copied from other companies
- **0-2:** No instrumentation, or numbers chosen to impress investors rather than drive decisions

## Framework

### 1. Good Metrics vs Vanity Metrics

**Core concept:** A good metric is comparative (versus last week, versus another cohort), understandable (the team can recall and debate it), a ratio or rate (not an ever-growing total), and behavior-changing — if a number won't change what you do, stop measuring it. Vanity metrics — total signups, page views, cumulative anything — only go up and only make you feel good.

**Why it works:** The output of analytics is decisions, not data. Ratios are inherently comparative and operable, while totals hide decay: total registered users rises even while the product bleeds actives. Forcing every metric through the "what will we do differently?" test converts reporting into learning.

**Key insights:**
- Work the lens pairs: qualitative vs quantitative (interviews reveal *why*, numbers reveal *how much*), exploratory vs reporting (exploration finds your unfair advantage; reporting keeps the lights on), leading vs lagging (complaints predict churn before churn happens), correlated vs causal
- Correlation finds the lever; only an experiment proves it — find metrics that move together, then change one for a randomized group to test causality
- Cohorts make time honest: compare users by signup month, or real improvement vanishes inside blended averages
- Segments make comparisons honest: split by channel, plan, and geography — a flat aggregate often hides one segment soaring and another collapsing
- Averages lie under skew: whales and lurkers are different businesses, so read medians and percentiles
- A cumulative up-and-to-the-right chart is the single most reliable vanity tell

**Applications:**

| Context | Application | Example |
|---------|-------------|---------|
| Dashboard audit | Rewrite each total as a ratio | Total signups → % of visitors activating within 7 days |
| Board reporting | Show cohorts, not cumulative curves | Retention by signup month replaces "users over time" |
| Feature decision | Demand a behavior-changing metric | "If D7 retention doesn't rise 10%, the feature comes out" |

**Ethical boundary:** Metrics exist to describe and serve users, not manipulate them — instrument only what you need and respect privacy in what you collect.

See: [references/good-metrics.md](references/good-metrics.md)

### 2. The One Metric That Matters (OMTM)

**Core concept:** At any moment there is one number that matters above all others — the one that tells you whether the current riskiest assumption is working. Pick it, display it everywhere, and let it drive every experiment until you graduate to the next stage.

**Why it works:** The OMTM answers the most important question you have right now, forces you to draw a line in the sand so "good" is defined before results arrive, and focuses the entire company. A dashboard of forty numbers diffuses accountability; one number creates a shared scoreboard and a culture of experimentation.

**Key insights:**
- The OMTM rotates — it is the metric that matters *now*, not forever; passing a stage gate or pivoting changes it
- Pair it with a counter-metric so it can't be gamed: activation speed paired with 30-day retention, sales velocity paired with refund rate
- A line in the sand has three parts: a target number, a date, and a pre-committed answer to "what do we do if we miss?"
- "Good enough" is a decision made in advance, not a discovery made after — otherwise the goalposts move
- If the team can't agree on the OMTM, you haven't agreed what the riskiest part of the business is — that argument is the valuable part
- Collect many metrics, but *watch* one — the rest live in drill-down reports, not on the wall

**Applications:**

| Context | Application | Example |
|---------|-------------|---------|
| Quarterly planning | One OMTM per stage; experiments ladder up to it | Stickiness stage → all bets target week-4 retention |
| Dashboard design | OMTM big, 4-6 supporting metrics small | Wall display: paid conversion 3.2% huge; CAC, churn, NPS below |
| Team alignment | Pre-commit the miss response | "Under 10% by March 1 → we pivot to the agency segment" |

**Ethical boundary:** The line in the sand disciplines the company's bets, not individuals — turning the OMTM into personal quotas invites gaming and hides truth.

See: [references/omtm.md](references/omtm.md)

### 3. Metrics by Business Model

**Core concept:** Your business m