Day 49 · Phase G

A number that can't change one action is decoration

Metrics that move an action · small samples are mostly noise · score your own forecasts · review the wins too·4 moves · 4 diagrams
"I've got a feeling this one closes" never sharpens with practice — not because you aren't smart, but because the feedback arrives too slowly and too noisily for your brain to extract an error signal from it. Numbers aren't for reporting upward. They're the replacement for an instinct this environment can't teach you.
The trouble with most sales dashboards isn't that the numbers are wrong. It's that even if every one of them were exactly right, you wouldn't change a single action because of them. Four moves turn numbers into decisions. One, ask what action would change if this number moved, then decide what to watch. Two, compute the noise before you draw the conclusion. Three, put a probability on every deal and score yourself at quarter end. Four, review the wins as well as the losses, and defuse the two biases that make reviews worthless.
MOVE 01

Ask what action would change if this number moved — if you can't answer, stop watching it A number that can't change one action is decoration.

CACLTVpayback
The principle in one line
Half the numbers on your dashboard have never once changed what you did — that isn't a metric, it's decoration. Only three earn their place, and all three answer the same question: if I spend another unit of money acquiring a customer, do I get it back, and how fast? CAC is every cost of acquisition, salaries included, divided by new customers in the same period. LTV is monthly gross profit times months retained — gross profit, never revenue. Payback is CAC divided by monthly gross profit. Watch payback hardest, because it's the only one of the three that requires no guess about the future: LTV asks you to predict how long someone stays; payback only uses money already collected.
When one customer turns cash-positive
Cumulative gross profit per customer: when acquisition pays itself back Illustrative: CAC 6,000 · gross profit 800 per month · average retention 24 months Payback = 6,000 ÷ 800 ≈ 7.5 months LTV (24 months) = 800 × 24 = 19,200 LTV / CAC = 3.2 — a convention to check against, not a pass mark 0 −6,000 payback point acquisition is paid in full in month 0 month 0 month 6 month 12
Checklist · four ways to stop computing these wrong

1. Put people into CAC. Sales and marketing salaries, contractors and tools, all divided by new customers in the same period. A CAC built from ad spend alone is routinely under half the real figure — and you'll conclude the channel is a bargain and keep pouring money in.

2. Compute it per channel, never blended. A blended CAC is an average held down by your best channel. Split it and there's usually one or two channels losing money outright, which is exactly where the budget has been growing.

3. Use gross profit and the median for LTV. Revenue treats delivery cost as profit. The mean lets a handful of enormous accounts drag the number to a place no real customer occupies. The median is your typical customer.

4. Don't estimate retention from cohorts that haven't aged. If nobody has been a customer longer than three months, you cannot produce "average retention: 24 months" — that's an assumption wearing a number's clothes. Write it honestly: "extrapolated from current monthly churn, about 24 months, not yet observed."

Why it works · mechanism and evidence
Why it works

Mechanism: these three numbers decide whether money can buy growth. Payback caps how fast you can spend — how long each unit of cash takes to come back. LTV over CAC decides whether the business is worth scaling at all. And the reason you need them rather than a feel for it is that sales happens to be exactly the kind of environment intuition cannot learn.

  • Hard evidence (theoretical synthesis, co-authored across opposing camps) · intuition is conditional: Kahneman & Klein (2009, American Psychologist) reduced a long dispute to two conditions — the environment must be regular, and the feedback must be fast and unambiguous. Chess and firefighting qualify. Sales fails both: a deal takes weeks to months, and the outcome is contaminated by forces you don't control. It isn't that you're slow — the environment itself can't teach you, so a record has to remember for you.
  • Hard evidence (meta-analysis of 136 studies) · simple rules usually beat expert feel: Grove et al. (2000, Psychological Assessment) compared expert clinical judgement with mechanical statistical rules. On average the mechanical rule was somewhat better, in roughly half the studies clearly better, and humans were clearly better in only a handful. In sales terms: three or four objective signals — has anyone met the actual decision-maker, is there a stated date, did the buyer raise budget unprompted — weighted into a score beats "I like the feel of this one" over time.
  • Industry convention (not research, don't treat it as law) · the thresholds: LTV/CAC ≥ 3 and payback under twelve months come from published operator and investor experience in software (David Skok, Bessemer and others). No control group anywhere. Capital costs, margins and retention vary enormously between businesses, so use them as a check-up reference, not a pass mark.

Honest boundary: with only a few dozen customers, LTV is largely fiction — the retention curve hasn't grown yet, so you're multiplying your own assumption by a big number. Watch payback and per-deal gross profit instead; they're enough. And all three answer whether the business scales, not whether this deal is worth chasing. That second question belongs to qualification, and no financial ratio will decide it for you.

MOVE 02

Twenty deals can't tell you anything — compute the noise, then conclude Twenty deals can't tell you anything. Compute the noise before you draw a conclusion.

small samplesregression to the meanprocess metrics
The principle in one line
"We closed 28% this month against 20% last month, so the new script works." At twenty-something deals, that sentence says nothing. The wobble in a close rate is computable by hand: standard error ≈ √(p × (1−p) / n). At p = 20% and n = 25 that's about eight percentage points, so a 95% interval spans roughly ±16 — the truth behind your observed 20% could sit anywhere between 4% and 36%. Worse is regression to the mean: an unusually bad month is usually followed by an ordinary one, so anything you did at the bottom will look effective — the new script, the morning stand-up, a different coffee mug.
The same 20%, at four sample sizes
All four are a 20% close rate. The smaller the sample, the wilder the truth behind it Bars = 95% interval, estimated from √(p(1−p)/n); the n=10 bar is clipped at 0% what you observed: 20% n = 10 deals 0 – 45% n = 25 deals 4 – 36% n = 100 deals 12 – 28% n = 400 deals 16 – 24% 0% 10% 20% 30% 40% 50% So "28% this month against 20% last month" can't even establish a direction at this volume. To detect a real 20% → 28% lift you'd need roughly four hundred deals per group. Most teams never get there in a year.
Checklist · three rules against noise

1. Ask how many deals the claim would need, before you make it. If you can't compute it, use the diagram's orders of magnitude: a few dozen shows direction and never a decimal point; hundreds are where comparisons begin. This one habit kills half your bad decisions.

2. When volume is short, move upstream to process metrics. Closing is the sparsest layer, so it's the noisiest. One layer up: meetings booked, second-meeting rate, share of deals where you reached the decision-maker, reply rate within seven days of a quote. Ten times the volume, far less noise, and they happen earlier, so a change shows up sooner.

3. Fix the definition and the window before you look. Whether you count by the month a lead entered or the month it closed has to be settled in advance. Choosing the window after seeing the result can make anything look effective — that isn't analysis, it's telling yourself a story.

What to say in the weekly meeting: "Is that lift real or is it noise? How many deals did we have this month — enough to see a difference that size?"

Why it works · mechanism and evidence
Why it works

Mechanism: a close rate is a proportion estimated from a string of won/lost draws, and the smaller the sample the more it jitters. The jitter carries no information — but people reliably supply a cause for it, preferably whatever they did most recently.

  • Hard evidence (classic, repeatedly replicated) · the law of small numbers: Tversky & Kahneman (1971, Psychological Bulletin) found that even statistically trained researchers treat small samples as though they were as reliable as large ones, and so overestimate how likely a result is to replicate. The sales version: two wins out of three attempts and the playbook is declared found.
  • Hard statistical fact (not a psychological hypothesis) · regression to the mean: Galton (1886) described it — an extreme observation tends to be followed by one closer to average. Kahneman's flight-instructor example shows how it deceives: shout at the cadet who flew badly and he does fly better next time; praise the one who flew well and he does worse, so instructors conclude criticism works and praise harms. It was regression to the mean all along. (A teaching case, not a controlled experiment — but the statistical phenomenon underneath is solid.)
  • Mechanical implication (arithmetic, not a finding) · why upstream helps: for the same proportion, a tenfold larger sample shrinks the standard error only about threefold — but that is often exactly the difference between "invisible" and "visible". That's the real advantage of process metrics: naturally larger samples, arriving earlier.

Boundary: a small sample isn't no information. A buyer's own words in a loss, or the same obstacle recurring, is qualitative signal worth having after a handful of cases — you can't build a percentage on it, but it tells you what to change next. The real error is turning five deals into a percentage and rebuilding the process around it.

MOVE 03

Put a probability on every deal and score yourself — no forecast, no error to learn from Put a probability on every deal, then score yourself. No forecast, no error signal, no learning.

calibrationprediction errorcommit vs forecast
The principle in one line
"I'd say this one's eighty percent there" is a feeling, not a forecast, because nothing about it can later be judged wrong. Make it a rule instead: every deal carries two written values — a close probability from a fixed ladder (10 / 30 / 50 / 70 / 90) and an expected close date — changed only on new information, and only with one line saying what that information was. At quarter end, pull every deal you marked 70% and count. If four in ten closed, your "70%" means 40% in the real world, and from now on you discount it. Your forecast becomes accurate immediately, and you learned no new skill — you simply stopped fooling yourself.
What you predicted vs what actually closed
Turn "I feel good about it" into a number and quarter end shows you the bias Horizontal: the probability you assigned Vertical: how many of that batch actually closed Dashed = perfect calibration: seven of every ten deals you called 70% Solid = a typical over-optimistic shape (illustrative, not real data) 0 50% 100% 10% 30% 50% 70% 90% Reading it: the whole line under the diagonal = systematic optimism. Treat your 70% as 45% and the forecast lands.
Template · a ten-minute calibration review

1. Count by rung: how many deals sat in each of the five, and how many closed. A dozen deals per rung is enough for the shape to show.

2. Look for two biases: probability — if every rung came in below its label, you're systematically optimistic, so discount as the diagram shows; dates — take the median number of days by which real close dates ran late, and add it to every future date from now on. Nearly everyone has both, and fixing them costs nothing.

3. Always report two numbers, never one blend: the commit is the 90% rung only, what you'll stand behind; the forecast is every rung weighted by your calibrated probabilities. Asked how the quarter looks: "Commit is 800k, forecast 1.3m, and historically my forecast runs about ten percent optimistic."

What changing a probability sounds like: "He brought finance into the call today — that's new information, so this moves from 50% to 70%." Without a sentence like that, a change is just a mood.

Why it works · mechanism and evidence
Why it works

Mechanism: learning runs on the gap between what you expected and what happened. If the expectation was never fixed as a number, your brain rewrites it after the fact to match the outcome, the gap collapses to zero, nothing looks like a mistake — and nothing is learned. Writing it down exists only to keep the error alive.

  • Hard evidence (neuroscience, specific mechanism) · brains learn from prediction error: Schultz, Dayan & Montague (1997, Science) recorded midbrain dopamine neurons (ventral tegmental area and substantia nigra) and found they encode not reward itself but reward prediction error: firing rises when an outcome beats expectation, stays near baseline when it matches, and drops below baseline when it disappoints. The signal projects to the striatum and nucleus accumbens, where it drives the adjustment of later behaviour. With no prior expectation, there is no error to encode. That's the hard reason to write it down first — but don't extrapolate it into "writing forecasts raises win rates by X%", which no study has shown.
  • Hard evidence (multi-year forecasting tournament) · calibration is trainable: Mellers et al. (2014, Psychological Science) found in the Good Judgment Project's geopolitical tournaments that roughly an hour of probabilistic-reasoning training, working in teams, and continuous scoring each significantly improved accuracy (measured by Brier score). Scoring is the active ingredient — a score is what makes feedback exist at all.
  • Hard evidence (theory plus experimental review) · three kinds of overconfidence: Moore & Healy (2008, Psychological Review) separate overestimating your own performance, believing you're better than others, and stating intervals that are too narrow. Sales forecasting mostly suffers the third, which is why you should give a range rather than one precise-looking figure.

Boundary: no more than five rungs — a 17% carries only the appearance of precision. More important: never tie forecasts to compensation. Once the number you report determines your reward, what gets reported stops being what you believe and becomes what is safe. It's the same logic as commission: whatever you measure, you get the performance of it. If you need both, keep the commit and the forecast separate and hold people to the commit only.

MOVE 04

Review the wins too — and defuse the two biases that turn a review into storytelling Review the wins too — and defuse the two biases that turn a review into storytelling.

hindsightoutcome biasloss interviews
The principle in one line
Review only your losses and you'll learn a great deal about how not to lose and nothing about why you win. Worse, the bad decisions buried inside a won deal get laundered by the good outcome and copied across the team as "what works". Two biases stand in the way. Hindsight: once you know the result, you sincerely feel the signs were there all along, and can no longer recall how genuinely unsure you were. Outcome bias: you grade the decision by how it turned out. There is exactly one defence — compare against what you wrote down at the time, never against memory. That's the second use of the two values from the previous move.
Review the decision, not the result
Four ways the same deal can turn out. Only one square is safe to copy Decision at the time: poor Decision at the time: good Result: won Result: lost The dangerous square The result laundered the mistake, so it became "experience" and got copied. The square worth copying Process right, result right. Still ask one question: how much of this was luck? The one people learn from Bad decision, bad result, so everyone accepts it — and the lesson is usually right. Bad luck. Change nothing You did the right things and still lost. Rewriting the process here mostly breaks what worked. One way only to tell which square you're in: read the probability and the next step you wrote at the time. Memory can't.
Template · three questions for a lost deal, two rules for inside

Ask the buyer (one email is enough — no defending, no re-pitching, you're here to learn):

1. "Which two options were you finally choosing between?" — this asks about the alternative, not for a reason.

2. "At the moment you decided, what was the one thing that settled it?" — asking about that moment is far more honest than asking why they didn't pick you.

3. "Which part of dealing with us took the most effort?" — this one most often surfaces something you can actually fix.

Internal rule one: read the next step and the probability recorded at the time before the discussion starts. Reverse that order and the meeting becomes an exercise in constructing a plausible account of a known result.

Internal rule two: each quarter, run a pre-mortem on two or three live deals — "assume this is dead in three months; everyone write the most likely cause." The same cause tends to keep appearing: nobody ever met the person who actually decides.

Why it works · mechanism and evidence
Why it works

Mechanism: once the outcome is known it reaches backwards and rewrites your memory of what you believed, so a review slides from examining a decision into assembling evidence for a conclusion you already have. Discipline doesn't fix this. A contemporaneous record does — it's the only thing the outcome can't edit.

  • Hard evidence (classic, extensively replicated) · hindsight bias: Fischhoff (1975, Journal of Experimental Psychology: Human Perception and Performance) asked people to judge how likely events were; those told the outcome substantially overestimated how much they had known in advance. Decades of replications and meta-analyses since support a stable effect. The sales version: after a deal dies, everybody remembers sensing it was wrong.
  • Hard evidence (clean experimental design) · outcome bias: Baron & Hershey (1988, Journal of Personality and Social Psychology) gave people identical decision information and varied only the outcome; their ratings of the decision's quality moved with the result. The sales version: the same pushy cadence is called "confident momentum" when it wins and "rushing the buyer" when it doesn't.
  • Moderate strength (single experiment, widely adopted) · the pre-mortem: Mitchell, Russo & Pennington (1989, Journal of Behavioral Decision Making) found that treating a future event as though it had already happened and explaining it (prospective hindsight) increased the number of causes people identified by roughly 30%. One study, so don't treat it as settled — but it costs nothing to try.

Boundary: discount what loss interviews tell you — buyers offer a presentable reason, and "budget" is the easiest shield to reach for. That's why you ask about alternatives and process rather than "why didn't you choose us". And a review stops working the moment it affects anyone's rating: people will only bring the version that survives scrutiny. If you want reviews to be useful, telling the truth has to be safe in that room.

Your Day 49 Action

Sixty minutes to replace a feeling with four checkable numbers.

One (15 min) · Compute a real CAC and payback: last quarter's full sales and marketing spend including salaries, divided by new customers in the same period; then divide by monthly gross profit per customer. The answer is usually uglier than you assumed, which is precisely what makes it useful.

Two (10 min) · Compute the noise behind your most trusted conclusion: write down how many deals it rests on. Under a few dozen, demote it today from "conclusion" to "hypothesis", and note how many deals would be needed to test it.

Three (20 min) · Add two values to every deal in the pipeline: a probability (10/30/50/70/90) and an expected close date. The deals where you can't produce a probability didn't belong in the forecast anyway. Put a calendar reminder at quarter end: pull the 70% rung and count what closed.

Four (15 min) · Review a deal you recently won: read the next step and probability you wrote at the time, then ask how much of it was luck. Then send those three questions to someone you recently lost — most people reply, precisely because you aren't selling.

A boundary worth keeping: data can tell you which layer is leaking and which conclusion won't hold. It can't tell you what to say next. Metrics choose the battlefield; the conversation is still practised in the room.
Think It Through
1. We only close a dozen deals a month. Is any of this statistics relevant to me?
The opposite: the lower your volume, the more you need to know your numbers can't be trusted, or you'll keep rebuilding the process around noise — four playbooks a year, none of them alive long enough to be tested.

At low volume, use it this way:
One, direction only, never decimals. "Second-meeting rate is clearly climbing" is usable. "It went from 21% to 24%" is not.
Two, swap statistics for counting. Don't compute a rate — categorise causes. Where did the last ten losses die? The same obstacle appearing four times is real even at ten deals. That isn't statistical inference, it's pattern-spotting, and a handful of cases is enough.
Three, find your sample upstream. A dozen closes a month may sit under fifty quotes and a hundred meetings. Watch the trend there.
Four, lengthen the window. Quarters or rolling twelve-week periods, not months.

And the highest-return habit of all: keep a decision log. For every non-obvious decision (switching channels, changing how you quote), write three lines: what I expect to happen, how long until I'd see it, and what result would make me admit I was wrong. A year later that stack is worth more than any dashboard — it's the only control group you'll ever have.
2. My boss wants accurate forecasts, but I get shouted at for reporting a low number.
That's a genuine structural conflict, so don't paper over it with "I'll try harder". The fix is to separate two numbers and say out loud that they are different things:

The commit: the 90% rung only. What you'll stand behind. Small and hard.
The forecast: the whole pipeline weighted by calibrated probabilities. It's an expected value, so it's supposed to move.

One sentence in the meeting: "Commit is 800k, forecast is 1.3m, and historically my forecast runs about ten percent optimistic." You aren't reporting a number, you're reporting a number plus its error. What actually draws the shouting is rarely a low figure — it's a figure that keeps changing while nobody knows how much to trust it.

Bring one more thing: your calibration record. "Last quarter I marked these 70% and 68% of them closed." Once you can show your numbers survive an audit, reporting a low one stops being an attitude problem and becomes a credibility asset.

If compensation is paid on the forecast itself, the forecast will be distorted, and that isn't a character flaw — it's a design flaw. What you can do is make the trade-off explicit: do you want an accurate forecast or a pleasant one? If the answer is both, use the two numbers above.
3. My LTV/CAC looks healthy. Why is cash still tight?
Because LTV is future money and CAC is money you paid today, and the payback period sits between them. Concretely: CAC 6,000, gross profit 800 a month, payback seven and a half months. Sign ten customers this month and you're out 60,000 today, whole again in month eight — so the faster you grow, the bigger the hole. That's the entire secret behind the counterintuitive experience of selling more and having less cash: you aren't unprofitable, you're financing your own future revenue.

Three things to do about it:
One, watch payback, not the LTV/CAC ratio. A business with 24-month payback can't be pushed hard however pretty the ratio looks, unless your money is very cheap.
Two, pull collection forward. Discount for annual prepayment, weight the first instalment. This is usually far easier than lowering CAC and it works immediately — the cash bought by a 10% annual-prepay discount is often worth more than the acquisition cost you could have saved.
Three, let cash set the pace of spending. How fast you invest should be governed by payback, not by market opportunity — opportunity will always exceed your cash.

One last caution: if retention has never been observed (nobody has been a customer more than a few months), the LTV you computed is your assumption multiplied by a large number. That's exactly when payback should be the number you steer by.