How scientific a theory is depends not on how much it can explain, but on how much it forbids. A theory that can absorb any outcome has, in fact, said nothing. This inverts the usual test: stop asking "how much evidence supports me" and start asking "what result would make me admit I was wrong." Supporting examples are cheap and unlimited — anyone can collect a basketful. What actually prices a claim is the counterexample it dares to bet against.
Information content is proportional to the possibilities excluded: "it may or may not rain tomorrow" rules out zero worlds, and therefore carries no content. The weightiest predictions are the suicidal ones — the number they commit to would be a different number under the rival theory. But note a wrinkle: when observation contradicts prediction, the fault may lie with the instrument or an auxiliary assumption rather than the core theory. So real falsification is never a single fatal blow; it is a readjustment of the whole theoretical web. Falsifiability is a discipline, not a verdict you can execute mechanically.
The 1919 eclipse observation became a landmark not because it "supported" general relativity, but because relativity had already handed over a number that could kill it: starlight grazing the sun's edge should bend by about 1.75 arcseconds, where Newtonian calculation gave only half that. The measurement could land on one or the other — no escape route. Conversely, those systems that look stunningly explanatory, able to rationalize events whichever way they turn, have precisely because they never risk anything never once been genuinely tested. Explanatory reach and credibility are routinely mistaken for each other.
Machine learning has an isomorphic phenomenon: a model that can fit anything on the training set is showing you it is under no constraint; the held-out test set is manufactured falsifiability — freeze the model first, then test, and never in the other order. In software engineering, a test that passes on any behavior is not a test; an assertion is worth exactly what it excludes. Investing is the same: a thesis that explains both rallies and crashes can never trigger a stop-loss, which is why serious practice is to write down, before entering, "what data would make me admit I read this wrong."
The most common unfalsifiable phrasings in a design review are "this option scales better" and "this architecture is cleaner" — no observation can refute them, so they carry no information. Rewrite them into forms that can slap you in the face: "P99 under 80 ms at a 50k-QPS peak, verified by quarter's end, roll back to option B if we miss." Once a judgment carries a number, a deadline, and an exit condition, the quality of the discussion changes immediately.
Pick the technical judgment you currently hold most firmly (say, "building it ourselves beats buying"). Can you write down, right now, one concrete observable result that would make you concede you were wrong? If you can't, is what you're defending a judgment — or a position?
A p-value below 0.05 is habitually read as "there's a 95% chance the conclusion is true." What it actually says is: if the effect did not exist at all, data this extreme would show up less than 5% of the time. More to the point, that 5% only holds when the analysis plan was fixed in advance. Once the researcher can decide how to analyze while watching the data, the threshold is toothless. The main driver of the replication crisis was never fraud — it was a chain of choices that each looked perfectly reasonable.
One dataset can legitimately grow hundreds of analyses: trim the outliers or not, which covariates to control, when to stop collecting, whether to look at subgroups. No single fork is cheating, but combined they amount to searching a forest for significance — and a search always finds something. A classic simulation showed that granting a researcher four common analytic flexibilities is enough to push the false-positive rate from 5% to over sixty percent. Layer publication bias on top: null results stay in the drawer, and journals pick only the pretty batch.
One collaborative study handed the same dataset — whether player skin tone correlates with receiving red cards — to 29 independent teams to analyze as each saw fit. The conclusions spanned the entire range: twenty teams judged the effect significant, nine judged it not. Identical data; the disagreement came entirely from analytic choices. Separately, a project that systematically replicated 100 psychology studies found only about a third produced a significant result on re-run, with effect sizes shrinking roughly by half. The most counterintuitive part: even when every single step is clean, a literature can still drift systematically away from the truth.
This is machine learning's "tuning on the test set": every time you use test-set numbers to revise the model, you quietly convert it into training data, and a meaningful share of leaderboard progress is overfitting to the leaderboard. Finance calls it backtest overfitting — try ten thousand strategies on the same stretch of history and a few beautiful equity curves are guaranteed. A/B testing is no different: peeking at interim curves, stopping when it looks good, slicing users after the fact — all the same false-positive machine. Wherever you can retry freely and report only the win, this mechanism switches itself on.
Your experimentation platform almost certainly lets anyone watch live curves, and that is the most dangerous degree of freedom of all: the moment "stop when it goes significant" is allowed, the advertised 5% stopped being 5%. The workable discipline is to force the analysis plan into the record first — lock the primary metric, sample size, stopping rule, and segmentation before launch, and mark every other finding as "exploratory, pending independent replication."
Think back to the last time you said "the data proves this change worked." Were the primary metric, the stopping time, and the user segmentation decided before launch, or after you'd seen the data? If it was after, what did you validate — the change, or your own expectation?
Significance only answers "is there an effect"; effect size answers "how big," and only the latter can be acted on. Worse, the two deceive each other: with a large enough sample, a difference of no practical meaning can be highly significant; with a small sample, the only results that squeeze past the threshold are the ones noise has inflated. So the problem with underpowered research isn't that it finds too little — it's that its findings are systematically too big. Which is why the literature is packed with small studies reporting spectacular effects that won't replicate.
Effect size (say, the difference in group means divided by the standard deviation) doesn't move with sample size, so it can be compared across studies and pooled into meta-analysis; a p-value can do neither. Statistical power is the probability of detecting a real effect when one exists: a study with only 40–50% power is essentially flipping a coin, and the curse runs both ways — it will likely miss real effects, and it inflates the estimates that do slip through, the winner's curse. The fixes are institutional: preregistration (filing the hypothesis, primary metric, sample size, and analysis plan publicly before seeing data) and registered reports (journals accepting or rejecting before results exist). Neither forbids exploration; they simply label conclusions honestly — this one was a bet placed in advance, that one was spotted afterward.
The power of preregistration can be quantified. An analysis of large drug trials funded by one national agency found that before clinical trials were required to register publicly, about 57% reported the drug as effective; after mandatory registration, that fell to 8%. The drugs didn't change and the researchers didn't collectively turn crooked — they simply could no longer adjust the primary endpoint after seeing the data. Likewise, studies published as registered reports support their original hypothesis roughly forty-odd percent of the time, against well over ninety percent in the conventional literature.
Engineering and business suffer a familiar version of this: ship anything that comes back "statistically significant," accumulate dozens of significant changes over a year, and watch the top-line metric stay flat — because every gain was small enough to be swallowed by noise and mutual cancellation. Medicine invented the "minimal clinically important difference" for exactly this: an improvement below that magnitude isn't worth changing treatment for, however significant. Any system that makes decisions from metrics has to agree first on how big counts as big.
Add a column to your experiment dashboard: "minimum effect size worth shipping." Then, before starting, run the sample-size math once — at current traffic, how long until you could detect that magnitude with 80% power? Many experiments will reveal on the spot that they have no capacity to answer the question they pose, and knowing that upfront is far cheaper than running two weeks and then interpreting a noise curve.
The last feature you shipped because "the data was significant" — what was the effect size? If you had agreed beforehand that anything under a 1% lift wouldn't ship, would it still have shipped? Do you care about the real gain, or about the reassurance of that significance marker?
Science is reliable not because scientists are more honest or smarter than everyone else. Individual researchers are just as biased, just as self-interested, and just as prone to falling in love with their own hypotheses. Reliability comes from a social error-correcting apparatus: openness, replication, competitive verification, retractability. Once you see this, how you evaluate any source of knowledge shifts — stop asking "is this person trustworthy" and ask "what error-correcting machinery is this claim exposed to."
Correction operates in deliberately redundant layers. Publication takes a claim out of its author's control; replication demands it hold up in someone else's hands; competing teams have strong incentives to find the flaw, which is the main engine here, far stronger than peer review — review is only sparse sampling, and experiments show reviewers routinely miss errors deliberately planted in a manuscript. Meta-analysis pools scattered effect sizes and can even infer publication bias from the shape of their distribution; retractions and corrections supply the rollback path. The point is that no single layer is reliable; only stacked do they shorten the lifespan of an error. That is a fundamentally different strategy from "find a batch of reliable people."
In 2011 a large physics experiment announced it had measured neutrinos traveling faster than light. Had it held, the foundations of modern physics would have needed rebuilding. The team didn't sit on it — they released the data and invited the world to check; months later the problem was traced to a loose fiber-optic connector, and they retracted the result themselves. It looks like a scandal; it was actually the machinery running at its most beautiful. Equally counterintuitive: retraction counts have risen noticeably in recent years. It feels like science getting worse, but it more likely reflects detection getting better — the way wider screening necessarily raises the number of "cases found."
The reliability philosophy of distributed systems is exactly isomorphic: you assume no single node is honest and crash-free, and instead buy overall reliability with redundancy, cross-replica reconciliation, failure detection, and rollback — Byzantine fault tolerance goes further and assumes outright that some nodes lie. Open-source code earns trust not from the author's skill but from "anyone can read it, file an issue, revert it." Inversely, a closed knowledge-production system that nobody can re-check and that never has to retract — an internal report, or a confidently worded model output — does not deserve science-grade trust.
Port the apparatus into your engineering organization: require independent replication for key conclusions, meaning a different person recomputing through a separate data pipeline before it counts; keep decision documents versioned and retractable; and in retrospectives, reward the person who volunteers that their own earlier conclusion no longer holds. This layer is scarcer still when working with AI — model output natively lacks error correction, and your core value is supplying the replication and refutation it can't.
The single most important number on your team (cost, latency, retention) — has a second person ever computed it through a fully independent path? If it were in fact wrong, how long until anyone noticed: a week, a year, or never?