We assume the abstract is a faithful compression of the body. But the abstract is the one part of a paper written to be read. In a study of index-membership effects, the abstract reports roughly +0.1% for deletions while the paper's own first table reports −0.62% — not a difference in magnitude, a difference in sign. In a study of exchange-traded funds, the abstract offers a "16%" increase in volatility; in the body that means 16% of a standard deviation, which works out to about twenty basis points — and the published version quietly dropped the sentence that explained this. Almost every secondhand citation stops at the abstract. So the distortion does not wait for the retelling; it is already complete inside the paper.
Three forces stack. First, division of labor: the abstract is usually written last and finalized by the corresponding author, while the tables come from whoever ran the analysis, and nobody owns the line-by-line reconciliation between them — peer review rarely does it either. Second, incentives: the abstract determines whether the paper gets opened and whether it gets a press release, so any phrasing that makes the conclusion cleaner is rewarded, and rewarded legitimately. Third, compression is lossy: squeeze "in this subsample, after controlling for these covariates, measured by this proxy" into a single declarative sentence and the conditions go first, the qualifiers second, leaving a claim that is both stronger and easier to misuse. What makes this hard to catch is that it leaves no trace — the abstract reads as entirely self-consistent, and only the table reveals the crack.
Repeated systematic audits of medical journals keep finding the same thing: in a substantial share of papers, the abstract does not match the results in the body — a prettier secondary outcome swapped in for the primary one, a marginal result written up as a firm conclusion, a number reported that appears nowhere in the full text. The finding is stable enough that journals wrote a dedicated reporting standard for abstracts. Note what that really means: this is not a signature of fraudulent papers. It is the normal condition of papers that passed peer review.
The software analogue is the README and the code: the README states intent, the code executes reality, and the two drift at different rates. In financial reporting, every adjustment behind an "adjusted" figure sits in the footnotes while the market reads only the headline line. Model cards work the same way: the benchmark score is printed at the top, the evaluation protocol is buried below.
When your team cites papers in technical decisions, adopt one nearly free rule: any number entering a decision must be located in the body — which table, which row, which specification. If it can't be located, it can't go in the document. What this filters out is abstract-grade evidence, and abstract-grade evidence is exactly what gets overturned hardest in architecture choices.
The last time a paper changed your technical judgment, did you open its results table? If not, what convinced you — the study, or its abstract?
Between the working paper and the published version sits a round of revision, and numbers move in that round: 15% becomes 11%, 37.8% becomes 33.5%. Authors treat this as ordinary manuscript improvement, not worth announcing. The trouble is that during the intervening year or two the working paper has already been covered by the press, pasted into slide decks, and written into other people's introductions. So one finding exists in two circulating versions — and the older one usually travels further, because it came first, landed louder, and carried the bigger number.
The lag is structural. Preprints and working papers go public one to two years before formal publication, precisely during the window when the topic is hottest and most citable; by the time the published version corrects the number, attention has moved on. Citation chains have inertia once formed: A cites the working paper, B cites A, C cites B, and by C nobody returns to the source — yet every layer appears to have one. Subtler still is the silent edit: the published version deletes a qualifier or switches a definition with no change log at all. A formal retraction at least carries a conspicuous mark. Version drift carries none.
Both versions of one study can coexist for years and both can be cited legitimately: the working paper has a stable number, the published article has another, and search tends to surface the more-cited older one first. Which means "this number has a source" and "this number is the current version" are two different claims — and the harder and more authoritative the source looks, the more readily people skip the check. Verifiability becomes the excuse for not verifying.
In technical documentation, old version pages outrank current ones in search for years, and teams break things by following a two-year-old config example. In data warehouses it appears as definition changes: the metric was redefined, the old dashboard still runs the old logic, and both numbers show up in the same meeting. Model evaluation is the same — which weights, which harness version produced a given score is usually recorded nowhere.
When an external number enters a decision document, write down which version it came from and when it was pulled. This is not academic fussiness; it leaves your future self an audit entry. Six months later someone asks "does this 30% still hold?" — a version-stamped number takes ten minutes to verify, an unstamped one has to be redone from scratch. The cost is three seconds of typing.
The external number cited most often in your team's documents — preprint or published version? When did you last check?
A concept usually has a family of proxy measures, and authors run all of them and list them in one table — a transparent practice. But an abstract holds only one number. When five proxies in that same table run from 13.31% all the way to 33.5%, and the one that reaches the abstract and the press release is the largest while the one the authors call their preferred specification in the body is somewhere in the middle, transparency is cancelled by the headline. Further: some "numbers" are not estimates at all but constructions. A percentage obtained by equal-weighting several measures looks like a measurement result; it is the product of a weighting choice. Normalize by standard deviation instead and the same underlying figures move from one quarter to one sixth.
The spread across a proxy family carries information in itself: a tight spread means the conclusion does not depend on the choice of measure, a wide spread means the conclusion is mostly driven by that choice. Reporting a single point erases exactly this — information often more important than the point estimate. Constructions go a layer deeper: every step of a composite index (which components, how to standardize, how to weight) is a researcher degree of freedom, and those steps are typically documented only in the methods or the appendix. Equal weighting embeds the strong assumption that every component matters equally and is commensurable, an assumption almost never tested. So the reader sees a percentage with two decimal places, instinctively treats it as a measurement, and it is in fact a model.
Notice that the authors hid nothing: all five proxies are in the table, the preferred specification is stated in the body, the normalization is spelled out in the methods. Everything is disclosed and the distortion happens anyway — because the transmission channel carries only the one number in the abstract. Which shows that "disclose the full analysis" is not sufficient against misreading; you also need "the headline conclusion may use only the pre-declared specification." That is exactly why clinical research requires preregistration to lock the primary outcome: what it constrains is not the freedom to analyze, but the freedom to choose after the fact.
Machine learning benchmarks are the nearest analogue: report the best of ten tasks, or report the mean of ten — the latter is equally a construction, and a different normalization reshuffles the ranking. University rankings flip when weights change, and the weight table never makes the headline. Composite performance scores work the same way: components and weights decide who comes out on top, while the discussion circles the final score.
Evaluating an AI coding tool will hand you a family of measures: time to completion, review rounds, defect rate, subjective satisfaction. Set one rule up front: before the data arrives, write down which one is primary and treat the rest as supporting. Choosing after the data lands does not produce a conclusion, it produces an act of self-persuasion. If you must report a composite, report the spread of its components alongside — the spread is the evidence for whether the composite deserves trust.
Take a composite score from your last review. Under a different but equally defensible weighting, does the ranking change? If it does, did you explain how the weights were set?
Some working papers contain an entire section on mechanism: through which channels the effect operates, which alternative explanations were ruled out. In the published version that section is deleted whole — usually for length, or because a referee judged the evidence there too weak. What remains is a clean reduced-form association: X correlates with Y by so much. And readers, especially secondhand readers, automatically read it as causal. What was cut is precisely the only part that could support a causal reading, and after the cut the causal reading flows more freely than before — nothing on the page reminds you the mechanism is still unestablished.
A reduced-form estimate answers "how large is the total effect," not "through what." The mechanism section decomposes the total, tests mediating paths, and rules out confounding channels; once it is gone, the causal claim rests entirely on the research design — natural experiment, instrumental variable, discontinuity — and the credibility of a design lives in the methods, which fewer people read. There is a selection effect on top: mechanism analyses often show a smaller effect than the reduced form, or an effect operating through only some channels, and precisely those unfavorable results are the easiest for referee comments and length limits to squeeze out. The publication process therefore leans systematically toward keeping the clean association and cutting the messy mechanism.
The result is a strange inversion: the more thoroughly a paper explains its mechanism, the more uncertainty it exposes, and the harder it becomes to publish and to circulate; the more it reports a single clean association, the more readily it is cited as causal evidence. That explains why so many widely cited "X causes Y" claims have no corresponding mechanism evidence — not because nobody did the work, but because the part that was done got trimmed off in the publication pipeline. The deletion leaves no trace: what the reader sees is a complete, crisply written paper.
In accident investigation, delete the root-cause analysis and leave a list of contributing factors, and readers supply the causation themselves — then change process on that basis. When an A/B test writeup reports the headline lift without the funnel decomposition, the team will invent a mechanism story and believe it. Surrogate endpoints in medicine are the same structure: the marker improved, and nobody validated the chain from marker to hard outcome. Observability too — metric curves without distributed traces, and people read temporal correlation as cause.
When you write up an experiment, keep the section titled "alternative explanations we checked," even when the answer is "could not rule out." It makes the document longer and the conclusion look weaker, and it is the only reason that document will still stand six months later. Reading someone else's report, look for that section first; if it isn't there, downgrade the conclusion to correlation. The downgrade is free. Mistaking correlation for causation usually takes a quarter to surface.
The last time your team said a change "drove" a metric improvement, was the mechanism articulated? If it wasn't, are you willing to rewrite "drove" as "correlated with"?
Before a figure counts as evidence, ask which part of the paper it is speaking from:
Day 70 asked how strict the ruler is, Day 72 asked which coordinates you are standing on, and this one asks which part of the paper is speaking. Together the three let you judge whether a number deserves a place in your decision — without redoing the research.