Where was the last place you saw the words "studies show"?
You probably did not go and look up the original. That is not only laziness — looking it up would not help much: you cannot read the methods section, and if you could, you still could not redo the work. What you are actually doing is something else: estimating how true-looking the conclusion is, on the basis of who said it, where it appeared and how many people repeat it.
This is how science actually runs inside a society. Which makes the question a hard one: a body of knowledge that almost nobody can verify for themselves — on what grounds is it, on the whole, reliable? The usual answers are "scientists are honest" and "there is a scientific method". The research in this issue puts the answer somewhere else: at the level of institutions. And once you see that it sits there, you also see where it will break.
In 1942 the American sociologist Robert K. Merton wrote a short essay with a very concrete background: Germany at that time was clearing relativity out of its classrooms as "Jewish physics" and promoting something called Deutsche Physik, under which whether a claim holds depends first on who made it. His question was what conditions this line of work needs institutionally in order to function at all.
He counted four. Universalism: whether a claim holds has nothing to do with the nationality, race or standing of whoever made it, only with whether it survives testing. Communism: findings must be handed over, not kept — you get your name on it, but the knowledge belongs to everyone. Disinterestedness: not a claim that scientists have no self-interest, but that the trade gives no prize for "this is the result because I need it to be". Organized skepticism: any conclusion may be challenged in public, and nothing is exempt.
What matters is not how upstanding the four sound but the way they lock together. Merton has another essay on priority disputes: those rather petty quarrels through the history of science about who found it first are not evidence that scientists are unusually vain, but of the fact that the only hard currency in the trade is having your name recognized. You have to hand the finding over completely before you can collect that payment — and the act of handing it over hands it, at the same time, to everybody who would like to find a hole in it. Personal ambition and public testability are tied to the same rope.
Norms, though, are norms. In 1974 Ian Mitroff published a study running three and a half years, based on interviews with forty-two of the scientists working on the Apollo moon rocks, and found a second set of four running alongside: strong emotional commitment to one's own hypothesis, secrecy before publication, and a real readiness to weigh who was speaking. His point was not that Merton was wrong but that scientists are pulled by both sets at once, and that the pull has a function — only someone partial to their own hypothesis will push a line all the way while the evidence is still thin. This is a small-sample interview study, to be generalized with care.
From 1975 to 1977 the Frenchman Bruno Latour spent close to two years in a neuroendocrinology laboratory at the Salk Institute in California. He did not understand the science, and that was deliberate: he took notes the way an anthropologist entering an unfamiliar village would, on what these people actually did all day.
The first thing he saw was that most of the lab's space and budget is instruments, and that what the instruments do takes one line to say: they turn matter into lines on paper. He called them inscription devices. A piece of brain tissue goes in; what comes out is a peak on a chart, and every argument that follows is about the peak.
The second thing matters more, and it is about sentences. A statement is born covered in qualifiers: "so-and-so's data suggest that, under these conditions, X may be related to Y." Its route onward is that the qualifiers come off one layer at a time, until it reads "X is Y" and nobody asks where it came from any more. That is exactly where Latour puts his definition of a fact: a fact is a statement stripped bare, whose origins nobody traces any longer. The reverse holds too — as soon as someone challenges it seriously, the qualifiers get put back on one by one.
So can you reproduce a result by following the paper? In the 1970s the British sociologist Harry Collins followed a set of labs trying to build a new kind of laser: the ones that succeeded had all had face-to-face contact with a lab that already had a working one, and not a single lab managed it from the published papers alone. What the papers cannot hold has a name — tacit knowledge: the feel in the hands, the temperament of the machine, the step where you wait ten seconds longer.
Out of that comes an uncomfortable loop that Collins calls the experimenter's regress. How do you know your apparatus is working properly? Because it gives the right result. What counts as the right result? The one a properly working apparatus gives. In a field still in dispute — his case was the early controversy over detecting gravitational waves — the loop cannot be broken by logic, only by the community deciding whose equipment counts, and that is a social process.
One misreading to guard against: saying a fact was made is not saying it is false — bridges still stand. "Made" refers to the people, money and instruments the process costs, not to any suggestion that the conclusion could just as well have been another one.
In 2011 three psychologists ran a real experiment: twenty undergraduates listened either to the Beatles' "When I'm Sixty-Four" or to another track, then reported their date of birth and their father's age. After adjusting for the father's age, the Beatles group came out nearly a year and a half younger, at the significance level people normally accept. Real participants, real data, every step above board — and they did it on purpose, as a demonstration.
What they were demonstrating is called researcher degrees of freedom: when to stop collecting data, which variables to measure, what to control for, whether to drop outliers. As long as those choices are made after seeing the data, a false-positive rate that should be five percent can be pushed very high. The crucial part is that no single step is fraud: each one, taken alone, could be written into a methods section.
Four years later there were systematic numbers. In 2015 a collaboration of more than two hundred researchers repeated a hundred published psychology studies: 97% of the originals had reported significant results; 36% of the repeats were still significant, and effect sizes were on average halved. That 36% depends on what you count as a successful replication, and other criteria give anywhere from the mid-thirties to the high sixties — but on every criterion the shrinkage is plain.
For the classics people keep retelling, the current state should be stated. Ego depletion (doing something that costs self-control makes you give up sooner on the next task): a 2016 preregistered multi-lab replication across twenty-three labs and more than two thousand one hundred participants measured an effect of almost exactly zero. The hormonal part of "power posing" likewise did not come out of a large replication in 2015, and one of the original authors said publicly in 2016 that she no longer believed in the effect.
The sociological reading takes us back to the rope in "nobody ever read out these rules". These are not crooks — it is the reward structure running: recognition pays only for the new and the positive, journals tend to filter out the studies that found nothing, nobody's career has ever been built on repeating someone else's experiment. Merton's organized skepticism has never had posts and budgets attached to it. A norm nobody is rewarded for is just a nice sentiment.
The shape of the repairs is worth noticing, because it is procedural: preregistration (hypotheses and analyses written down and filed before the data come in) and registered reports (the journal reviews the design first and commits to publishing whatever the result turns out to be). In 2021 somebody compared more than a hundred and fifty papers from the standard psychology literature with all seventy-one registered reports that existed at the time: 96% of the standard literature reported results supporting the hypothesis, against 44% of the registered reports. That does not mean half the registered reports "failed"; it means that under a different arrangement, negative results can finally get out. The crisis was found by science using its own tools, and what is being repaired is a process, not people's honesty.