How to tell whether data really follow a power law — and why most of the time they do not
Over the past thirty years, "this quantity follows a power law" has become an easy thing to say. City sizes, citation counts, web links, surname frequencies, war casualties, protein interactions — all have been declared so. The usual procedure is: plot the data on log-log axes (each step on each axis multiplies by ten), see something straight, fit a line, report the slope, done.
The trouble is that every step of that procedure is flawed, and all the flaws lean the same way: it tends to classify non-power-laws as power laws. This page answers what an honest procedure looks like, and why "it looks straight" carries almost no evidential weight on its own.
The method given by Clauset, Shalizi and Newman in 2009 is the current standard. It has four steps, and each one plugs a specific hole:
Set this beside "plot it and draw a line" and the difference is clear: the latter only establishes that a fit is possible, while the former asks whether the fit is good enough and whether something else explains it better.
Start with a purely graphical trap, which arrives earlier than the statistical problems and is easier to fall into. The two panels below use the same data — a thousand draws from a genuine power law — and differ only in how they are drawn.
The left panel uses the most common method: slice the horizontal axis into equal-width bins and count. Out in the tail each bin holds zero or one sample, so every non-empty bin lands on the "one sample" floor and the plot becomes a meaningless horizontal row of dots. Those dots dominate both the visual impression and the fit, and they carry nearly no information.
The right panel plots the same numbers differently: size x still runs along the horizontal axis, but the vertical axis now shows what fraction of samples exceed x. That quantity is the complementary cumulative distribution function, abbreviated CCDF hereafter. It needs no bins, uses every data point, and so keeps the tail clean. If you are going to plot, plot the CCDF; equal-width histograms are essentially unusable on heavy-tailed data.
With the plotting fixed, the statistics arrive, and they are harder. Running the Clauset procedure on real data usually ends one of two ways: either the goodness-of-fit test rejects the power law outright, or the power law survives but the log-normal survives just as well and the likelihood ratio test returns "indistinguishable".
This is not a failure of effort. Power laws and log-normals are intrinsically very similar over a limited range of magnitudes — one says "each doubling multiplies the probability by a fixed factor", the other says "the logarithm is bell-shaped", and across two or three decades the gap between them is smaller than typical sampling noise. Separating them usually takes another one or two decades of data, and tail data are inherently scarce.
What this method explains is not a natural phenomenon but a systematic bias in a stretch of academic history: why, around the year 2000, power laws appeared to be everywhere.
Three causes compounded. Graphically, equal-width histograms and least-squares fits both tend to produce power-law-looking results. Statistically, almost nobody ran goodness-of-fit tests, so "can be fitted" was taken as "holds". And in publishing, "we found a power law" travels further than "we found it was actually log-normal".
After 2009 the wind changed. As the procedure spread, many classic power-law claims were re-examined and a substantial share failed; network science's signature claim that real networks are scale-free later met a systematic rebuttal of its own. The episode is a useful specimen in itself: a sufficiently convenient shape can stay popular for twenty years while the methodology catches up.