中文 →
Blog Deep Read · 17 · muratbuffalo.blogspot.com

Murat Demirbas
Brains Are Ordinary, So Build Instruments

2007 to present · 839 posts · 16-year span (steady weekly cadence since September 2010) · One clear pivot: left tenured academia around 2020 for industry (three years at AWS, now MongoDB Research) · Still active, most recent post 12 September 2026 · Entirely free, no paywall · This page draws on 19 posts read in full, spanning January 2011 to September 2026, four of them from the earliest period, with samples on both sides of the pivot

To write this page I crawled the body text of all 839 posts on Murat Demirbas's blog and counted the internal links, wanting to know which posts are the load-bearing walls of a blog that has run for sixteen years.

The top result is not Paxos. It is not any paper review. It is a short piece published on New Year's Day 2018, half of which is about where the question mark came from and whether apes can ask questions. Forty-six inbound links.

Second place goes to his own invention, hybrid logical clocks (41). Third is the follow-up to that same New Year's piece (33). Six of the top ten are about how to ask questions, how to read a paper, how to write, how to show your thinking. Only four are technical.

A leading expert in distributed systems, and the foundation of his blog is a set of posts about how to think.

This is not an accident. At the end of 2017, after a decade as a professor and seven years of blogging, he sat down and gave himself a fairly unsparing diagnosis: his questions had gone blunt, they "often come out as shallow/predictable." Then came the line that really stings:

But yeah, I cannot be sure whether I have gone compliant/docile recently, can I?Mad questions, 2 January 2018

His New Year's resolution was not "ask more questions." He knew resolutions do not work. His move was to install a system: from then on, every blog post would end with a mandatory section called "MAD questions," plus a new blog tag to collect them.

A year later he reviewed the experiment. Across 70 posts, that section averaged 25% of the length of the main body and was frequently longer than the post itself. More surprising: a lot of half-finished drafts only became publishable because the section forced him to keep thinking. And his explanation for why something so cheap worked so well is the best move in the whole story — it works because

our default mode is to not question things. We have been implicitly taught this at our family, school, and work.Master your questioning skills, 17 December 2018

So "MAD" is not decoration, it is a permission slip: calling them mad questions granted him immunity from the usual assumptions. He also gives one unusually practical rule: do not be embarrassed by your first, stupid questions. "Bad questions lead to good questions." Exhaust them and the good ones surface. And how do you know a good one when it arrives? A good question is like a good joke — it has an unexpected element, it changes the perspective, and you feel it in your body.

All of this can read like self-help until you see the hard premise it hangs on, which is the real key to the man. He nailed it down in 2012:

We all have humble brains; don't let anybody tell you differently. Falling short of superpowers (genius/giftedness), we invent processes/systems to succeed, and the good thing about these processes are they are replicable (unlike being genius/gifted).Tell me about your thought-process, not just your results, 11 January 2012

Six years later he compressed the same axiom into five words, in an entirely different context: "Being smart does not scale." That was about why you cannot design a distributed protocol by reasoning it through in your head, and need exhaustive model checking instead.

Once you hold that key, his apparently unrelated subjects line up. They are all versions of one move: how does an ordinary mind get reliable understanding of a system too big to hold in the head? The answer is never "be smarter." It is always "build an instrument."

His most famous instrument is the hybrid logical clock, from 2014. Time in distributed systems was a stalemate then. Lamport's 1978 logical clocks record causal order but are divorced from real time, so you cannot ask what the database looked like last Tuesday. NTP-synchronized physical clocks have real time but cannot be tightly synchronized across regions, and come with leap seconds and non-monotonic jumps. Google's 2012 Spanner threw money at it — GPS receivers and atomic clocks to push the uncertainty down to 6ms — at the cost of building an entire timing infrastructure, and of having to explicitly wait out that uncertainty interval on every commit.

Three roads, each charging a toll. The question he and Sandeep Kulkarni asked ran sideways: why must we choose? Could one timestamp preserve causality, fit in constant space, and stay within a bound of the physical clock?

The naive construction fails: copy Lamport and take the max, and the logical part runs ahead of physical time without bound. The HLC trick is a small counter c plus a guarantee that it keeps getting reset — either you receive a larger logical time, and c goes to zero, or the physical clock catches up and pushes the logical value along, and c goes to zero again. Both roads lead to a reset, so the gap stays trivially bounded. Then the engineering flourish: let the logical part track only the high 48 bits of the physical clock (still microsecond granularity) and give the remaining 16 bits to c. The line that actually sells the design sits in a parenthesis:

(In our experiments c mostly stayed in single digits.)Hybrid Logical Clocks, 27 July 2014

Room for 65,536, single digits used. CockroachDB, MongoDB and YugabyteDB all run on it today.

Ten years later he wrote a five-part survey whose conclusion is worth more than the algorithm, because it redefines what time is for: synchronized clocks are not about knowing what time it is, they are a shared reference frame.

When nodes operate independently, they essentially exist in their own temporal universes.Use of Time in Distributed Databases (part 5), 14 January 2025

Align the frame and a great deal that used to require a round trip — consistent snapshots, conflict detection, leases — can be computed locally instead. But he holds one line, inherited from Barbara Liskov in 1991 and in his view unchanged since: use time for performance, not for correctness. When a clock misbehaves you should get slower, not wrong.

His sharpest recent judgment is from May 2026. He has taught TLA+ for years, always saying abstraction is the most important and hardest step. But a dissonance kept nagging: aren't computer scientists supposed to be good at abstraction already? Abstract data types, APIs, layering — staples of every CS curriculum. So why does every formal methods person report a gaping skill gap?

His answer: two opposite things hide under one word.

Modularity abstraction hides. Modeling abstraction reduces.The Two Abstractions of System Design: Hide or Reduce, 8 May 2026

Modularity abstraction draws boundaries and encapsulates, cutting vertically to hide the layer below so the module is easy to use — which means hiding concurrency and pretending operations are atomic. Joel Spolsky's famous complaint that all abstractions leak is entirely about this kind.

Modeling abstraction goes the other way. It cuts crosswise, reducing the system to the minimal behavioral skeleton for the one property you care about and discarding everything else. And — the counterintuitive part — it deliberately lets things leak: it exposes the fine-grained actions and every possible interleaving, then proves the invariants hold anyway. The payoff is not usability. It is harvesting the maximum safe concurrency the system has to offer.

The examples he reaches for are a potted history of the field, each readable as "what got thrown away": Lamport clocks discard wall-clock time and keep happens-before; HLC keeps both real time and causality and discards the rest; TrueTime turns time into a bounded-uncertainty interval; linearizability discards replication, caching and retries; log-is-the-database discards materialized state as the source of truth; MapReduce discards orchestration, parallelism, scheduling and fault tolerance.

If you want one sentence for the man, take the advice he gave distributed systems designers in 2023 — from someone who has spent half his career on consensus protocols:

Even as a distributed coordination researcher and Paxos expert, my advice is to reduce coordination.Hints for Distributed Systems Design, 2 October 2023

And his most effective instrument, in the end, is the blog itself. He holds a weekly quota, not for output but for "noticing" — staying in a frame of mind where things register. The quota, he says, lowers his standards for writing, which paradoxically raises them. Rereading his own posts from years ago "refires the same neurons," letting him reload the understanding he once strained to build.

The blog became his external memory. 839 posts, over five million views: one person's replicable, ungifted record of digging wells.