CS Papers Deep-Read

Paper 1Attention Is All You Need — one attention-only architecture that became the foundation of every modern large modelVaswani et al. · 2017 Paper 2Deep Residual Learning (ResNet) — one skip connection let networks go hundreds of layers deep; the backbone of almost every deep netHe et al. · 2015 Paper 3AlexNet — a deep net that learned its own features from a million images, won 2012 by a landslide and ignited the deep learning revolutionKrizhevsky et al. · 2012 Paper 4Word2Vec — a machine that teaches itself word vectors from massive text; "king − man + woman ≈ queen"; where the embedding era beganMikolov et al. · 2013 Paper 5ViT (An Image is Worth 16×16 Words) — slice an image into patches as "words" for a standard Transformer; vision and language now share one architectureDosovitskiy et al. · 2020 Paper 6CLIP — a matching game over 400M image–text pairs puts pictures and words in one shared meaning space; the foundation of multimodal AIRadford et al. · 2021 Paper 7LSTM (Long Short-Term Memory) — a memory conveyor plus two learned gates cured vanishing gradients and ruled sequence modeling for twenty yearsHochreiter & Schmidhuber · 1997 Paper 8Seq2Seq (Sequence to Sequence Learning) — two LSTMs in relay, "understand then re-say," establishing the encoder-decoder paradigmSutskever, Vinyals & Le · 2014 Paper 9Bahdanau Attention (Jointly Learning to Align and Translate) — let the decoder glance back at the source by relevance for every word it writes; attention's birth certificateBahdanau, Cho & Bengio · 2014 Paper 10BERT — swap "guess the next word" for "fill in the blank" to read text both ways and teach itself language, founding the "pre-train + fine-tune" eraDevlin et al. · 2018 Paper 11GPT-3 — scale "guess the next word" to 175B parameters and it picks up new tasks from a few examples; the start of the prompting eraBrown et al. · 2020 Paper 12Scaling Laws — a model's skill falls along a smooth power law in "model + data + compute," turning big models from alchemy into engineeringKaplan et al. · 2020 Paper 13Emergent Abilities — some skills appear "suddenly" once a model crosses a scale threshold and can't be predicted; it also drew the famous "is it a mirage?" rebuttalWei et al. · 2022 Paper 14InstructGPT / RLHF — align GPT-3 to human preference so it answers you thoughtfully, shifting the focus from "make it bigger" to "align it"; ChatGPT's recipeOuyang et al. · 2022 Paper 15Constitutional AI — write the values into a short "constitution" and let the AI self-critique and rewrite against it (RLAIF), using almost no human harmfulness labels; a cornerstone of training ClaudeBai et al. · 2022 Paper 16GAN (Generative Adversarial Networks) — a "forger" makes images and an "inspector" judges them; two adversaries push each other stronger, igniting a whole wave of generative modelingGoodfellow et al. · 2014 Paper 17DDPM (Denoising Diffusion Probabilistic Models) — sprinkle an image into static over a thousand steps, then train a network to spot the noise added at each one and run the chain backwards — where diffusion overtook GANsHo et al. · 2020 Batch Normalization — standardize each layer's inputs with the current mini-batch's own mean and variance, and training deep networks stops being tightrope walkingIoffe & Szegedy · 2015 Dropout — send half the neurons home at random on every training step, so none can lean on a partner and one weight set buys a whole ensembleSrivastava et al. · 2014 Adam (Adaptive Moment Estimation) — keep two ledgers per parameter, "which way on average" and "how big and jittery the slope is," to auto-scale each step; deep learning's default optimizer for a decadeKingma & Ba · 2014 DQN (Deep Q-Network) — feed only screen pixels and score, use a convolutional network to estimate a "total future score" per action, and stabilize training with "experience replay"; one program learns 7 Atari games and founds deep reinforcement learningMnih et al. · 2013 AlphaGo — a policy network cuts away most bad moves and a value network skips reading to the end, guiding a tree search to first beat a professional in full-board GoSilver et al. · 2016 Paper 1The Google File System (GFS) — one master keeps the ledger, data sliced into big chunks stored three times; petabyte-scale storage from failure-prone cheap machines, big-data's first foundation stoneGhemawat et al. · 2003 Paper 2MapReduce — write just two functions, map + reduce, and the framework auto-parallelizes across thousands of machines with end-to-end fault tolerance, opening the big-data eraDean & Ghemawat · 2004 Paper 3Bigtable — a sparse sorted table keyed by (row, column, timestamp), spanning thousands of machines to petabyte scale; ancestor of wide-column NoSQL and the LSM engineChang et al. · 2006 Paper 4The Chubby Lock Service — wraps error-prone Paxos into a "files + coarse-grained locks" service where whoever grabs the lock is master; the shared ancestor of GFS/Bigtable leader election and of ZooKeeper/etcdBurrows · 2006 Paper 5Percolator (Caffeine) — cross-row transactions with primary-lock two-phase commit plus observer notifications on Bigtable, turning Google's search index from full rebuilds into incremental updates and doubling freshnessPeng & Dabek · 2010 Paper 6Pregel — a vertex-centric, BSP model for graph processing: write only "what one vertex does in one round," and the system handles partitioning, synchronization and fault tolerance at billion-vertex scaleMalewicz et al. · 2010 Paper 7Dremel — columnar storage for nested data (repetition/definition levels shred and reassemble losslessly) plus a search-engine-style serving tree over thousands of machines, for second-scale interactive aggregation over trillions of rows; the basis of BigQueryMelnik et al. · 2010 Paper 8Spanner — each clock honestly reports "the time is somewhere in this small interval" (TrueTime), and a short commit-wait waits the error out, so a planet-spanning database gains a transaction total order everyone agrees on (external consistency)Corbett et al. · 2012 Paper 9F1 — a distributed SQL layer built on Spanner: it re-fits a "scalable + strongly consistent" storage foundation with full SQL, transactions, and consistent indexes, spreading cross-data-center latency thin via hierarchical clustering and optimistic transactions; it ran AdWordsShute et al. · 2013 Paper 10Dapper — give each request a trace id that travels with it, record each unit of work as a span, reassemble a call tree; transparent via shared-library probes, near-zero overhead via sampling and out-of-band collection — the blueprint for distributed tracing and observabilitySigelman et al. · 2010 Paper 11The Tail at Scale — you wait on the slowest machine, so rare slowness gets amplified by scale; tolerate the latency tail the way we tolerate faultsDean & Barroso · 2013 Paper 1Time, Clocks, and the Ordering of Events — with no clock everyone trusts, define order by "a message is sent before it's received," give each machine an ever-increasing ticket number, and machines a world apart produce one ordering everyone agrees onLamport · 1978 Paper 2The Byzantine Generals Problem — reaching agreement when nodes may lie and send conflicting messages: solvable only if traitors are under one-third, via relaying plus majority vote; with unforgeable signatures any number of traitors is tolerable — the origin of every anti-cheating consensusLamport, Shostak & Pease · 1982 Paxos Made Simple — make crash-prone machines that only talk over an unreliable network agree on one value, uniquely and irreversibly: draw an ever-larger ticket, ask a majority first, honor the bigger ticket, and reuse any already-accepted value before pushing your own; since any two majorities overlap, a chosen value can never be overturnedLamport · 2001 In Search of an Understandable Consensus Algorithm (Raft) — rewrote a consensus algorithm no one had grasped for thirty years into rules you can read and implement correctly: single leader + randomized elections + commit-on-majority, now the consistency core of etcd, CockroachDB, and countless systemsOngaro & Ousterhout · 2014 Dynamo: Amazon's Highly Available Key-value Store — built so the cart never refuses a write: it trades immediate consistency for perpetual availability, using consistent hashing for placement, a tunable R+W quorum, and vector clocks that hand version conflicts to the app — igniting a whole generation of eventually consistent NoSQL like CassandraDeCandia et al. · 2007 Harvest, Yield, and Scalable Tolerant Systems — split availability into two dials, yield and harvest, for graceful degradation — the idea that became CAPFox & Brewer · 1999 Chord: A Scalable Peer-to-peer Lookup Service — turns "among millions of nodes coming and going, who owns this key" into a provable minimal primitive: nodes and keys sit on a consistent-hashing ring, a key goes to the nearest node clockwise, a "distance-doubling" finger table locates it in O(log N) hops, and a stabilization protocol self-heals — bedrock of a generation of NoSQL and P2P systemsStoica et al. · 2001 Kafka — remade the message queue as an append-only partitioned log: the broker tracks no consumption and deletes nothing on read, so many parties share one stream at their own pace and can rewindKreps et al. · 2011 Distributed Snapshots — no pause and no synchronized clocks: one marker down every channel draws the line, each process records only locally, and a causally consistent global photograph assembles itselfChandy & Lamport · 1985 Impossibility of Distributed Consensus with One Faulty Process — a crash is indistinguishable from extreme slowness, so one possible crash rules out any deterministic algorithm that guarantees consensus terminatesFischer, Lynch, Paterson · 1985 Consistent Hashing and Random Trees — pin nodes and keys on one ring and let the nearest node clockwise own each key, so adding or removing a node disturbs a single arc and moves only 1/N of the dataKarger et al. · 1997 Paper 1A Relational Model of Data — lay data out as tidy tables linked by shared values rather than physical pointers, turning queries from "teach it how to walk" into "state what you want"; the theoretical root of nearly every modern database and of SQLE. F. Codd · 1970 ARIES — log every change before touching the data, then after a crash repeat all of history exactly before undoing the half-finished transactions: the blueprint behind crash recovery in nearly every relational databaseMohan et al. · 1992 A Critique of ANSI SQL Isolation Levels — the standard's plain-English definition of isolation reads two ways and misses real accidents; this paper turns each level into a checklist of what it actually prevents, and names snapshot isolationBerenson, Bernstein, Gray et al. · 1995 C-Store: A Column-oriented DBMS — store by column, keep the same data in several sort orders, and compute on the compressed bytes without unpacking: the turning point for analytical databasesStonebraker et al. · 2005 Resilient Distributed Datasets — log how data was computed instead of replicating it, rebuild whatever a dead node took, and memory becomes a safe place to keep itZaharia et al. · 2012 Paper 1The UNIX Time-Sharing System — a few minimal ideas (everything is a file, small tools joined by pipes, a tree of folders) build a small yet general OS; Linux, macOS, Android, and iOS are all its descendantsRitchie & Thompson · 1974 Paper 2End-to-End Arguments in System Design — some functions only the two ends can get right (like confirming a file arrived byte-for-byte); the network in the middle can't do them completely and drags everyone down trying, so keep the clever work at the ends and the network simple — the intellectual foundation of TCP/IP's "dumb pipes, smart endpoints"Saltzer, Reed & Clark · 1984 Paper 3Congestion Avoidance and Control — with no central dispatcher, let each sender pace its packets to the returning echoes (ACKs), nudge up while things go well, and halve on any loss; this shared "nudge up, halve down" self-discipline keeps the internet from collapsing together under loadVan Jacobson · 1988 A Protocol for Packet Network Intercommunication (the origin of TCP/IP) — nobody has to change their network: a gateway translates at each border, one common envelope wraps the data, and reliability is pushed to the two endsCerf & Kahn · 1974 New Directions in Cryptography — split the key into a public and private half so two strangers can agree on a secret in the open; the bedrock of modern encryptionDiffie & Hellman · 1976 RSA — the first practical public-key scheme, betting encryption and digital signatures on the hardness of factoring large numbersRivest, Shamir & Adleman · 1978 Bitcoin (A Peer-to-Peer Electronic Cash System) — "solve a hard puzzle to record + longest chain wins + reward the recorder in coin" lets anonymous strangers stop double-spending with no bank, the origin of the blockchainSatoshi Nakamoto · 2008 Reflections on Trusting Trust — hide a backdoor inside the "compiler compiles itself" step so it self-reproduces and leaves not one word in any source, proving "you can't trust code you did not totally create yourself," the intellectual origin of the supply-chain attackKen Thompson · 1984 The Protection of Information in Computer Systems — security is not patching after the breach but eight principles held at design time: deny by default, check every access, grant only what is neededSaltzer & Schroeder · 1975 As We May Think — in 1945 imagined a desk called the Memex that ties knowledge into named, shareable "trails" by association, the intellectual source of the hyperlink and the WebVannevar Bush · 1945 The Anatomy of a Search Engine — treat every link as a weighted recommendation (PageRank) and describe a page by "what others call it" (anchor text) to surface the page most worth seeing in an exploding web, where Google beganBrin & Page · 1998 A Mathematical Theory of Communication — weighing information in "bits" and proving two iron laws: every source has a compression floor (entropy), every channel a speed limit (capacity) under which noise can be corrected to near-zero errorClaude Shannon · 1948 On Computable Numbers (The Turing Machine) — defining "computing" as a minimal tape machine, building the universal machine that "reads an instruction sheet to become any machine," then proving some problems no machine can ever decideAlan Turing · 1936 Computing Machinery and Intelligence (The Turing Test) — replacing the unanswerable "can machines think?" with an imitation game: if a judge chatting by teleprinter can't tell machine from human, call it thinking — the founding text of AI philosophy and its first touchstoneAlan Turing · 1950 The Complexity of Theorem-Proving Procedures (NP-completeness) — making "hard to solve" precise: proving SAT the "hardest representative" so cracking it cracks a whole class of "easy to check, brutal to solve" problems, turning P vs NP into an exact million-dollar open questionStephen Cook · 1971 Go To Statement Considered Harmful — a two-page letter arguing goto unmoors "where you read" from "where the run is"; keep only sequence/branch/loop, and structured programming is bornEdsger Dijkstra · 1968 On the Criteria To Be Used in Decomposing Systems into Modules — split modules by the design decisions most likely to change, not by processing step; the source of encapsulation and OOPDavid Parnas · 1972 No Silver Bullet: Essence and Accident in Software Engineering — software's difficulty splits into essence and accident; past advances only shaved accident, now nearly spent, so no bullet gives a 10× gainFred Brooks · 1986 Communicating Sequential Processes (CSP) — abolish shared memory and let processes hand messages over, so one exchange does the data transfer and the synchronization at once; the source of Go's channelsC. A. R. Hoare · 1978 Hints for Computer System Design — the judgment behind getting it right, fast and still working, compressed into thirty-odd slogans he honestly called hints: they may be wrong, go checkButler Lampson · 1983 Large-scale Cluster Management at Google with Borg — allocating a whole datacenter as one machine: mixing latency-sensitive services with batch work and reclaiming what was requested but never used, saving 20–30% of the fleet, and the direct blueprint for KubernetesVerma et al. · 2015 Autopilot — cloud waste comes mostly from people mis-declaring resource requests: let a fading histogram of real usage set every quota instead, and slack falls from 46% to 23%Rzadca et al. · 2020 Paper 3A Berkeley View on Serverless Computing — pins down what serverless actually is (FaaS + BaaS, billed for what you use) and lists the four things it structurally cannot doJonas et al. · 2019 Firecracker: Lightweight Virtualization for Serverless Applications — shave a VM's emulated hardware down to four devices, and VM-grade isolation arrives at container-grade density and a 125 ms bootAgache et al. · 2020 Pivot Tracing: Dynamic Causal Monitoring for Distributed Systems — a happened-before join pairs the cost measured at the bottom with the identity known only at the top, so a live system can finally answer who is eating this disk right nowMace, Roelke, Fonseca · 2015