System Design

BigCat's Architecture Lab

> Distributed · Scalability · Real-world Case Studies
Day 01Scalability FoundationsVertical/Horizontal · L4/L7 LB · Stateless · Capacity Planning Day 02CachingCache Layers · LRU/LFU/ARC · Cache-aside · Invalidation Day 03Database SelectionSQL/NoSQL · OLTP/OLAP · ACID/BASE · Decision Tree Day 04Database ShardingHash/Range · Consistent Hashing · Hot Spots · Resharding Day 05ReplicationLeader/Multi/Leaderless · Sync/Async · Replication Lag · Failover Day 06ConsistencyLinearizable/Causal/Eventual · CAP/PACELC · HLC · Per-domain Choice Day 07Distributed Transactions2PC/3PC · Saga · Outbox · Idempotency Day 08Message QueuesKafka/RabbitMQ/SQS · Delivery Semantics · Backpressure · DLQ Day 09API DesignREST/GraphQL/gRPC · Pagination · Versioning · Rate-Limit Contract Day 10Rate LimitingToken/Leaky/Sliding/GCRA · Distributed Counters · Multi-dim · 429 Contract Day 11Unique ID GenerationUUIDv4/v7 · Snowflake · ULID/KSUID · Clock Skew Day 12Search SystemsInverted Index · Lucene Segments · Scatter-Gather · Vector HNSW Day 13Recommendation SystemsTwo-Tower Retrieval · Multi-stage Funnel · Cold Start · Generative Rec TIGER Day 14Feed SystemPush/Pull/Hybrid · Fanout Amplification · Timeline Storage · Ranking Pipeline Day 15Chat SystemWebSocket Connections · Bucketed Storage · Delivery Guarantees · E2E Encryption Day 16Video StreamingTranscode Pipeline · Per-Title/Per-Shot · ABR · CDN/Open Connect Day 17Payment SystemsIdempotency Recovery Points · Double-Entry Ledger · Saga/Outbox · 3-Way Reconciliation Day 18Subscription & BillingLifecycle State Machine · By-Second Proration · Usage Metering · Multi-currency & Tax Day 19Geospatial SystemsGeohash/S2/H3 · Grid vs R-Tree · haversine/ETA · Uber DISCO Geo-Sharding Day 20Data Processing (Batch vs Stream)Batch/Stream Unify · Lambda/Kappa · Event-time/Watermark · Exactly-once Day 21Monitoring & ObservabilityMetrics/Logs/Traces · OpenTelemetry Sampling · SLO/Error Budget · Percentile Aggregation Day 22Deployment & ReleaseRolling/Blue-Green/Canary · Canary Analysis · Feature Flags · Expand-Contract Migration Day 23ReliabilityCircuit Breaker FSM · Backoff+Jitter Retry · Bulkhead Isolation · Graceful Degradation & Load Shedding Day 24SecurityAuthN/AuthZ Boundary · OAuth2/OIDC + PKCE · JWT vs Session Revocation · Secret Dynamic Creds Day 25System Design InterviewScoping · 4-Step Framework + Time-boxing · Think Out Loud & Deep Dive · Articulating Trade-offs Day 26Capacity EstimationPowers of 2 + Latency Numbers · DAU→QPS Peak · Storage/Bandwidth/Memory · Assumption Discipline Day 27Cost & Capacity EngineeringQueueing Headroom · Reactive/Predictive Autoscaling · FinOps Attribution · Rightsizing/Spot Tiers Day 28CDN & EdgeAnycast/BGP · Tiered Cache Funnel · Purge/SWR Invalidation · Edge Compute V8 Isolate Day 29File & Object StorageObject vs Block vs File flat keyspace · Replication vs Erasure Coding 11 nines · Multipart + presigned direct upload · Strong metadata consistency & hot partitions Day 30Authorization & Account SystemsRBAC/ABAC/ReBAC relation tuples · Hierarchical inheritance & Zookie revocation · Multi-tenant RLS isolation · Hash-chained tamper-evident audit Day 31Hybrid Search & RerankingBM25 vs dense vectors complementary · Recall/rerank multi-stage funnel · RRF fusion & cross-encoder reranking · Query-biased snippet highlighting Day 32LLM ServingTwo-phase inference & continuous batching · KV Cache/PagedAttention memory bottleneck · Prompt caching & prefix-aware routing · Cost/Latency/Quality triangle Day 33AI Product BackendRAG recall quality contextual+hybrid+rerank · Durable resumable agent loops · Embedding service model/index version locking · Human-in-the-loop approval gate Day 34Real-time SystemsTCP head-of-line blocking & UDP/WebRTC/QUIC channels · Client prediction + server reconciliation + interpolation · Authoritative server tick/delta/AOI state sync · Collab editing OT/CRDT/LWW (Figma) Day 35IoT & EdgeMQTT pub/sub & tiered QoS · Edge reduction + offline buffering · Kafka peak-shaving/decouple/backpressure · TSDB Gorilla compression & cardinality explosion Day 36Blockchain & Distributed LedgerPoW/PoS/BFT consensus & Sybil resistance · Smart-contract determinism + gas + reentrancy · Hash chain/Merkle/off-chain storage availability · Trilemma & L2 Rollups (Optimistic vs ZK) Day 37Multi-tenant SaaSSilo/Pool/Bridge isolation spectrum · Shard by tenant_id + RLS for leak/skew control · Noisy-neighbor per-tenant limiting + fair queues · Idempotent recomputable usage metering Day 38Data Lake & LakehouseLake/warehouse/lakehouse selection & open columnar · ACID on S3 with Iceberg/Delta/Hudi (immutable files + atomic snapshot swap + OCC) · Layered-metadata pruning + time travel + small-file ops · Unified batch/stream COW vs MoR & incremental ETL Day 39Workflow EngineDurable execution: event-sourcing + deterministic replay · Orchestration vs choreography · Saga compensation (semantic undo) + backward/forward recovery · Activity at-least-once + idempotency key + heartbeat/timeout Day 40Feature Platform & ML InfraDual store: offline warehouse vs online KV, one definition materialized twice · Point-in-time join prevents label leakage (log-and-wait vs backfill) · Model serving latency-budget split + fetch optimization · A/B stable bucketing + guardrails + two-sided-market switchback Day 41Failure Faster Than HumansGet humans out of the fast loop: auto-detect + contain (breaker/kill switch) · Bound blast radius with cells/bulkheads/shuffle sharding · Change safety: canary + auto-rollback + error budget (guardrails before scale) · Observability-driven automation, stable cognition/variable action (Knight Capital/Perrow) Day 42Multi-region ArchitectureGeoDNS/Anycast locality routing + home-region failover · Active-active consistency: LWW vs single-writer home vs Spanner TrueTime · Data residency via geo-partitioning for GDPR (whole lifecycle incl. backups/logs) · Cross-region latency: read-local write-home, cut trans-oceanic round trips Day 43Privacy & ComplianceRight to erasure: crypto-shredding deletes a key to void backups/warehouse/Kafka (vs physical delete/tombstone) · Anonymization: pseudonymized ≠ anonymous, k-anonymity/DP & re-identification · Audit trail: append-only tamper-evident hash chain (skeleton + payload split) · Consent: purpose limitation, consent travels with data (IAB TCF) Day 44Chaos Engineering & Resilience TestingSteady-state hypothesis: control/experiment groups disprove (cancels time noise) · FIT layered fault injection propagated per-request (vs Chaos Monkey killing instances) · Escalating blast radius + auto-abort kill switch (production-experiment guardrails) · Game Day/Google DiRT tests people & process MTTD/MTTR Day 45Collaborative EditingOT central serialization buys plain text with zero bloat (TP1/TP2 · Jupiter/Google Docs) · CRDT dense position ids commute at the cost of metadata bloat + interleaving (Yjs/Automerge columnar 100:1→1:1) · Offline-first version-vector causal merge (Figma reconnect replay + LWW properties) · Per-document single-writer authority process + snapshot truncation & tombstone GC Day 46Distributed Consensus & CoordinationMajority quorum intersection = safety bedrock (Raft/Paxos/ZAB · etcd/Chubby/KRaft) · Monotonic term + majority election kills split-brain mathematically · Distributed locks: fencing token cures GC-pause double-ownership · Linearizable reads ReadIndex/lease/follower (Jepsen: ~80% of etcd reads non-linearizable) Day 47Database Internals & Storage EnginesB+Tree vs LSM: pick 2 of read/write/space amplification (RUM · RocksDB/InnoDB) · WAL log-first + ARIES three-phase crash recovery, RPO=0 · MVCC reads don't block writes + version reclaim (VACUUM/undo · Postgres vs InnoDB write amp) · Index internals + CBO optimizer: leftmost-prefix / covering index / selectivity Day 48Networking FundamentalsTCP reliable-ordering's cost = HOL blocking vs UDP zero-handshake · HTTP/1.1→2→3 three sieges on HOL blocking (HTTP/2 single-TCP hurts more on loss → QUIC per-stream) · TLS 1.2 2-RTT→1.3 1-RTT→0-RTT, replay means idempotent-only · DNS/Anycast nearest + L4 for volume vs L7 TLS-terminating smart routing Day 49Containers & OrchestrationContainer = plain process + namespaces slice the view / cgroups slice resources (shared kernel trades density for weaker-than-VM isolation) · Declarative level-triggered reconcile loop: failure isn't a special case, just level drift · Scheduling filter/score bin-packing & the utilization↔availability opposition · Service Mesh Envoy sidecar trades zero code change for per-hop proxy cost (→ambient/eBPF sidecar-less) Day 50Code Review as a Signal-Detection SystemEffective FP <10% decides the tool's fate (Google Tricorder NOT USEFUL) · Triage layer severity×confidence gating + nitpicks to autofix, never ask a human · Diff-time vs batch decides fix rate (Meta Infer near-0→substantial) · FP budget auto-downgrade like an SLO error budget + activity vs outcome metrics Goodhart trap Day 51Alerting Under a Low Base RateBase-rate fallacy: 0.1% per-check FP × 10M checks drowns 5 real reports, P(true│alert)≈0.05% (Axelsson Bayes) · Three exits: cut FPR / raise base rate by shrinking the domain / chain independent layers to multiply · Independent oracle backstop vs LLM self-judge — shared blind spots degrade multiply to no-drop · Report-level vs decision-level FP metrics trap + on-call trust is the true SLO Day 52Adding a Correctness Oracle to Chaos EngineeringUnder a fault "alive" ≠ "correct": skipping a shard silently returns the partial sum, zero alerts (gray failure's differential observability) · Differential oracle: fault-free reference path vs fault path compared after normalization, ground truth self-generated from traffic · Normalization (total order / quantize money to cents / mask now·rand) or FPs spam you off · Three-state verdict PASS / acceptable degradation / correctness bug — honestly-declared partial=🟡, falsely-complete=🔴 · Independent reference vs correlated failure + AI only in the generator seat, never judge (builds on Day 44/51) Day 53Managing Project Risk by the Fat TailIT overruns are power-law fat-tailed: median project on budget, mean 73pp above it, 1-in-6 a black-swan +200%, worst 18% average +447% (Flyvbjerg's 16k-project database) · Watch the tail not the mean: reference-class forecast at P80 for reserves + mechanical stop-loss + staged commitment (real options) · "All-or-nothing" success definitions manufacture false failure rates (CHAOS 16.2%, Flyvbjerg 0.5% — a framing trap, per Day 51) · Same root as capacity planning's "provision for P99, not the mean" Day 54Fail Obviously — Designing Failure Visibility into AI WorkflowsBainbridge 1983's "Ironies of Automation" becomes its hardest prescription in the LLM era: LLMs "fail fluently" and never self-announce · Failure-visibility engineering: structured assertions / executable self-check / diff highlighting, refuted by an independent oracle that turns "silent wrong result" into a red-flagged assertion (same-model self-review is correlated ≈ no check) · Verification-cost engineering: explanations reduce overreliance only when they truly lower verification cost (the Vasconcelos CSCW 2023 / Zhang FAT* 2020 boundary) · Final-seat four parameters: base rate / verification cost / practice dose (seeded faults) / failure visibility, extending Day 51 low-base-rate alerting