System Design Crash Course: Scaling 100 to 100M Users

A complete system design crash course on scaling: caching, sharding, queues, load balancing, resilience, and multi-region, from 100 to 100 million users. This crash course starts on a single t3.micro serving 100 people and climbs, one bottleneck at a time, to an architecture serving a hundred million. It installs the instruments first: RED and USE, why an average hides the p99 users actually feel, and how one traced request tells you which service is slow. Then it names the map: every scaling problem you will ever diagnose is compute, reads, writes, or async work. You buy the bigger box, watch a real Datadog case where one SQL line beat the hardware upgrade, and find where vertical scaling dies, with Amdahl's Law explaining why throughput eventually falls. Going horizontal brings statelessness, the DNS, L4 and L7 traffic layers, health checks that cascade into fleet-wide outages, and autoscaling on the saturated signal instead of CPU. The read wall covers cursor pagination, read replicas and the read-your-own-writes bug replication lag creates, cache-aside hit-rate math, materialized views, fan-out on write versus on read, thundering herds, hot keys with a real Twitter incident, and the speed-of-light floor a CDN works around. The write wall covers connection exhaustion, write amplification, functional partitioning, sharding and hot partitions, resharding, polyglot persistence, and CQRS. Async work brings queues versus topics, idempotency and why exactly-once is marketing, what ordering costs in throughput, silent message loss, the transactional outbox, and backpressure. Then the video turns on itself: past a certain size, scaling makes outages worse. Retry storms, cascading failure, deadline propagation, circuit breakers, load shedding, rate limiting, and deciding what is allowed to fail. It closes on multi-AZ versus multi-region, active-active versus active-passive, CAP, cell-based blast radius, the real bill including egress and data tiering, and a four-question decision tree for 2am. ━━━━━━━━━━━━━━━━━━━━━━━ CONNECT WITH ME 🔗 LinkedIn:   / joud-awad   📝 Medium:   / joudwawad   ━━━━━━━━━━━━━━━━━━━━━━━ Chapters: 0:00 What you'll learn: 100 to 100 million users 1:25 Latency, throughput, capacity: three different things 2:15 Vertical vs. horizontal: the only two directions 2:48 The four axes: compute, reads, writes, async 3:57 100 users: the monolith baseline 4:34 RED and USE: two methods for watching a system 5:08 Why averages lie: P90, P99 and the tail 6:01 Traces: finding which part is actually slow 6:51 10K users: buying the bigger box 7:53 The missing index that beat a hardware upgrade 8:47 Where vertical scaling actually dies 9:25 The Universal Scalability Law: when a server makes you slower 10:34 100K users: statelessness before you duplicate 11:43 Traffic layers: DNS, L4 and L7 12:38 When the health check causes the outage 13:38 Autoscaling on the right signal, not CPU 14:45 Pagination: offset vs. cursor 15:28 Read replicas and replication lag 16:04 Read-your-own-writes 16:42 Cache-aside and the hit-rate math 17:28 Cache invalidation: TTL, write-through, explicit bust 18:12 Materialized views: cache the answer, not the row 18:57 Fan-out on write vs. fan-out on read 20:05 Thundering herd and cache stampede 21:04 Hot keys, and the load-bearing cache question 22:21 CDN and the speed-of-light floor 23:39 5M users: connection pooling and exhaustion 24:46 Batching and write amplification 25:31 Functional partitioning: split by capability first 26:18 Sharding and hot partitions 27:09 Resharding and consistent hashing 28:09 Right tool for the job: search, time-series, graph 28:46 CQRS: one model to write, another to read 29:25 20M users: what the user actually needs right now 30:23 Queues vs. topics 31:13 Idempotency, and why exactly-once is marketing 32:12 Ordering costs throughput 33:00 Silent message loss and the DLQ nobody checks 33:52 The dual-write problem and the outbox pattern 34:42 Backpressure and bounded queues 35:40 100M users: retry storms, backoff and jitter 37:18 Cascading failure from one slow dependency 38:27 Deadline propagation: one budget, not three timeouts 39:14 Circuit breakers: closed, open, half-open 40:03 Load shedding: drop 10%, keep 90% healthy 41:03 Rate limiting: token bucket vs. sliding window 41:58 Graceful degradation: what's allowed to fail 42:34 Multi-AZ vs. multi-region 43:27 Active-active vs. active-passive 44:16 CAP theorem and conflict resolution 45:06 Cell-based architecture and blast radius 45:45 Cost per request and the over-provisioning tax 46:39 Reserved, spot, and on-demand 47:28 Data tiering: most of your data is cold 48:10 Egress: the arrows that bill you 48:56 Regional failover: DNS TTLs and untested runbooks 49:29 The payoff: every box, named 50:03 The four questions: a scaling decision tree 50:53 Recap: the whole climb 51:17 Sign off #systemdesign #scalability #distributedsystems #backend #softwarearchitecture