Identifying Scaling Bottlenecks

Amdahl's Law, the NALSD method, and how Google does back-of-envelope on a whiteboard.

Overview

Bottlenecks come from growth patterns

Scaling problems rarely look like "we need bigger machines." They look like "this design works at 10K QPS but breaks at 100K." The bottleneck is a component whose growth pattern the original design didn't anticipate.

NALSD: Non-Abstract Large System Design

SRE Workbook Ch 12 defines NALSD as "the ability to assess, design, and evaluate large systems." Its thesis: "all systems will eventually have to run on real computers in real datacenters using real networks." You can't hand-wave the math.

The worked example: AdWords CTR dashboard at 500K search QPS

1 — Single machine — Fails: at 2KB per query event, 86.4 TB/day before indexes; 1,563 machines with 64GB RAM just for storage, and 2,500 disks for I/O
2 — MapReduce — Breaks the 5-minute freshness SLO
3 — Sharded LogJoiner — Survives at single-DC scale
4 — Multi-DC with Paxos — Handles regional failure

> [!IMPORTANT] Separate components by growth pattern. Storage, compute, join state and replication bandwidth each scale independently if you let them; merged together, the slowest caps the whole system.

Amdahl's Law: the serial fraction

If 5% of your work is serial, you can't parallelize past a 20× speedup no matter how many cores you add.

> [!TIP] Find the serial fraction, eliminate it, repeat.

Key concepts

NALSD (Non-Abstract Large System Design): Workbook Ch 12 method: "the ability to assess, design, and evaluate large systems" using actual numbers — KB per event, RTT per RPC, ops/sec per host.
Four-question loop: Possible? Better? Feasible (at scale)? Resilient (under failure)? — the scaffolding for any system-design conversation.
Amdahl's Law: Speedup limit = 1 / (serial_fraction + parallel_fraction/N). With 5% serial, max speedup is 20× regardless of N. The serial fraction is the real bottleneck.
Separate components by growth pattern: Storage / compute / join-state / replication-BW each have their own scaling curve. Merging them creates a single slowest-growing component that caps the system.
Hot shards: Power-law distributions are everywhere — 80% of traffic to 20% of keys. Hash partitioning alone doesn't solve hot shards; you need consistent hashing + bounded-load or virtual nodes.

When to use it

System-design interviews — NALSD's four-question loop is the canonical interview answer scaffold
Capacity planning — when does the current architecture stop working?
Pre-launch reviews — has the design been stress-tested past expected load?
Performance investigations — what's the actual bottleneck under load?

Interview questions

How do you approach a scaling-bottleneck question on a whiteboard?
How does Amdahl's Law apply to scaling bottlenecks?
I've scaled my service horizontally and one shard is overloaded. Why?