Content Delivery Network

Edge caching at global scale

Design a content delivery network: caches in about 100 locations that serve customers' files near users, send few requests back to origins, and remove content everywhere within seconds.

Overview

A content delivery network (CDN) is a fleet of caching proxy servers spread over many points of presence (PoPs): racks of servers in data centers close to users. A customer points its domain, say static.example.com, at the CDN. A user's request reaches a nearby PoP; if a server there has the file, it answers in a few milliseconds; if not, it fetches the file, keeps a copy, and answers.

This page designs the CDN itself. When a site should use a CDN at all, and what is safe to cache, are in the CDN vs Direct Serving topic; the HTTP rules a CDN follows (Cache-Control, Vary, stale content, cache keys) are in the CDN Deep Dive topic; how users are steered to a location is in the GSLB, GeoDNS and Anycast topic. Here we size the fleet and design the parts an interviewer probes: how a PoP avoids storing the same object on every server, what to keep on limited disks, how misses are funneled so the origin sees few of them, and how a purge reaches hundreds of servers quickly and verifiably.

Questions to ask first

Who are the customers? — Many websites and apps, each with its own origin
What content? — Mostly static: images, scripts, downloads, video segments
How much traffic? — 20 Tbit/s at peak, 8 Tbit/s on average; objects average 100 KB
How many locations? — About 100 PoPs, grouped under about 10 regional parent PoPs
How fast must a purge land? — Everywhere within 5 seconds (a target we chose), with a status the customer can check
HTTPS? — Yes, with each customer's certificate at the edge
Dynamic or personalized pages? — Passed through without caching

What you will be able to do

Size a CDN from peak egress: servers, storage and the traffic that reaches parents and origins.
Make a PoP store each object once, and keep popular objects despite limited disk.
Explain tiered caching and request collapsing with the origin load they save.
Design a purge system that reaches every server fast and knows when it is done.
Say how the CDN behaves when a PoP, a parent or the origin fails.

Key concepts

Points of presence
Anycast or DNS steering
Rendezvous hashing in a PoP
Admission on second hit
Tiered caching (shield)
Request collapsing
Purge fan-out with acks
Versioned URLs
Serve stale on error

Functional requirements

Serve GET and HEAD for customer content from the PoP the user reaches
On a miss, fetch from a regional parent, and on its miss from the origin
Follow HTTP caching rules: store only what may be stored, for as long as allowed; revalidate with conditional requests
Purge by exact URL, by prefix or by tag; report when every server has done it
Terminate TLS with each customer's certificate
Give customers per-minute statistics and raw logs

Non-functional requirements

A cache hit answers in under 20 ms at p50 for users near a PoP (a target we chose)
At least 90% of bytes served from the edge for typical static sites (a target we chose)
A purge reaches every server within 5 seconds at p99 (a target we chose)
Losing a whole PoP only moves its users to the next nearest one
If an origin fails, content still in cache keeps being served

Interview questions

How many servers does the CDN need?
Why hash requests to one server inside a PoP?
What does an origin shield do, and what does it cost?
How do you purge content from every server within seconds?
Anycast or DNS to route users?
What happens when the customer's origin goes down?
How do you decide what to keep on the edge disks?