Fundamentals

The ideas every system design answer is built on: how systems grow, how fast they answer, how often they stay up, and what they can promise about data.

Overview

System design is the work of deciding how the parts of a software system fit together so that it keeps working as more people use it. Almost every design question comes back to the same few ideas: how the system grows, how fast it answers, how often it is up, whether its answers are correct and current, and what all of that costs.

This chapter explains each of those ideas in plain terms, with one picture per idea and the numbers you need to reason about it. The later chapters on caching, databases, load balancing and messaging assume you know these words and these numbers, so it is worth reading end to end once.

For a sense of scale: in 2025 Google reported more than 5 trillion searches a year, about 160,000 every second on average. Netflix passed 325 million paid memberships at the end of 2025. During Prime Day 2025, Amazon DynamoDB peaked at 151 million requests per second. None of these run on one machine, and all of them are built from the ideas below.

What you will be able to do

  • Explain when to make one machine bigger and when to add more machines.
  • Estimate traffic, storage and server counts in a few minutes, out loud.
  • Work out how many nines of availability a design can reach, and what the next nine costs.
  • Choose between strong and eventual consistency for each part of a design, and say why.
  • Walk through what happens when one request crosses DNS, a CDN, a load balancer, a cache and a database.

Key concepts

  • Scale up and scale out
  • Latency and throughput
  • Availability and reliability
  • CAP and PACELC
  • Consistency models
  • DNS, CDNs and load balancers
  • Proxies
  • Idempotency
  • Estimation

Interview questions

  • When would you scale up instead of scaling out?
  • What does the CAP theorem actually say, and what does PACELC add?
  • Estimate the traffic and storage for a service with 10 million daily users.
  • What is the difference between latency and throughput, and how does Little's Law link them?
  • How do you work out the availability of a system made of several parts?
  • What is the difference between availability, reliability and durability?
  • Walk through what happens when a user loads a page of your product.
  • What is the difference between a forward proxy and a reverse proxy?
  • How do you make a payment API safe to retry?
  • When is eventual consistency acceptable, and when is it not?

Sign up free to read the full topic

More System Design topics

Start free on Cariara · Coding problems · System design studio