Soft vs Hard TTL

Two expiry times for one cache entry: a soft TTL after which the value is refreshed in the background while still being served, and a hard TTL after which it must never be served again.

Overview

A TTL (time to live) is how long a cached copy may be used before the cache must fetch it again. Most caches have one TTL per entry, and it is a hard TTL: when it runs out, the entry is gone and the next read waits for the origin (the database or service that holds the real data).

A soft TTL adds a second, earlier deadline. After the soft TTL the entry is stale, but it can still be served: the reader gets the old value at once, and the cache refreshes it in the background. The hard TTL still sits behind it as an absolute limit on how old a served value may be.

The pair turns one sharp edge into two gentle ones. Users almost never wait for a miss on a popular key, the origin sees one refresh instead of a burst, and if the origin is down the cache keeps answering until the hard limit. HTTP has standard names for this, stale-while-revalidate and stale-if-error, and CDNs such as CloudFront and Cloudflare support them.

What you will be able to do

  • Explain what a read gets before the soft TTL, between the two TTLs, and after the hard TTL.
  • Implement a soft and hard TTL cache that refreshes each key once, and keeps the old value when a refresh fails.
  • Write the HTTP Cache-Control header for the same behavior and say what each directive means.
  • Choose the two numbers for a given kind of data, and add jitter so keys do not expire together.
  • Decide when TTLs are enough and when writes must invalidate the cache.

Key concepts

  • Hard TTL
  • Soft TTL
  • Stale-while-revalidate
  • Stale-if-error
  • Single-flight refresh
  • TTL jitter
  • Invalidation on write
  • Probabilistic early refresh

Interview questions

  • What is the difference between a soft TTL and a hard TTL?
  • How does stale-while-revalidate work, and how is stale-if-error different?
  • Implement get() for a cache with soft and hard TTLs. What must it get right?
  • Why add jitter to TTLs, and how much?
  • When should you invalidate on write instead of relying on TTLs?
  • Your origin database is down for 10 minutes. How does your cache design keep the product page working?
  • How do soft TTLs relate to the cache stampede problem?

Sign up free to read the full topic

More System Design topics

Start free on Cariara · Coding problems · System design studio