Admission Control

The SRE Book's approach to handling server overload: CPU-based throttling, per-client quotas, and criticality signals that prevent overload from destroying service quality for all users.

Overview

When a server cannot even afford to say no

SRE Book Chapter 21, Handling Overload, addresses a failure mode that circuit breakers and load balancers alone cannot prevent. A server can be so saturated that it cannot process requests fast enough to serve its own error responses.

At extreme overload it spends all its capacity on requests it will reject anyway, and enters a death spiral.

> [!IMPORTANT] Chapter 21: "Servers should protect themselves from overload by explicitly throttling requests, rather than attempting to handle all requests and failing unpredictably."

The three layers

Client-side adaptive throttling — Each client tracks its recent request and accept counts and throttles new requests as errors climb, before they reach the server; SRE Book Chapter 21 gives the formula: throttle probability = max(0, (requests - K × accepts) / (requests + 1)), with K typically 2
Server-side admission — Per-client quotas; excess requests get 429 before resources are allocated, not after processing
Criticality-based shedding — When saturated, shed by criticality carried in request metadata

Shedding order

Predictable — Serve fewer users at full quality
Unpredictable — Serve all users at random quality

> [!TIP] Predictable degradation beats unpredictable degradation.

Key concepts

Client-side adaptive throttling: Client tracks its own request/error ratio and throttles new outbound requests before they reach the server, proportional to the recent rejection rate. Prevents retry storms from amplifying overload.
Per-client quota enforcement: Server tracks request rate per authenticated client and returns 429 (Too Many Requests) immediately — before resource allocation — for quota-exceeding requests.
Criticality: A metadata field on each request classifying its business importance. During overload, the server sheds lowest-criticality requests first, preserving quality for high-criticality user-facing traffic.
Load-based admission: Server-side gate that rejects new requests when CPU or queue depth exceeds a threshold — measured in utilization, not arrival rate. Prevents the overload death spiral.
Backend response estimation: When a backend is too overloaded to serve a response, returning a cached stale response or a degraded response is better than timing out — serves the user, reduces retry pressure.

When to use it

"What happens to your service when it receives 10× its normal traffic?" — walk through admission control layers
Designing a public API that must stay stable when clients misbehave or traffic spikes unexpectedly
"How do you prevent a retry storm from taking down your service?" — client-side adaptive throttling
Explaining why a server should reject requests early rather than queueing them indefinitely

Interview questions

Explain client-side adaptive throttling and why it outperforms pure server-side rate limiting during overload.