Retry, Jitter & Backoff

The exponential-backoff formula, why jitter is mandatory, and AWS's "full jitter" recommendation.

Overview

Retries: simple idea, subtle implementation

Retries are the most common cause of cascading failure. The intuition, "if a request fails, try again," is fine; the implementation is subtle.

> [!IMPORTANT] Retry only idempotent operations, bound the attempts, and always add jitter.

The three knobs

1. Retry count — Bounded or unlimited — Bounded, typically 2-3 attempts; unlimited retries cause retry storms
2. Backoff — Linear (N s x attempt, slow ramp), exponential (base × 2^attempt, standard), fixed (rarely correct) — Exponential
3. Jitter — None or random — Without jitter all clients retry together, a thundering herd; with jitter retries spread out

Backoff formulas

Plain exponential — wait = base × 2^attempt — 100ms, 200ms, 400ms, 800ms, 1600ms
Full Jitter (AWS recommendation) — wait = random(0, base × 2^attempt) — random(0, 100ms), random(0, 200ms), random(0, 400ms), ...
Equal Jitter — wait = (base × 2^attempt / 2) + random(0, base × 2^attempt / 2) — 50-100ms, 100-200ms, 200-400ms; slower than full jitter, same spreading
Decorrelated Jitter (AWS) — wait = min(cap, random(base, prev × 3)) — 100, then 100-300, then 100-900; slightly faster but harder to reason about

The AWS Architecture Blog post "Exponential Backoff and Jitter" found that random offsets reduce load on the failing service. Full jitter spreads retries best.

The retry-storm anti-pattern

A service is degraded for 5 seconds. With naive backoff, 10,000 clients retry at exactly T+200ms, then T+400ms, and each pulse is larger than the original load. Every backoff window ends with another tsunami of requests.

Idempotency requirement

Idempotent — Same result no matter how many times it runs — PUT /users/123 sets state; GET reads
Non-idempotent — Each call has a side effect — POST /payments charges; POST /tweets posts

Idempotency keys — Client generates a unique key; server deduplicates by it
Conditional updates — If-Match headers proceed only if state is unchanged
Do not retry — For some financial operations, fail and let the user retry manually

Where to retry

Client side: browser, mobile, internal service client — Network failures, 5xx, 429
Service side: proxies and gateways — Transient errors such as network and timeout, not application errors like most 4xx
Anti-pattern — Retrying on the same proxy that made the original call amplifies load on the same path

HTTP retry guidance

5xx, except 501 and 505 — Retry; the server says it is temporary
429 Too Many Requests — Retry per the Retry-After header
408 Request Timeout — Retry
Other 4xx — Do not retry; it is a client error and will repeat
Network errors: ECONNRESET, timeout — Retry
3xx redirects — Follow, do not retry

The retry budget

A service should have a global retry budget: at most N retries per second across all clients. Without one, retries during an incident can saturate even healthy capacity.

Key concepts

Exponential backoff: wait = base × 2^attempt. Standard formula. NEVER use without jitter.
Full Jitter (AWS): wait = random(0, base × 2^attempt). Mathematically optimal for spreading retries. Industry standard.
Retry storm: Synchronized retries multiply load on a recovering service; prevent it from recovering. Solved by jitter.
Idempotency: Operation can be retried safely. Required for retries; non-idempotent ops need idempotency keys or no retry.
Retry budget: Global cap on retries per second. Token bucket. Prevents retry waves from exhausting healthy capacity.
Bounded retries: 2-3 attempts max. Unbounded is the canonical retry-storm cause.

When to use it

Designing every retry policy — explicit jitter is non-negotiable
Reviewing existing retries — are they bounded, jittered, idempotent?
Postmortem with retry-storm component — was the policy correct?
Service mesh / API gateway config — global retry settings affect every service

Interview questions

Why is jitter mandatory for retries?
How do you handle retries for non-idempotent operations?
What's a "retry budget" and when do you need one?