Timeouts & Deadlines

Why every external call needs a timeout, deadline propagation across services, and choosing the right values.

Overview

Every external call needs a timeout

This is non-negotiable. A call without a timeout can hang forever, and under load, hung calls are how services collapse.

SRE Book Chapter 22: "RPCs inherit and decrease deadlines as they descend the stack so a server doesn't waste cycles on a request whose caller already gave up."

> [!IMPORTANT] Pass the deadline down the stack and cancel downstream work when it expires. No work should happen after the user has given up.

The default-no-timeout problem

Many HTTP clients in Python, Java and Node default to no timeout or a very long one, so a call can wait minutes or hours. Each hung call holds a thread; hold 100 threads for 30 minutes and the service is unavailable.

Choosing timeout values

The right timeout is just longer than the call's p99 latency, with a small buffer. If a dependency's p99 is 200ms, set 500ms-1s: tight enough to detect hangs, loose enough for legitimate slow calls.

DB query — 1-5 s
Cache lookup — 50-200 ms
Internal microservice RPC — 100 ms-2 s, depending on the call
External third-party API — 5-30 s
Long-running batch operation — 30-300 s, or async with status

The timeout pyramid

When A calls B, A's timeout must exceed B's total downstream waits, or A times out before B can detect its own failure.

A times out at 1 s; B calls 5 downstreams with 0.5 s timeouts each — B may spend 0.5 s x 5 = 2.5 s plus processing
A gives up at 1 s and retries — B is still working; the work doubles

Deadline propagation

gRPC — Built in via context: grpc.WithDeadline(time.Now().Add(1 * time.Second)); the server reads it for downstream calls
HTTP — Less standard; some use an X-Deadline header with an absolute timestamp
OpenTelemetry — Trace context can carry deadline information; an emerging standard

Hard vs soft timeouts

Behavior — Kill the operation at the deadline; the call returns an error — Cancel, but the operation may still finish in the background
Used for — Synchronous operations — Some async patterns
Risk or pairing — The operation may have succeeded while the response was late — Pair with idempotency keys, so a retry returns the same result

The cancellation cascade

A times out at 5 s, but its calls to B, C and D have no timeout and keep running, holding threads. A returns an error to the user while its thread pool is still spent on abandoned calls.

In practice, pass a cancellation context; downstream calls check it periodically and abort if canceled.

Timeouts as configuration

Keep timeouts in configuration, not hard-coded, so they are tunable per environment, dependency and release.

When a dependency gets faster, slower or has a new version, tune without redeploying.

Common mistakes

No timeout, the most common — HTTP client defaults hang forever
Too long, such as 5 minutes — Threads still hang for minutes
Too short, such as 50ms against a 100ms p99 — False-positive timeouts, and retries amplify load
One timeout for every dependency — Ignores different latency profiles
No cancellation propagation — Timed-out calls still consume downstream resources

Key concepts

No timeout = no protection: A call without timeout can hang forever; threads exhaust; service collapses. Mandatory.
Right timeout value: Just longer than dep's p99, with buffer. Tight enough to detect hangs; lax enough for legitimate slow calls.
Deadline propagation: Caller passes deadline; downstream services use it for their own timeouts. SRE Book Ch 22 verbatim.
Cancellation cascade: When a call times out, cancel all downstream in-flight calls. Free resources immediately.
Hard vs soft timeout: Hard: kill operation. Soft: cancel but allow background completion. Idempotency keys make soft safe.
Timeout pyramid rule: A's timeout > sum of B's downstream waits. Otherwise A times out while B is still working.

When to use it

Every single external call — timeouts are mandatory
Reviewing existing code — search for HTTP clients without timeout config
Postmortem with hung-thread component — was timeout missing or wrong?
Microservice design — propagate deadlines via gRPC or context

Interview questions

Walk me through deadline propagation in a microservice call.
How do you choose a timeout value?
I've added timeouts everywhere. What ELSE is needed?