On-call Structure

How rotations work, the primary/secondary backup model, follow-the-sun for global teams.

Overview

Someone is always available

On-call is the discipline of "someone is always available to respond to production issues." How it is organized matters: bad on-call drives engineers out of SRE, and good on-call is sustainable.

> [!IMPORTANT] Google SRE Book Chapter 11: if on-call is persistently painful, that is a bug, not a fact of life. Find the noisy services, alerts and patterns and fix them.

The two-tier model

Primary — Acknowledges within ~5 min, investigates, mitigates — First
Secondary — Backup — Primary misses the ack in 10 min, parallel incidents, or primary needs help
Tertiary, manager or IC — Escalation — SEV-1, primary and secondary need help, or decision authority is needed for rollback approval or customer comms

Page routing

1 — Routed to primary via PagerDuty, Opsgenie or similar
2 — Phone call, SMS and push, escalating in intensity
3 — No ack in 5-10 min: routed to secondary
4 — No ack from secondary in 10 min: routed to the manager or on-call lead
5 — Every escalation is visible in the incident channel

Rotation patterns

Weekly — 7 days on, then N weeks off; standard for most teams — Predictable; full focus on on-call — A bad week lasts 7 days; harder to schedule around
Daily — One engineer per day; smaller teams — Short bad days; easier scheduling — Less context per shift; constant handoffs
Follow-the-sun — Rotations across US, EU and APAC daytimes — Nobody on-call at night — Needs teams in several regions; handoff complexity

PagerDuty and Opsgenie support follow-the-sun schedules natively.

Shift length

Single 7-day shift — Most common; the engineer is on for the whole week
Split shifts, such as M-W-F primary and Tue-Thu secondary — Less common; less fatigue at the cost of complexity
24-hour rotations — Only for ops-heavy teams such as NOC and support; burns engineers out

Compensation

None, assumed part of the role — Smaller US tech companies
Hourly bonus while assigned — Larger tech companies and consulting
Per-page payment — Companies with very lean on-call duty
Time in lieu, N hours of TOIL per on-call hour — European pattern

Compensation rises with load and falls with rotation length: tight rotations with heavy load pay better.

Page volume targets

Real, action required — Burnout
Solvable within 15-30 min — Lost pattern recognition
Documented with runbooks — More mistakes

Key concepts

Primary / secondary / escalation: Standard tiered model. Primary first; secondary if no ack; escalate up the chain for severity.
Follow-the-sun: Rotation across regions so no one is on at night. Requires multi-region team; tools like PagerDuty support.
Rotation pattern: Weekly (most common), daily (small teams), or split shifts. Match to team size.
Compensation: No-pay (small companies), hourly bonus (mid), per-page (lean), time-in-lieu (EU). Match to load.
Page volume target: 5 pages/shift unhealthy. Drives sustainability.
Sustainability rule: Painful on-call is a bug. Fix the noisy services / alerts; don't accept the pain as default.

When to use it

Setting up on-call rotation for a new team
Reviewing existing on-call — is it sustainable?
Hiring SREs — communicate the on-call expectations clearly
Quarterly review — page volume per engineer

Interview questions

Walk me through how on-call escalation works.
How do you implement follow-the-sun on-call?
What's a sustainable on-call load?