Cost vs Reliability

Why each additional 9 costs 10×, FinOps for SRE, and the right cost-of-downtime conversation with finance.

Overview

Reliability has a cost curve

Each additional 9 of availability costs roughly 10x the engineering investment of the previous one. Understanding the curve is essential to setting SLOs that match business reality.

> [!IMPORTANT] Pick the SLO where the downtime you save is worth more than the engineering it costs, not the highest number you can reach.

The cost of 9s

The cost-of-9s framing comes from the SRE Book's Embracing Risk chapter plus industry data. Costs below are relative to 99.9%.

99% — 3.65 days — Basic redundancy, monitoring and response — Most startups, for non-critical services
99.9% — 8h 45min — Multi-AZ redundancy, a dedicated SRE function, basic automation — Any company that takes uptime seriously
99.95% — 4h 22min — Tighter dependencies, faster detection, well-practiced response; ~3-5x cost
99.99% — 52 min — Multi-region active/active or warm standby, deep automation, 24/7 NOC, formal incident command; ~10x cost
99.999% — 5 min — Cell-level isolation, formal failover, no SPOFs, reliability engineering at scale; ~100x cost — Financial settlement, life-safety, telecom
Beyond five 9s — Seconds — Specialized hardware, formal verification, redundant control planes — Undersea cables, nuclear control, large financial infrastructure

Framing the trade-off for product and finance

1 — What does downtime cost? — Revenue x (downtime / total time) for direct cost, plus churn, brand damage and SLA penalties
2 — What does another 9 cost? — About 10x the previous investment; 99.9% to 99.99% might be $1M/year of infrastructure plus 2-3 senior SRE FTEs
3 — Is it worth it? — Only when saved downtime cost exceeds the engineering cost

Worked examples

Revenue per hour — ~$11K — ~$114K
Downtime cost at 99.9% — $96K/year — ~$1M/year
Downtime cost at 99.99% — $9.5K/year — ~$98K/year
Savings from the move — $86K/year — ~$900K/year
Engineering cost — ~$1M/year plus 2-3 FTEs ($600K) — Same $1M-$1.5M
Verdict — Do not do it; stay at 99.9% — Marginal; weigh customer trust and SLA penalties

FinOps for SRE

The cloud bill itself is an SRE concern. The 2024-2026 FinOps movement names the common savings patterns.

Right-sizing — Match instance types to actual usage — 20-40%
Reserved capacity or Savings Plans — 1-3 year commitments on the baseline — 30-70% discount
Spot or preemptible — Interruptible workloads such as batch and dev/test — 60-80% discount
Storage tiering — Hot vs cool vs archive — Often 10x between tiers
Network egress — Cross-region, cross-cloud and internet egress are priced differently — Surprising for new teams
Idle resources — Forgotten dev clusters, orphan volumes, untagged resources — Audits find 5-15%

> [!TIP] The Bezos test: if a cost reduces customer pain, pay it. If it adds capability, pay it only when the value exceeds the cost.

Cost monitoring discipline

Attribution — Per-service cost via tagging and namespace-based billing
Reviews — Monthly cost review per service
Anomaly detection — A sudden 2x spike usually means runaway autoscaling or runaway logs
Ownership — Cost is a budget that SRE owns like an SLO

Key concepts

Cost of 9s curve: Each additional 9 costs ~10× the previous. From 99.9% to 99.99% is a 10× investment.
Cost of downtime: Direct revenue lost + indirect costs (churn, brand, SLA penalties). The metric that justifies reliability investment.
FinOps: Discipline of cloud-cost management. Right-sizing, reserved capacity, spot instances, tiered storage.
Bezos test: Cost that reduces customer pain: do it. Cost that adds capability: only if value > cost.
Reserved capacity: Commitment to specific instance types for 1-3 years. 30-70% discount vs on-demand. For predictable baseline.
Spot / preemptible instances: 60-80% discount; can be reclaimed at any time. For interruptible workloads (batch, dev/test, ML training).

When to use it

Setting SLO targets — pick the SLO that matches business cost-of-downtime
Annual budgeting — plan reliability investments against revenue impact
Cost reviews — am I paying for reliability we don't need?
Finance conversations — translate reliability investments to dollar terms

Interview questions

How do you decide whether to target 99.9% vs 99.99%?
What's the most common cloud-cost mistake SRE teams make?
How do you set up cloud-cost attribution per team?