Risk vs Feature Velocity

The economic + UX argument for setting reliability targets below 100%, and how to make the tradeoff legible to product.

Overview

The tension

Maximizing reliability and maximizing feature velocity pull against each other. The SRE response is not to pick a winner. It is to make the tradeoff quantitative through SLOs and error budgets, so both sides negotiate against the same number.

> [!IMPORTANT] SRE Book Ch 3, verbatim: "Extreme reliability comes at a cost: maximizing stability limits how fast new features can be developed and dramatically increases costs." Ch 1's epigraph: "Hope is not a strategy."

The cost curve

Each additional 9 costs roughly 10× the engineering investment of the previous one. This is an industry rule of thumb, not a Google quote.

The user-side limit

Ch 3, verbatim: "a user on a 99% reliable smartphone cannot tell the difference between 99.99% and 99.999% service reliability."

  • User's device and network — Often the weakest link for consumer products
  • Your service — Cannot exceed the weakest link
  • Five 9s on flaky mobile users — Money spent to no effect

Who decides risk tolerance

  • Product — Owns the decision: how much downtime can we afford, what does a 30-minute outage cost?
  • SRE — Provides the operational reality: what is achievable without burning out the team?

> [!TIP] Risk tolerance is a business question. SRE informs it; SRE does not decide it alone.

Key concepts

Hope is not a strategy
SRE Book epigraph. The discipline replaces hope with measurable targets and explicit policies.
Cost of a 9
Each additional 9 of availability costs roughly an order of magnitude more than the previous (industry rule-of-thumb). 99.9% → 99.99% is much cheaper than 99.99% → 99.999%.
Chain of reliability
A service's effective reliability is bounded by the reliability of every component in the request path. A 99.99% service running on a 99.9% smartphone is a 99.9% experience.
Reliability tax on innovation
Every reliability investment is engineering time NOT spent on features. The cost is not just dollars; it's velocity.
Risk-tolerance ownership
Set jointly by product (business cost of downtime) and SRE (operational sustainability), not by SRE alone.

When to use it

  • Pushing back on "we need 100%" from product or executive leadership
  • Defending the velocity tradeoff implied by a tighter SLO
  • Sizing a service's SLO based on user perceptibility rather than abstract perfection
  • Setting expectations during dependency-chain reliability conversations

Interview questions

  • Why is 100% reliability not the right target?
  • Product wants 99.999% on a service that has dependencies running at 99.9%. What do you say?
  • How do you make the reliability-vs-velocity tradeoff legible to product or executive leadership?

Sign up free to read the full topic

More SRE topics

Start free on Cariara · Coding problems · System design studio