Design for Understandability

Building Secure and Reliable Systems Ch 6 — designing systems whose behavior is predictable, whose failure modes are obvious, and whose mental models do not decay over time.

Overview

You cannot fix what you cannot understand

*Building Secure and Reliable Systems* Chapter 6, Design for Understandability, argues that the main cause of extended incidents is not technical complexity but operational incomprehensibility. The on-call engineer cannot diagnose and remediate what they cannot understand.

> [!IMPORTANT] Design APIs that are hard to misuse. A function whose caller must understand implementation details will eventually be misused in production.

Three properties of understandable systems

Invariants — Properties that should always hold, in a checkable form — "The number of active sessions should never exceed the number of active users"; a violation points to a specific class of bug
Mental models — The engineer's internal picture of how the system works — A good design's model stays correct as the system evolves; a bad one is correct only when its author built it
Least surprise — The system behaves as an experienced engineer expects from its interfaces and docs — No hidden state transitions, no implicit side effects, nothing explainable only by reading the implementation

Practical design guidance

Explicit state machines — Implicit state spread across correlated global variables
Typed errors, distinct per failure mode — Opaque error codes or catch-all exceptions
Synchronous operations where correctness needs it — Async flows that are hard to reason about under failure
APIs that are hard to misuse — APIs that require implementation knowledge

Key concepts

System invariant: A property that should always be true about the system state, expressible as a checkable assertion. Violated invariants indicate a specific bug class and narrow diagnosis to the component responsible for the invariant.
Mental model: The engineer's internal representation of how the system works. A reliable system has a mental model that remains accurate as the system evolves. Complexity that invalidates mental models causes diagnostic errors under incident conditions.
Least surprise: The system behaves as an experienced engineer would expect based on its interfaces and documentation. No hidden state transitions, implicit side effects, or behavior explainable only by reading the implementation.
Explicit state machine: System state managed through a formal state machine with named states and named transitions. State is auditable, transitions are logged, and invalid transitions are rejected — vs. implicit state through correlated global variables.
Typed errors: Distinct error types for distinct failure modes, rather than opaque error codes or catch-all exceptions. Enables callers to handle specific failure modes programmatically and on-call engineers to diagnose failures without reading source code.

When to use it

"Why is this system so hard to debug when it breaks?" — use understandability principles to audit the design
Reviewing a new system design for operability before it reaches production
"What makes a system easy or hard to operate at 3am?" — mental models and invariants
Explaining why simplicity is a reliability property, not just an engineering aesthetic

Interview questions

What makes a distributed system "understandable" and why does understandability reduce MTTR?