Phi Accrual Failure Detection

A failure detector that reports a suspicion level instead of alive or dead, computed from each node's own heartbeat history, so every application can pick how sure it needs to be before it acts.

Overview

A classic failure detector answers one question with yes or no: is this node alive? It usually works with a fixed timeout, and the timeout bakes one trade-off into the detector for every user at once. The phi (φ) accrual failure detector, published by Naohiro Hayashibara, Xavier Défago, Rami Yared and Takuya Katayama in 2004, changes the interface. It outputs a number, phi, that keeps rising the longer a node stays silent, measured against how that node's heartbeats have arrived recently. Each application picks its own threshold.

Phi has a direct meaning: it is the minus log (base 10) of the chance that the next heartbeat would arrive this late or later. A threshold of 1 accepts roughly a 10% chance of being wrong, 2 about 1%, 3 about 0.1%, and so on. Akka Cluster and Apache Cassandra both use a phi detector for cluster membership.

The Heartbeats and Failure Detection topic introduces phi alongside timeouts and SWIM. This topic goes further: why the accrual interface matters, how the detector is measured and tuned, the different probability models real systems use, and what the threshold turns into in seconds.

What you will be able to do

  • Explain the accrual idea and why it separates monitoring from decisions.
  • Compute phi from a window of heartbeat gaps, under a normal and an exponential model.
  • Convert any threshold into the timeout it really implies.
  • Tune the window, the threshold and the guards, using detection time and mistake rate.
  • Say where phi helps and where it does not.

Key concepts

  • Accrual vs binary detectors
  • Suspicion level phi
  • Sampling window
  • Detection time and mistake rate
  • Equivalent timeout
  • Normal vs exponential model
  • Acceptable pause
  • Several thresholds, several actions

Interview questions

  • What is an accrual failure detector, and why is it better than a boolean one?
  • How is phi computed?
  • Cassandra's phi_convict_threshold is 8. How long does it actually wait?
  • Why does raising the threshold from 8 to 12 change so little in Akka?
  • How would you choose a threshold for a new system?
  • What problems does phi accrual not solve?
  • How does phi relate to SWIM and gossip?

Sign up free to read the full topic

More System Design topics

Start free on Cariara · Coding problems · System design studio