AIOps Root Cause Analysis

How AIOps systems decompose root cause analysis into failure localization, failure category classification, and natural-language report generation. Covers open-source libraries, benchmarks, commercial tooling, and the critical distinction between correlation-based candidate surfacing and proven causal analysis.

Overview

Root cause analysis in three subtasks

Root cause analysis (RCA) is one of the most commercially valuable and technically difficult problems in AIOps. Research in arxiv 2406.11213 splits it into three subtasks, each narrowing the problem for the next.

Failure localization — Not "auth service is slow" but "pod auth-service-7x4k2 in us-east-1a is the origin of the latency spike" — Causal graph traversal, anomaly score propagation, trace-based analysis, from service down to instance
Failure category classification — CPU shortage, memory shortage, software configuration error, network issue, disk I/O saturation, connection pool exhaustion — Multiclass classification
Root cause report generation — Timeline, affected components in dependency order, probable cause with confidence, evidence chain, next steps — LLM generation; where LLMs add the most value

> [!NOTE] The category picks the fix: CPU shortage leads to horizontal scaling or limit review; connection pool exhaustion leads to pool size tuning or query optimization.

PyRCA: one API for metric-based RCA

PyRCA (arxiv 2306.11417, open-sourced by Salesforce at github.com/salesforce/PyRCA) gives every metric-based RCA algorithm for microservices the same interface, so you can swap algorithms without changing pipeline code.

Causal discovery — PC (Peter-Clark) — Constraint-based; tests conditional independence between variables
Causal discovery — GES (Greedy Equivalence Search) — Score-based; optimizes a scoring function over the graph space
Root cause scoring — Bayesian Network Inference — Propagates anomaly probability through the graph
Root cause scoring — Random Walk — Propagates anomaly scores along graph edges
Root cause scoring — Hypothesis Testing — Statistical tests rank candidate nodes

RCAEval: the standard benchmark

RCAEval (arxiv 2412.17015) provides 15 reproducible baselines in three families. It scores each at two granularities: coarse-grained (which service) and fine-grained (which metric on which service).

Knowledge graph RCA

KGroot and similar approaches propagate anomaly signals over the service dependency graph. Unlike purely statistical methods, the graph encodes the actual direction of dependencies, which makes these methods more reliable in large, well-documented microservice architectures where topology is known.

Correlation is not causation

Most production tools find temporal and topological correlations, not proven causal chains.

> [!IMPORTANT] A correlation tool cannot prove which service started the problem. "Root-cause assistance" or "candidate surfacing" is the more honest framing.

Commercial tooling

Honeycomb BubbleUp — Surfaces dimensions where failing requests differ from the baseline population
Datadog Watchdog Insights — Surfaces recent deploys and config changes correlated with the anomaly window
Dynatrace Davis — Problem tree via Smartscape topology graph traversal; deterministic, the most topology-aware commercial approach
Splunk ITSI — Rule-based event correlation with configurable temporal windows

Interview questions

Walk through the three-subtask RCA taxonomy and explain why each subtask requires a different technical approach.
How does PyRCA separate the causal discovery problem from the root cause scoring problem, and why does that separation matter?
Explain the correlation versus causation problem in AIOps RCA and how it affects how you should interpret commercial tool outputs.
Compare the metric-based, trace-based, and multi-source RCA approaches in the RCAEval benchmark. When should you prefer each?