LLM-Powered AIOps

Large language models address the two core structural limits of traditional AIOps ML models -- inability to generalize across systems and inability to handle multiple task types -- and introduce a five-level auto-remediation taxonomy from assisted questioning through fully autonomous execution.

Overview

LLMs as a layer on top of AIOps

LLM-powered AIOps applies large language models to operational intelligence. It does not replace ML-based anomaly detection and alert correlation; it is a layer that fixes two structural limits traditional AIOps has had since the category began.

The two structural limits

Cross-platform generality — A detector trained on Netflix's metrics does not transfer to Uber's. Each org collects labels, tunes thresholds, retrains and keeps one model per environment — Smaller organizations without ML teams, which cannot do this at all
Cross-task flexibility — Each model does exactly one task. The anomaly detector cannot mine log patterns; the log clusterer cannot answer a runbook search — Anyone running the full stack: N separate models for N tasks

LLMs address both at once. They arrive pretrained on ops documentation, Stack Overflow, GitHub infrastructure-as-code, incident post-mortems and vendor runbooks, and they switch tasks by prompt instead of by model change.

The five-level auto-remediation taxonomy

Research in arxiv 2406.11213 defines five levels of LLM involvement in remediation. Each level hands the model more of the work.

1. Assisted questioning — Hypotheses for "what could cause high p99 latency on this service?" — Engineer investigates; LLM takes no action
2. Mitigation solution generation — Prioritized options: rolling restart of auth pods, check connection pool settings, verify upstream database health — Engineer picks the path
3. Command recommendation — kubectl rollout restart deployment/auth-service -n prod — Engineer reads, validates and copies it
4. Script generation — A complete, runnable remediation script — Engineer reviews, then executes; approval is the critical gate
5. Automatic execution — Generated and executed fix — LLM, end to end from detection through fix to verification

The AgentOps paradigm

AgentOps moves AIOps from "assist humans" to "act autonomously" by giving an LLM agent the whole incident lifecycle.

Reliability tiers in practice

Not every LLM use in AIOps performs the same. The deciding factor is how much ground truth the model has for the task.

1 — The LLM organizes structured input into narrative prose instead of inferring causes it cannot observe, so hallucination risk is low
2 — Output is usually right but needs a human check before use
3 — No ground truth about your infrastructure, and the model cannot signal uncertainty

Guardrails for Level 4-5 automation

Earn trust: human approval for the first 20 executions of any new automation class.
Blast radius: no production database schema changes without a multi-step approval chain.
Rollback first: every remediation script ships with a rollback script before execution is allowed.
Audit: log every LLM-generated action taken.

> [!WARNING] Level 5 removes the human gate entirely. Keep it for bounded, well-characterized incident classes that have already passed the approval period.

Interview questions

Explain the two structural limitations of traditional AIOps ML and how LLMs address each.
Walk through the five-level LLM auto-remediation taxonomy and identify where human approval should be mandatory.
What is the AgentOps paradigm and how does it differ architecturally from traditional AIOps?
Explain the three reliability tiers for LLM AIOps features and why each tier behaves the way it does.