Online and Continual Learning

Incremental learning from streaming data, catastrophic forgetting, concept drift detection, and strategies for real-time model adaptation.

Overview

The case for online learning

Most production systems retrain in batches: collect data, retrain, evaluate, deploy, repeat. That is enough for weekly recommendations or monthly fraud updates. When the data distribution changes faster than the retraining cycle, as in real-time fraud detection, CTR prediction for news or dynamic pricing in spot markets, the model needs to update as observations arrive.

Problem — Keep up with a changing data stream — Learn a sequence of tasks without losing old ones
Update — One example or mini-batch at a time — Train on each new task
Main risk — Concept drift — Catastrophic forgetting

Online learning algorithms

Online algorithms update parameters incrementally without revisiting past data. The key property is O(1) memory: each example is seen once and discarded, so unbounded streams are fine.

Online Gradient Descent (OGD) — One SGD step per example with a decaying learning rate
Vowpal Wabbit (Microsoft) — Production-grade logistic regression, contextual bandits and neural networks; sub-millisecond updates per example
River (Python) — Naive Bayes, Hoeffding Trees and adaptive ensembles for data streams

Catastrophic forgetting

Fine-tuning a neural network on a new task with standard gradient descent overwrites the weights that encoded the old task. This is catastrophic forgetting (or catastrophic interference).

Three ways to prevent forgetting

Elastic Weight Consolidation (EWC) — Adds a loss term penalizing changes to parameters important for past tasks; importance = Fisher information — Shown to keep 90%+ of previous task performance
Progressive Neural Networks — New column of neurons per task, linked to frozen old columns by lateral connections — Zero forgetting by construction; size grows linearly with tasks
Experience Replay — Buffer of past examples mixed into each mini-batch — Reservoir Sampling keeps a fixed-size uniform buffer; Dark Experience Replay (DER) also stores soft labels (logits)

Concept drift

Concept drift is a change over time in the statistical properties of the data stream.

Sudden — Abrupt shift — A new fraud pattern appears overnight
Gradual — Slow change — Seasonal purchasing behavior
Recurring — Cycles — Weekday versus weekend traffic
Incremental — Steady one-way shift — A monotonic trend

ADWIN (adaptive windowing) tests whether the recent part of a sliding window has a significantly higher error rate than the older part, and drops pre-drift data when it does.

ADWIN — Compares error rates in sub-windows of an adaptive window — Error-rate drift in streams
Page-Hinkley — Sequential test on cumulative sums — Continuously monitored metrics
DDM (Drift Detection Method) — Watches the online learner's error rate — Warning and drift alarms on significant increases

Online versus batch retraining

Labels arrive in seconds to minutes (clicks, transactions, sensor readings) — Labels are delayed by weeks or months
The task changes faster than the retraining cycle — Quality must be checked on a held-out test set before deploy
Memory cannot hold a growing training set — The architecture needs full-dataset statistics (normalization layers, tree splits)

> [!TIP] Many production systems combine both: a daily or weekly batch retrain of the base model, plus continuous online updates to a small set of user-level or context-specific parameters.

Interview questions

Design an online learning system for CTR prediction in a real-time ad serving platform that handles millions of events per second.
Explain the three main approaches to mitigating catastrophic forgetting in continual learning and discuss their practical tradeoffs.
How do you detect and respond to concept drift in a production fraud detection model?

More MLOps & LLMOps topics

Start free on Cariara · Coding problems · System design studio