Autoscaling Strategies

The Kubernetes autoscaling toolkit, when each fits, and predictive vs reactive scaling.

Overview

Why autoscaling exists

Autoscaling automates capacity decisions. It is never the whole story, since you still need capacity planning, but it absorbs predictable variance and rare spikes.

> [!IMPORTANT] The 2026 best practice layers predictive and reactive scaling. Predictive handles the daily peak baseline; reactive handles unpredictable spikes.

The Kubernetes autoscaling toolkit

HPA (Horizontal Pod Autoscaler) — Pod count — CPU and memory by default; custom Prometheus metrics such as request rate, queue depth, latency; external cloud metrics such as CloudWatch and Azure Monitor — The most common autoscaler
VPA (Vertical Pod Autoscaler) — Pod resource requests — Observed usage; recommends or applies new requests and limits — Right-sizes services with unpredictable needs; in some modes it cannot resize a running pod without a restart
KEDA (Kubernetes Event-Driven Autoscaling) — Pod count — Kafka lag, SQS depth, Pub/Sub messages, RabbitMQ queues, PostgreSQL row counts, custom HTTP triggers — 60+ built-in scalers; standard for worker pools and batch processors
Karpenter — Nodes — Pending pods' resource requests — Built by AWS, now a Kubernetes SIG Autoscaling project; picks matching instance types; provisions in 30-60 s vs 5-10 min; aggressive bin-packing cuts cost
Cluster Autoscaler (CA) — Nodes — Pending pods — Older and generic across AWS, GCP and Azure; slower than Karpenter on AWS
Predictive autoscaling — Any — ML forecast of load — AWS Predictive Scaling, GCP Compute predictive autoscaling, custom solutions; best for daily or weekly cycles

Scaling strategies

Reactive — Scale on current metrics; standard HPA — Responds to any pattern — Lag: spike at T0, HPA reacts at T+30s, pods ready at T+90s
Predictive — Scale on forecasted load — Capacity in place before load arrives — Only for predictable patterns; a bad forecast gives wrong capacity
Scheduled — Cron: "50 pods at 8am Monday-Friday" — Simple, predictable — Does not adapt to actual load
Step — Add N pods when a threshold is crossed, then cool down — Smooth, low thrash — Slow on large spikes
Target tracking — Hold a target metric, such as 50% CPU; default in HPA, AWS and GCP — Stable, easy to reason about — Assumes the metric correlates with load

Common autoscaling failures

Slow scale-up vs spike — Spike in 30 s, HPA reacts in 90 s — Predictive scaling plus faster scrape intervals
Thrashing — Repeated up and down — Tune cooldowns and hysteresis
Downstream saturation — Scaling app servers when the DB is the bottleneck makes it worse — Scale on the real bottleneck
Cold start — New pods take 30-180 s to be ready — Plan for ramp-up
Missing dependencies — Backends scale but the load balancer, broker or DB does not — Scale the whole path

The metric that scales matters

CPU — Default, works for CPU-bound work; often misleading, since a service can burn CPU on JIT compilation while idle
Memory — Tricky; does not correlate with load for many services
Request rate — Better; direct measurement of incoming work
Queue depth — Best for async work; KEDA-friendly
p95 latency — Lagging: fires when the service degrades, not when it is busy; better as a guard rail than a primary signal

> [!TIP] Pick the metric that measures the work coming in, not the side effects of doing it.

Key concepts

HPA: Horizontal Pod Autoscaler. Scales pod count based on metric (CPU default; custom metrics common).
KEDA: Event-driven autoscaler. 60+ built-in scalers (Kafka, SQS, PubSub, etc.). Standard for event-driven workloads.
Karpenter: Faster, more aggressive node autoscaler for AWS. Provisions in 30-60s vs Cluster Autoscaler's 5-10 min.
Predictive scaling: ML-driven scaling ahead of forecasted load. Best for predictable patterns; combines with reactive.
Target tracking: Scale to maintain target metric (e.g., 50% CPU). Default for HPA, AWS, GCP.
Cooldown / hysteresis: Cooldown between scale events; asymmetric thresholds (scale up at 70%, down at 50%). Prevents thrashing.

When to use it

Designing autoscaling for a new service — pick metric, set bounds, test
Cost optimization — am I over-provisioned at off-peak?
Pre-event prep — can autoscaling react fast enough?
Postmortem — was scale-up too slow?

Interview questions

When do you use HPA, KEDA, and Karpenter together?
How do you avoid autoscaling thrashing?
When does predictive scaling help vs reactive?