Pods Stuck Pending
Scheduler events, insufficient resources, taints/tolerations, affinity rules, and PVC binding failures.
Overview
Overview
A pod stuck in Pending means the scheduler cannot find a node to place it on. The scheduler checks every node against the pod's requirements (resources, taints, affinity, topology constraints) and filters out nodes that fail any of them, then scores the rest and binds the pod to the best one. If no node qualifies, the pod stays Pending indefinitely.
Checks and what each tells you
- kubectl describe pod, Events section — Exactly why no node fits, for example "0/5 nodes are available: 3 Insufficient cpu, 2 Insufficient memory", with a count per failed constraint
- kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints — The taints on every node
- PVC status — Whether the claim the pod needs is still Pending
> [!TIP] "0/5 nodes are available: 3 Insufficient cpu, 2 Insufficient memory" is the most common message, and it tells you how many nodes failed each constraint.
Causes
- Resource exhaustion — No node has enough CPU or memory left to satisfy the pod's requests
- Taints and tolerations — Every node has a taint (e.g., node-role.kubernetes.io/control-plane:NoSchedule) and the pod has no matching toleration
- NodeAffinity and PodAffinity — No node matches the required labels, or spreading constraints cannot be satisfied
- PVC in Pending — The PVC is not yet bound to a PV: no PV matches (wrong storage class, insufficient capacity, wrong access mode) or dynamic provisioning failed
> [!IMPORTANT] The scheduler places pods by requests, not limits or real usage. A node at 30% actual CPU but 100% CPU request allocation still rejects new pods with CPU requests.
Key concepts
- Resource requests (scheduling)
- Scheduler uses requests (not limits) for bin-packing. A node can run pods that exceed its requests in practice, but scheduler will not place a pod on a node where requests exceed allocatable.
- Taint/toleration
- Taints on nodes repel pods without matching tolerations. Common: node.kubernetes.io/not-ready, node-role.kubernetes.io/control-plane. Check with kubectl describe node | grep Taint.
- PodDisruptionBudget (PDB)
- Minimum number of pods that must be available. If eviction would violate PDB, the scheduler will not evict the pod even during a drain. Can block node drain indefinitely.
- TopologySpreadConstraints
- Spread pods across zones, nodes, or other topology keys. If constraints cannot be satisfied (too few nodes in a zone for the required maxSkew), pods are Pending.
- Cluster Autoscaler
- Watches for Pending pods and adds nodes to satisfy requests. If CA is enabled but pods remain Pending for >5 minutes, check CA logs for why it is not scaling up.
When to use it
- Diagnosing why a deployment scales up but new pods stay in Pending state
- Explaining why pods are Pending after adding taints to all nodes
- Debugging a pod that requires a specific node label that no current node has
- Diagnosing why a StatefulSet pod is Pending because its PVC is Pending
Interview questions
- A pod is stuck in Pending and kubectl describe shows "0/3 nodes are available: 3 Insufficient memory." How do you resolve it?
Sign up free to read the full topic
More Troubleshooting topics
- Missing Traces in Distributed Tracing
- Multi-Region High Latency
- NAT Gateway Bandwidth and Throttling
- Network Bandwidth Saturation
- OOMKilled Pod Diagnosis
- Packet Loss and Network Degradation
- PVC Mount Failures in Kubernetes
- RDS CPU at 100%
- RDS Read Replica Replication Lag
- S3 High Latency Spikes
- Slow Query Diagnosis
- Staging Passes, Prod Fails
Start free on Cariara · Coding problems · System design studio