Infra Notes

Kubernetes Pod Pending: troubleshoot insufficient CPU and memory

Diagnose Kubernetes Pods stuck Pending by reading scheduler events and checking requests, taints, affinity, storage, quotas, and autoscaler capacity.

Published

KubernetesPod PendingInsufficient CPUInsufficient MemoryScheduler

Problem

A Pod stays Pending because the scheduler cannot find an eligible node—or because a required volume has not been provisioned. The Pod phase alone does not distinguish capacity, placement, policy, and storage failures.

Quick answer

Read scheduler events first:

bash

kubectl describe pod <pod> -n <namespace>
kubectl get events -n <namespace> \
  --field-selector involvedObject.name=<pod> \
  --sort-by='.lastTimestamp'

A message such as 0/3 nodes are available: 3 Insufficient cpu means no eligible node currently has enough unallocated requested CPU. It does not necessarily mean node CPU utilization is 100%.

Requests determine scheduling

The scheduler evaluates resource requests, not live dashboard utilization. Inspect every container, including init containers:

bash

kubectl get pod <pod> -n <namespace> \
  -o jsonpath='{range .spec.containers[*]}{.name}{" cpu="}{.resources.requests.cpu}{" memory="}{.resources.requests.memory}{"\n"}{end}'
kubectl describe nodes

Compare requests with node allocatable capacity and already allocated requests. Correct an unrealistic request only when workload measurements justify it; lowering requests merely to force placement can create node pressure later.

Other scheduler constraints

If capacity appears available, examine the rest of the event:

  • untolerated taint means the Pod lacks a matching toleration.
  • didn't match Pod's node affinity/selector means placement rules exclude the nodes.
  • pod anti-affinity rules may prevent replicas from sharing a topology domain.
  • unbound immediate PersistentVolumeClaims means storage binding blocks scheduling.
  • Namespace ResourceQuota or LimitRange may reject or alter resources before scheduling.

Check these inputs explicitly:

bash

kubectl get nodes --show-labels
kubectl describe node <node>
kubectl get resourcequota,limitrange -n <namespace>
kubectl get pvc -n <namespace>

When cluster autoscaling should help

An autoscaler can add a node only if a compatible node group or provisioner exists and the pending Pod would fit on a possible node. Impossible affinity, an unavailable instance type, exhausted cloud quota, subnet IP shortage, or a PVC constraint can prevent scale-out.

Inspect autoscaler or Karpenter logs and events after confirming the scheduler reason. Do not start there; otherwise an invalid Pod specification can be mistaken for an autoscaler defect.

Prevention

Measure real workload usage, set explainable requests, test topology rules, monitor allocatable headroom, and alert on unschedulable Pods with their reason. Capacity planning should include DaemonSet overhead and system reservations.

If the Pod is scheduled but restarts, use Kubernetes CrashLoopBackOff debugging. For autoscaling design, read the Kubernetes platform engineering blueprint.

References