DevOpsInterviews logo
← Incident Response & Production Debugging / 04
easy★ EssentialFlipkartTCSAccenture

A pod has been Pending for ten minutes. Walk me through your diagnosis.

The gentlest scheduling question there is, and it still separates people who read the events from people who guess. Check whether the pod has a node before choosing between scheduling and startup diagnosis.

Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Pending means the pod has been accepted but its containers are not yet fully set up. Check PodScheduled and spec.nodeName: an unbound pod needs scheduling diagnosis; a bound pod may be pulling images, mounting volumes or running initialization. Events and container status identify the branch.

How to approach it

Say what Pending means before saying what you would run, because the meaning bounds the search. Then go to events first. Logs help only when a container has started, including an init container; an unscheduled pod has no container logs.

A strong answer

Read the full pod description, including its assigned node, conditions, init-container status and events. If spec.nodeName is empty and PodScheduled is false, focus on scheduling constraints. If a node is assigned, inspect image pulls, volume setup, the container runtime and init containers on that node. The phase remains Pending during these startup steps.

The events say why:

kubectl get pod api-7f9c -o wide
kubectl describe pod api-7f9c
# 0/6 nodes are available: 3 Insufficient cpu, 2 node(s) had untolerated taint,
# 1 node(s) had volume node affinity conflict.

That message is the answer, and it is worth reading carefully because it reports failed scheduling checks and affected node counts. A node can fail multiple checks, so the counts are not necessarily disjoint groups.

Insufficient CPU or memory means the incoming pod's requests, combined with already allocated requests, do not fit the node's allocatable resources. Requests, not limits, and not usage, and the gap between those is wide enough to be worth a number. A node with 8 allocatable cores running 8 pods that each request 900m has 8 × 0.9 = 7.2 cores committed, leaving 8 − 7.2 = 0.8 for anything new. If those pods actually use 100m each, real utilisation is 8 × 0.1 / 8 = 10 percent. The dashboard shows a node that is 90 percent idle and the scheduler correctly refuses to place a pod requesting one core, because it is reading the claim rather than the usage. The fix is right-sizing requests or adding capacity, based on measured demand and workload constraints.

An unsatisfiable selector or required affinity can involve missing node labels, conflicting pod placement rules or topology constraints. Check the pod's selector against kubectl get nodes --show-labels, and expect a typo or a node group that was scaled to zero.

An untolerated taint means the only nodes with room are reserved. Control plane nodes and GPU pools are the usual ones. Either tolerate it deliberately or schedule elsewhere.

A volume conflict means the PVC is bound to a PersistentVolume in one availability zone and the only free capacity is in another, or the PVC is not bound at all because no StorageClass matched. This is the one that looks most mysterious and it is common on StatefulSets.

Finally, if the cluster autoscaler is present, Pending is sometimes correct and temporary: it is the signal that triggers a scale-up, and the pod stays Pending for the provisioning time. Check whether a node is being added before treating it as a fault.

Pending means two different things, and the first check separates them UNSCHEDULED SCHEDULED, NOT STARTED 1 Is a node assigned this splits the whole problem 2 No node: read the message it counts nodes, per reason 3 Insufficient resources against requests, not usage 4 Untolerated taint a placement decision, not capacity 5 Volume affinity conflict the volume is in another zone 6 Node assigned: pull registry, credentials, size 7 Then volumes attach, then mount 8 Then init containers each to completion, in order Pending covers both an unscheduled pod and a scheduled pod whose containers are still being set up. They share a phase and share nothing else, so this check is the whole diagnosis. The scheduler's message is specific and people skim it. It names how many nodes failed for each reason, which is a complete answer to "why not this cluster" in one line.

Explain why adding the wrong node changes nothing

Assume the pod requires zone A because its bound volume has that node affinity. All zone-A nodes lack the requested memory, while zone B has ample room. A new node in zone B cannot satisfy the complete placement requirements. Inspect the PersistentVolume's affinity and the eligible node pool together before requesting capacity.

The same reasoning applies to an autoscaler. Look for its decision on this pod and pool, including pool limits or unavailable instance capacity. “Autoscaling is enabled” does not establish that it can provision a suitable node. Avoid relaxing volume affinity to force a bind; the storage may be physically inaccessible from the other zone.

After capacity arrives, follow the pod beyond scheduling. A populated node name followed by a FailedMount event means placement succeeded and the current failure has moved to volume setup. Report that transition explicitly so responders stop changing scheduler settings and investigate the attachment path.

What interviewers probe next

"The node shows 40 percent CPU used but the pod will not schedule. Explain." Scheduling is on requests, monitoring shows usage. The gap between them is the whole answer, and it is the most common misunderstanding on this topic.

"How would you prevent this class of problem?" Sensible defaults through LimitRange, requests derived from observed usage rather than guessed, and alerting on pods Pending beyond a threshold so you find out before a user does.

"What if events are empty?" Events can expire or be unavailable. Check pod conditions, scheduling gates and any custom schedulerName, then inspect the responsible scheduler if the pod is unbound. For an assigned pod, inspect kubelet and container status instead.

Common mistakes

Treating every Pending pod as unscheduled, or asking for application logs before checking whether a container has started.

Confusing requests with limits, which turns the entire answer inside out.

Adding nodes as the reflex fix. Check requests and placement constraints before buying capacity that cannot resolve the actual restriction.

That one was free, and so are 18 answers per topic without an account. Signing in doubles that to 28, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

Nothing here yet. Say how you would answer it.