Walmart Global Tech Incident Response & Production Debugging interview questions

The gentlest scheduling question there is, and it still separates people who read the events from people who guess. Check whether the pod has a node before choosing between scheduling and startup diagnosis.

Intermittent DNS is the most-reported and least-understood Kubernetes failure. There are three well-known causes and each leaves a different fingerprint.

Tests the mitigate-before-diagnose reflex under time pressure, plus whether you know the two ways a disk can be full and the trap that makes freed space not come back.

An idle database with a starved application is the tell. The connections are held, not used. More database capacity does not release leaked connections.

One disk metric explaining a whole cluster's weirdness is the point of this question. Restarting etcd first turns a slow cluster into a down one, because a restarted member has to catch up before it can vote.

An unchanged pipeline can fail because of environment state, external inputs or a latent artifact defect. Failures that start on an expiry boundary narrow the field to two suspects fast.

Nothing errors because dropping unmatched lines is documented behaviour. Counting the tagged failures converts an invisible leak into a number, and numbers have owners.

The backup is doing its job and the database is paying for it. Attribution first, then enforcement that actually binds on modern kernels, because priority support depends on the active kernel and IO scheduler.

Duplicate-IP claims need evidence, and one command provides it. Then the real skill: telling a genuine two-owner clash apart from ARP flux, because the fixes could not be more different.

Bandwidth graphs look healthy and the partner swears they are fine. The wall is ports, not bits: the available source-port budget depends on the gateway and its address allocation.

Exit codes said green while the warehouse went hollow. Where detection should have sat, why retries could not help, and the backfill discipline finance will actually accept.

Stateful firewalls keep established flows alive after their rule vanishes, so cleanup day passes every check and lag appears days later. Flow logs name the hour; inventories prevent the repeat.

A health endpoint that returns 200 because the process is running tells you the process is running. Interviewers want to hear where you draw the line between a check that pages and a check that removes traffic.

One partition has one consumer, by design. If the key distribution sends a third of the traffic to one partition, the parallelism you bought is unavailable to the work that needs it.

A deceptively deep question, because there are at least six places a version can get stuck and they span the pipeline, the cluster and the edge. Work the path in order.

Status codes describe an HTTP failure category. Locate the component that emitted the response, then use its logs to distinguish connection, capacity and application failures.

Three places, and which one depends on how the thing was started. Knowing which logs the runtime collects and which the application writes itself, and that journald may not persist across reboots, saves the first twenty minutes.

A common scenario question, and the one where a list of commands most often stands in for an argument about what each one eliminates. Empty logs is the clue, not the obstacle.

The pods are Running, the dashboard is green, and users get errors. The gap between Running and receiving traffic is where this question lives, and it has a precise answer.

A senior favourite, because the naive answer is to look at averages and the average is exactly what is hiding the problem. Tail latency has a short list of causes and they are all structural.