TL;DR: 500 reports an unexpected server condition; 502 reports an invalid upstream response at a gateway; 503 reports temporary inability to serve; 504 reports an upstream response deadline exceeded at a gateway. Their concrete causes and emitters require evidence.
How to approach it
Say who emits the code first, because that is the diagnostic value. A 500 can come from any server component that encounters an unexpected condition; locate its emitter. A 502, 503 or 504 usually comes from the load balancer or ingress, and the application may have logged nothing at all, which is the thing that confuses people into believing the logs are broken.
A strong answer
500 Internal Server Error. The responding server encountered an unexpected condition. In an application, examples include an unhandled exception, a null dereference, a failed database call with no error handling. If the application generated it, its request logs or error telemetry may contain the cause. Check the application log for that request, ideally by trace ID.
502 Bad Gateway. A proxy tried to reach your application and got something it could not use: connection refused, connection reset, a malformed response, or the process died mid-response. Common causes are the application not listening on the port the proxy expects, the container crashing on startup so nothing is behind the service, a worker process being killed while handling the request (often the OOM killer), or a protocol mismatch such as the proxy speaking HTTPS to a plaintext listener. In Kubernetes a 502 frequently means the pod is running but the container is not actually serving yet.
503 Service Unavailable. The proxy had nothing to send the request to, or deliberately refused it. In Kubernetes this is the classic "no ready endpoints" case: pods exist, readiness probes are failing, so the Service's endpoint list is empty. It also comes from a proxy's own queue being full, from a circuit breaker opening, or from rate limiting. A 503 does not establish whether an upstream attempt occurred; inspect the emitting component's reason.
504 Gateway Timeout. A gateway did not receive a timely upstream response, potentially including a timeout establishing the connection. This is the one that points at your application being slow rather than broken: a query with no index, a downstream call with no timeout of its own, a thread pool exhausted so the request queued. The number to check immediately is the proxy's timeout against your application's own p99, and the reason is arithmetic. A p99 near 28 seconds warrants inspecting the slower tail, but it does not determine the fraction beyond a 30-second deadline.
Small fractions are not small counts. At two million requests a day, even a tenth of a percent is 2,000,000 × 0.001 = 2,000 timeouts daily, spread with no pattern anyone can see, which is exactly why this presents as random and why the application logs show nothing, since those requests completed successfully from its point of view a few seconds after the proxy gave up.
The margin is what you are really setting. A timeout must fit the caller's deadline and resource budget; a multiple of p99 is not a universal safety rule. Pick it from the measured distribution rather than from the round number the config template came with.
The practical sequence:
# is it every request or some?
kubectl logs -n prod deploy/ingress-nginx --tail=100 | grep ' 50[234] '
# does the Service have anything behind it?
kubectl get endpointslices -n prod -l kubernetes.io/service-name=checkout
# is the app itself logging the request at all?
kubectl logs -n prod deploy/checkout --since=10m | grep <trace-id>
If the ingress logs show the code and the application logs show nothing, investigate both the request path and logging coverage; missing application logs alone do not prove non-arrival.
Two others worth knowing because they get confused with these. 499 in nginx logs means the client gave up and closed the connection first, which usually means you are slower than the caller's timeout rather than failing. And 429 is rate limiting, which is a deliberate refusal rather than a fault.
Identify the emitter before assigning a cause
In a hypothetical request path, the CDN, edge proxy and application can each generate an error response. Correlate a request ID across their logs and inspect the component-specific reason code. A 503 generated by application overload differs from an edge 503 caused by an empty eligible backend set.
A 504 can also occur during connection establishment; it does not prove the application accepted the connection. If application logs are absent, check sampling, buffering and logging configuration before concluding the request never arrived. Use proxy connection and upstream timing fields according to that proxy's documented definitions.
Set deadlines from the user operation's budget and measured stage costs, including retries and cancellation. A p99 of 28 seconds alone cannot tell you how many requests exceed 30 seconds; inspect the distribution beyond that boundary. Raising the gateway timeout may merely leave more work running after the client has given up. HTTP semantics defines these codes without making them unique root-cause diagnoses.
What interviewers probe next
"502 or 503, which worries you more?" Neither status alone establishes impact scope. Check the proportion of requests affected before deciding which is worse.
"Why would 504s appear only at peak?" Queueing. Utilisation rises, requests wait for a worker, total time crosses the proxy timeout. The application may be processing at the same speed per request; the waiting is what changed.
"How do you find which hop is slow?" Compare the proxy's upstream response time with the application's own request duration. If the proxy sees 8 seconds and the application reports 200 milliseconds, the time is in connection setup, queueing or the network between them.
Common mistakes
Treating all 5xx codes as one condition, which discards the information about which hop failed.
Searching the application log after a 503 and concluding logging is broken, when the request never reached the application.
Raising the proxy timeout to stop 504s, which hides a slow path and moves the failure to the client.
Ignoring readiness probes as a cause of 503, which is the most common explanation in Kubernetes.