autoscaling

Turns the standard autoscaling definition into the problem it actually causes. Almost every candidate can define an HPA; far fewer can say why theirs did not save them.

Fleet-wide replacement is a traffic-shaping problem: bake once, wave through in controlled batches, watch the alarms, and keep the exit open. In-place patching is the alternative and usually the regret.

Guaranteed capacity and discounted capacity are different products. Peak season needs both, arranged in layers and verified with a game day before the traffic arrives.

A specific configuration trap with a clean explanation. Both controllers act on the same signal from opposite directions, which can produce unstable feedback depending on their policies and workload.

Utilization-based scaling is the wrong sensor when work arrives as backlog. The design questions are threshold math against drain rate, reaction-time bounds from polling, and why lag-driven fleets oscillate without generous targets and idempotent consumers.

Kubelet resource metrics to metrics-server to aggregated API to HPA controller: every hop adds lag and averaging. Walking the path and naming where it distorts demand separates operators from dashboard readers.

Changing pod resources historically required recreation, which made vertical autoscaling disruptive for workloads with expensive recovery. In-place resize changes that, with limits worth knowing.

The Indian e-commerce interview question, and autoscaling is the wrong first answer. A 10x spike arriving in seconds outruns any scaling loop, so the design has to absorb rather than scale.

The difference between scaling node groups and provisioning nodes is the whole answer. One picks from shapes you defined in advance; the other reads the pending pods and buys what they need.

The fastest-growing design round in infrastructure hiring. GPU economics break the assumptions CPU autoscaling is built on, and the answer has to start from that rather than from Kubernetes.

CPU memory intuition fails on inference servers. The scarce resource is KV-cache, requests arrive with unseen context lengths, and the fix is queue-depth scaling plus admission control, not bigger pods.

Requests-per-second is not the load unit, cold starts are minutes not seconds, and scale-in can kill paying users mid-sentence. Everything you know about HPA needs re-deriving here.

Separate scheduling delay from image pull, artifact transfer, GPU loading and warm-up. The autoscaler needs the full request-to-ready time.

Scale to zero is correct for some workloads and an outage generator for others, and the deciding number is what the first request after idle experiences. Cold start turns an availability promise into a queue.