capacity

Turns the standard autoscaling definition into the problem it actually causes. Almost every candidate can define an HPA; far fewer can say why theirs did not save them.

Priority is one number with three separate effects: scheduling order, preemption victim selection, and eviction order under node pressure. Conflating those is the trap; designing tiers that survive both a full cluster and a hot node is the pass.

The classic napkin-math question. The number in the prompt is the question. Drawing the architecture before doing the arithmetic produces a design the physics forbids, and the arithmetic takes a minute.

Operational load is not a discipline problem. It is the interest payment on everything the team shipped without an owner, and it compounds.

Every capability a platform accepts is a permanent operational commitment. Teams that cannot say no end up operating six things badly instead of two things well.

A mixed fleet is the normal state, not a transitional one, because capacity arrives when it arrives. The failure is a scheduler that treats two very different cards as the same countable unit.

It reads as a configuration change and it is a capacity change. Cache memory per request grows with the limit, prefill cost grows with what people actually send, and concurrency falls in proportion.

Scale to zero is correct for some workloads and an outage generator for others, and the deciding number is what the first request after idle experiences. Cold start turns an availability promise into a queue.

GPUs are the default for reasons that hold for large generative models and not for everything else. A small classifier on CPU can be cheaper, more available and easier to operate, and knowing where the crossover sits is worth more than knowing the newest card.