inference

The fastest-growing design round in infrastructure hiring. GPU economics break the assumptions CPU autoscaling is built on, and the answer has to start from that rather than from Kubernetes.

Per-token pricing looks expensive until you compute what an idle GPU costs at 3am. Break-even depends on workload volume and duty cycle, measured throughput, and the engineering cost on both sides.

GPUs are the default for reasons that hold for large generative models and not for everything else. A small classifier on CPU can be cheaper, more available and easier to operate, and knowing where the crossover sits is worth more than knowing the newest card.