DevOpsInterviews logo
← Platform Engineering & FinOps / 04
mediumAmazonDatadogUber

Your cluster autoscaler adds nodes slowly and they are the wrong size. What would you change?

The difference between scaling node groups and provisioning nodes is the whole answer. One picks from shapes you defined in advance; the other reads the pending pods and buys what they need.

Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Cluster Autoscaler scales pre-defined node groups, so it can only add more of a shape you chose in advance and it simulates whether pending pods fit the available group templates before selecting a group. Karpenter reads the pending pods directly and provisions an instance sized for them, then consolidates when utilisation drops. Compare scheduling-to-ready latency and node cost under the actual constraints.

How to approach it

Explain what each one is actually deciding, since the difference in outcome follows entirely from that. Then cover consolidation, which is where the cost saving comes from and which candidates usually omit.

A strong answer

Cluster Autoscaler works with node groups: an autoscaling group per instance type and zone, each with a min and max. When pods are Pending it simulates which group would fit them and increases that group's desired count. The constraints follow: you must define the shapes in advance, a group is homogeneous, and scaling means launching another of that shape whether or not it fits the workload well. A pod needing 2 cores and 32GB on a cluster of general-purpose nodes gets a node with far more CPU than it needs, and you pay for the whole thing.

Karpenter removes the group. It watches unschedulable pods, computes their combined requirements including affinities and topology constraints, and provisions an instance chosen from a broad set of permitted types that actually fits. Fewer, better-sized nodes, and no advance combinatorial planning of groups across types and zones.

Worked example: where the money actually is

The shape difference is easier to judge with a workload attached. Take 100 pods each requesting 2 vCPU and 32 GB, which is memory-heavy, and illustrative prices rather than current ones.

Under Cluster Autoscaler with a general-purpose node group of 16 vCPU and 64 GB, each node fits 64 / 32 = 2 pods before memory runs out, so 100 pods need 50 nodes. Those nodes use 2 × 2 = 4 of their 16 vCPU, which is 25 percent, and you are billed for all of it: 50 × 12 = 600 vCPU provisioned and idle, because the group's shape was chosen before the workload existed.

Karpenter picks the instance from the pods. A memory-optimised type at 8 vCPU and 64 GB still fits 2 pods, so the node count is unchanged at 50, but each node now wastes 8 − 4 = 4 vCPU instead of 12. At illustrative hourly prices of $0.77 and $0.50, the fleet moves from 50 × 0.77 = $38.50 to 50 × 0.50 = $25.00 an hour, about 35 percent, or roughly 13.50 × 730 ≈ $9,855 a month for a workload nobody changed.

That is the provisioning half. Consolidation is the other half and it addresses a different waste: fragmentation that accumulates after the fact, as deployments scale down and pods reschedule, leaving nodes at 60 percent packed. Fifty nodes at that density hold work that fits on 50 × 0.6 = 30, so the saving is available only to something that revisits the decision rather than only reacting to Pending pods.

Two things the arithmetic does not settle. It assumes the memory-optimised type is available in your region and zones at the moment you need it, which is the constraint that makes instance-type breadth valuable in its own right. And it ignores the churn cost of consolidation, which is real and lands on workloads rather than on the bill.

Two mechanisms matter beyond the initial provisioning.

Consolidation is the cost lever and it is the part people miss. Karpenter continuously evaluates whether the current pods could run on fewer or cheaper nodes, and if so it cordons, drains and replaces. That reclaims the fragmentation that accumulates as workloads come and go, while Cluster Autoscaler can also remove underutilized nodes when their pods can move elsewhere. It is also the behaviour you must understand before enabling it, because it deliberately moves running pods: PodDisruptionBudgets and graceful termination are prerequisites rather than nice-to-haves, and a do-not-disrupt annotation can restrict voluntary disruption. It cannot prevent forced Spot reclamation; verify the configured termination and disruption policies.

Spot handling. Karpenter can select across many instance types, which materially improves spot availability because you are not tied to one shape, and it handles interruption notices by draining ahead of reclamation. Pool diversity reduces concentration risk; fallback capacity, quotas and interruption recovery still matter.

The honest limits. It is tightly coupled to the cloud provider's API, so it is not a portable abstraction. Giving a controller permission to launch instances is a real privilege that needs scoping. And consolidation churn can surprise a team that has never seen pods move for reasons unrelated to deployment, which is a communication problem as much as a technical one.

See the Cluster Autoscaler FAQ and Karpenter disruption documentation for the deployed versions.

Where Cluster Autoscaler still fits: environments with strict node standardisation for compliance reasons, or clusters small and static enough that the complexity is not worth it.

What interviewers probe next

"Why is scheduling faster?" No group selection step and no waiting for an autoscaling group to reconcile. It calls the instance API directly for a shape it already knows fits.

"How does this interact with requests?" Completely. Karpenter provisions against requests, so inflated requests buy larger instances than the workload needs. Right-sizing requests is a prerequisite for the cost benefit, not an alternative to it.

"What breaks if you enable consolidation blindly?" Anything without a PodDisruptionBudget, anything with long graceful shutdown, and anything stateful that assumed it would stay put.

Common mistakes

Assuming one controller always scales faster. Both evaluate fit and can remove unneeded capacity; their provisioning models and supported constraints differ.

Enabling consolidation without PDBs, which produces disruption nobody expected.

Expecting savings while requests are wrong. It will faithfully buy capacity for what you asked for.

That one was free, and so are 18 answers per topic without an account. Signing in doubles that to 28, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

Nothing here yet. Say how you would answer it.