DevOpsInterviews logo
← Platform Engineering & FinOps / 03
mediumUberDatabricksSnowflake

Finance wants to know what each team costs on a shared Kubernetes cluster. How do you answer them?

A shared cluster produces one bill and no answer. Attribution is a measurement problem first and a cultural one second, and the number that changes behaviour is not the one finance asked for.

Grounded in researched DevOps, SRE and platform engineering interview loops, written to a senior-engineer editorial bar, and never padded to hit a word count.

TL;DR: Define allocation explicitly: an OpenCost-style model uses the maximum of requested and used resources, with documented unit rates and shared-cost treatment. Requests consume scheduler accounting capacity; ordinary CPU requests do not reserve exclusive physical cores. Show allocated cost and utilization together, then reconcile to the bill.

How to approach it

Separate the measurement question from the behaviour question, because they have different answers and the second is the one that matters. Then be honest about the allocation choice, since there is no perfect one.

A strong answer

The measurement first. A node costs a fixed hourly rate whether or not the pods on it are busy, so the question is how to divide that cost among pods. Two defensible bases:

Requests affect scheduling admission. A request of four CPUs can stop other pods being scheduled even if the pod uses one, but those idle CPU cycles are not necessarily inaccessible to already-running pods. Report this gap as a scheduling-efficiency signal rather than exclusive core ownership.

Usage shows consumption above or below requests. Use a declared allocation basis such as the maximum of request and usage over each interval, with separate CPU and memory rates. This is the basis documented by the OpenCost specification. Reconcile workload, idle and shared allocations so they do not double-count the same node cost.

Then the parts that are not workload:

  • Idle capacity: nodes running with nothing scheduled, plus the headroom you deliberately keep. Someone owns this and it is usually the platform team, because it is a platform decision. It is also larger than people expect, so compute it rather than assuming it rounds away: 40 nodes at $0.50 an hour is 40 × 0.5 × 730 = $14,600 a month, and if pods request 5 of each node's 8 allocatable CPUs then 1 − 5/8 = 37.5 percent of that, about 14,600 × 0.375 = $5,475 a month, belongs to no workload at all. Spreading it across teams by usage makes the platform's own capacity decisions invisible and inflates every team's unit cost by a factor they cannot influence; holding it as a named line makes headroom a reviewable choice.
  • System overhead: kube-system, the CNI, log shippers, the mesh. Distribute proportionally rather than absorbing it, or the platform team's bill grows with everyone else's usage and nobody sees it.
  • Non-compute: load balancers, persistent volumes, and cross-zone data transfer, which is frequently the surprise line item on a chatty multi-zone estate.

Mechanically this needs labels that are enforced rather than requested. A namespace per team with required labels, and an admission policy that rejects workloads without an owner label, because a voluntary convention produces an unallocated bucket that grows until it is the largest line on the report. OpenCost or Kubecost implement the arithmetic; the labelling discipline is the part you have to own.

Now the half that actually reduces spend. A monthly report sent to finance changes nothing, because finance cannot resize a deployment. The number has to reach the team that controls it, weekly, next to something they can act on: your requests are 3.2x your p95 usage, here are the five deployments driving it. Chargeback (real budget transfer) works in some organisations and creates gaming in others; showback (visibility without transfer) is usually enough and is far less politically expensive.

Dividing one node's bill among the pods that share it THE WORKLOAD SHARE EVERYTHING ELSE 1 The node bills hourly busy or idle, the same 2 Pick an allocation basis and declare it out loud 3 Per interval, per pod CPU and memory, separately 4 Sum to workloads then to teams 5 Idle capacity remains and somebody owns it 6 Headroom is a decision so it belongs to the decider 7 Shared services divided on a rule people agreed 8 Reconcile to the bill once, and only once Requests affect scheduling admission: a request for four CPUs can stop other pods landing even when the pod uses one. Report that gap as a scheduling-efficiency signal rather than as exclusive ownership of idle cores. Nodes running with nothing scheduled, plus the headroom you deliberately keep. Someone owns this, and it is usually the platform team, because it is a platform decision. Workload, idle and shared allocations have to reconcile against the node cost without double-counting it. A model whose parts do not sum to the invoice is an argument waiting to happen with finance.

What interviewers probe next

"How do you attribute a spot interruption or an autoscaler event?" Use events to explain capacity changes, but do not assign the entire shared-node cost to whichever pending pod happened to trigger scale-up. Allocate the node over its lifetime under the agreed usage/request and shared-capacity policy. An interruption is a reliability event, not a separate cost allocation unit.

"What is the first thing you would fix?" Request accuracy, almost always. The gap between requests and usage is typically the largest single recoverable number on the report and it needs no architectural change.

"How do you stop teams gaming it?" The stated max(request, usage) allocation limits under-requesting as a billing tactic. Separately monitor contention and eviction risk; the allocation formula does not enforce resource safety.

Common mistakes

Allocating on usage, which produces a comfortable report and no behaviour change.

Ignoring shared overhead, so the platform team absorbs a cost that grows with everyone else's decisions.

Reporting to finance rather than to engineers. The bill goes to the person who can move the number.

That one was free, and so are 18 answers per topic without an account. Signing in doubles that to 28, keeps your bookmarks, and tracks which topics you keep getting wrong.one Google click · no card · nothing to cancel
HOW DID IT GO?
0
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

Nothing here yet. Say how you would answer it.