Requests are what the scheduler believes, limits are what the kernel enforces

TL;DR: Resource-fit scheduling uses requests. On Linux, CPU requests also influence relative CPU weights under contention; limits bound CPU time and memory usage through cgroups. Requests too high strand capacity. Requests too low overload nodes the scheduler thinks are fine. The declarations affect QoS class, while node-pressure eviction also depends on actual usage and priority.

Requests: a claim, not a measurement

When a pod is created, the scheduler filters nodes with enough unreserved capacity and scores what remains. It reads requests. The default resource-fit calculation uses declared requests rather than live utilization; placement also considers other constraints and scheduling plugins.

That single fact explains a great deal. A fleet where every pod requests four cores and uses one will look full at 25 percent real utilisation, until you right-size requests or add capacity. Invert it and the scheduler happily packs a node whose pods collectively want far more than exists, then the kernel sorts it out, badly.

So requests are the most consequential number in a manifest, and they are usually copied from another manifest.

Limits: enforcement, and the two resources behave differently

CPU is compressible. When the cgroup exhausts its CPU quota, runnable work is throttled until quota is available again. The period is commonly 100ms but is configurable. CPU quota enforcement itself does not kill a process. Latency does, in a way that hides from average CPU graphs because most periods end under quota.

At a memory limit, the kernel first attempts reclaim. If it cannot satisfy an allocation within the cgroup boundary, a cgroup OOM can kill processes; it is not an immediate kill for every transient approach to the limit. Cgroup v2 also supports memory.high reclaim pressure and throttling, whose use depends on runtime and cluster configuration.

For a predictable service working set, equal memory requests and limits can reserve capacity conservatively. Bursty workloads may justify a lower request with a measured limit; budget aggregate node demand and the resulting eviction risk. CPU limits are contested, and the honest position is that they cost latency to buy predictability, so they belong where you have multi-tenant or cost-attribution reasons and not by reflex.

The gap between them is a policy

With container-level CPU and memory declarations, QoS classes are:

• Guaranteed requires a nonzero CPU request/limit pair and memory request/limit pair, equal within each pair, for every container.
• BestEffort has no CPU or memory requests or limits.
• Burstable covers the remaining configurations. A missing request can also default from a declared limit, so inspect the admitted pod.

QoS is not a strict eviction queue. For resource pressure, kubelet considers whether usage exceeds requests, pod priority, then usage relative to requests. Inode and PID pressure have no corresponding requests, so priority matters directly. Guaranteed pods can still be evicted when the node cannot recover. Check version-specific rules when using pod-level resource declarations.

A pod with no resource fields is not neutral, it is volunteering. That is worth saying to anyone who thinks omitting the section is the safe default.

Where the numbers should come from

Observed usage, not intuition. Request near the sustained working set with headroom for real variance, using a percentile from actual data rather than the peak, since a request sized for the once-a-quarter peak strands capacity for the other three months.

The mechanism for getting that data without letting anything act is a vertical autoscaler in recommendation mode. It watches, it suggests, it changes nothing. Comparing its recommendations against what teams actually requested is usually the fastest way to find out whether your cluster is oversubscribed or half empty, and the answer is rarely what people expect.

The failure this explains

Nodes at 90 percent while others sit idle. Pods evicted on a node with free memory. A cluster that cannot schedule anything while its real utilisation is 30 percent. p99 latency spikes on a service whose CPU graph is flat.

Compare requests, limits and observed demand when investigating these symptoms, alongside placement constraints and node-pressure signals. Similar symptoms can have different causes.

Worked example: scheduling fits, runtime demand does not

A node has eight allocatable CPUs. Eight pods each request one CPU and have a two-CPU limit. Their requests fit exactly, so resource-fit scheduling can admit that placement if other constraints allow it. If all eight demand two CPUs at once, the node cannot supply sixteen CPUs. Individual limits are ceilings, not a reservation of simultaneous capacity.

Now change each request to two CPUs while observed steady usage remains one. Only four such pods fit by requests, leaving apparent spare CPU on the dashboard. That may be deliberate headroom for correlated bursts, or it may be over-allocation. Historical averages alone cannot distinguish those cases; inspect peak concurrency, latency and the consequence of contention.

Include rolling-update surge and node-loss recovery in the capacity calculation. A service that fits in steady state may have nowhere to place replacements during maintenance. VPA recommendations provide evidence about observed usage, but they do not decide the required failure headroom or application latency objective for you.

Self-check

A cluster has 20 nodes with 8 allocatable CPUs each. Every pod in it requests 2 CPUs, 80 pods are running, and nothing new will schedule. Metrics show sustained CPU usage of about 400m per pod. Separately, a Guaranteed pod on one of those nodes has just been evicted, and its memory usage was below its limit. Work out the real utilisation, what right-sizing requests would buy, why the Guaranteed pod was evicted anyway, and what you would refuse to conclude from a vertical autoscaler's recommendations. Work it through before reading on.

The capacity arithmetic first. The cluster has 20 × 8 = 160 allocatable CPUs, and 80 pods requesting 2 each have claimed 80 × 2 = 160, so every CPU is reserved and the scheduler correctly reports no room. Actual demand is 80 × 0.4 = 32 CPUs, which is 32 / 160 = 20 percent. The cluster is full and four fifths idle at the same time, and both statements are true because they are about different numbers: the scheduler is reading a claim about the future and the dashboard is reading the present.

Right-sizing is therefore the largest lever available and it costs nothing but attention. Requesting 600m, which is the observed 400m with 50 percent headroom for variance, gives 160 / 0.6 ≈ 266 pods on the same hardware, against 80 today. That is more than three times the density, and the honest caveat is that 600m has to come from a percentile of real data rather than from the mean, because a request sized at the average is under-provisioned half the time by construction.

The Guaranteed eviction is the part that surprises people, and the mistake is treating QoS as an eviction queue. Guaranteed means requests equal limits for every container, which buys the strongest position under node pressure and not immunity. Kubelet evicts when the node itself cannot recover, and it considers usage relative to requests together with pod priority, so a Guaranteed pod can be evicted when the node is under pressure and there is nothing better to reclaim. The other likely explanation is that the pressure was not memory at all: inode and PID pressure have no corresponding request to be protected by, so the class provides no protection from them whatsoever. That is where the investigation starts, with which signal the kubelet actually reported.

On the recommender, the refusal is specific. A vertical autoscaler in recommendation mode is evidence about observed usage, and observed usage is exactly what the 80 pods above were missing, so it is the right instrument for the first problem. What it cannot decide is how much headroom the failure case needs. Sizing requests to the recommendation leaves no room for a rolling update's surge or for the loss of a node, and a cluster that fits perfectly in steady state may have nowhere to place replacements during routine maintenance. The recommender also knows nothing about a latency objective, so it will happily recommend a request that keeps a service alive and slow. Take the usage figure from it and take the headroom from the failure you intend to survive.

Next: CFS quotas and throttling is what the limit half of the pair actually enforces, and saturation and utilisation targets supplies the headroom arithmetic the recommender cannot give you.

Use QoS and eviction evidence to distinguish a pod removed by node pressure from a container OOM or CPU throttling.