This is the third year we’ve published our report on the real CPU and memory utilization in Kubernetes clusters. CPU utilization fell to 8%, down from 10% last year. Memory dropped from 23% to 20%. This year, we added GPU utilization to the mix – and across the clusters we analyzed, it stood at just 5%.
Note that these aren’t estimates. They’re direct measurements from tens of thousands of production clusters running on AWS, GCP, and Azure before any optimization was applied.
The expectation is that things will improve each year as cloud usage and Kubernetes footprints mature. Our report shows this simply isn’t true.
The overprovisioning problem is structural, not accidental
CPU overprovisioning jumped from 40% to 69% year over year. Memory overprovisioning sits at 79%. Organizations are paying for infrastructure that their workloads don’t even request.
The mechanics are simple: teams pad resource requests to avoid throttling and OOM evictions, the cost of that padding is invisible to the team in charge of the platform, and there’s no systematic process to revisit those definitions after deployment.
Helm charts use conservative estimates across services. Cluster autoscalers respond to inflated requests as if they were genuine demand, provisioning nodes to match the demand. The gap becomes structural.
The counterintuitive part is reliability.
The common assumption is that more headroom means fewer crashes. It doesn’t work that way. One cluster we looked at averaged 40–50 OOM kills per measurement interval with generous resource padding. After automated rightsizing was deployed – which also cut provisioned CPUs by roughly half – OOM kills dropped to near zero.
The rightsizing agent increases memory limits for workloads under genuine pressure, which is the part that static overprovisioning consistently misses.
The lesson? You don’t have to choose between efficiency and reliability. Automated rightsizing delivers both because the same mechanism that eliminates waste also captures workloads humans miss.
GPUs deserve a separate conversation
The GPU economics are very different. An idle CPU core costs cents per hour. An idle GPU costs dollars. And for the first time since EC2 launched in 2006, GPU prices are rising, not falling. AWS raised H200 Capacity Block prices by 15% in January 2026. That broke a two-decade precedent.
At an average utilization of 5%, the math doesn’t work. And the hoarding instinct – holding capacity you might not get back – feeds the scarcity loop that drives prices higher.
Spot adoption for GPU workloads was essentially nonexistent through most of 2025 – fewer than 2% of GPUs ran on Spot. Some of that is availability: for most of last year, Spot capacity for GPU instances simply wasn’t there. That started changing in early 2026 for lower-end hardware.
T4s in specific US regions are showing survival rates above 90% over 30 minutes.
The regional variance in Spot reliability is worth understanding. For T4s on AWS, eu-west-3 maintains a survival probability above 0.9 for a full 24-hour window. eu-central-1 and us-east-1 drop below 0.2 in the same window – roughly an 80% chance of interruption within a day.
Region selection for GPU Spot workloads is a reliability decision. The same instance type, in a better region, is the difference between a training run that completes and one that requires constant recovery.
By selecting the most favorable US region in any given period, teams could achieve 2–5x cost differences on Spot pricing alone. No team can monitor that in real time manually. This is exactly where automation creates practical value.
GPU sharing is well understood, and almost nobody uses it
The standard deployment model gives each model its own dedicated GPU instance. For most inference workloads, that’s wasteful – request rates are bursty, with long idle periods between them. On dedicated instances, those idle periods cost the same as full utilization.
What GPU sharing actually looks like in practice: ALLEN Digital was running 7 models on SageMaker: 3 open-source and 4 custom. GPU instances ran continuously but served an intermittent load.
After moving to Kubernetes with GPU time-slicing enabled, a 50/50 on-demand/Spot split, and node bin-packing, utilization improved dramatically. This led to 20% savings immediately from time-slicing, 30–40% after consolidating models onto shared instances, and more than 70% total savings versus SageMaker after rightsizing CPU and memory alongside the GPU changes. Latency held throughout.
High GPU utilization is achievable
One cluster in our dataset – 136 H200s sustaining 49% GPU utilization – shows the ceiling isn’t theoretical. The fleet average is 5%. The gap is 10x. That gap is almost entirely technique, not hardware.
What the organizations that closed the gap did differently
They automated continuously:
- Not a one-time rightsizing pass at deployment – continuous monitoring and adjustment of resource requests against actual consumption and workload optimization.
- Not manual Spot placement decisions – automated selection across instance pools, availability zones, and regions, with fallback to on-demand when availability drops.
- Not one-time GPU sizing decisions – shared instances with intelligent scheduling that places multiple workloads based on actual compute and memory needs.
The data challenges a narrative we hear constantly: that overprovisioning is the price of reliability. It isn’t. The teams that stopped overprovisioning didn’t get less reliable; they got more reliable. And they’re paying significantly less.
The inefficiencies described in this report are not new. They’re last year’s story, compounded. The pattern is consistent: Kubernetes adoption scales, efficiency declines proportionally, and the gap between what organizations pay for and what they consume keeps widening. That trajectory doesn’t correct itself.
The organizations that closed the gap didn’t wait for it to improve on its own – they built systems that continuously enforce efficiency, just as they build systems that enforce uptime. Treat it as an operational property, not a periodic project, and the numbers move.