Why your Kubernetes cluster wastes 40% of its budget — and how to fix it

TL;DR

• Most Kubernetes clusters run at 35-50% real utilization — the rest is requested-but-unused CPU and memory that you’re paying for at full price.
• Four leaks account for almost all of it: over-requested resource limits, poor bin-packing, idle non-prod environments, and orphaned storage/load balancers.
• Karpenter (or Cluster Autoscaler with care), VPA in recommendation mode, and a deliberate spot/on-demand mix typically recover 30-50% of compute spend in 4-8 weeks.
• None of this requires touching your application code or accepting worse reliability — it’s almost entirely a scheduling and rightsizing problem.

The problem: your cloud bill says one thing, your dashboards say another

Here’s a pattern we see in nearly every cost audit: the finance team sees a Kubernetes compute bill that’s grown 3x in eighteen months. The platform team’s Grafana dashboards show CPU utilization hovering around 30-40%. Both are telling the truth — and the gap between them is the budget leak.

The mechanism is simple and almost invisible day-to-day. Engineers set resource requests defensively — “2 CPU, 4Gi memory” for a service that uses 400m and 800Mi on a normal day — because nobody wants to be the reason a pod gets OOMKilled at 2am. The Kubernetes scheduler honors those requests when placing pods, which means your cluster autoscaler provisions nodes to fit the requested capacity, not the used capacity. You end up paying for headroom that exists only on paper.

Multiply that across 200 services, three environments, and a scaling event or two, and a cluster that should cost $40K/month costs $110K/month — and nobody can point to the exact moment it happened, because each individual decision looked reasonable.

The framework: four leaks, four fixes

1. Over-requested resources (the biggest leak, usually 15-25% of spend)

Run the numbers: compare kubectl top pods actual usage against resources.requests across your workloads for two weeks of real traffic, p95 not average. In most clusters we audit, requested CPU runs 2-4x actual p95 usage.

Fix: Deploy the Vertical Pod Autoscaler in recommendation mode (not auto — you don’t want it restarting pods unsupervised on day one). Let it observe for two weeks, then work through services in batches, starting with the highest-spend, lowest-risk ones. Most teams can safely cut requests by 30-50% on the majority of services without a single incident.

2. Poor bin-packing (5-15% of spend)

Default cluster autoscaler behavior tends to spread pods across nodes rather than consolidate them — which means you’re running six half-empty m5.2xlarge nodes instead of three full ones. Each “half-empty” node still costs full price.

Fix: Replace (or augment) Cluster Autoscaler with Karpenter. It provisions right-sized nodes for the pods that are actually pending, consolidates aggressively when utilization drops, and supports mixed instance types natively — so it can pick the cheapest instance that fits, not just the first one that’s available. Teams moving from Cluster Autoscaler to Karpenter typically see node count drop 20-35% with the same workload.

3. Idle non-production environments (5-10% of spend)

Dev and staging clusters running 24/7 at the same size as production is one of the most common — and most fixable — leaks. Nobody is running load tests against staging at 2am on a Saturday.

Fix: Scheduled scale-down (a CronJob that scales deployments to zero replicas outside business hours, or a tool like Kube-downscaler) for non-prod namespaces. This alone often recovers 60-70% of non-prod compute cost — roughly 5-8% of total cluster spend for a typical multi-environment setup.

4. Orphaned storage and load balancers (1-5% of spend, but pure waste)

PersistentVolumeClaims and LoadBalancer-type Services that outlive the workloads that created them. They don’t show up in compute dashboards, so they’re invisible until someone audits the cloud bill line-by-line.

Fix: A monthly automated sweep — kubectl get pvc --all-namespaces cross-referenced against running pods, and the same for orphaned ELBs/NLBs via cloud provider tags. Fifteen minutes of scripting, recurring savings.

Real-world example

This is exactly the pattern behind our AWS Cost Optimization case study — a SaaS company whose AWS bill had grown from $35K to $180K/month over 18 months, with engineering headcount growing far slower than spend. The audit found the same signature we see repeatedly: CPU requests running well above actual p95 usage, nodes sitting at low utilization, and non-production environments running full-size around the clock.

A ten-week engagement — tagging and cost dashboards first, then right-sizing, Spot migration, and storage/networking cleanup — brought the bill to $65K/month. That’s a 64% reduction within 90 days, sustained for the following two quarters, with zero customer-facing incidents.

Trade-offs and what we’d avoid

• Don’t run VPA in auto mode on stateful or latency-sensitive services on day one. It restarts pods to apply new resource values, which is fine for stateless web services and risky for anything with in-memory state or strict startup-time SLAs.
• Spot instances are not free lunch. They’re excellent for batch jobs, CI runners, and stateless services with good graceful-shutdown handling — and a bad fit for anything that can’t tolerate a 2-minute interruption notice. Mix capacity types deliberately by workload class; don’t blanket-apply spot and hope.
• Don’t chase the last 5%. The first 30% of savings costs you a few weeks of focused work. The last 5% often costs more in engineering time and operational risk than it returns. Know where to stop.

What to do next

1. Run the comparison — actual p95 usage vs. requested resources, two weeks of data, per-namespace. This single number tells you how big your leak is before you spend a dollar on tooling.
2. Start with VPA in recommendation mode on your top 10 highest-spend services. No risk, immediate visibility.
3. Talk to us about a structured cost audit — our Cloud Cost Optimization engagements typically pay for themselves within the first quarter, and our Kubernetes practice handles the Karpenter migration and bin-packing work end to end.

If you want a second pair of eyes on where your cluster spend is actually going, book a 30-minute call — we’ll tell you, roughly, what we’d expect to find before we start.

Related reading: GPU node pools on Kubernetes — five sharp edges to know if your waste is concentrated in AI/ML workloads rather than general compute.