Karpenter: Can’t Live With It, Can’t Live Without It

In this webinar, Adi and Or from Komodor explore the evolution of Kubernetes node autoscaling, comparing the traditional Cluster Autoscaler with the newer Karpenter project. It details the fundamental differences in their provisioning mechanisms, highlighting Cluster Autoscaler’s reliance on rigid node groups and Karpenter’s dynamic, pod-driven approach. Finally, the session introduces Komodor’s cost optimization layer, which addresses the inherent reliability and efficiency gaps in both systems through predictive placement and proactive consolidation.

TL;DR

• Speakers: Or Sher (R&D Team Lead) and Adi Fayer (Product Manager) from Komodor lead a deep dive into Kubernetes cost optimization and autoscaling mechanisms.
• Focus: The core focus is on the operational, architectural, and efficiency differences between Cluster Autoscaler and Karpenter.
• Core Concepts: Cluster Autoscaler scales predefined node groups resulting in overprovisioning and latency, whereas Karpenter provisions optimal instances directly based on pod requirements but doesn’t have awareness of future state of workloads.
• Includes: The webinar includes detailed walkthroughs of scale-up and scale-down loops, common failure modes like availability degradation and underutilized nodes, and the delicate balance between cost and reliability.
• Wrap-up: Komodor presents its Cost Optimization platform that with capabilites that help bridge this gap by providing rightsizing, a unified autoscaler add-on, actionable insights based on Capacity Intelligence, and Predictive Placement to ensure efficient and safe scaling without replacing the underlying tools.

Key Takeaways

• Cluster Autoscaler scales predefined node groups rather than individual nodes, which frequently leads to operational overhead and costly overprovisioning.
• Karpenter bypasses node groups entirely to provision the lowest-cost instance that fits a pod’s specific requirements, drastically reducing provisioning latency from minutes to seconds.
• Despite its speed and efficiency, Karpenter’s reactive consolidation can lead to availability degradation and underutilized nodes due to its lack of predictive cluster state awareness.
• Achieving the perfect balance between cost savings and system reliability requires continuous tuning, as overly aggressive downscaling can disrupt stateful workloads, long-running jobs, and CI/CD pipelines.
• Komodor’s optimization layer enhances existing autoscalers by proactively simulating cluster states to prevent bad placement decisions, ultimately reducing costs by 40-80% while improving reliability.

Webinar Transcript 

Please note that the following text may have slight differences or mistranscription from the audio recording.

Ilan: Let’s start today’s webinar, “Karpenter: Can’t live with it, Can’t live without it.” A few housekeeping issues before we get started. Yes, this webinar is being recorded, so you will receive a copy at the end. Please feel free to use the Q&A section to ask any questions, and we will have a Q&A section at the end. This webinar is going to be about 45 minutes long.

Right now, I’m going to introduce our fabulous speakers for today. We have with me Adi Fayer, Product Manager for Cost Optimization, and Or Sher, the R&D Team Lead for Cost Optimization at Komodor. I’m going to hand it over to them and let them do their magic.

Or Sher: Great. Thank you, Ilan. Again, my name is Or, and I am the R&D Team Lead for Cost Optimization. I’m super glad to be here and very excited.

To understand why Karpenter matters, you have to feel the problem it was built to solve. We will spend a few minutes on Cluster Autoscaler. We’ll talk about what it is, how it works in terms of scale-up and scale-down, and where it runs out of road. Only then will the problems Karpenter is meant to solve become much clearer.

Cluster Autoscaler is what most teams have run for node autoscaling for years. It is Kubernetes-native, has been around since 2016, and works across AWS, GCP, Azure, and more cloud providers. The key thing to hold onto is that it scales node groups, not individual nodes. It doesn’t pick a machine; it changes a group’s desired count. Think of it as a bridge. When pods can’t be scheduled, Cluster Autoscaler tells the cloud provider’s VM group API to add more capacity—an Autoscaling Group on AWS, an Instance Group on GCP, or a VM Scale Set on Azure. In any case, it translates pending pods into adding more machines.

Here is the catch: those VM group APIs were built for a different job. AWS’s Autoscaling Group, for instance, dates back to 2009, before Kubernetes even existed. They were designed to scale a fleet of identical VMs on infrastructure metrics like CPU behind a load balancer. Cluster Autoscaler reuses these old primitives to place pods. Every limitation we are about to see traces back to that same mismatch.

To understand how Cluster Autoscaler works, we need to understand its scaling decisions first. Here is the scale-up loop end-to-end. A pod can’t be placed, so it sits in pending. Cluster Autoscaler scans for pending pods about every 10 seconds. When it finds one, it simulates adding a node from each of your predefined groups and keeps the ones that actually host that pod, checking CPU, memory, affinity rules, taints, etc. When several groups can host it, an expander breaks the tie. Then, Cluster Autoscaler raises the chosen group’s desired count, and the cloud launches that node. It joins, and from here on, the scheduler is responsible and places the pod.

Notice what just happened: Cluster Autoscaler never chose an instance for the pod. It chose a compatible group from the groups you predefined in advance. Hold that thought; it’s the root of the next few slides.

Now, let’s discuss how scale-down happens. The approach here is deliberately conservative. Cluster Autoscaler prioritizes stability over cost. A node only comes off when all of these conditions are met:

1. Utilization should be below a threshold—CPU and memory requests against allocatable. The default is 50%.

2. Every pod on the node can move somewhere else. Cluster Autoscaler needs to simulate that move first and respect Pod Disruption Budgets (PDBs) and affinity rules.

3. All the pods on the node are evictable.

4. This entire state should stay like that past the timeout, which is 10 minutes by default.

Let’s look at a specific cluster in this example. We have Node A, which is under 50% utilization. Every pod on Node A can move to Node B, which has enough room. Cluster Autoscaler drains it. Beautiful.

Node C is just as empty, with 30% utilization, but it has one unevictable pod—either because it’s marked with an annotation, has local storage, or a PDB that says “don’t touch me.” That single pod pins that entire node. Cluster Autoscaler sees it and moves on.

Now let’s look at Node D, sitting at around 60%. A human being would look at this and say, “Okay, we can merge Node C and Node D together. We have enough space. We can move them onto a single node.” But Cluster Autoscaler won’t even look at Node D. Its candidacy is above the threshold. It’s not a judgment call.

To sum it up, all four conditions must be met, and only after 10 minutes is it scaled down. It’s careful by design and still leaves the cluster overprovisioned.

Now that we understand how scaling happens, we can discuss where Cluster Autoscaler runs out of road. Let’s start with the operational cost of the node groups. Cluster Autoscaler can only scale groups you predefined ahead of time. To cover real workload needs precisely, you need a separate group for every combination of requirements.

Take one example: you have x86 and ARM architectures, small and large sizes of workloads or nodes, and spot and on-demand capacity. That is already eight different node groups, and that’s even before multiplying by availability zones. Each of those is a resource you need to own and maintain forever. On top of that, the number of groups multiplies fast. Teams cope by collapsing into fewer, broader groups, which sets up the next problem.

We just saw teams collapse into fewer, broad, general-purpose groups just to stay sane. Here is what that costs. Cluster Autoscaler never invents a node size; it can only pick from the predefined groups. Say you kept two general-size nodes: a c5.2xlarge and a c5.4xlarge. We have a pending pod that needs only 1 CPU and 2 GB of memory. The smallest group that fits is the c5.2xlarge. The pod uses only 1 CPU, but you pay for the entire eight. Roughly 88% of the node is idle but still billed, and the size was locked in before the pod even existed.

The third limitation on the scale-down side is that consolidation is purely reactive. A node’s utilization drops below 50%, but Cluster Autoscaler doesn’t act immediately. It waits for the node to stay underutilized for 10 minutes. Only then does it remove it. This entire waiting window is wasted spend. The node sits half-empty but fully billed. The deeper limit is that Cluster Autoscaler only removes nodes. It never tries to replace an existing node to improve its utilization. It reacts to waste, never designing density in.

The last limitation is that scale-up is slow, and not just because of boot times. Look at the path: a pod goes pending, Cluster Autoscaler polls every 10 seconds, then hands it to the VM group API, which calls the instance fleet API to boot the node. That middle layer—the VM group—is pure overhead. Even on the happy path, we’re talking about minutes of cold starts.

It gets worse with prioritized groups. Imagine a spot-only group set as the top priority, but there’s no spot capacity available right now. Cluster Autoscaler waits for that group’s attempt to time out before failing back to the next group. Stack a few prioritized groups, and a single scale-up burns minutes of stacked timeouts while pods sit pending. That extra VM group layer is exactly what Karpenter removes.

Before we move on, here’s a quick recap of the four areas where Cluster Autoscaler runs out of road: operational overhead, overprovisioning, reactive consolidation, and provisioning latency. These are exactly the problems Karpenter is built to solve.

Let’s introduce Karpenter. Karpenter didn’t tune Cluster Autoscaler; it rethought the entire question. We’ll see what it is, how it works, how to configure it, and where it still stops. It’s a real leap forward, but it has its own sharp edges. We will cover both.

Karpenter is an open-source Kubernetes node autoscaler. Originally built by AWS, it was later contributed to the CNCF. It’s vendor-neutral now. There are providers for AWS, Azure, and a community one for GCP. Karpenter does not follow the node group constraints we saw before. Instead of asking which of your predefined groups it needs to grow, Karpenter asks: “What does this pod actually need?” It reads the full spec of the pod—CPU, memory, taints, topology spread, capacity type, etc.—and provisions the lowest-cost instance that fits.

To understand where Karpenter shines, we need to understand its scaling and consolidation logic. Here is how scale-up happens. The same trigger applies: pods can’t schedule. Karpenter batches the pending pods, bin-packs them by their real requirements, and then picks an instance from the whole cloud catalog—not a node from a group you predefined. It calls the instance fleet API directly. There’s no ASG or VMSS layer in the middle. Provisioning drops from minutes to seconds.

On the scale-down side, consolidation is not just about removal. Both Cluster Autoscaler and Karpenter can remove an empty node. But only Karpenter can swap a node for a cheaper one. Take this scenario: an m5.2xlarge sits at 30% utilization. Its pods need about 3 CPUs. Nothing else in the cluster can move to it. Cluster Autoscaler is stuck. Five out of eight CPUs are idle but billed indefinitely.

In contrast, Karpenter launches an m5.xlarge first, waits until it’s ready, moves the pods, and then drains the 2xlarge. What you get is a cheaper node and denser packing. Overprovisioning doesn’t just get fixed once at launch; it dynamically consolidates and keeps it fixed as the cluster changes.

Let’s talk about configuration. Configuring Karpenter is mainly just these two objects, and they split the work very cleanly. The NodePool is the “what” and the rules: which instances are allowed, disruption and lifecycle settings, resource limits, priority, etc. It expresses pure Kubernetes intent. The NodeClass is the “how”: the image, subnets, security groups, IAM identity, disk configuration—the cloud-specific mechanics. They link through the NodeClassRef. There’s a many-to-one relationship. You define the cloud wiring once in a NodeClass and reuse it across multiple NodePools.

Let’s look at a NodePool configuration. I won’t walk through the entire spec, just the fields that matter.

• Requirements: This is the box of allowed nodes. Every constraint you add removes packing options Karpenter could have used. You need to keep it as broad as you can.

• Disruption: This is where you set scale-down behavior. You have consolidation policies which decide what is eligible to remove. The WhenEmpty policy only touches nodes with no pods. It’s safe, with no disruption, but leaves savings on the table. The WhenUnderutilized policy also repacks underused nodes onto fewer and cheaper nodes. This yields more savings but more pod movement.

• ConsolidationAfter: This is the cooldown before acting. Zero seconds means act the moment the node is drainable. That’s the most aggressive setting. If your load is spiky, you probably want to set it higher, like a few minutes, so it acts only when a node stays drainable for a while.

• Limits: This is your cost guardrail. It’s a hard ceiling on the total CPU and memory the pool can consume.

Broad requirements, deliberate disruption settings, and the cost ceiling—that’s most of getting a NodePool right. With that, I’ll leave it to Adi to show you where Karpenter still falls short and what you can do about it.

Adi Fayer: Thank you, Or. Hi everyone. Up until now, we discussed the upside of Karpenter, and it’s genuinely good. But Karpenter comes with a few challenges and two main failure modes. Let’s dive into that.

First, same as the Cluster Autoscaler, Karpenter is hard to operate. There are a lot of moving parts that drive consolidation and provisioning decisions: your workload specs, your NodePool and disruption configuration, and of course, live cluster traffic and scheduling patterns.

Second, tuning configuration to resolve cost and reliability tradeoffs is a continuous process. It’s not a one-off. Striking the right balance demands deep expertise and ongoing observability as your workloads change.

Third, Karpenter is reactive, just like Cluster Autoscaler. It is downstream of the scheduler. It inherits whatever placement decisions the scheduler made upstream.

Here is the binding underlying truth: Karpenter’s consolidation solves one problem really well. It removes empty or underutilized nodes and repacks them onto fewer, cheaper ones. But it optimizes each decision in isolation. It doesn’t understand how your workloads behave over time. Therefore, what you have to do is reach for guardrails. You set disruption budgets, do-not-disrupt annotations, or longer consolidate-after windows. Essentially, the same guardrails that prevent churn or pod evictions are also the ones that block savings and keep your capacity underutilized. You end up choosing between cost and reliability.

Now let’s move to the failure modes of Karpenter, or how it looks in real life.

The first failure mode is where scale-down degrades availability. Turning on underutilized consolidation actually saves money, but it introduces real reliability risks. This stems from churn. Bursty workloads can trigger repeated node removals and recreations, resulting in pod evictions. Pods get restarted on different nodes, which could disrupt stateful workloads, long-running jobs, and CI/CD pipelines. Also, there’s a critical situation regarding noisy neighbors. Because your nodes are tightly packed, one pod that spikes can step on others. The more aggressively you consolidate for cost optimization, the more you put availability at risk. That’s the first edge of the tradeoff.

The second failure mode quietly creates significant waste, stemming from underutilized nodes that essentially never go away. There are three main reasons this happens:

1. The Scheduler vs. Autoscaler: The scheduler optimizes for what’s happening right now. When a pod needs to be placed, it finds a node that fits and puts it there. It has no awareness of future cluster states. On the other side, the autoscaler is trying to consolidate, but it inherits the placement decisions the scheduler already made. It finds unevictable pods scattered across nodes it wants to drain. These two systems are not coordinated; they’re working against each other.

2. Strict Guardrails: Even when the autoscaler wants to remove a node, it can only scale down if every pod on it can relocate. There’s a long list of reasons a pod cannot move: anti-affinity rules, topology spread constraints, host path requirements, or unsatisfied PDBs. Any of these pins the node in place.

3. Policy Restrictions: Even when pods could relocate, your disruption controls often forbid that action. Your consolidation policy might be set to only touch fully empty nodes. The cooldown before consolidation might be too long. Disruption budgets might prevent too many nodes from being disrupted at once. Capacity is reclaimable in theory, but your own guardrails say it’s not allowed.

Put those three together, and the result is that 30 to 40% of your cluster capacity is structurally wasted. Not because anything is broken, but because the system was never designed to consolidate proactively.

Let’s put it all together. We’ve seen what Cluster Autoscaler is and where it falls short. We saw how Karpenter rethinks the problem and fixes most of it. But we also saw the challenges that come with running Karpenter, especially in production environments. Some of these are structural, not bugs you can configure away. That’s where Komodor comes in to close the gap. Not by replacing your autoscaler, but by adding a layer of optimization on top of it across both reliability and cost. Whether you run Karpenter, Cluster Autoscaler, or a managed autoscaler, we make it work better without vendor lock-in.

We close these gaps with three layers of optimization:

1. Rightsizing: Each autoscaler provisions nodes based on pod resource requests, not actual usage. The requests are almost always wrong—either overprovisioned, wasting money, or underprovisioned, causing throttling and OOMs. Our rightsizing offering is bidirectional. We resize down to eliminate waste and resize up to improve reliability. Every recommendation we generate is informed by reliability signals. We analyze actual usage patterns, spike behavior, and Quality of Service (QoS) classes.

2. Autoscaler Add-on: Any autoscaler gives you almost no view into its own behavior. You’re largely guessing whether it’s churning too hard or quietly failing to scale down. The autoscaler add-on continuously analyzes your environment and surfaces insights. It looks at your autoscaler configuration, NodePools, pod specs, and metrics, moving you from “Is scaling actually happening?” to “Is it happening efficiently and safely?” It surfaces insights like suboptimal instance types, disruption settings that need tuning, or oversized nodes holding idle capacity, along with one-click fixes.

3. Predictive Placement: This is the deepest layer. The structural problem we described, where the scheduler places greedily and the autoscaler inherits the mess, is an upstream problem. The fix is a better scheduler. We built Predictive Placement with four pillars:

   • Continuous Simulation: Every minute, we simulate a full cluster drain.

   • Intelligent Placement: We classify every node into states (e.g., removable, blocked) and steer new pods away from drain candidates before bad placement happens.

   • Unevictable Consolidation: We steer unevictable pods onto designated keeper nodes, concentrating blocker nodes together to keep other nodes free to drain.

   • AI Pattern Recognition: The simulation is based on a point in time, but AI makes it predictive. We learn workload patterns, scheduling frequencies, and lifespan behavior to anticipate future cluster states.

Our findings from analyzing thousands of clusters show that you can achieve 40% to 80% lower costs, up to a 50% reduction in issues, and around 70% faster recovery—all using the same autoscaler you already run.

Now, it’s time for the Q&A. Or, do we have any questions?

Or Sher: Yeah, I see Samir wrote a few questions. Samir asks: “Does Karpenter support scaling based on custom metrics such as network usage or the number of API requests?” That’s a good question. The short answer is no. Karpenter doesn’t scale on metrics at all. Not CPU, network, or request count. It’s driven purely by pending pods. When the scheduler can’t place a pod, Karpenter comes in and provisions a node for it.

Let me ask Adi a question we usually get asked at events: “Does Predictive Placement interfere with existing scheduling rules?”

Adi Fayer: No. Essentially, we use preferred affinity rules—preferred, never required. So pods are soft-scheduled to the ideal node. All hard constraints, such as PDBs, taints, and tolerations, are completely honored throughout. We steer; we never block.

Or Sher: Another common question is: “What happens to Komodor if the Komodor agent goes down? Does my cluster stop scaling?”

No, definitely not. Komodor is not in the critical path of your autoscaler. If our agent loses connectivity somehow, your cluster will keep running as normal. Karpenter or Cluster Autoscaler will handle scaling. Existing pods will keep running, and they’ll keep their current resources. There will just be no new optimizations from us until connectivity is restored, but there is no single point of failure introduced.

I think that’s it for the questions. Thank you, everyone, for attending. Thank you, Adi. We will have another webinar next month centered around CPU and memory. If you’re interested in Komodor’s cost optimization capabilities, you’re welcome to stop by our website, schedule a call with our experts, or take it out for a test drive. Thank you so much!

Adi Fayer: Thank you, Or. Thank you, everyone!

Frequently Asked Questions

Q: How does Karpenter differ from traditional Cluster Autoscaler when making scaling decisions?

• A: Cluster Autoscaler looks at pending pods and attempts to adjust the “desired count” of pre-configured, rigid cloud node groups (like AWS ASGs or Azure VMSS). This often leads to overprovisioning since pods are forced onto whatever predefined node size is available in that group. Karpenter, on the other hand, bypasses node groups entirely. It evaluates the exact resource requirements of pending pods (CPU, memory, taints, topology constraints) and dynamically provisions the single most cost-effective instance type straight from the cloud provider’s entire catalog.

Q: Will implementing Komodor’s Predictive Placement layer overwrite or break my existing Kubernetes scheduling rules?

• A: No. Komodor’s Predictive Placement operates using preferred (soft) affinity rules rather than required (hard) constraints. This ensures that your native Kubernetes scheduling rules, Pod Disruption Budgets (PDBs), taints, and tolerations are always fully respected. The platform guides your cluster toward optimal node density and placement without ever blocking or forcing an invalid scheduling state.