Kubernetes Autoscaling: Karpenter & KEDA
Traditional Kubernetes autoscaling relies on static Horizontal Pod Autoscalers (HPA) driven by CPU/memory utilization and the legacy Kubernetes Cluster Autoscaler (CA). When worker pods experience massive traffic spikes or queue backlog surges, CPU metrics react too slowly, while CA takes 3 to 5 minutes to negotiate fixed ASG node group expansion from cloud providers.
Modern cloud-native autoscaling decouples pod event triggers from compute node provisioning. KEDA (Kubernetes Event-driven Autoscaling) scales application pods from 0 to N instantly based on real-time messaging queue depth (Kafka, SQS, RabbitMQ), while Karpenter bypasses static node groups to provision right-sized compute nodes (EC2/GCE instances) in under 60 seconds. This guide details Karpenter just-in-time node provisioning, KEDA event triggers, and spot instance consolidation.
Mental Model: Legacy Cluster Autoscaler Node Groups vs Karpenter Just-in-Time Provisioning
Legacy Cluster Autoscaler (CA) requires creating pre-configured Auto Scaling Groups (ASGs) with fixed instance types (e.g. m5.large). When unschedulable pods request 6 CPU cores, CA must add multiple 2-core nodes sequentially, leaving un-utilized CPU capacity.
Karpenter Just-in-Time (JIT) Node Provisioning evaluates unschedulable pod resource requirements directly against cloud provider instance catalogs.
Instead of fitting pods into rigid node groups, Karpenter evaluates pod CPU, RAM, GPU, architecture (arm64 vs amd64), and topology spread constraints to launch the exact right-sized instance type (e.g. launching a single c6g.2xlarge instance) within 45 seconds. For Kubernetes production infrastructure, review implementing database sharding vitess kubernetes and building custom k8s admission webhooks open policy agent.
Quick reference
- Bypasses rigid cloud provider Auto Scaling Groups (ASGs) for direct instance API provisioning.
- Evaluates unschedulable pod resource requirements to select optimal right-sized EC2/GCE VM types.
- Provisions new worker nodes in under 60 seconds (compared to 3-5 minutes for CA).
- Supports dynamic architecture matching (Graviton ARM64, x86_64, NVIDIA GPUs).
- Powers cloud-native Kubernetes clusters at AWS, Datadog, Mercado Libre, and CoreConcept.
Remember this
Deploy Karpenter to replace legacy CA node groups with sub-minute just-in-time node provisioning.
Karpenter NodePools, Provisioners, & Spot Instance Consolidation
Karpenter is configured via declarative NodePool custom resources:
1apiVersion: karpenter.sh/v1beta12kind: NodePool3metadata:4 name: default-pool5spec:6 template:7 spec:8 requirements:9 - key: karpenter.sh/capacity-type10 operator: In11 values: ["spot", "on-demand"]12 - key: kubernetes.io/arch13 operator: In14 values: ["arm64", "amd64"]15 disruption:16 consolidationPolicy: WhenUnderutilized17 expireAfter: 720h # 30-day node rotation### Automatic Consolidation When workload traffic decreases, Karpenter continuously evaluates active nodes. If running pods can fit onto smaller or fewer instances, Karpenter cordons, drains, and terminates underutilized nodes automatically, reducing cloud infrastructure costs by up to 50%.
Quick reference
- NodePools declare instance family restrictions, Spot/On-Demand balances, and ARCH constraints.
- ConsolidationPolicy: WhenUnderutilized replaces larger nodes with cheaper instances automatically.
- Spot instance termination handler replaces interrupted Spot VMs in sub-30s windows.
- expireAfter node rotation enforces security patching and eliminates long-lived node drift.
- Achieves up to 50%+ cloud compute cost reductions through Spot & consolidation optimization.
Remember this
Configure Karpenter NodePools with Spot instances and consolidation policies to minimize infrastructure spend.
KEDA Event-Driven Autoscaling (Kafka, SQS, Prometheus Triggers)
While Kubernetes HPA only reacts after CPU utilization hits thresholds, KEDA drives pod scaling based on external event sources via ScaledObject resources:
1apiVersion: keda.sh/v1alpha12kind: ScaledObject3metadata:4 name: order-processor-scaler5spec:6 scaleTargetRef:7 name: order-processor-deployment8 minReplicaCount: 0 # Scales to ZERO when queue is empty!9 maxReplicaCount: 10010 triggers:11 - type: aws-sqs-queue12 metadata:13 queueURL: https://sqs.us-east-1.amazonaws.com/123/orders14 queueLength: "10" # Scale 1 pod per 10 pending messagesQuick reference
- KEDA ScaledObjects drive HPA scaling based on real-time external event metrics.
- Supports 60+ event scalers including Kafka, RabbitMQ, AWS SQS, GCP Pub/Sub, and Prometheus.
- Scales deployments down to ZERO replicas when event queues are empty to save resources.
- Triggers rapid pod scaling BEFORE CPU utilization rises, eliminating queue processing backlog.
- Natively handles secret authentication via KEDA TriggerAuthentication CRDs.
Remember this
Implement KEDA ScaledObjects to scale application pods dynamically based on real-time queue depth.
Combining KEDA Event Metrics with Karpenter Fast Node Provisioning
Combining KEDA and Karpenter establishes a complete end-to-end reactive autoscaling architecture:
1. Event Spike: A sudden surge of 100,000 messages lands in an AWS SQS queue.
2. KEDA Rapid Pod Scaling: KEDA detects the queue length and immediately scales order-processor deployment from 0 to 50 pods.
3. Karpenter Fast Provisioning: Kubernetes scheduler marks 40 pods as Pending. Karpenter detects the pending pods, selects optimal EC2 Spot instances, provisions nodes in 45 seconds, and binds all pods.
4. Workload Drain: Once SQS queue empties, KEDA scales pods back to 0, and Karpenter consolidates and terminates the worker nodes.
Quick reference
- Pairs event-driven pod scaling (KEDA) with fast just-in-time node provisioning (Karpenter).
- Eliminates pod pending delays caused by slow cloud provider ASG node group scaling.
- Provides true zero-to-hero scaling for batch workers, queue processors, and web APIs.
- Reduces idle cluster standby costs by scaling down both pods and nodes during off-peak hours.
- Establishes a resilient, self-healing cloud-native Kubernetes infrastructure engine.
Remember this
Combine KEDA event-driven pod scaling with Karpenter node provisioning for instant reactive autoscaling.
Key takeaway
To test KEDA locally, install KEDA via Helm (helm repo add keda https://kedacore.github.io/charts && helm install keda keda/keda --namespace keda --create-namespace). Deploy a test ScaledObject for a local Redis queue.
Related Articles
Explore this topic