Optimizing Kubernetes Autoscaling with KEDA, HPA, and Karpenter

Three independent control loops

KEDA, the Horizontal Pod Autoscaler, and Karpenter do not execute a guaranteed sequence. They are independent reconcilers observing different signals:

• KEDA polls event sources and exposes external metrics. It directly handles activation between zero and one replica when scale-to-zero is enabled.
• KEDA creates and manages an HPA. The Kubernetes HPA controller calculates the desired pod count from one to many replicas.
• The scheduler places new pods. If suitable capacity is unavailable, pods remain pending.
• Karpenter observes unschedulable pods and provisions compatible nodes, subject to NodePool requirements and limits.

Because each loop has its own polling, stabilization, startup, scheduling, and cloud-provider latency, fixed promises such as “pods in 30 seconds” or “nodes in 60 seconds” are unsafe. Measure end-to-end recovery time for the actual workload.

One autoscaler owns one workload

Do not attach both a hand-written HPA and a KEDA ScaledObject to the same Deployment. KEDA already creates an HPA under the hood; two HPAs can race while writing the same replica field.

If the workload needs event, CPU, and memory signals, define them as triggers in one ScaledObject. The generated HPA evaluates the metrics and selects the highest recommended replica count.

Step 1: Make the workload schedulable

Autoscaling depends on accurate requests and a graceful lifecycle:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: frontend
          image: registry.example.com/frontend@sha256:replace-with-tested-digest
          ports:
            - containerPort: 8080
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              memory: 1Gi
          readinessProbe:
            httpGet:
              path: /readyz
              port: 8080
            periodSeconds: 5
          lifecycle:
            preStop:
              exec:
                command: ["sh", "-c", "sleep 10"]

CPU utilization scaling uses resource requests as its denominator. Missing or unrealistic requests make both HPA decisions and Karpenter bin-packing unreliable. Derive requests from load tests and production telemetry instead of copying these sample numbers.

Step 2: Define Karpenter capacity boundaries

Current Karpenter APIs use karpenter.sh/v1 for NodePool. A NodePool also references a cloud-provider node class; on AWS that is normally an EC2NodeClass managed separately.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: application
spec:
  template:
    metadata:
      labels:
        workload-tier: application
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: application
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
      expireAfter: 720h
  limits:
    cpu: "200"
    memory: 800Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 5m
    budgets:
      - nodes: "10%"

Pin and verify Karpenter before applying this manifest because fields evolve between releases. Avoid arbitrary maxPods values: pod density also depends on the VPC CNI, instance networking limits, prefix delegation, and kubelet configuration.

NodePool limits are eventual consistency controls, not an instantaneous hard quota during rapid parallel provisioning. Pair them with AWS service quotas, account budgets, alerts, and admission policies.

Spot and On-Demand in one pool allow Karpenter to choose available compatible capacity; this does not guarantee a particular discount. There is no standard karpenter.k8s.aws/spot-discount-rate scheduling requirement. Use supported requirements, price/capacity behavior, and separate pools or weights when workloads need a stronger capacity policy.

Step 3: Let KEDA own the HPA

The example below combines Prometheus request rate and CPU utilization in one scaling owner. Keep minReplicaCount above zero for latency-sensitive HTTP services unless cold starts are acceptable.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: frontend-scaler
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: frontend
  pollingInterval: 30
  cooldownPeriod: 300
  minReplicaCount: 3
  maxReplicaCount: 50
  advanced:
    horizontalPodAutoscalerConfig:
      name: frontend-hpa
      behavior:
        scaleUp:
          stabilizationWindowSeconds: 0
          selectPolicy: Max
          policies:
            - type: Percent
              value: 100
              periodSeconds: 60
            - type: Pods
              value: 5
              periodSeconds: 60
        scaleDown:
          stabilizationWindowSeconds: 300
          selectPolicy: Min
          policies:
            - type: Percent
              value: 20
              periodSeconds: 60
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus-server.monitoring.svc:9090
        metricName: frontend_http_requests_per_second
        query: sum(rate(http_requests_total{namespace="production",app="frontend"}[2m]))
        threshold: "100"
        activationThreshold: "10"
    - type: cpu
      metricType: Utilization
      metadata:
        value: "70"

Test the PromQL independently. It must return one numeric sample, have stable labels, and reflect capacity demand rather than merely total traffic. A global request-rate threshold usually needs to be interpreted as target demand per replica; confirm the resulting HPA math under load.

When the external metric source fails, scaling can stop changing while the workload remains at its current count. Alert on scaler errors and decide whether a supported KEDA fallback configuration, a minimum-replica safety margin, or another application-specific response is appropriate. Validate fallback compatibility with every configured scaler and metric type before enabling it.

Memory utilization often stays high after load subsides and may cause slow or unstable scale-down. Add the KEDA memory scaler only when tests show that memory is a meaningful capacity signal.

Step 4: Protect availability during scale-down

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: frontend
  namespace: production
spec:
  maxUnavailable: 1
  unhealthyPodEvictionPolicy: AlwaysAllow
  selector:
    matchLabels:
      app: frontend

A PDB governs voluntary disruption; it does not prevent every outage or control HPA replica reductions. An overly strict PDB can block node consolidation and maintenance. Combine it with topology spread, readiness gates, graceful termination, application retry behavior, and enough spare capacity.

Karpenter disruption respects several scheduling controls, but validate the exact behavior for consolidation, expiration, interruption, and your Karpenter release. Separate critical On-Demand workloads from interruptible capacity when availability requirements demand it.

Step 5: Observe every layer

kubectl -n production get scaledobject frontend-scaler
kubectl -n production describe scaledobject frontend-scaler
kubectl -n production get hpa frontend-hpa --watch
kubectl -n production get pods --watch
kubectl get nodeclaims,nodes --watch

Monitor:

• KEDA scaler errors, metric latency, READY, ACTIVE, and fallback state.
• HPA current/desired replicas, missing metrics, and time at maximum replicas.
• Pending pods grouped by scheduling reason, not just a raw count.
• NodeClaim launch/registration failures, cloud quota errors, and provisioning duration.
• Application queue age, request latency, errors, saturation, and cold-start time.
• Node utilization, disruption, Spot interruption, and cost.

Load and failure tests

Run a staged test that proves the whole control system:

1. Establish steady-state requests, resource use, and replica count.
2. Increase demand until KEDA's metric exceeds its target.
3. Confirm the generated HPA requests the expected replicas.
4. Exhaust current node capacity and verify pending pods contain actionable scheduling reasons.
5. Confirm Karpenter creates compatible NodeClaims and pods become ready.
6. Remove load and verify stabilization prevents oscillation.
7. Fail Prometheus and confirm the documented replica behavior plus alerting.
8. Simulate insufficient EC2 quota, unavailable instance types, and Spot interruption.
9. Verify PDB and graceful termination during consolidation.

Measure time from demand arrival to application-ready capacity. That end-to-end result is more useful than the reaction time of any individual controller.

Common failure modes

• A separate HPA competes with KEDA for the same Deployment.
• CPU requests are missing, so utilization cannot be calculated correctly.
• PromQL returns multiple series, no data, or stale data.
• maxReplicaCount is below peak demand or creates more pods than node quotas can support.
• NodePool requirements exclude all available instance types or zones.
• DaemonSet overhead and topology constraints were omitted from capacity planning.
• Scale-down terminates work that cannot retry or resume.
• Strict PDBs prevent consolidation indefinitely.
• Spot capacity is treated as guaranteed capacity.

Further reading

• Installing KEDA on Kubernetes
• KEDA scaling concepts
• KEDA ScaledObject specification
• Kubernetes HPA behavior
• Karpenter NodePools
• Karpenter disruption

Conclusion

Reliable autoscaling comes from clear ownership and bounded feedback loops. Let one KEDA-managed HPA own each workload, give the scheduler accurate requests, constrain Karpenter with supported NodePool policies, and validate the complete path from demand to ready application capacity under both normal and failed dependencies.