Autoscaling and Resource Management in Kubernetes

This is the fifth and final post in our Kubernetes fundamentals series, building on Pods and Deployments, networking, ConfigMaps and Secrets, and storage.

A Kubernetes cluster with no resource requests or limits configured on any of its Pods will, eventually, run into a specific and unpleasant failure mode: one Pod consumes more memory than the node actually has available, and the node itself becomes unstable — not just that one Pod. Requests and limits are how you prevent that, and they're also the foundation everything about autoscaling is built on. Because this connects scheduling, stability, and cost into one coherent system, it's worth covering in real depth.

Requests and limits: two different guarantees

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: my-app:1.0
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              cpu: "500m"
              memory: "512Mi"

These two fields answer genuinely different questions:

• requests is what the scheduler uses to decide which node a Pod can even be placed on — it will only schedule a Pod onto a node that has at least this much CPU and memory currently unreserved. This is a scheduling guarantee, not a hard ceiling.
• limits is the actual ceiling enforced at runtime. Exceed the CPU limit, and the container is throttled — slowed down, not killed. Exceed the memory limit, specifically, and the container is killed outright (an "OOMKilled" event), because unlike CPU, memory can't be safely throttled after the fact.

kubectl get pod web-7d9f8c6b5d-2xk9p

NAME                   READY   STATUS      RESTARTS   AGE
web-7d9f8c6b5d-2xk9p   0/1     OOMKilled   1          2m

Why "no limits set" is not the same as "unlimited, and safe"

A Pod with no requests set gets scheduled onto any node, regardless of actual available capacity, because the scheduler has nothing to reason about. A handful of such Pods landing on the same node, all growing their memory usage simultaneously under real load, can exhaust that node's actual physical memory — at which point the Linux kernel's OOM killer starts terminating processes on that node somewhat unpredictably, potentially including Pods that had nothing to do with the actual problem. Setting requests and limits deliberately on every Pod is what keeps one workload's resource usage from being able to destabilize everything else sharing that node.

Quality of Service classes: the consequence of how you set requests and limits

Kubernetes derives a QoS class for every Pod automatically, based purely on how its requests and limits relate to each other — and that class determines which Pods get evicted first under real node-level memory pressure:

• Guaranteed: every container's requests equal its limits, for both CPU and memory. These Pods are the last to be evicted under memory pressure — Kubernetes treats them as having a fully predictable resource footprint.
• Burstable: requests are set but are lower than limits (the example above). These Pods can use more than their request when spare capacity exists, but are evicted before Guaranteed Pods if the node comes under real memory pressure.
• BestEffort: no requests or limits set at all. These are evicted first, with no meaningful guarantee at any point.

kubectl get pod web-7d9f8c6b5d-2xk9p -o jsonpath='{.status.qosClass}'

Burstable

This is worth setting deliberately rather than discovering by accident: a genuinely critical workload (a payment processor, a primary database) is a strong candidate for Guaranteed QoS — set requests equal to limits — specifically so it's the last thing evicted if a node runs low on memory, while a less critical batch job might reasonably run as Burstable or even accept BestEffort if being evicted under pressure is an acceptable outcome for it.

The Horizontal Pod Autoscaler: more replicas under load

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

The Horizontal Pod Autoscaler (HPA) watches a metric — CPU utilization here, though custom metrics like requests-per-second are equally common — and adjusts the Deployment's replica count within the given range to keep that metric near the target. Average CPU usage across all web Pods climbing above 70% triggers additional replicas, up to maxReplicas; usage dropping well below target scales back down, no lower than minReplicas.

This directly depends on requests being set correctly. "70% utilization" is measured relative to the CPU request — a Pod with no request set, or a wildly inaccurate one, gives the HPA a meaningless number to scale against, and it will make correspondingly bad scaling decisions.

Scaling on a custom metric, not just CPU

CPU utilization is a reasonable default, but it's frequently the wrong signal for the workload that actually matters — a queue-processing service is better scaled on queue depth than on CPU, and an HTTP service is often better scaled on requests-per-second than on CPU alone:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: worker-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: worker
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: External
      external:
        metric:
          name: sqs_queue_depth
          selector:
            matchLabels:
              queue: order-processing
        target:
          type: AverageValue
          averageValue: "30"

This requires a metrics adapter (the Kubernetes Metrics Server for basic resource metrics, or something like Prometheus Adapter or KEDA for genuinely custom and external metrics such as an SQS queue depth) actually feeding that metric into the HPA. Choosing the metric that reflects genuine load for a given workload — not defaulting to CPU because it's the example every tutorial uses — is often the difference between an HPA that scales sensibly and one that reacts to the wrong signal entirely.

Scaling behavior: controlling how fast, not just when

The default HPA behavior can react to a brief spike by scaling up aggressively, then scaling back down just as quickly once it passes — a pattern called "flapping" that adds real replica churn for a spike that may not have warranted it. behavior lets you control this directly:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 3
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Pods
          value: 1
          periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 30

stabilizationWindowSeconds: 300 on scaleDown means the HPA looks at the last five minutes of recommended replica counts and picks the highest one, before actually scaling down — meaning a brief dip doesn't immediately trigger scaling in, while scaleUp reacting with no stabilization window (and a policy allowing the replica count to double every 30 seconds) means genuine load spikes still get responded to quickly. This asymmetry — cautious scaling down, aggressive scaling up — is a deliberate, common pattern: the cost of scaling up briefly too much is usually smaller than the cost of scaling down too eagerly and then immediately needing to scale back up.

The Vertical Pod Autoscaler: right-sizing the request itself, over time

Where the HPA changes how many replicas exist, the Vertical Pod Autoscaler (VPA) adjusts the requests and limits values themselves, based on a Pod's actual observed usage history — useful for workloads where the right resource request isn't obvious upfront, or drifts over time as the application evolves.

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  updatePolicy:
    updateMode: "Auto"

updateMode: "Off" is also worth knowing: it runs the VPA in recommendation-only mode, computing what it would set requests to without actually applying anything — a genuinely useful way to learn what a workload's real resource footprint looks like before trusting an autoscaler to change it live.

The real caveat: HPA and VPA both actively managing CPU/memory for the same workload conflict with each other — VPA changing a Pod's resource requests can trigger the Pod being recreated, right as HPA is independently trying to reason about replica count based on those same values changing underneath it. Most real deployments pick one axis to automate — horizontal scaling for stateless, request-driven services (the common case), vertical for a smaller set of workloads with genuinely unpredictable per-instance resource needs — rather than running both simultaneously on the same Deployment.

Cluster Autoscaler: the layer above both of these

HPA and VPA both operate within the existing set of nodes. If every node is already fully allocated and a new Pod (or a scaled-up existing one) has nowhere to fit, the Cluster Autoscaler is what actually provisions a new node from the cloud provider to make room — and scales nodes back down when they're sitting mostly idle. This is the layer that connects Kubernetes' scheduling decisions to real infrastructure cost, and it's why requests accuracy matters beyond just the HPA: overly generous requests waste real money on unnecessarily provisioned nodes; overly stingy ones risk the instability covered earlier in this post.

What to actually remember from this series

• requests drive scheduling; limits are the enforced ceiling — exceeding a memory limit kills the container; exceeding a CPU limit throttles it.
• Unset resource requests aren't "unlimited and safe" — they're a genuine risk to overall node stability under real load, and they place a Pod in the BestEffort QoS class, evicted first under pressure.
• QoS class (Guaranteed, Burstable, BestEffort) is derived automatically from how requests relate to limits — set it deliberately for genuinely critical workloads.
• The HPA scales replica count based on a metric measured relative to requests — inaccurate requests produce bad scaling decisions, not just bad scheduling, and the metric chosen should reflect genuine load, not just default to CPU.
• behavior policies control scaling speed asymmetrically — cautious scale-down, responsive scale-up is a common, deliberate pattern.
• HPA and VPA generally shouldn't manage the same workload simultaneously; Cluster Autoscaler operates a layer above both, provisioning actual nodes when existing capacity runs out.

That closes out our Kubernetes fundamentals series — from Pods and Deployments through networking, ConfigMaps and Secrets, storage, and now resource management. If your team is running — or planning to run — real production workloads on Kubernetes, this is exactly the kind of hands-on work we build our cloud and DevOps training around.