Optimizing Resource Usage in Kubernetes with HPA

Table of Contents

1. Understanding HPA: What It Is and Why It Matters
2. How HPA Works: Core Components and Workflow
3. Prerequisites for Implementing HPA
4. Configuring HPA: A Step-by-Step Guide
5. Advanced HPA Configurations
6. Best Practices for HPA
7. Troubleshooting Common HPA Issues
8. Conclusion
9. References

1. Understanding HPA: What It Is and Why It Matters

What Is HPA?

The Horizontal Pod Autoscaler (HPA) is a Kubernetes API resource that automatically scales the number of pod replicas in a deployment, statefulset, or replica set. Unlike vertical scaling (which increases resources for individual pods), horizontal scaling adds or removes pods to match demand. HPA is part of the Kubernetes autoscaling API group (autoscaling/v2) and is managed by the Kubernetes controller manager.

Why HPA Matters

• Cost Efficiency: Avoid overprovisioning by scaling down during low traffic, reducing cloud spend.
• Performance: Scale up during traffic spikes to maintain responsiveness and avoid downtime.
• Automation: Eliminates manual intervention, freeing DevOps teams to focus on higher-value tasks.
• Resilience: Ensures applications can handle unexpected load without manual adjustments.

How HPA Differs from Other Autoscaling Tools

HPA is just one of several autoscaling mechanisms in Kubernetes:

• Vertical Pod Autoscaler (VPA): Adjusts CPU/memory requests for individual pods (complements HPA).
• Cluster Autoscaler: Scales the number of nodes in the cluster (works with HPA to add nodes when pods can’t schedule).
• Vertical Node Autoscaler: Adjusts node resources (less common; cloud-provider specific).

HPA focuses on pod count, making it ideal for stateless applications with shared workloads (e.g., web servers, APIs).

2. How HPA Works: Core Components and Workflow

HPA operates through a closed-loop control mechanism, relying on three key components:

Core Components

1. Metrics Server: A cluster-wide aggregator of resource metrics (CPU, memory) from kubelets. It exposes metrics via the Kubernetes API (metrics.k8s.io), which HPA queries to make scaling decisions.
2. HPA Controller: Runs in the Kubernetes controller manager and periodically (every 15 seconds by default) checks metrics for target resources. It calculates the desired number of replicas and updates the deployment/statefulset.
3. Target Resource: The deployment, statefulset, or replica set that HPA scales (e.g., a nginx-deployment).

HPA Workflow

The HPA controller follows this process:

1. Fetch Metrics: Queries the Metrics Server (or custom metrics APIs) for metrics like CPU usage, memory usage, or custom metrics (e.g., requests per second).
2. Calculate Desired Replicas: Compares current metrics to user-defined targets (e.g., “keep CPU usage at 70%”). The formula for desired replicas is:

   Desired Replicas = ceil(Current Replicas × (Current Metric Value / Target Metric Value))  

   Example: If current CPU usage is 80% (target: 40%), and there are 2 replicas, desired replicas = 2 × (80/40) = 4.
3. Enforce Bounds: Clamps the desired replicas to the minReplicas and maxReplicas limits defined in the HPA spec.
4. Update Target Resource: Adjusts the replicas field of the target deployment/statefulset to match the desired count.

3. Prerequisites for Implementing HPA

Before deploying HPA, ensure your cluster meets these requirements:

1. Metrics Server Installed

HPA relies on the Metrics Server to fetch CPU/memory metrics. Most managed Kubernetes clusters (EKS, GKE, AKS) include Metrics Server by default, but self-managed clusters require manual installation:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml  

Verify installation:

kubectl get pods -n kube-system | grep metrics-server  

2. Resource Requests and Limits Defined

HPA requires resource requests (minimum resources a pod needs) to calculate utilization. Without requests, HPA cannot determine if pods are over/under-utilized.

Example deployment with resource requests:

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: nginx-deployment  
spec:  
  replicas: 2  
  selector:  
    matchLabels:  
      app: nginx  
  template:  
    metadata:  
      labels:  
        app: nginx  
    spec:  
      containers:  
      - name: nginx  
        image: nginx:1.21  
        resources:  
          requests:  # Required for HPA  
            cpu: 100m  # 100 millicores  
            memory: 128Mi  
          limits:  # Optional but recommended  
            cpu: 500m  
            memory: 256Mi  

3. API Version Compatibility

Use autoscaling/v2 (stable) for HPA configurations. Older versions (autoscaling/v1) only support CPU/memory metrics and lack advanced features like scaling policies.

4. Custom Metrics (Optional)

For non-resource metrics (e.g., requests per second, queue length), install a custom metrics adapter like the Prometheus Adapter to expose metrics via the Kubernetes API (custom.metrics.k8s.io or external.metrics.k8s.io).

4. Configuring HPA: A Step-by-Step Guide

Let’s walk through deploying HPA for a sample Nginx deployment. We’ll start with basic CPU-based scaling, then add memory and custom metrics.

Step 1: Deploy a Sample Application

First, create a deployment with resource requests (as shown in Section 3.2). Apply it:

kubectl apply -f nginx-deployment.yaml  

Step 2: Create a Basic HPA (CPU-Based)

Define an HPA that scales the deployment based on CPU utilization. Save this as hpa-basic.yaml:

apiVersion: autoscaling/v2  
kind: HorizontalPodAutoscaler  
metadata:  
  name: nginx-hpa  
spec:  
  scaleTargetRef:  
    apiVersion: apps/v1  
    kind: Deployment  
    name: nginx-deployment  # Target deployment  
  minReplicas: 2  # Minimum pods  
  maxReplicas: 10  # Maximum pods  
  metrics:  
  - type: Resource  
    resource:  
      name: cpu  
      target:  
        type: Utilization  
        averageUtilization: 70  # Target CPU usage (70% of request)  

Apply the HPA:

kubectl apply -f hpa-basic.yaml  

Step 3: Verify HPA Configuration

Check the HPA status:

kubectl get hpa nginx-hpa  

Output:

NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE  
nginx-hpa    Deployment/nginx-deploy  30%/70%   2         10        2          5m  

• TARGETS: Current CPU usage (30%) vs. target (70%).
• REPLICAS: Current pod count (2, since 30% < 70%).

Step 4: Test Scaling

To trigger a scale-up, generate load on the deployment. Use kubectl run to create a pod that sends traffic:

kubectl run -it --rm load-generator --image=busybox /bin/sh  

In the load-generator pod, run:

while true; do wget -q -O- http://nginx-deployment.default.svc.cluster.local; done  

After a minute, check HPA again. CPU usage will spike, and HPA will scale up replicas:

NAME         REFERENCE               TARGETS   MINPODS   MAXPODS   REPLICAS   AGE  
nginx-hpa    Deployment/nginx-deploy  85%/70%   2         10        4          10m  

Step 5: Add Memory Metrics (Optional)

Update the HPA to scale based on memory usage. Modify hpa-basic.yaml to include a memory metric:

metrics:  
- type: Resource  
  resource:  
    name: cpu  
    target:  
      type: Utilization  
      averageUtilization: 70  
- type: Resource  
  resource:  
    name: memory  
    target:  
      type: Utilization  
      averageUtilization: 80  # Target memory usage (80% of request)  

HPA will scale up if either CPU or memory exceeds its target.

Step 6: Use kubectl autoscale (Alternative)

For quick setup, use kubectl autoscale to create an HPA without a YAML file:

kubectl autoscale deployment nginx-deployment --min=2 --max=10 --cpu-percent=70  

5. Advanced HPA Configurations

HPA offers advanced features to fine-tune scaling behavior, handle custom metrics, and integrate with external systems.

Scaling Policies: Stabilization and Delays

Prevent rapid scaling (“thrashing”) with stabilization windows and scale-down delays. Add a behavior section to the HPA spec:

spec:  
  # ... (scaleTargetRef, min/maxReplicas, metrics)  
  behavior:  
    scaleUp:  
      stabilizationWindowSeconds: 60  # Wait 60s before scaling up  
      policies:  
      - type: Percent  
        value: 50  # Scale up by 50% of current replicas  
        periodSeconds: 60  
    scaleDown:  
      stabilizationWindowSeconds: 300  # Wait 5min before scaling down  
      policies:  
      - type: Percent  
        value: 30  # Scale down by 30% of current replicas  
        periodSeconds: 120  

Custom Metrics (e.g., Requests Per Second)

To scale based on application-specific metrics (e.g., RPS), use the Prometheus Adapter to expose metrics from Prometheus.

Example HPA with a custom “requests per second” metric:

metrics:  
- type: Pods  
  pods:  
    metric:  
      name: http_requests_per_second  # Custom metric from Prometheus  
    target:  
      type: AverageValue  
      averageValue: 1000  # Target 1000 RPS per pod  

External Metrics (e.g., Cloud Queue Depth)

Scale based on metrics from external systems (e.g., AWS SQS queue depth). Use the External metric type:

metrics:  
- type: External  
  external:  
    metric:  
      name: sqs_queue_depth  
      selector:  
        matchLabels:  
          queue: my-app-queue  
    target:  
      type: Value  
      value: 5000  # Scale up if queue depth > 5000  

HPA with StatefulSets

HPA works with statefulsets (e.g., databases) but requires careful planning:

• Ensure the statefulset has a headless service for stable networking.
• Avoid scaling statefulsets with persistent storage if data consistency is critical (e.g., leader-follower databases).

6. Best Practices for HPA

To maximize HPA effectiveness, follow these best practices:

1. Set Accurate Resource Requests

HPA uses resource requests to calculate utilization. Underestimating requests leads to false scaling triggers; overestimating wastes resources. Use tools like kube-resource-report to analyze historical usage and set requests.

2. Choose the Right Metrics

• CPU/Memory: Best for general workloads (e.g., web servers).
• Custom Metrics: Use for application-specific load (e.g., RPS, queue length).
• External Metrics: Ideal for workloads tied to external systems (e.g., message queues).

3. Avoid Thrashing

Use stabilization windows (Section 5.1) to prevent rapid scale-up/scale-down cycles. For example, set a 5-minute scale-down delay to avoid scaling down during transient lulls.

4. Monitor HPA Itself

Track HPA decisions with tools like Prometheus and Grafana. Key metrics to monitor:

• horizontalpodautoscaler_status_current_replicas
• horizontalpodautoscaler_status_desired_replicas
• horizontalpodautoscaler_status_current_metrics

5. Test Under Load

Simulate traffic spikes (e.g., with k6) to validate HPA behavior. Ensure scaling is fast enough to handle peak load and doesn’t exceed maxReplicas.

6. Combine with Cluster Autoscaler

HPA scales pods, but if the cluster lacks nodes to schedule new pods, use the Cluster Autoscaler to add nodes dynamically.

7. Troubleshooting Common HPA Issues

Issue 1: HPA Shows “Metrics Unavailable”

Cause: Metrics Server is not running or misconfigured.
Fix:

• Check Metrics Server logs: kubectl logs -n kube-system metrics-server-<pod-id>
• Ensure Metrics Server has permissions to access kubelet metrics (common in self-managed clusters).

Issue 2: HPA Not Scaling Up/Down

Cause: Metrics are below/above target, but minReplicas/maxReplicas are blocking changes.
Fix:

• Verify TARGETS in kubectl get hpa: If current < target, scale-down is blocked by minReplicas.
• Check for pod disruptions (e.g., kubectl describe pod <pod-id> for errors).

Issue 3: Incorrect Target Utilization

Cause: Resource requests are set too low/high, or metrics are miscalculated.
Fix:

• Validate requests with kubectl describe pod <pod-id> | grep Requests.
• Use kubectl top pod to check actual resource usage vs. requests.

Issue 4: Custom Metrics Not Found

Cause: Prometheus Adapter (or custom metrics server) is not installed or misconfigured.
Fix:

• Check adapter logs: kubectl logs -n monitoring prometheus-adapter-<pod-id>.
• Verify metrics are exposed via kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1".

8. Conclusion

The Horizontal Pod Autoscaler (HPA) is a cornerstone of Kubernetes resource optimization, enabling dynamic scaling to match real-time demand. By automating pod replica adjustments based on metrics, HPA reduces costs, improves performance, and minimizes manual intervention.

To succeed with HPA:

• Start with CPU/memory metrics, then add custom/external metrics.
• Use stabilization windows to avoid thrashing.
• Combine HPA with Cluster Autoscaler for end-to-end scaling.
• Monitor HPA and application metrics to refine configurations.

As Kubernetes evolves, HPA will continue to gain features (e.g., predictive scaling with AI/ML), making it even more critical for modern cloud-native applications.

9. References

• Kubernetes HPA Documentation
• Metrics Server Installation Guide
• Prometheus Adapter for Custom Metrics
• Cluster Autoscaler
• Kubernetes Autoscaling Best Practices