coderain guide

Kubernetes AWS EKS: A Complete Beginners’ Guide

In today’s cloud-native world, containerization has revolutionized how applications are built, shipped, and scaled. At the heart of this revolution is **Kubernetes** (K8s), an open-source container orchestration platform that automates deployment, scaling, and management of containerized applications. However, managing a Kubernetes cluster from scratch—especially its control plane—can be complex, time-consuming, and error-prone. Enter **Amazon Elastic Kubernetes Service (EKS)**. EKS is AWS’s managed Kubernetes service that simplifies running Kubernetes on AWS by handling the heavy lifting of cluster management. With EKS, AWS manages the Kubernetes control plane (masters), leaving you free to focus on deploying and scaling your applications. Whether you’re new to Kubernetes or looking to migrate existing workloads to AWS, this guide will walk you through everything you need to know to get started with EKS—from core concepts to deploying your first application.

Table of Contents

  1. Introduction to Kubernetes
  2. What is AWS EKS?
  3. Benefits of Using AWS EKS
  4. Prerequisites for Getting Started
  5. Setting Up Your AWS Account and Tools
  6. Creating Your First EKS Cluster (Step-by-Step)
  7. Deploying a Sample Application on EKS
  8. Managing EKS Clusters: Key Operations
  9. Monitoring and Logging in EKS
  10. Cost Considerations for EKS
  11. Common Challenges and Troubleshooting Tips
  12. Conclusion
  13. References

1. Introduction to Kubernetes

Before diving into EKS, let’s recap the basics of Kubernetes:

  • Kubernetes is an open-source container orchestration platform designed to automate deploying, scaling, and managing containerized applications (e.g., Docker containers).
  • Key Concepts:
    • Cluster: A set of machines (physical or virtual) that run Kubernetes.
    • Control Plane: The “brain” of the cluster, managing scheduling, scaling, and overall cluster health (components: API Server, etcd, Scheduler, Controller Manager).
    • Nodes: Worker machines (VMs or physical servers) that run containerized applications. Each node has a kubelet (to communicate with the control plane) and a container runtime (e.g., Docker).
    • Pods: The smallest deployable units in Kubernetes, containing one or more containers that share resources (network, storage).
    • Deployments: Declarative configurations for managing pods and replica sets (ensures a specified number of pod replicas are running).
    • Services: Expose pods to internal or external traffic (e.g., LoadBalancer for external access, ClusterIP for internal).

2. What is AWS EKS?

Amazon Elastic Kubernetes Service (EKS) is a fully managed Kubernetes service that allows you to run Kubernetes clusters on AWS without managing the control plane.

  • Managed Control Plane: AWS hosts and manages the Kubernetes control plane (masters) across multiple Availability Zones (AZs) for high availability (HA). This includes automated upgrades, patching, and scaling of the control plane.
  • Worker Nodes: You manage the worker nodes (EC2 instances) or use AWS Fargate (serverless compute for containers) to run pods without managing nodes.
  • Compatibility: EKS is certified Kubernetes-conformant, meaning it works with existing Kubernetes tools (e.g., kubectl, Helm) and applications.

3. Benefits of Using AWS EKS

EKS simplifies Kubernetes management while leveraging AWS’s ecosystem. Key benefits include:

  • Managed Control Plane: AWS handles control plane maintenance (upgrades, backups, HA), reducing operational overhead.
  • AWS Integration: Seamlessly connects with AWS services like:
    • IAM: For role-based access control (RBAC) to clusters and pods.
    • VPC: Isolate clusters with private subnets and security groups.
    • RDS/ElastiCache: Attach managed databases and caches to applications.
    • CloudWatch: Monitor cluster metrics and logs.
    • ELB/NLB: Expose applications via AWS load balancers.
  • Scalability: Scale clusters up/down with node groups or Fargate.
  • Security: Built-in AWS security features (encryption at rest/in transit, network policies, AWS WAF).
  • Compliance: Meets certifications like SOC, HIPAA, and GDPR, critical for regulated industries.

4. Prerequisites for Getting Started

Before creating your first EKS cluster, ensure you have the following:

  • AWS Account: Sign up at aws.amazon.com. You’ll need administrative access or permissions to create EKS clusters, IAM roles, and EC2 resources.
  • IAM Permissions: The user/role creating the cluster needs policies like AmazonEKSClusterPolicy, AmazonEKSServicePolicy, and permissions to manage VPCs, subnets, and security groups.
  • Tools Installed:
    • AWS CLI: To interact with AWS services via the command line.
    • kubectl: The Kubernetes command-line tool to manage clusters and applications.
    • eksctl: A CLI tool (built by AWS and Weaveworks) to simplify EKS cluster creation and management (recommended for beginners).

5. Setting Up Your AWS Account and Tools

Let’s walk through installing and configuring the required tools:

Step 1: Install AWS CLI

  1. Follow the AWS CLI installation guide for your OS (Windows, macOS, Linux).
  2. Configure the AWS CLI with your AWS credentials: 
    aws configure
    Enter your Access Key ID and Secret Access Key (created via IAM in the AWS Console), default region (e.g., us-west-2), and output format (e.g., json).

Step 2: Install kubectl

kubectl is required to interact with Kubernetes clusters. Install it via:

  • macOS/Linux: Use Homebrew or curl: 
    # Homebrew (macOS)  
brew install kubectl  

# Linux  
curl -LO "https://dl.k8s.io/release/v1.28.0/bin/linux/amd64/kubectl"  
chmod +x kubectl  
sudo mv kubectl /usr/local/bin/
  • Windows: Use Chocolatey: choco install kubernetes-cli.

Verify installation:

kubectl version --client

Step 3: Install eksctl

eksctl is the official CLI for EKS, simplifying cluster creation. Install it via:

  • macOS/Linux: 
    # Using Homebrew (macOS)  
brew tap weaveworks/tap  
brew install weaveworks/tap/eksctl  

# Linux (AMD64)  
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp  
sudo mv /tmp/eksctl /usr/local/bin
  • Windows: Use Chocolatey: choco install eksctl.

Verify installation:

eksctl version

6. Creating Your First EKS Cluster (Step-by-Step)

We’ll use eksctl to create a cluster, as it automates VPC, subnet, and IAM setup.

Step 1: Create a Cluster with eksctl

Run the following command to create a basic EKS cluster with a node group (adjust region, node type, and count as needed):

eksctl create cluster \  
  --name my-first-eks-cluster \  
  --region us-west-2 \  
  --nodegroup-name standard-workers \  
  --node-type t3.medium \  # Instance type (2 vCPU, 4GB RAM)  
  --nodes 2 \               # Initial number of nodes  
  --nodes-min 1 \           # Minimum nodes (for scaling)  
  --nodes-max 3             # Maximum nodes (for scaling)

What Happens Next?

  • eksctl creates a Kubernetes control plane across 3 AZs (for HA).
  • It provisions a VPC with public/private subnets, security groups, and IAM roles.
  • A node group (EC2 instances) is created in private subnets (for security).
  • The cluster configuration is added to your kubeconfig (so kubectl can connect).

Step 2: Verify Cluster Creation

Once the cluster is created (takes ~10–15 minutes), verify it with:

# Check cluster details  
eksctl get cluster --name my-first-eks-cluster --region us-west-2  

# Check nodes (worker nodes should be "Ready")  
kubectl get nodes

You should see output like:

NAME                                         STATUS   ROLES    AGE   VERSION  
ip-10-0-12-34.us-west-2.compute.internal   Ready    <none>   5m    v1.28.0-eks-abc123  
ip-10-0-56-78.us-west-2.compute.internal   Ready    <none>   5m    v1.28.0-eks-abc123

7. Deploying a Sample Application on EKS

Let’s deploy a simple Nginx web server to your EKS cluster.

Step 1: Create a Deployment

A deployment ensures Nginx pods are running and replicated. Create a file named nginx-deployment.yaml with:

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: nginx-deployment  
spec:  
  replicas: 2  # Run 2 pod replicas  
  selector:  
    matchLabels:  
      app: nginx  
  template:  
    metadata:  
      labels:  
        app: nginx  
    spec:  
      containers:  
      - name: nginx  
        image: nginx:1.21  # Official Nginx image  
        ports:  
        - containerPort: 80  # Nginx listens on port 80

Apply the deployment:

kubectl apply -f nginx-deployment.yaml

Step 2: Expose the Deployment with a Service

To access Nginx externally, create a LoadBalancer service. Create nginx-service.yaml:

apiVersion: v1  
kind: Service  
metadata:  
  name: nginx-service  
spec:  
  selector:  
    app: nginx  # Matches labels in the deployment  
  ports:  
    - protocol: TCP  
      port: 80        # Service port  
      targetPort: 80  # Pod port (Nginx)  
  type: LoadBalancer  # Provisions an AWS ELB

Apply the service:

kubectl apply -f nginx-service.yaml

Step 3: Access the Application

Get the external IP of the load balancer:

kubectl get service nginx-service

Output (wait a minute for the EXTERNAL-IP to populate):

NAME            TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)        AGE  
nginx-service   LoadBalancer   10.100.23.45    a1b2c3d4e5f6.us-west-2.elb.amazonaws.com   80:30080/TCP   2m

Visit the EXTERNAL-IP in your browser—you’ll see the Nginx welcome page!

8. Managing EKS Clusters: Key Operations

Now that your cluster is running, here are essential management tasks:

Scaling Node Groups

Scale the node group up/down with eksctl:

# Scale to 3 nodes  
eksctl scale nodegroup --cluster my-first-eks-cluster --name standard-workers --nodes 3

Upgrading the Cluster

EKS regularly releases Kubernetes versions. Upgrade the control plane and node groups with:

# Check available versions  
eksctl upgrade cluster --name my-first-eks-cluster --region us-west-2 --dry-run  

# Upgrade to the latest version  
eksctl upgrade cluster --name my-first-eks-cluster --region us-west-2

Adding a Node Group

Add a new node group (e.g., for GPU workloads) with:

eksctl create nodegroup --cluster my-first-eks-cluster --name gpu-workers --node-type p3.2xlarge --nodes 1

Deleting a Cluster

To avoid AWS charges, delete the cluster when done:

eksctl delete cluster --name my-first-eks-cluster --region us-west-2

9. Monitoring and Logging in EKS

EKS integrates with AWS CloudWatch for monitoring and logging.

Enable Container Insights

Container Insights collects metrics (CPU, memory, disk) and logs from pods and nodes. Enable it during cluster creation with eksctl:

eksctl create cluster --name my-cluster --region us-west-2 --enable-irsa --with-oidc --managed-nodegroups Default --enable-container-insights

Or enable it post-creation via the AWS Console.

View Metrics in CloudWatch

  • Go to the CloudWatch Console.
  • Navigate to Metrics > Container Insights > EKS Clusters.
  • View metrics like CPUUtilization, MemoryUtilization, and PodCount.

View Logs in CloudWatch Logs

Logs from pods, nodes, and the control plane are stored in CloudWatch Logs. To view pod logs:

# Stream logs from a pod  
kubectl logs <pod-name> -f  

# Or view in CloudWatch: Log Groups > /aws/eks/my-cluster/containers

10. Cost Considerations for EKS

EKS costs depend on the control plane, worker nodes, and AWS services used.

Key Costs:

  • Control Plane: $73/month per cluster (flat fee, regardless of size).
  • Worker Nodes: EC2 instance costs (e.g., t3.medium is ~$30/month per node).
  • Fargate: Pay per vCPU and memory used by pods (e.g., $0.04048 per vCPU hour + $0.004445 per GB hour).
  • Data Transfer: Outbound data from AWS (e.g., $0.09 per GB for the first 10TB).
  • CloudWatch: Metrics and logs (e.g., $0.01 per metric per month, $0.50 per GB of logs ingested).

Cost Optimization Tips:

  • Use Auto Scaling: Scale node groups down during off-hours.
  • Spot Instances: For non-critical workloads, use EC2 Spot Instances (up to 90% cheaper than On-Demand).
  • Fargate for Bursty Workloads: Avoid over-provisioning nodes for variable traffic.
  • Right-Size Nodes: Choose instance types that match your workload’s resource needs.

11. Common Challenges and Troubleshooting Tips

Cluster Creation Fails

  • Cause: Insufficient IAM permissions, invalid region, or resource limits.
  • Fix: Verify IAM policies (attach AdministratorAccess for testing), use a supported region (e.g., us-west-2), and check AWS Service Quotas.

kubectl Can’t Connect to Cluster

  • Cause: kubeconfig not updated or invalid AWS credentials.
  • Fix: Update kubeconfig with: 
    eksctl utils write-kubeconfig --cluster my-first-eks-cluster --region us-west-2

Pods Stuck in Pending State

  • Cause: Insufficient node resources (CPU/memory), or missing permissions to pull images.
  • Fix: Scale node groups, reduce pod resource requests, or check IAM roles for service accounts (IRSA) if using private images.

Node Group Not Joining Cluster

  • Cause: Security group rules blocking communication with the control plane.
  • Fix: Ensure node security groups allow inbound traffic on port 443 (Kubernetes API) from the control plane.

12. Conclusion

AWS EKS simplifies running Kubernetes on AWS by managing the control plane, integrating with AWS services, and providing scalability and security. In this guide, you learned:

  • The basics of Kubernetes and EKS.
  • How to set up tools like eksctl, kubectl, and AWS CLI.
  • To create an EKS cluster and deploy a sample application.
  • Key management tasks (scaling, upgrading) and monitoring with CloudWatch.

EKS is a powerful tool for building cloud-native applications. As you progress, explore advanced topics like Fargate, IAM Roles for Service Accounts (IRSA), Helm charts, and Kubernetes Network Policies.

13. References