Darren Rabbitt

Writing / / Interactive

Amazon EKS topology, explained

Amazon EKS runs the Kubernetes control plane for you and wires it into AWS networking, IAM and load balancing. These two diagrams show how the pieces fit, from the VPC down to the kubelet.

The AWS side

The AWS side: the VPC, load balancing, NAT and the worker nodes. Select a highlighted area, or use the buttons under the diagram.

Select a part of the diagram to see what it does.

Inside the cluster

Here, we take a closer look at the key Kubernetes components, such as the API server, etcd, scheduler, and kubelet, which work together to manage the lifecycle of applications within the cluster. Select a component to see its role.

Select a part of the diagram to see what it does.

Component notes

Amazon EKS Components

1. VPC (Virtual Private Cloud)

The EKS cluster is hosted within a VPC, ensuring network isolation and security by segregating resources. The VPC contains both public and private subnets:

  • Public Subnets: Used for resources requiring internet access, like Elastic Load Balancers (ELBs).
  • Private Subnets: Hosts internal services, such as worker nodes, ensuring they remain isolated from direct internet access for security.

2. Elastic Load Balancer (ELB)

The ELB distributes incoming application traffic across multiple targets, such as worker nodes, improving fault tolerance by preventing any single node from being overwhelmed.

  • Integration with Auto Scaling: ELB integrates with auto-scaling groups to dynamically adjust the number of worker nodes based on real-time traffic demands, ensuring resilience and high availability.

3. NAT Gateway and Bastion Host

The NAT Gateway and Bastion Host help ensure secure access and connectivity for your private subnet resources:

  • NAT Gateway: Allows instances in private subnets to securely access the internet (for updates, external services) while blocking inbound traffic.
  • Bastion Host: Provides secure SSH access to private instances without exposing them directly to the public internet.

4. Kubernetes Control Plane

Amazon EKS provides a fully managed Kubernetes control plane, handling key operations and maintaining the cluster state through various components:

  • API Server: Exposes the Kubernetes API for interacting with the cluster (e.g., using kubectl).
  • etcd: A distributed key-value store that stores the cluster’s configuration and state. Managed by AWS to ensure redundancy and availability.
  • Scheduler: Assigns pods to the most appropriate worker nodes based on resource requirements.
  • Controller Manager: Monitors and manages cluster operations like pod replication, node failures, and maintaining the desired state.

For more details on this, have a read of the user guide.

5. Worker Nodes (EC2 Instances)

Worker nodes are EC2 instances that run containerized applications inside Kubernetes pods. They are managed by the control plane and dynamically scaled as needed.

  • Auto Scaling Group: Automatically adjusts the number of worker nodes based on resource demand.
  • Kubernetes Pods: Worker nodes run containerized applications within pods, the smallest deployable units in Kubernetes. Each node can host multiple pods that are dynamically managed by the control plane.

For more details, refer to the AWS documentation

6. Kubernetes Networking

Kubernetes uses a flat network architecture, allowing seamless communication between all pods, regardless of the node they are on. EKS offers various networking solutions:

  • AWS VPC CNI Plugin: This plugin assigns IP addresses to pods directly from the VPC range, allowing them to function as native AWS resources, fully integrated into the AWS networking ecosystem. For more details, refer to the EKS networking requirements

Additionally, EKS supports Third-Party CNI Plugins, such as Calico, which provide enhanced network policy features and security.

Kubernetes Architecture Components

1. Controller Managers

Kubernetes uses two types of controller managers to manage key aspects of the control plane and cloud resources within the Kubernetes cluster:

  • Kube Controller Manager: Handles Kubernetes controllers like node controllers, replication controllers, and other tasks necessary for maintaining the cluster’s state. For more details, read kube-controller-manager
  • Cloud Controller Manager: Manages cloud-specific integrations (e.g., load balancers, volumes) and ensures smooth interaction between Kubernetes and cloud services. Read this article for more details on cloud-controller-manager

2. etcd

etcd is the distributed key-value store that stores all data and state related to the Kubernetes cluster.

  • Data Store: Holds cluster configuration, workloads, and metadata.
  • Fault Tolerance: Provides redundancy by replicating data across multiple nodes, ensuring recovery from failures.
  • State Persistence: Critical for persisting the cluster state and configuration across reboots and failures.

Additional details on etcd

3. API Server

The API Server acts as the gateway to the Kubernetes control plane and handles all administrative requests.

  • REST Interface: Receives and validates requests (via kubectl or other tools) before updating the cluster state in etcd.
  • Interaction Hub: Coordinates with other components (controllers, schedulers) within the control plane to execute changes and maintain the cluster’s state.

Additional details on api-server

4. Scheduler

The Kubernetes Scheduler is responsible for allocating workloads (pods) to the most suitable worker nodes based on resource availability and policies.

  • Resource Management: Ensures pods are assigned to nodes that meet their resource requirements (CPU, memory).
  • Continuous Monitoring: Regularly checks cluster resource usage to optimize workload distribution.

Additional details on scheduler

5. Kubelet

Kubelet is the node agent responsible for managing pods and containers on each worker node.

  • Pod Lifecycle: Ensures containers within pods are running as expected.
  • Cluster Communication: Kubelet interacts with the API Server to receive pod instructions, monitor node health, and report status updates back to the control plane.
  • Container Management: Manages container lifecycles, ensuring they meet the specifications defined in the pod manifest.

Additional details on kubelet

6. Kube-proxy

Kube-proxy handles networking across the cluster, maintaining rules for routing traffic between pods and services.

  • Network Management: Kube-proxy ensures that both internal and external network requests are forwarded to the appropriate pod or service across nodes.
  • Service Discovery: Maintains service discovery and load balancing within the cluster to ensure connectivity and resource access.

Additional details on kube-proxy