Kubernetes on AWS: EKS vs. EC2 and How to Get Started

Kubernetes, an open-source container orchestration platform, has become a go-to solution for managing containerized applications at scale. It simplifies deployment, scaling, and the management of applications, providing a flexible framework for deploying software at speed. When integrated with AWS, a leading cloud services provider, Kubernetes can be easier to install, manage and scale.

AWS offers several options for running Kubernetes in the cloud. First, AWS provides the necessary infrastructure to host the Kubernetes clusters, eliminating the need to purchase and maintain servers on-premises, making it easier to scale your clusters, and providing redundancy and high availability.

In addition, Amazon provides the popular Elastic Kubernetes Service (EKS), which sets up a Kubernetes cluster in the cloud, with the Kubernetes control plane fully managed by Amazon. EKS handles many administrative tasks, such as installing and updating Kubernetes, and managing the API Server and etc database, reducing the learning curve and saving time for DevOps teams.

This is part of a series of articles about Kubernetes tools.

Benefits of Running Kubernetes on AWS 

Complete Control Over Your Servers

When running Kubernetes on AWS, you can choose the type of instances participating in your Kubernetes cluster, their sizes, and the region in which they’re hosted. This level of control allows businesses to optimize their resources based on their specific requirements, whether it’s compute-intensive applications or data-heavy workloads. Kubernetes adds an extra layer of control by managing how these servers are utilized.

Enhanced Security

One of the most significant advantages of running Kubernetes on AWS is the enhanced security. AWS offers several security features including Identity and Access Management (IAM), Security Groups, and Virtual Private Cloud (VPC). These features, combined with Kubernetes’ own security measures, provide a secure environment for your applications.

Portability

Kubernetes’ container-centric approach allows for high portability. Containers can run anywhere, regardless of the underlying infrastructure—be it on-premises, in a public cloud, or even a combination of both. This means you can develop locally, test on the same cluster, and deploy to production globally, all using the same toolset.

Scalability and Cloud Bursting

When running Kubernetes clusters in AWS, you can add more nodes to your cluster, or remove nodes, with the click of a button. You can also use tools like Cluster Autoscaler to automatically adjust the number of nodes to the requirements of your workloads. 

Another benefit of Kubernetes on AWS is the ability to burst into the cloud during peak demand. This means that you can utilize on-premises resources for standard demand, but when demand spikes, you can leverage AWS resources to scale up, without having to expand your on-premises data center. You pay for the extra resources only as long as you use them.

Kubernetes on AWS: Two Options and Their Pros and Cons 

There are three main ways to run Kubernetes on AWS and pros and cons you should be aware of:

Amazon EC2

Amazon Elastic Compute Cloud (EC2) is a web service that allows businesses to run application programs in the Amazon Web Services public cloud. Amazon EC2 allows scalable deployment of applications, allowing users to boot an Amazon Machine Image (AMI) to create a virtual machine, which Amazon calls an instance.

Running Kubernetes on AWS using Amazon EC2 gives you full control over your Kubernetes infrastructure. It allows you to choose the type of instances you want to use, the region and availability zones where your nodes will be located, and the Kubernetes version you want to run. This level of control is ideal for businesses with specific infrastructure requirements.

However, running Kubernetes on EC2 also means you are responsible for managing and maintaining the Kubernetes control plane. This includes tasks such as upgrading the control plane to new Kubernetes versions and scaling the control plane to handle more clusters.

Pros of running Kubernetes on EC2:

• Full control: You decide on instance types, regions, and Kubernetes versions.
• Optimized Infrastructure: Suitable for businesses with specific infrastructure demands.
• Flexibility: Easily accommodate specific requirements and configurations.

Cons of running Kubernetes on EC2:

• Maintenance overhead: You’re responsible for managing the Kubernetes control plane.
• Upgrades: Need to handle updates to newer Kubernetes versions manually.
• Scaling challenges: Scaling the control plane to manage more clusters can be intricate.

Amazon EKS

Amazon Elastic Kubernetes Service (EKS) is a managed service that makes it easy to run Kubernetes on AWS without needing to install, operate, and maintain your own Kubernetes control plane or nodes. Amazon EKS runs up-to-date versions of the open-source Kubernetes software, so you can use all the existing plugins and tooling from the Kubernetes community.

Running Kubernetes on AWS using Amazon EKS is a great option for businesses looking to offload the administrative burden. Amazon EKS takes care of the heavy lifting involved in managing the Kubernetes control plane. This includes patching, updates, and backups.

With Amazon EKS, you can also take advantage of AWS integrations with other AWS services. This includes features such as load balancing with ALB and NLB, IAM for RBAC, and AWS CloudTrail for logging and auditing.

Pros of running Kubernetes on EKS:

• Managed control plane: AWS handles tasks like patching, updates, and backups.
• Up-to-date Kubernetes versions: Utilize the latest features with automated upgrades and security patches.
• Seamless AWS integrations: Utilize AWS services like ALB, IAM, and CloudTrail efficiently.

Cons of running Kubernetes on EKS:

• Less control: Less granular control compared to an EC2-based solution, only supports some Kubernetes versions.
• Cost: Has an additional charge of $0.10 / hour / cluster for the control plane.
• Learning curve: For those already using Kubernetes, understanding AWS-specific integrations and configurations can take time.

Using Amazon ECR to Manage Container Images for Kubernetes

Amazon Elastic Container Registry (ECR) is a fully-managed Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. Amazon ECR is integrated with Amazon Elastic Container Service (ECS), simplifying your development to production workflow.

Running Kubernetes on AWS using Amazon ECR allows you to easily store, manage, and deploy Docker container images. You can use the Docker CLI to push, pull, and manage images. Amazon ECR provides a secure, scalable, and reliable registry on which to store your images.

While Amazon ECR is a great tool for managing Docker images, it is not a standalone option for running Kubernetes on AWS. Instead, it complements the other options (Amazon EC2 and EKS) by providing a robust and scalable solution for managing Docker images.

Related content: Read our guide to GKE vs AKS vs EKS (coming soon)

Tips from the expert

Itiel Shwartz

Co-Founder & CTO

Itiel is the CTO and co-founder of Komodor. He’s a big believer in dev empowerment and moving fast, has worked at eBay, Forter and Rookout (as the founding engineer). Itiel is a backend and infra developer turned “DevOps”, an avid public speaker that loves talking about things such as cloud infrastructure, Kubernetes, Python, observability, and R&D culture.

In my experience, here are tips that can help you better run Kubernetes on AWS:

Use Amazon EC2 Spot Instances Wisely

Incorporate Spot Instances for non-critical and fault-tolerant workloads to reduce costs significantly, but ensure you have a strategy for handling interruptions gracefully.

Implement Infrastructure as Code (IaC)

Use IaC tools like Terraform or AWS CloudFormation to define and manage your Kubernetes clusters and AWS resources. This ensures consistency and makes it easy to replicate environments.

Optimize EKS Node Group Configuration

Customize EKS node groups by mixing instance types and sizes to optimize costs and performance. Use instance types that match your workload requirements.

Enable IAM Roles for Service Accounts (IRSA)

Use IRSA to assign AWS IAM roles to Kubernetes service accounts, providing fine-grained permissions and avoiding the use of node IAM roles with overly broad permissions.

Leverage AWS Bottlerocket for Worker Nodes

Use Bottlerocket, a Linux-based OS purpose-built by AWS for running containers, which can improve security and operational consistency for EKS worker nodes.

Quick Start: Setting Up a Kubernetes Cluster on EKS 

Prerequisites

Make sure you have the following in place:

• An existing Virtual Private Cloud (VPC) and subnets that conform to the Amazon EKS criteria.
• The command-line tool, kubectl, installed on either your device or AWS CloudShell.
• The AWS Command Line Interface (CLI) version 2.12.3 or later, or 1.27.160 or later, installed and set up on your gadget or AWS CloudShell.

Creating an Amazon EKS Cluster

You can use the utility eksctl to generate a new cluster. The following command builds an Amazon EKS cluster that supports IPv4, utilizing the default Kubernetes variant from Amazon EKS within your default AWS region:

eksctl create cluster --name my-cluster --region region-code --version 1.27 --vpc-private-subnets subnet-ExampleID1,subnet-ExampleID2 --without-nodegroup

Before executing the command, please make the following adjustments:

• Substitute region-code with the AWS Region where you plan to create your cluster.
• Change my-cluster to your preferred cluster name. Note that this name should only include alphanumeric characters (sensitivity to case) and hyphens. It must start with a letter and should not exceed 100 characters. The name should be unique within the AWS Region and AWS account you’re setting the cluster up within.
• Replace 1.27 with a version that Amazon EKS supports.
• Modify the vpc-private-subnets values. Be sure to list at least two subnet IDs. 
• If you prefer to specify public subnets (which is less secure), amend --vpc-private-subnets to --vpc-public-subnets. Keep in mind that public subnets come with a route table linked to an internet gateway.

Verifying Cluster Deployment

It will take several minutes to provision your cluster. There will be several lines of output in your terminal detailing various steps of provisioning. The final line should be similar to this:

[✓]  EKS cluster "my-cluster" in "region-code" region is ready

Next, to verify communication with your cluster, execute the following command:

kubectl get svc

The example output should look something like this:

NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.100.0.1   <none>        443/TCP   28h

Best Practices for Deploying Kubernetes on AWS 

Design for High Availability

One of the key benefits of deploying Kubernetes on AWS is its high availability. However, to truly take advantage of this, it’s crucial to design your setup correctly:

• Deploy Kubernetes nodes across multiple Availability Zones (AZs): This prevents a single point of failure and ensures that your applications remain accessible even if one zone goes down.
• Use AWS Auto Scaling groups: This automatically adjusts the number of Amazon EC2 instances in response to traffic patterns, ensuring that your applications have enough resources to meet demand.

Use AWS Elastic Load Balancing (ELB): ELB automatically distributes incoming application traffic across multiple targets, such as Amazon EC2 instances, helping to increase the availability of your application.

Secure Your Cluster

Securing your Kubernetes cluster on AWS is of paramount importance. Here are some ways to enhance your cluster’s security:

• Use AWS Identity and Access Management (IAM): Carefully restrict access to your AWS resources. You can create and manage AWS users and groups and use permissions to allow or deny their access to AWS resources.
• Ensure that all communication is encrypted: This can be done using AWS Certificate Manager, which handles the complexity of creating, storing, and managing public and private SSL/TLS certificates.
• Use AWS Shield for DDoS protection: The service provides always-on detection and automatic inline mitigations that minimize application downtime and latency.

Backup and Disaster Recovery

Having a robust backup and disaster recovery plan is vital when deploying Kubernetes on AWS. Here’s how you can implement this:

• Regularly backup your entire AWS environment: This includes your EC2 instances, EBS volumes, and any databases you may be using. AWS Backup is a fully managed backup service that makes it easy to centralize and automate the backup of data across AWS services.
• Consider using AWS Snapshot: This will enable point-in-time recovery, allowing you to quickly restore your applications to a previous state in case of a disaster.
• Use AWS disaster recovery: Amazon provides a set of cloud-based disaster recovery services that enable rapid recovery of your IT infrastructure and data.

Optimize Costs

When deploying Kubernetes on AWS, cost optimization is crucial. Here are some ways to keep your costs in check:

• Use AWS Cost Explorer: This free tool can help visualize, understand, and manage AWS costs and usage over time. This tool gives you detailed insights into how much you are spending on your Kubernetes deployments and can help identify areas where you can cut costs, for example by right-sizing instances.
• Consider using AWS Savings Plans: These offer significant savings on AWS usage and are a flexible alternative to Reserved Instances.
• Use AWS Spot Instances: These offer spare Amazon EC2 compute capacity at discounts of up to 90% compared to On-Demand prices. However, be aware that these instances can be interrupted by AWS with two minutes of notification when AWS needs the capacity back.

Storage Optimization

Ensuring optimal storage use is another crucial aspect of deploying Kubernetes on AWS. Here are some tips to help optimize your storage:

• Use Amazon Elastic Block Store (EBS): EBS is designed for application workloads that require fine-tuned performance, and it provides the ability to scale up and down quickly.
• Use Amazon Elastic File System (EFS): EFS is a fully managed, elastic NFS file system that you can use with your AWS Cloud services and on-premises resources.
• Use Amazon S3 for object storage: S3 provides a simple, scalable, durable, and secure way to store and retrieve data at any time, from anywhere.