How Does Load Balancing Work in Kubernetes?
Kubernetes, an open-source platform for automating deployment, scaling, and operations of application containers, has become increasingly popular in managing containerized applications across various environments. In such a dynamic environment, load balancing is a critical component. Load balancing in Kubernetes refers to the method of distributing network traffic across multiple servers or pods to ensure that no single pod gets overwhelmed, thereby enhancing the overall efficiency and reliability of applications.
Kubernetes facilitates load balancing in two primary ways: internally within the cluster, and externally for accessing applications from outside the cluster. Internally, you can use a ClusterIP service to direct traffic to pods, ensuring smooth operation even if pods are replaced in a Deployment. Externally, you would typically use services of type NodePort or LoadBalancer, or the more powerful Ingress controllers, to manage inbound traffic from outside the cluster.
Keep in mind that Kubernetes does not provide a built-in load balancer. It can provide basic traffic routing functionality, but for full load balancing, you will need to use an external load balancer. On most cloud providers, this is done automatically when you set up a Service of type LoadBalancer. An Ingress object provides more control and lets you integrate with the load balancer of your choice.
This is part of a series of articles about Kubernetes management
Types of Load Balancers in Kubernetes
Internal Traffic Routing with ClusterIP Service
ClusterIP is the default type of Service in Kubernetes. A Service is a way to provide a stable network address for applications running in one or more pods within the cluster. ClusterIP services allow you to expose an application internally, making it accessible only within the cluster and not from the outside world. This can be particularly useful for multi-tier applications where some parts of the application need to communicate with others but should not be exposed externally.
External Traffic Routing with NodePort
NodePort is another type of Kubernetes Service that allows you to expose your application externally. It does this by opening a specific port on each node in the cluster, and any traffic that is sent to this port is forwarded to the Service. This makes your application accessible from outside the cluster, which can be particularly useful for applications that need to be accessed by external users or systems.
External Load Balancing with a LoadBalancer Service
LoadBalancer is another type of Service that provides an external IP address. Kubernetes does not provide built-in load balancer functionality—on most cloud providers, creating a LoadBalancer of Service automatically provisions a load balancer for the Service. This load balancer can then distribute incoming traffic to multiple pods, ensuring that the workload is evenly distributed.
Using an Ingress with an External Load Balancer
An Ingress is a Kubernetes resource that manages external access to the services in a cluster. It can provide load balancing, SSL termination, and name-based virtual hosting, among other things. When combined with an external load balancer, it can provide a flexible solution for managing traffic to your applications.
Tips from the expert
Itiel Shwartz
Co-Founder & CTO
In my experience, here are tips that can help you better manage Kubernetes load balancers:
Use multiple Ingress controllers
Deploy multiple Ingress controllers to handle different types of traffic and improve scalability.
Enable session persistence
Ensure session persistence (sticky sessions) for stateful applications to maintain user session continuity.
Implement health checks
Configure health checks for your load balancers to ensure traffic is only routed to healthy pods.
Use DNS-based load balancing
Leverage DNS-based load balancing for geographically distributed clusters to route traffic efficiently.
Utilize service meshes
Implement service meshes like Istio or Linkerd for advanced traffic management and load balancing features.
Should You Use a LoadBalancer Service or Ingress?
The question of whether to use a LoadBalancer service or Ingress often arises when dealing with Kubernetes systems.
A LoadBalancer Service is a viable option when you only have a handful of services you need to reveal. As the number of Services grows, you might run into a few difficulties. For example, restricted ability to specify URLs, dependence on cloud providers, lack of SSL termination, and no support for URL rewriting or rate limiting
An Ingress is more suitable when you need to evenly distribute loads across a large number of services. Ingress sets rules for incoming traffic, managing routing based on the URL path and hostname. This allows you to use a LoadBalancer to expose multiple services while controlling the inbound network flow to your cluster.
In addition to the basic functionality of the Kubernetes LoadBalancer service, Ingress controllers typically provide enhanced URL routing, URL path-oriented routing, routing based on hostname, SSL termination, URL rewriting, and rate limiting. It’s important to note that there are multiple Ingress controllers you can use according to your use case—there are ingress controllers available from NGINX, Envoy, Traefik, and other providers.
How to Define a LoadBalancer Service
Let’s assume you need to establish an NGINX server network cluster with three instances, and you want to load balance between them. Let’s see how to create the pods and a LoadBalancer Service that sets up an external cloud load balancer.
Note: This will only work if your cluster is deployed in a cloud provider that supports load balancing for Kubernetes, for example AWS, Azure, or Google Cloud.
To accomplish this, create a ReplicaSet composed of three pods, each running the NGINX server. Here is an example Deployment manifest that can set this up:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
ports:
- containerPort: 80
Here is how to define a LoadBalancer Service:
apiVersion: v1
kind: Service
metadata:
name: nginx-loadbalancer
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- name: http
port: 80
targetPort: 8080
protocol: TCP
When you apply both the Deployment and the Service in your cluster, the Service will use a selector to associate itself with the corresponding three pods that share the same labels.
You can run the following command to see the running pods:
kubectl get pods --output=wide
The output looks something like this:
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-deployment-1569f5bf4-fgthj 1/1 Running 0 18h 10.244.2.5 k8s-node01 <none> <none> nginx-deployment-1569f5bf4-hjklv 1/1 Running 0 18h 10.244.1.4 k8s-node02 <none> <none> nginx-deployment-1569f5bf4-zxcvb 1/1 Running 0 18h 10.244.0.6 k8s-node03 <none> <none>
Kubernetes uses Endpoints as objects to keep the IPs of each pod updated. Each service creates its own Endpoint object, which allows the cluster to keep track of each matching pod’s IPs automatically.
You can describe your service to see how Kubernetes Endpoints enumerate the matching pods:
kubectl describe service nginx-loadbalancer
The output looks something like this:
Name: nginx-loadbalancer Namespace: default Labels: <none> Annotations: <none> Selector: app=nginx Type: LoadBalancer IP Families: <none> IP: 10.100.200.0 IPs: 10.100.200.0 LoadBalancer Ingress: 192.0.2.1 Port: http 80/TCP TargetPort: 8080/TCP NodePort: http 31234/TCP Endpoints: 10.244.2.5:8080,10.244.1.4:8080,10.244.0.6:8080 Session Affinity: None External Traffic Policy: Cluster Events: <none>
Kubernetes Endpoints keep the IPs updated, making traffic forwarding smoother via the Service. If pods are added, modified, or removed from the service selector, the Endpoints will update automatically. This functionality enables the Service to direct network traffic accurately to the right pods, because the Endpoints register the application’s necessary routing destination.
Each endpoint has the pod’s network IP and port and carries the same name as the Service. You can see this by describing the endpoints:
kubectl describe endpoints nginx-loadbalancer
The output looks something like this:
Name: nginx-loadbalancer
Namespace: default
Labels: <none>
Annotations: <none>
Subsets:
Addresses: 10.244.2.5, 10.244.1.4, 10.244.0.6
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
http 8080 TCP
Endpoints establish a stable network identity for the pods. They dynamically adjust as pod state changes, ensuring balanced distribution of network traffic among a dynamic set of pods.
Kubernetes Troubleshooting & Reliability with Komodor
Kubernetes load balancing is complex and involves multiple components; you might experience errors that are difficult to diagnose and fix. Without the right tools and expertise in place, the troubleshooting process can become stressful, ineffective and time-consuming. Some best practices can help minimize the chances of things breaking down, but eventually something will go wrong – simply because it can.
This is where Komodor comes in – Komodor is the Continuous Kubernetes Reliability Platform, designed to democratize K8s expertise across the organization and enable engineering teams to leverage its full value.
Komodor’s platform empowers developers to confidently monitor and troubleshoot their workloads while allowing cluster operators to enforce standardization and optimize performance.
By leveraging Komodor, companies of all sizes significantly improve reliability, productivity, and velocity. Or, to put it simply – Komodor helps you spend less time and resources on managing Kubernetes, and more time on innovating at scale.
If you are interested in checking out Komodor, use this link to sign up for a Free Trial.
