Komodor is a Kubernetes management platform that empowers everyone from Platform engineers to Developers to stop firefighting, simplify operations and proactively improve the health of their workloads and infrastructure.
Proactively detect & remediate issues in your clusters & workloads.
Easily operate & manage K8s clusters at scale.
Reduce costs without compromising on performance.
Empower developers with self-service K8s troubleshooting.
Simplify and accelerate K8s migration for everyone.
Fix things fast with AI-powered root cause analysis.
Explore our K8s guides, e-books and webinars.
Learn about K8s trends & best practices from our experts.
Listen to K8s adoption stories from seasoned industry veterans.
The missing UI for Helm – a simplified way of working with Helm.
Visualize Crossplane resources and speed up troubleshooting.
Validate, clean & secure your K8s YAMLs.
Navigate the community-driven K8s ecosystem map.
Kubernetes 101: A comprehensive guide
Expert tips for debugging Kubernetes
Tools and best practices
Kubernetes monitoring best practices
Understand Kubernetes & Container exit codes in simple terms
Exploring the building blocks of Kubernetes
Cost factors, challenges and solutions
Kubectl commands at your fingertips
Understanding K8s versions & getting the latest version
Rancher overview, tutorial and alternatives
Kubernetes management tools: Lens vs alternatives
Troubleshooting and fixing 5xx server errors
Solving common Git errors and issues
Who we are, and our promise for the future of K8s.
Have a question for us? Write us.
Come aboard the K8s ship – we’re hiring!
Hear’s what they’re saying about Komodor in the news.
Kubectl is a command line tool for Kubernetes that allows you to communicate and control Kubernetes clusters.
Kubectl works by communicating with the Kubernetes API server: It authenticates with the Master Node of your cluster and makes API calls to perform management actions. You can use kubectl to create, inspect, update, and delete Kubernetes objects, deploy applications, inspect and manage cluster resources, and view logs.
Kubectl offers three techniques.
kubectl create pod
kubectl delete service
kubectl create -f manifest.yaml
kubectl apply -f manifest.yaml
The object notation is usually in the form of object type, followed by a slash, followed by the object name. For example, to address the “mypod” pod, the notation will be “pods/mypod”. Some commands accept different notations (e.g., “kubectl get pod mypod”), so you might see different notations used throughout this article.
For more information, including a complete list of kubectl operations, see the kubectl reference documentation.
This is part of an extensive series of guides about DevOps.
This is the TL;DR section of this article—a quick-access section to remind you of the most important commands. No explanations are given, but the commands are explained further down in the article.
To get the current contexts configured in your kubeconfig file:
$ kubectl config get-contexts
To switch context:
$ kubectl config use-context minikube
To get the name of the containers of a running pod:
$ kubectl get pod MYPOD -o 'jsonpath={.spec.containers[*].name}'
To get the value of a secret (if you have the base64 command available):
$ kubectl -n mynamespace get secret MYSECRET -o 'jsonpath={.data.DB_PASSWORD}' | base64 -d SuperSecretPassword
If you don’t have the base64 command available, you can use the go-template:
$ kubectl -n mynamespace get secret MYSECRET -o ‘go-template={{.data.DB_PASSWORD | base64decode}}’
An example of a filter in jsonpath:
$ kubectl get pod nginx -o 'jsonpath={.spec.containers[?(@.name=="nginx")].image}' nginx:1.9.1
To create a secret from your current Docker credentials to pull images from a private registry:
$ kubectl create secret generic SECRETNAME --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson
To forward pod port 8080 to your local computer on port 8888:
$ kubectl port-forward MYPOD 8888:8080
To test RBAC rules:
$ kubectl --as=system:serviceaccount:MYNS:MYSA auth can-i get configmap/MYCM yes
To login to a specific container within a pod:
“kubectl exec -it <podname> -c <containername> — /bin/bash”
This will open a Bash shell inside the specified container, allowing you to run commands and interact with the container’s file system.
Itiel Shwartz
Co-Founder & CTO
In my experience, here are tips that can help you better utilize kubectl:
Create aliases for frequently used kubectl commands to save time and reduce errors.
Manage multiple clusters efficiently by using context switching with kubectl config set-context.
Extract specific fields from kubectl output using JSONPath or jq for more precise information.
Use –dry-run to test configurations before applying changes to your cluster.
Create shell scripts for repetitive kubectl tasks to streamline your workflows.
Kubectl uses “contexts” to know how to communicate with the cluster. Contexts are stored in a kubeconfig file, which can store multiple contexts. Contexts will usually be provided by some other commands related to the control plane or some other management commands.
Such commands will usually add context to your kubeconfig file. The default kubectl config file is located at $HOME/.kube/config. You can use a different kubectl config file by specifying the --kubeconfig=PATH arguments on the kubectl command line.
$HOME/.kube/config
--kubeconfig=PATH
To list the contexts available in your kubeconfig file, use “get-contexts”:
$ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE * arn:aws:eks:... arn:aws:eks:... arn:aws:eks:... minikube minikube minikube default
To switch from one context to another, use “use-context”:
$ kubectl config use-context minikube Switched to context "minikube". $ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE arn:aws:eks:... arn:aws:eks:... arn:aws:eks:... * minikube minikube minikube default
If you have only one context (i.e. you are managing a single cluster), then you are not concerned about contexts. If you are working with two or more clusters, you might want to consider using the --context command line option.
--context
The reason is that if you often switch between clusters, you are pretty much guaranteed that one day you will run a kubectl command that you intended for another cluster, and the consequences might very well be dramatic.
With this in mind, you might want to set your default context to something innocuous, like minikube, and force yourself to explicitly provide the --context option on every kubectl command.
Kubectl get is probably the command you will use the most, so it deserves its own section. Kubectl get can retrieve information about all Kubernetes objects, as well as nodes in the Kubernetes data plane. The most common Kubernetes objects you are likely to query are pods, services, deployments, stateful sets, and secrets.
Kubectl get
Kubectl get offers a range of output formats:
-o wide
-o yaml
-o json
-o jsonpath
-o go-template
Here are some examples of commonly used commands:
List all pods in the default namespace (in this case, there are no pods in the default namespace):
default
$ kubectl get pod No resources found in default namespace.
Get more information about a given pod:
$ kubectl -n mynamespace get po mypod-0 -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mypod-0 2/2 Running 0 4d3h 192.168.181.98 node1.lan [none] [none]
The READY column shows how many containers are in the “ready” state in the given pod. The IP column shows the allocated IP address of this pod inside the Kubernetes cluster. The NODE column shows on which node the pod is running (or is scheduled).
Get the full state in YAML of the same pod as above:
$ kubectl -n mynamespace get pods/mypod -o yaml apiVersion: v1 kind: Pod metadata: [ --- snip --- ]
To get the services in the default namespace:
$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.100.0.1 [none] 443/TCP 24d mysql ClusterIP 10.100.78.115 [none] 3306/TCP 22d mysql-headless ClusterIP None [none] 3306/TCP 22d
The kubernetes service is used to access the Kubernetes API from within the cluster and is usually located in the default namespace.
kubernetes
To get the value of a secret:
$ kubectl -n mynamespace get secrets MYSECRET -o 'jsonpath={.data.DB_PASSWORD}' | base64 -d SuperSecretPassword
Kubernetes stores secrets as base64-encoded values, hence the | base64 -d at the end to show a human-readable value. The base64 command is available on many operating systems, but you might need to change for your OS.
| base64 -d
There are many other commands to retrieve information besides kubectl get. In no particular order, here is a selection of the most important:
kubectl get
Retrieve the version strings for both the kubectl program and the Kubernetes server:
$ kubectl version --short Client Version: v1.21.2 Server Version: v1.21.2-eks-0389ca3
To retrieve a lot of information about Kubernetes objects that are not shown by kubectl get, use kubectl describe:
kubectl describe
$ kubectl -n mynamespace describe pods/mypod-0 Name: mypod-0 Namespace: mynamespace [ --- snip --- ] Containers: myapp: Container ID: docker://9958565f45e7ddd27a5a7ca88254d3d27c162bb7612f0bb4c04781f154b33fd9 Image: myapp:3 Port: 8080/TCP Host Port: 0/TCP State: Running Started: Thu, 26 Aug 2021 13:31:13 +0100 Ready: True [ --- snip --- ] Events: [none]
This command provides a lot of information and is very useful to understand why a pod is having problems. It shows the detailed status of all the containers, the mounted volumes, and the events associated with the pod.
In most cases, running this command is all you need to understand why your pod is not working the way you want. If the problem is still not apparent, however, you might want to run kubectl describe node, which will provide you with useful information about the Kubernetes nodes, such as the cumulative allocated resources like CPU and memory. Also, make sure to check out the “Conditions” section, which might flag some issues.
kubectl describe node
$ kubectl describe nodes Name: minikube Roles: control-plane,master [ --- snip --- ] Conditions: Type Status LastHeartbeatTime LastTransitionTime Reason Message ---- ------ ----------------- ------------------ ------ ------- MemoryPressure False Mon, 11 Oct 2021 07:36:08 +0545 Sun, 29 Aug 2021 22:00:50 +0545 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Mon, 11 Oct 2021 07:36:08 +0545 Sun, 29 Aug 2021 22:00:50 +0545 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Mon, 11 Oct 2021 07:36:08 +0545 Sun, 29 Aug 2021 22:00:50 +0545 KubeletHasSufficientPID kubelet has sufficient PID available Ready True Mon, 11 Oct 2021 07:36:08 +0545 Sun, 29 Aug 2021 22:01:03 +0545 KubeletReady kubelet is posting ready status [ --- snip --- ] Allocated resources: (Total limits may be over 100 percent, i.e., overcommitted.) Resource Requests Limits -------- -------- ------ cpu 790m (39%) 6 (300%) memory 390Mi (5%) 3242Mi (41%) ephemeral-storage 100Mi (0%) 0 (0%) hugepages-2Mi 0 (0%) 0 (0%) Events:
To get the logs from a container:
$ kubectl logs mypod-0 -c myapp
The -c argument is used to specify which container in the pod to query; if the pod has only one container, this argument is optional. Please note that kubectl logs will show the logs from the current log file on the node that is running the pod. Such log files are subject to rotation, so kubectl logs will show only the most recent log entries.
-c
kubectl logs
Generally speaking, you would use kubectl logs only for debugging during the development stage. For production environments, you will most likely use a log aggregator and query for the logs from there. Nevertheless, this command is one of your first ports of call to troubleshoot misbehaving pods and connectivity problems.
It should be noted that in many cases, you can get the logs of system pods running in the kube-system namespace, such as the API server, the DNS server, kube-proxy pods, etc. Some clusters have been set up to run those services directly on the control node(s) rather than pods, so in such cases you will need to check the logs directory on the control node(s).
kube-system
kube-proxy
These commands directly instruct Kubernetes to perform a specific operation on a given object. This section will show the most common such commands in no particular order. These are useful during the development stages, but you should definitely avoid them in a production environment.Use kubectl create to create a Kubernetes object, except for pods that are created using the kubectl run command. So to create a pod directly:
kubectl create
kubectl run
$ kubectl run debug --image=busybox -- sleep infinity pod/debug created $ kubectl get pod NAME READY STATUS RESTARTS AGE debug 1/1 Running 0 6s
Using the run command is good enough for running simple pods. For more complex pods with multiple containers, persistent volumes, mounted ConfigMaps or Secrets, and a lot of environment variables, this method will very quickly become intractable, and it’s much easier to use a manifest file.To delete the pod:
run
$ kubectl delete pod/debug pod "debug" deleted
You can directly create all other Kubernetes objects using the kubectl create command. Here is a (silly) example for creating a Deployment:
$ kubectl create deployment nginx --image=nginx --replicas=2 deployment.apps/nginx created $ kubectl get pod NAME READY STATUS RESTARTS AGE nginx-6799fc88d8-6clhp 1/1 Running 0 9s nginx-6799fc88d8-cjz56 1/1 Running 0 9s
Again, you can delete the deployment using the delete command:
delete
$ kubectl delete deployment nginx deployment.apps "nginx" deleted
You can modify the deployment like so (this will kick off your favorite text editor):
$ kubectl edit deployment nginx
If you don’t want kubectl to start a text editor, you can use the replace command. You will need to obtain the object’s state in YAML format first:
replace
$ kubectl get deploy/nginx -o yaml ] tmp.yaml $ # edit tmp.yaml $ kubectl replace -f tmp.yaml
Please note that not all elements of a Kubernetes object can be modified after it has been created.
Another method to modify a Kubernetes object is to use the patch command. With the above deployment object, such a command could look like this:
$ kubectl patch deployment/nginx -p '{"spec": {"replicas": 3}}' deployment.apps/nginx patched $ kubectl get pod NAME READY STATUS RESTARTS AGE nginx-6799fc88d8-8jtw9 1/1 Running 0 11m nginx-6799fc88d8-dgkqr 1/1 Running 0 24s nginx-6799fc88d8-vqnkj 1/1 Running 0 24s
The -p argument accepts either JSON or YAML, although JSON tends to be a bit easier to type in. In any case, you are probably thinking that this isn’t a user-friendly way of doing things, and you would be right. This is one example where working with manifest files and using the declarative style make things much easier.
-p
You can also use the kubectl set command to easily modify certain objects, but it is limited in scope and not generic. It is mostly used to modify the image of pods or deployments, for example:
$ kubectl set image deploy/nginx nginx=nginx:1.9.1 deployment.apps/nginx image updated $ kubectl get pod nginx-684c8b4f65-c6ws6 -o 'jsonpath={.spec.containers[?(@.name=="nginx")].image}' nginx:1.9.1
BTW, the jsonpath used above contains a filter on the “containers” array to extract only the information we want. Another way to modify objects is to add or change the labels and annotations attached to it, for example:
$ kubectl label deploy/nginx --overwrite app=frontend deployment.apps/nginx labeled $ kubectl get deploy nginx -o jsonpath='{.metadata.labels}' {"app":"frontend"} $ kubectl annotate deploy/nginx test=yes deployment.apps/nginx annotated $ kubectl get deploy/nginx -o 'jsonpath={.metadata.annotations}' {"deployment.kubernetes.io/revision":"2","test":"yes"}
Related content: Read our guide to kubectl patch
When you want to work in declarative mode, you will have a manifest file (or a set of them), and essentially use only one command:
$ kubectl apply -f X
Here, X is a file or a directory (you can add multiple -f arguments if necessary). If you want to make some changes to your existing Kubernetes objects, just modify the manifest files and run kubectl apply again. This will compute the differences between your desired state and the existing state and make the necessary changes to reconcile them. The only caveat is that it won’t delete objects that you removed from your manifest files.
-f
kubectl apply
If you use kustomization files, you should use the -k option instead, and the argument must be a directory:
$ kubectl apply -k DIR
There is a special case where the reconciliation is problematic, which has to do with the Kubernetes authorization system. The reasons for this are quite obscure and are touched on here if you are interested. If you make changes to RBAC resources or roles, you should use the kubectl auth reconcile command. You can find more information on this in the official Kubernetes documentation.
kubectl auth reconcile
You can delete objects declared in manifests files by using the following command:
$ kubectl delete -f X
Related content: Read our guide to kubectl apply
Here are some useful commands to manage deployments (in a general sense, so that also includes stateful sets and daemon sets).
When you update a deployment (or stateful set or daemon set), you can view the status of the update using kubectl rollout status deploy/myapp. You can cancel a rollout using this command: kubectl rollout undo statefulset/myapp and get a history of changes using kubectl rollout history deployment/myapp. In practice, however, these commands are seldom used because you would manage deployments using Helm or some other similar software.
kubectl rollout status deploy/myapp
kubectl rollout undo statefulset/myapp
kubectl rollout history deployment/myapp
You can modify the number of pods running for a given deployment using kubectl scale --replicas=N deploy/myapp, where N is the new desired number of replicas. The end result is the same as using kubectl edit deploy/myapp and modifying the number of replicas there.
kubectl scale --replicas=N deploy/myapp
kubectl edit deploy/myapp
Again, in practice, you are likely either to use Helm to perform static, manual changes or the pod autoscaler and not perform any manual operation. BTW, you can set up some basic autoscaling using kubectl autoscale, although that works only with one metric: CPU utilization.
kubectl autoscale
If your Docker registry is private, you will need to pass some credentials to Kubernetes. The easiest way to do this is to first login using the Docker command line, for example, on AWS:
$ aws ecr get-login-password | docker login -u AWS --password-stdin ACCOUNT_ID.dkr.ecr.REGION.amazonaws.com
Once logged in, you can manually create the secret like so:
You can then use the secret SECRETNAME in the pod specification inside your manifest file like so:
imagePullSecrets: [name: SECRETNAME]
Please note that such secrets are usually valid only for a limited period of time.
It is very common to forget to set imagePullSecrets, in which case your pods will show a “pull image error”. So, if you see that error and you are using a private registry, you might want to check that you did set imagePullSecrets, and that the secret is not expired.
imagePullSecrets
Related content: Read our guides to kubectl rollout and kubectl config.
This section will list a bunch of commands that are very useful to interact with your pods.
Nothing beats getting a shell to a running pod! Here’s how to do it (you can skip the -c argument if you have only one container running inside the pod):
$ kubectl -n NS exec -it POD -c CONTAINER -- sh
Please note that the shell will be executed as the default user specified by the container’s image. Kubectl currently doesn’t allow you to run the shell as another user (e.g., root), although there are some plugins that can do that. Also note that you need an executable shell available inside the container image (so you can’t use it on distroless images, for example).
Consequently, you might need to think about how such containers will be troubleshooted. You might want to have two versions of such containers: One for development (with the shell and maybe other programs) and one for production (which would be distroless).
You can quickly create a link between a given port number on a given pod running inside the Kubernetes cluster and your computer (this is called port forwarding – learn more in our detailed guide on kubectl port forwarding). Here is how to do it, assuming your pod exposes port 8080:
You can then open up port 8888 on your local computer and that will forward the traffic to the pod MYPOD on its port 8080. This is very useful to do a quick check on your pod to verify that it looks OK.
You can copy files and directories using the kubectl cp command. We won’t go into detail here, but it is sometimes better for inspecting large and/or binary files on your computer rather than battling with the shell and a couple of utilities in an exec session.
kubectl cp
Finally, you can use kubectl attach to attach to the container terminal, although this is usually of limited use because you get the output from kubectl logs and, in general, apps don’t read stdin.
kubectl attach
Related content: Read our guide to kubectl logs
It is useful to be able to complete your shell commands, and kubectl has a ready-made solution for that:
$ source kubectl completion bash
Kubectl supports a number of shells, and you will need to consult Kubernetes documentation for more details. Please note you will also need to install the bash-completion package for your Operating System (if not already installed).
bash-completion
Also, the above command will enable autocompletion only for the current session; to make it permanent, add this command to your shell initialization file.
Pro tip: If, like me, you use bash and like to have an alias to abbreviate “kubectl” into “k”, add the following line into your ~/.bashrc file to make autocompletion work when using just “k”:
~/.bashrc
complete -F __start_kubectl k
You can show the APIs and resource types available on your cluster like so:
$ kubectl api-versions admissionregistration.k8s.io/v1 admissionregistration.k8s.io/v1beta1 apiextensions.k8s.io/v1 [ --- snip --- ] $ kubectl api-resources NAME SHORTNAMES APIVERSION NAMESPACED KIND bindings v1 true Binding componentstatuses cs v1 false ComponentStatus configmaps cm v1 true ConfigMap endpoints ep v1 true Endpoints events ev v1 true Event [ --- snip --- ]
Finally, a very useful command to test your RBAC rules is kubectl auth can-i. Here is an example to test whether a given service account can read a certain config map:
kubectl auth can-i
You will obviously need to replace the capitalized placeholders with the names that make sense for your setup.
Related content: Read our guide to kubectl autocomplete.
Visit these official resources for more information about the usage of kubectl:
Kubernetes is a complex system, and often, something will go wrong, simply because it can. In situations like this, you’ll likely begin the troubleshooting process by reverting to some of the above kubectl commands to try and determine the root cause. This process, however, can often run out of hand and turn into a stressful, ineffective, and time-consuming task.
This is the reason why we created Komodor, a tool that helps dev and ops teams stop wasting their precious time looking for needles in (hay)stacks every time things go wrong.
Acting as a single source of truth (SSOT) for all of your k8s troubleshooting needs, Komodor offers:
If you are interested in checking out Komodor, use this link to sign up for a Free Trial.
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of DevOps.
Authored by Codefresh
Authored by Granulate
Authored by Swimm
Share:
and start using Komodor in seconds!