Kubernetes Migration – Expectations vs. Reality

The following is an AI-generated transcript:

Hey everyone, welcome back to the channel. This is, I think I just realized this, it’s the first episode of Open Source Cafe for 2024. The one and only Itiel from Komodor, so I’m very excited. Thanks a lot for joining Itiel, and how’s it going?

I’m good. I’m honored to be the first guest for 2024, so quite happy. New plans, doing some very nice things for the community. By the way, shout out to the Platformers Community, if you want to check that out, check out the links in the description below. And I’m in my new apartment, still settling in, so new life, new year. Very excited about this year. And today we’re talking about a very interesting topic, which is Kubernetes migration, and I believe we’re going to talk about expectations versus reality. So if you’re planning to, because this is what I hear, I go to a lot of conferences and I talk to people, and even if you go to Kubernetes conferences they’re like, “Are you using Kubernetes?” And they’re like, “No, we have heard about it, very cool, but we are not using Kubernetes because we don’t have the resources to migrate, and it’s just complex and we are not trained,” or whatever. These questions arise. So the first point that I would like to address is, can you maybe, before we move forward with the migration process, elaborate on the primary motivations that are driving organizations to migrate to Kubernetes? And I think that’s the question.

So let’s talk a bit about that. I think Kubernetes in general had a lot of promises, and the biggest ones are like, increase the velocity of your deployment, meaning you will be able to deploy features faster to customers, and easily scalable infrastructure because Kubernetes is promising you the HPA and the ability to add nodes, then scaling becomes less of an issue. Increase uptime, Kubernetes has a lot of mechanisms to be auto-healing and to make sure that your system is always up. Standardization, because the way Kubernetes works, which is basically Docker orchestration or container orchestration, then all of the different applications can look the same. And reduce cost and be a bit more, I will say, both cost and security are also very big parts of that. Those are derived from the fact that everything is standardized. So once everything is standardized, it’s easier to do this kind of cost allocation and to do a single security policy. So Kubernetes in general provides a lot, or it offers you a lot of promises. And I would say maybe the only thing that I didn’t mention is maybe it’s sexy right now, so it’s more like, “Okay, it’s hot, it’s sexy, we should migrate to Kubernetes, we don’t even know why but we heard that everyone else is doing that.” So it’s maybe not the best or real reason, but I think a lot of companies are migrating simply due to the popularity of Kubernetes.Is Kubernetes right for you?

I have another question. Can you maybe share a scenario where Kubernetes might not be the right fit, because that’s something I get asked quite a lot. With microservices you have like, you need a bigger team, and I believe there are projects and companies that are solving this issue, but just for the audience, just so that they can also understand, what are some of the scenarios? How do you know, that’s essentially like the summary of the question, how do you know that you are ready for the move?

It’s a great question, Kunal. So I will split the answer into two: if you are just starting your company now or doing some small project, Kubernetes is not the right answer for you. Kubernetes requires a lot of knowledge, a lot of understanding, it takes time to master, and it probably will be much easier working with AWS Fargate, or maybe serverless, or even hosting one single machine and uploading your application on top of that, because all of the cloud providers already have the ability to run a container. I think Kubernetes makes sense when you have some requirements that are very important to you, and it might be around scaling, or hybrid and cloud, and in cloud or cross-cloud. Those kinds of scenarios scream Kubernetes. And I think from a certain maturity of a company, Kubernetes, simply because its ability to standardize and also it’s easier to attract talent, it becomes a no-brainer. If you reach a certain size, maybe 100 developers to the north, you probably need to deep dive into Kubernetes if you are not doing that already, because the industry is going there. It is going to be the next big thing, and it will be harder for you to retain talent or hire talent by saying, “Oh, I’m running with bare metal E2 chef and everything is manually built,” or things like that. So if you are small, probably don’t use Kubernetes. If you are large, then you are probably already investigating migrating into Kubernetes. And in between, I love simple things, so if you can work without Kubernetes in the beginning, I think it can be a valuable thing for you. That being said, when you scale to a full-blown company, it’s probably a good way or time to use Kubernetes. I do recommend almost everyone to use Docker and containers. This will make the migration to Kubernetes easier on one hand, and it’s not that hard on the other. You can already use containers around AWS Fargate and Google has its container platform something, and you can even do serverless with Docker, so I do recommend containers for everyone. Fargate is from a certain scale. 

Couldn’t agree more, and I think it’s about what your use case is. There was a recent post that went viral that was about cloud providers, so like, “We saved millions and millions of dollars moving away from cloud providers.” I think it was the founder of hey.com. Yeah, yeah, that was a very popular post, and I think that’s exactly, if you have the resources, if it’s, whatever your scenario is, your current team is, and how big your company is, and so on and so forth. Speaking of cloud providers, when we talk about migration, the migration process, the number one question is, what are some of the key factors that you consider when choosing a cloud provider and the Kubernetes distribution for the migration?

I think when you choose the Kubernetes distribution and cloud provider, you should think first of all about the use case and your company and where you are going. So we have the big ones, like the hyperscalers like clouds, which is Google, AWS, Azure, and Ali if you’re running on China. But those are if you need all of these bells and whistles, basically to be able to grow very fast with a very big team and very complicated setups. Even GKE is a bit simpler. On the other hand, you have the more easy-to-use and much more friendly for newcomers, and I know Civo and where you work at Kunal, and also Digital Ocean has a good offering around that. So I think it depends on the type of company, and also if you are already using one of the cloud providers, it is much easier to stay on the cloud that you are in. You don’t need to migrate to a different cloud just to use Kubernetes. When it comes to choosing the relevant flavor, if you’re running on the cloud, then just choose whatever the cloud provider is offering you. And if you are running on-prem, there are three main options, which is Tanzu, I will say OpenShift and Rancher, and I think the majority of players are now in favor of OpenShift, as Rancher is doing, SUSE is doing a bit of changes in their offering.

Thanks for sharing. I believe there was recent news about some price changes for the on-prem funds. Yeah, yeah, of Tanzu.

Yeah, ever since Broadcom bought Tanzu, I feel that something is not like, they’re not sure how to handle it, so a lot of things are a bit weird there. I would also say SUSE acquired Rancher, and I think also in that regard, things changed a bit ever since the acquisition, or at least that’s what we see because I do see more people deciding on going to OpenShift, or on the other hand, not needing any of the cloud providers at all, any of the distribution at all, like running vanilla. 

It’s also about when we talk about the migration process. By the way, thanks a lot for sharing, and couldn’t agree more. Can you maybe also walk us through the assessment and the planning phase of a Kubernetes migration? Like, okay, now you decide, “Okay, now I want to do it.” What does the planning phase look like? What are some of the critical components?

I think it depends and varies between companies, but first of all, you need to choose the cloud, you need to choose the distribution, and then you need to choose what is the first application that I’m going to migrate and what would success look like. Will success be all of the applications running in the cloud? Will success be only the computer running Kubernetes and the rest not? You need to understand what’s a good win for you, and I really suggest picking a single application or a service and migrating it throughout all of the stages, which is the QA environment, the stage environment, the production environment, making everything Kubernetes. So usually it’s about what do we want to migrate, what are the timelines that we have, and then to choose the relevant service to migrate. Again, my recommendation is to choose something which is not super monolithic, super legacy, because those kinds of applications are hard, and at Komodor we meet a lot of customers that already tried a migration, they failed, and now they’re doing another migration with a much smaller scope. So I will also start with the smallest scope possible so my team can learn Kubernetes and we can see how it works without having the pressure of migrating all of the system. Migration in a well-established company is probably a couple of years process, like everything to migrate everything to Kubernetes.

Nice. Would you say Kubernetes migration can give you a migraine? 

I think that the amount of white hairs that Kubernetes has created for so many of our customers is crazy, and you meet very talented people, super talented people that are like, “The CTO told us that we need to migrate to Kubernetes, he said that it should take six months or something like that, and because we don’t have a lot of time, let’s do a shift and lift and make sure that it works.” But in reality, the majority of Kubernetes migrations I’m seeing did fail on the first occasion, mainly because they try to bite off more than they can chew and they try to migrate too many things at once without having this plan around the migration itself, and very clear steps. So what would you say are some of the common challenges encountered during the migration of applications and data to Kubernetes?

I think when migrating to Kubernetes there are five different things that make it especially hard. There is usually a lack of knowledge inside the team; they already know how to run the old infrastructure, not the new one. Lack of tools, a lot of the tools that you might be using can’t be applied to Kubernetes. Maybe it’s Chef, maybe it’s Puppet, maybe it’s how you monitor; those kinds of tools sometimes don’t work anymore. Lack of processes: your current setup works, right? If you work in a big company, it already works. You now need to reinvent the wheel in some ways around how do we handle issues, how do we handle new features, bugs, and so on. Multiple stakeholders is another problem. Kubernetes and all of the platform engineering concept is like a single team that is giving services to a lot of different personas, and it bites the FinOps guys, or the security, or the compliance, or the developers. So everyone has a stake and everyone is trying to push towards their direction. And the last thing I will say is hard deadlines. I see more and more that people underestimate how much time the migration will take. I can share with you that I used to work in a different company, a company named Rookout that was acquired by Dynatrace, and we met a lot of companies. I met a lot of companies when I worked there, it was six years ago, and when I met one of our customers, he told me that they have a Kubernetes migration plan. This was a very well-established company. I asked him, “How much time will it take you?” And he said, “We hope to do it by six months, in reality it will take us a year and that’s it.” And I told him that sounds very optimistic for such a big company that has so many different pieces, maybe it will take you a bit longer because Kubernetes is complex. And I can tell you that six months, six years later, they still didn’t fully finish the migration. So I think that’s a bit of a rare case, but for a company of their size, which is a couple of thousands of developers, I would estimate that migrating everything is maybe a five-year plan or something like that. Five years? In my experience, for banks that we meet, and some of our customers who are banks, it’s usually a 10-year plan for migrating to Kubernetes, so maybe even five years is optimistic. Again, migration is hard. Anyone that tells you that migration is easy is simply lying to you, because when you do the migration you find how fragile your system was to begin with. You dig in all of the buried bodies that someone put there to cope with different circumstances and to basically solve different issues. So migration is always hard. Migration to Kubernetes is even harder because it’s like migrating to a whole new technology step. 

So what would you say are some of the effective strategies for optimizing and continuously improving a Kubernetes environment? If you spend five years already, so post-migration, how would you continuously improve?

I think first of all, having the right tools. I’ll do a shameless plug here for Komodor, but make sure that you have the right tools in place. And in Kubernetes you have two main personas, as I see it. You have on one hand the platform engineers, or SRE, or infrastructure guys, depends how you call them. Those are the kind of people who need to operate large amounts of clusters and services with minimal downtime, minimal cost, and minimal toil on their hand. And those people need to make sure that they have the right tools in place to support them. On the other hand, we have the developers, and the developers usually are forced to use Kubernetes. It’s not like it’s in their roadmap; someone in the company decided Kubernetes is a good idea for them and now they are forced to use it. And for them, I think what’s important is education. Explain to them what is Kubernetes, why is it good for them, and why they should migrate to Kubernetes. And also, again, giving them the right tools to make sure that they are effective. Again, Komodor, but you can also do it with your own set of services or spend time and money on that. I can share that I recently, in my podcast, I interviewed the platform engineer group lead, the platform engineer at Forter, and he told me that they gave a lot of “candies” for the developers. They told them that they’re going to get auto-healing, they’re going to get Argo CD, which enables them to deploy faster. So you need to think, “What are the candies? What’s in it for me?” Not only for you as a platform engineer, but also for the end users of the platform.

That’s interesting. One thing that developers don’t like is working with YAML, for example. 

And I see a lot of people trying to solve that challenge, and now with the whole platform engineering thing, I mean, it’s not like new; platform engineers have been doing it previously as well. It’s not like it’s replacing DevOps. But can you maybe, for the folks who are just watching, touch a little bit on that as well and the community as well? So maybe you can share about that as well.

Great point. I think most mature companies have some sort of abstraction layer on top of Kubernetes, and it might be using Helm, it might be using templates in Backstage or anything else. And the goal is to abstract the creation of services from the developers. For example, in Rookout, again, my previous company, if a developer wanted to deploy a new service, all he needed to do is to have a file, maybe it was a YAML or a JSON, service name, type of programming language, and where is this Docker file or whatever. And that’s it. He didn’t need to understand or to know all of the different aspects of Kubernetes. And I think when you want to gain developer adoption when it comes to the creation phase, developers don’t really need to know all of the bits and bytes of “is this PVC with an SSD or is it configured a bit differently or whatever.” I think there are some important things that they need to control and tweak, but the majority of things should be delivered to them as a platform, as a product, and to make sure that they are able to onboard and to understand that as easily as possible. 

And when we talk about, we were talking about the challenges right now, can you also address some of the common pitfalls that companies face when they’re performing migration and how can they be mitigated? And by the way, for the Platformers Community, the joining link is in the description below so you can check that out.

I will say two or three biggest ones are, first of all, eating more than you can chew, meaning trying to move too fast without having the appropriate knowledge and tools. Sometimes you are trying too many different things. I meet a lot of companies that are migrating to Kubernetes and adding Istio on Day Zero, and Istio is very complex by itself. For example, combining both Istio and Kubernetes for someone who doesn’t know Kubernetes, it’s like a suicide for most people. So don’t do that. And another one is creating hate inside the company. You should find those people who will be your, how do you say it, the ones that will spread the news about Kubernetes inside your organization. So you need to find those early adopters and to harness them on your side. If you are a couple of months into the process and you don’t have any strong champion inside the team, you are doing something wrong. Go back to the drawing board and understand what you can bring them. And iteration, iteration, iteration. At the end of the day, it is a very long journey, migrating to Kubernetes. You need to treat it as one. It’s not like a sprint. I start, I finish. It’s, “Okay, now I have my service in staging, now I have my service in prod, now I have the CD.” So make sure that you have different stages and every stage can be completed and you have a clear definition of what success looks like.

So talk a bit more about some of the tools that you recommend for the migration journey. 

Yeah, so obviously Komodor, because we see quite a lot of companies using migration, choosing Komodor. But other than Komodor, I will say that the default stack is quite good. The classical one is Prometheus and Kibana, maybe Prometheus and Loki if you want to be only on the Grafana stack, or Elastic and Elastic APM if you want the Elastic stack. I’m a big fan of Datadog for monitoring. They are very costly, but they are a great product. And other than that, I think those are, make sure that you have the right monitoring in place. With Komodor you are getting also the management capabilities. Other nice tools are KLens or K9s to view the clusters. And I see more and more companies that are adding policies on top of their clusters, so OPA or Kyverno, or again, Komodor has those capabilities. Maybe one last very important family of tools are cost reduction tools. So you have again Komodor, but you have Cube Cost and you have Aperture. So make sure that you have all the right cost tools in place, and if not, your FinOps is probably going to be very angry at you very, very fast.

Very expensive, yeah. It can get pretty expensive pretty fast. Yeah, it’s very easy to get expensive. 

With Kubernetes, only changes in one YAML from 10 to 100 and that’s it, you pay 10x more and it was a typo.

Yeah, and it’s just, I think the first step of cost reduction is visibility. You don’t see where the money is going, you can’t reduce it. And as your infrastructure grows, it can get a bit more complex. So yeah, there are tools out there that you can try out to work with that. But when we talk about migration, obviously there are various stages, right? Day Zero, Day One, Day Two. So what are some of the essential elements for success and how do priorities and strategies shift across various stages? Maybe can you talk about the stages a little bit?

I think in Day Zero your goal is to make sure the application is running in production. That is a success, running, serving your own customers and everything is working in a reliable fashion. That’s like the Day Zero thing, and probably also secure and compliant per the company, if you need compliance, yes or no. Day Two is more on how do I reduce the toil, meaning how can I be much more effective as a platform engineer and how can I take all of those manual tasks, cluster upgrades, deprecated API and so on, that take me a lot of time and automate them? How do I bring value to other people inside the company, again, the security, the compliance, the developers? How do I empower them so they can also enjoy the benefits of Kubernetes? And cost is also something that usually should be addressed on Day Two and not Day One. It’s a luxury. Observability on the other hand should be in Day One because everything is going to crash. It’s the beginning, so you need to make sure that you have the right set of tools to begin with. So I think that’s it. 

Is there anything that you can hold off on?

I think things like cost, automated upgrades, and policies can also be, depending on the company, if you need those guardrails, yes or no. Even things like autoscaling, Carpenter, all of those tools help you mainly to save money or save time, but at the beginning, just make sure that you have this very thin pipeline that is working end to end. Even Argo, for example, can be added later on. You don’t need Argo from Day Zero. If it makes your life easier, then do it, but I think it can also be a bit later. I really love Argo and Helm, but it’s not a must. It’s more like I feel like a luxury. Even today, Argo is so popular, so maybe it’s even easier if you are using Argo from Day One.

Yeah, makes sense. When we talk about the current trends, can you maybe explore and share with us the layers of modern Kubernetes management? 

We can talk about data collection and aggregation, smart governance, for example.

Yeah, I think the most interesting thing that we’re seeing is a lot of hybrid environments. All of the big companies are running multiple Kubernetes flavors across different cloud providers and probably hybrid as well. And in order to do that, you need to make sure that you have the ability to manage dozens of different clusters across those different environments and having those tools. So like a centralized place for logging, a centralized place for metrics, a tool that allows you to manage those clusters, it might be Komodor, it might be OpenShift, it might be Rancher. But when you reach a certain level of maturity, you probably need one of those tools. I think governance, again, it’s like big boys problems, but governance, cluster hygiene are things that once you have a couple of dozens of clusters, they became from a nice to have to a must have. And in the industry, I see that every company is just increasing and getting more and more clusters. Like, if six years ago spinning up a cluster was super hard and you tried to push everything inside a single cluster, now everyone is like, “Yeah, sure, he gets a cluster and he gets a cluster and he gets a cluster.” So it changes how you treat clusters and how you monitor and operate clusters.

Amazing. Thanks a lot for sharing, Itiel, and I really learned a lot, and I’m sure the viewers, and myself as well, from this conversation. Very interesting topic indeed, we can go on and on, but one last question I’d like to ask you is, you mentioned Komodor being a helpful tool in Kubernetes migration. So where does Komodor fit in the picture, at which stage, and how do you help with Kubernetes migration? 

Yeah, sure. So great question, Kunal. So what we do is, we’re a tool that a lot of companies already used us to migrate, if it’s from Docker Swarm, if it’s from bare metal, if it’s from Cloud Foundry. We have a lot of success stories from companies using Komodor in order to expedite the migration process. This is because, first of all, we provide the right guardrails just out of the box, meaning best practices, configuring of the clusters and application, make sure everything works as it’s expected. Then what we do is reduce the complexity of Kubernetes, and that in turn allows you to troubleshoot much faster and allows you to bring more people, and it reduces the barrier for Kubernetes. Instead of learning a year of Kubernetes before being effective, with Komodor you can do it in a couple of days. So by simplifying most of the Kubernetes operation and troubleshooting, people don’t require that level of expertise in order to use it. And we help again with the setting up of the cluster, cost management, policy enforcement, and more particularly, our secret sauce is the ability to also help non-experts to troubleshoot their code and application. So think about it, a developer that doesn’t know Kubernetes is quite clueless around how do I fix issues. With Komodor, we have a guided troubleshooting experience that walks him hand-in-hand around where to look, what happened, why did it happen, and what you should do now for every Kubernetes issue. So yeah, I think that a tool like Komodor can be very, very meaningful when doing this migration, and we already have a lot of experience from huge companies that use us to expedite their migration process.

Amazing. Well, thanks a lot for sharing, Itiel, and all the links can be found in the description below. 

I’m very excited for this year and what Komodor does next. But very interesting because I think you started a Kubernetes community with a K, which went really, you know, really nicely, thousands of people joining, and then now there’s a dedicated community for Platformers. So it’s very interesting. Looking forward to seeing what you folks do, and always here to support as well. But for folks who are watching, if you don’t already know, go check out Komodor, and yeah, they have some nice open source projects as well. Yeah, thanks a lot for sharing, Itiel, really appreciate it and I’ll see you in the next one. Bye. Thank you. Meet KubeCon Paris. Bye. Bye. Entire team scrambling to figure out what went wrong.