#051 – Surviving the Shift: From Legacy Monoliths to Day 2 Chaos with Hayato Shimizu (Digitalis)

[Music] Kubernetes for Humans.

Itiel Shwartz: Hello everyone. My name is Itiel Shwartz and I’m your host today in the Kubernetes for humans podcast. Today I have in the show Hayato. I hope I pronounce it correctly. How are you?

Hayato Shimizu: Perfect. Thank you very much. Yeah, that’s absolutely perfect pronunciation. Thank you very much. and all good thanks and happy to be here.

Itiel Shwartz: So maybe tell us a bit about yourself. What do you do? Where do where do you work? When did you start working with Kubernetes? And let’s start with some basic intro.

Hayato Shimizu: Oh, this could take a very long time. So, but I hope you’re ready. but, so my name is Hayato Shimizu. I’m one of the co-founders of a company called Digitalis. what we do is essentially a kind of a technology consultancy company around DevOps data platforms. we do everything from the kind of a cloud infrastructure, DevOps operations. So from taking from day zero to day two of the platforms we focus on some of the most difficult problems and that that’s how kind of we made our name. So you know infrastructure you know security some of the most difficult security problems in infrastructure etc and data as well. So we are experts around we have we

Itiel Shwartz: You say you say the buzz word like data and Kubernetes and security so like so many like different hard stuff all like blended together. maybe like share a bit like how did you get started and arrive at where you are today as a company?

Hayato Shimizu: Yeah. Okay. Well, let me just kind of just go back for quite a few years now. so my career really started in the kind of late 90s when I was in the postgraduate around it was kind of going on neural nets machine learning side of things back then and the computers are really slow and the training used to see take weeks on these large scale machines and it was all we knew back then but then I kind of you know doing a lot of coding C and Perl and on Unix systems and R and so on and and then joined a company called Citrix. I don’t know if you know them but they’re a kind of application delivery company.

Itiel Shwartz: Yeah, they’re huge, aren’t they? Or am I trying like confusing with Citrix?

Hayato Shimizu: Yeah, they they they’re very large now. And I think I was employee 1300 or something like that early late 90s. I was kind of focusing a bit on the network protocol side of things there. but then after that I joined a large a very large bank and writing a kind of a what we called CI/CD about you know full kind of application delivery automation taking source code into kind of all the build builds and test pipelines into a large scale WebSphere application kind of clusters right

Hayato Shimizu: That’s probably the equivalent of you know Kubernetes back then in terms of being applications in a standardized fashions with a you know standardized messaging protocol the start standardized kind of application deployment containers etc. It’s not the containers that we know today but that’s that that was kind of you know that’s how applications used to be deployed probably still is in in

Itiel Shwartz: Is I would guess most of the people who started with WebSphere didn’t really left that or like most of them

Hayato Shimizu: That’s right I mean back then you know I was kind of introduced to some of these kind of a you know the containerization technologies back then and but I was kind of you massive Linux advocate as well. They used to follow the kernel update and I remember like OpenVZ came in and introduced kind of a resource isolation kind of Linux kernel 2.4 back in in those days and later on in kernel 2.6 six I think it was the concept cgroups was introduced into Linux. again very much excited reading about this all but u I think it was Docker you know really kind of you know standardized the use of cgroups and containerization into into Linux yeah

Itiel Shwartz: When was like 2.4 before like I feel it’s like a lot of time ago like what what what year range are we you look quite young if it’s okay to say in the podcast

Hayato Shimizu: 2006 maybe yeah around you know I started using Linux in two 1996

Itiel Shwartz: Okay

Hayato Shimizu: Yeah yeah that’s when my Windows machine crashed and destroyed my university documents u project I was working on and I said I vowed never to use Windows at that point. Although things have turned around since then, but

Itiel Shwartz: Yeah. Okay. So like you were in the bank, you were doing all of those things. Then what like how did you like ended up like founding your own company and when did you start playing with Kubernetes?

Hayato Shimizu: Yeah. So I think even I joined a company called Sky in 2008 and I think the concept of platform as a service was started to emerge. so being able to deploy applications more in a flexible way rather than just the web Java applications. So and then there was services like Heroku and if you remember Hioku they’re still going on. are still there like I think there

Hayato Shimizu: That’s right

Hayato Shimizu: But then Cloud Foundry was the open-source kind of a platform service that was out was written in Ruby at the time and we actually adopted this to deploy a lot of the internal applications back back then and so just the whole concept of you know being able to deploy applications and democratize the developers so that they can deploy the applications themselves rather than having to go to the ops say hey we got this app and can you deploy it and get it all you know wired up everything now actually kind of you know plat I suppose the the term knowledge is like platform engineering came out later but we were trying to just say to the devs you know look after your own applications here’s the platform to do it there’s the API etc and then so

Itiel Shwartz: So like you’re in sky right like just doing a recap you’re starting like to work on like a platform Platform engineers back then it wasn’t called right like platform engineers but the same concept of and mentality of basically trying to empower other people right inside organization many developers and then what happens or and then what

Hayato Shimizu: Yeah so I actually kind of moved on from that side of things and joined a company called DataStax focusing on Apache Cassandra and u you know large scale data platforms

Hayato Shimizu: And I was there involved two two and a half years on or so and then but my itch to go back to kind of as helping enterprises was there. So I decided to leave DataStax and set up my own consultancy company focusing around DevOps and data problems and essentially Kubernetes was really part of the delivery of you know DevOps services and so on. So

Itiel Shwartz: Maybe let’s do like a stop for a second cuz I think you know a lot of our listeners are platform engineers they are operation people and you after like years of you know working in like quite big companies and like smaller companies you decide you know to like to do a consultancy right so maybe like share a bit about that like why how was it scary maybe like share a bit about like moving from you know working as like a normal employee into like a indep independent worker basically and like your own business owner. So like why how

Hayato Shimizu: Yeah it was very scary to set up my own but I had a great partner to set this up with really important that you know it’s not you know one person trying to set up a company like this as well but it was really kind of you know we’re helping each other setting this company up. There’s a lot of work to do setting up a company up anyway. It’s not tech as you may know it’s not just there’s a whole bunch of thing that goes with it. but also you know I just had a my second child born around that times as well. So it was a very scary thing to move on from you know someone paying me salary to setting up our own company. But I think I’ve always had the itch to help enterprises because you know there there’s always big problems in especially around platforms and you know I really wanted to kind of leverage my experience knowledge to help enterprises to kind of you know introduce some of the kind of possibly what more scary tech like Kubernetes at the time you know in into these enterprises and help them get it set up and So that’s what we did and luckily there were customers who wanted to take our services.

Itiel Shwartz: How did you get the first customer just you know I think like that’s like the hardest thing right like or or maybe not.

Hayato Shimizu: It was was quite interesting because when we were at DataStax and I was running the consultancy arm of a part of the DataStax and luckily they were able to introduce some customers as we left the company. they said look these customers need help and so we kind of got them engaged so it’s really kind of an introduction through a previous employer but I think you know initially when you’re starting out you really need friends and well even when you establish you know it’s not you know you need friends you need to be part of the ecosystem you know we’re a services company but it’s you know we can’t just be services company we need to be friends with product companies as and which is why we’re having you know this lovely session right now to talk to people about you know the setting up companies and and having friends is so important.

Itiel Shwartz: Okay. I think I think it’s like very cool that you got your first customer through the employer like through your previous employer. I guess it’s not the most standard way of like starting like your own consultancy but it’s like a it’s a great story. So you start your own business, you start to consult to with pe to people. Is it still the same company or like so like what what happened since then?

Hayato Shimizu: Yeah, it’s still the same company and you know we’ve grown you know organically over time. We haven’t actually taken any funding as such. So, so we’ve grown organically through kind of our customer acquisitions and I trust that all of our you know customers are happy with our services and we do consultancy like you know the day zero to day one but we also do day two so we have a very sticky situation where you know some of our customer being with us for many many years in terms we we provide managed service of the operations. So you know that that’s part of what we do and looking after things like the Kubernetes and databases and DevOps automation and also cloud infrastructure and and so on.

Itiel Shwartz: Okay. Okay. So you know like we didn’t had a lot of consultants over like the podcast. So maybe like sure as I guess you see quite a lot of companies. So first question maybe like who who is your customer like when we are now talking about your customer is it like SMB enterprises and so on and what is the most common challenges that that you see like I think that it’s quite interesting.

Hayato Shimizu: Yeah I think that’s really good to talk about. So our customers have typically been well so it ranges from kind of you know medium scale enterprises to large enterprises like banks, insurance companies and so on where you know probably some of the biggest challenges are not technologies themselves but the kind of people aspect of it when you’ve got multiple vendors involved looking after these large scale platforms and then how we you know work with other organizations etc. And you know things like Kubernetes is now taking up a lot of the applications into these platforms involving application developers from over here the operations from team from another vendor you know Kubernetesmemes from subject matter experts from us and so on and and so on. But generally speaking, you know, when when things are okay, it’s it it’s great. It’s when the incidents happen and that’s you know when we go on the triage calls and how we try and identify the root causes etc. And you know people have different experiences and you know different companies have different expertise and trying to bring all of that together to come up with a root cause analysis can be quite challenging and

Itiel Shwartz: Yeah that that’s for sure like do you see most of the challenges in like the day two or the day zero or like the day one what what what part is the trickier for companies? I think you know every stage is always going to challenge brings different challenges of course but I think you know in the context of say Kubernetes I think you know it’s just made it so easy for the application developers to deploy you know containerized applications to platform so that that’s great I think you know the largest challenge is once you go into production and the scale that it hits them with can also be a challenge. How how to make sure that you know the platforms are designed to scale and sometimes unprecedented load comes in etc. and you know or it it just you know you can scale applications horizontally in combination that’s what it’s designed to do but then applications themselves may may not be designed for scale sometimes as well. So those are some of the challenges there. data is always a a big challenge in terms when it comes to scale as well. yeah and and of course you know these kind of challenges al always brings in incidents and and and also you know the platform engineering teams generally have got enormous backlog of you know things to do whilst they’re firefighting in in production systems and I’ve been in that situation many times whereby you know the Jira list is you know so long and you know the application team go hey can we do this and can we do this and we’d love to service all of them there’s you know always certain amount of resources available to deliver all this and then the production problems happen firefighting so that’s always you know a big challenge there to prioritize production problems and the demand Maybe maybe like share some tips right like let’s say I’m a big company I don’t know I have WebSphere I want to use Kubernetes until now I didn’t had any and you know I can call you right and then my life will be easier but let’s say like our customers our listeners that are trying that themselves like what is the biggest mistakes people are doing when are trying to migrate to Kubernetes and like what’s your suggestions on how to make your life a bit a bit better

Hayato Shimizu: Yeah I think you know the there’s there’s a bit of homework that people should do before kind of you know thinking about the migration into Kubernetes and I think of course you know there’s a training things that people go on to available but it’s really knowing what are the you know sharp corners are around deploy into Kubernetes and understanding you know what it’s really really good at what is the the fit with my existing applications and just doing that upfront work before just like you know just containerizing your applications deploying it and see what you know see the problems later on anti anticipate the problems yeah what could happen with your application that’s currently running in WebSphere and you’re so dependent on WebSphere and then you rewrite the application to be deployed into Kubernetes some unexpected events may happen. like yeah

Itiel Shwartz: Oops Okay and and and maybe like I get it so like that’s the day zero like let’s do your homework try to understand let’s say I already migrated something to Kubernetes like now it’s like a day one problem what is the biggest challenge in that regards that you see customers are struggling and where do you see maybe like the line between day one to day two if any right like day zero is easy I have zero Kubernetes now I have something let’s say running production and now I’m like in day one or day two So when do I graduate from like day one to day two if any and like what are the biggest challenges in day one?

Hayato Shimizu: Yeah. And so the biggest challenges I suppose is the kind of you know the norm of the Kubernetes applications in general is you know stateless nature of the application right you want to make sure that the and applications can handle restarts application can handle kind of scaling horizontally adding new nodes to the to the cluster you know StatefulSet if you have to do do that But, you know really because a lot of the u you know traditional applications have been written with a very static kind of set of you know running environments in mind. So a lot of the codes are being written with a lot of in-memory dependencies and then when the application restarts you know like suddenly but killed by the you know Kubernetes because it was exceeding the CPU utilization and then just suddenly killed and things like that and then oh but we we just lost something that was in memory sort of thing and so that those are some probably some of the day-one challenges and rewriting the application to be compatible with the way the Kubernetes works, right?

Itiel Shwartz: Yeah. No, like the lift and shift that a lot of people are hoping that will work. Like I rarely see see those things working like usually there’s a lot of like hidden magic in how applications are currently working with outside of Kubernetes and you forgot that someone is handling all of this like memory and networking and storage and you just don’t think about it and then all of the sudden you are forced to rethink all of your bad decision or like even like good decision from back in the day. H okay so now let’s talk maybe about day two. I rewrite my application, right? Like I’m already in Kubernetes, I have production workload. Like now what’s the biggest challenge or like what’s the biggest like difficulties?

Hayato Shimizu: Yeah. Yeah. I think you know the the biggest challenge might be you know there’s so many challenges that enterprises face with Kubernetes but let’s talk about capacity for example right u yeah so as you know kind of Kubernetes when you’re deploying applications you have set say set c set c set c set c set c set c set c set c set c set c set amount of resources and things like that and make you know so determining what sort of resources can be used by these applications. You have to run in production. obviously you got to do the load testing in in in non-production environments first like but you never know what the load is going to be in production until you actually hit it. You can you can estimate. I’ve been in so many situations where the business will tell us oh you’ll be this much load and we see nowhere near the load or sometimes way over the load and

Itiel Shwartz: No one knows your loads like you know I worked quite a lot like in the industry like even now if you ask me like what will be common load in a year from now or like in three months from now like I don’t know like everything is possible it’s it’s probably going to be worse than I thought that’s like the only thing that is is almost always guaranteed that it is going to different and and then like how do customers like adopt to that like to this uncertainty or this unknown?

Hayato Shimizu: Yeah. So I mean you know you got the platform level scaling so if you’re in the you know one of the public clouds it’s kind of easy to set up the Kubernetes so that you can auto scale you’ve got a whole bunch of technologies like you know Karpenter or similar tools to be able to scale but you know every single bit of technology requires a bit of learning right and then you know it’s also the experience of knowing how these technologies behave in these situations where when the subtle load comes in and how the application behaves and you know obviously the monitoring is very very important to make sure that you’re monitoring the applications so that you know when the you know your API responses breaching the service level agreements around the you know 99 percentile latencies and so on right but so having that in place and and then when that is breached then where’s the why didn’t the autoscaling happen properly or what what’s going on there and then you know you usually it could be the config issues limits that you may have set u there’s so many different tuning parameters with any kind of the application deploy was that so what is the tuning parameter or what is the knob that I need to dial to kind of you know to adapt to these type types of you know node that just came in or

Itiel Shwartz: Okay. Okay. So, we are like almost running out of time. Maybe like share a bit about like your predictions for the future like where are we going as an industry? What’s hot and what’s not? Like give your take on like where are we going to be in the upcoming few years?

Hayato Shimizu: Well, I think you know what’s obviously industry is kind of you know adopting the AI technologies and it’s it’s absolutely amazing what these technologies can do and I’ve been we’ve been kind of you know trying Komodor and what one of the interesting things is the is the root cause analysis that it provided with and u we’ve run this against a number of our Kubernetes clusters and it’s actually found problems that you know we knew about but its analysis was correct and I think you know it if we had these kind of tools what we were discussing earlier about multiple people trying to work on on on an instance and then understand well having you know having these kind of systems suggesting the root causes quickly allows us do to do the resolution very quickly and you know stops potentially the heated conversations that may happen in a very stressful situation. Everyone’s stressed. Everyone wants to fix the problem, but everyone’s got different opinions. But sometimes, you know, a machine-assisted analysis you know can be kind of a a a good thing. I’m really looking forward to these kind of technologies you know helping us drive the enterprise platform stability going forward.

Itiel Shwartz: Okay. Okay. That’s like a super like I’m with you on the take like overall I think we’re like investing quite a lot in that area. Anything else that you want to say is like you have a new book, a new lecture, a new anything that you want to give a shout out to.

Hayato Shimizu: So no I think you know I’ I’ve got just a you know just completely different conversation but I do run a different company called Axonauts which is a a product company that came out of digitalis so this is focusing more around Apache Cassandra and Kafka platform so that you know it’s essentially a day two tooling for for those two those you know data platforms specifically not not Kubernetes related but you know I thought that might be quite interesting.

Itiel Shwartz: Okay, that sounds quite cool. So we’ll add it to the show notes if we can and I think with that we’ll conclude the episode. Thank you very much. Pleasure having you and goodbye.

Hayato Shimizu: Thank you for hosting me.

[Music] Kubernetes for Humans.