#053 – The Road to Distributed AI and Kubernetes Infrastructure with Matt Butcher (Fermyon) & Ari Weil (Akamai)

Itiel Shwartz: Hello everyone and welcome to another episodes of the Kubernetes for Humans podcast. My name is Itiel Shwartz and today I have in the show two different guests. please introduce yourself.

Matt Butcher: I’m Matt Butcher. I was the creator of Helm in the Kubernetes ecosystem and also the illustrated children’s guide to Kubernetes. in 2021 I started a company called Fermyon and as of January 1st of 2026 we are now part of Akamai.

Itiel Shwartz: Okay. It’s like a interesting story that we we need to to unfold both hand both illustration so many different things and yeah and uh

Ari Weil: yeah and I’m Ari at Akamai technologies I manage our product marketing and developer advocacy teams and been working really closely with the CNCF and the Linux Foundation on our open source strategy and how we can basically maintain a really good community around Kubernetes within Akamai and then across the entirety of the landscape. So, it was great meeting Matt through some of that partnership and working with Helm together with some of our co- customers. Um, and we’re excited to talk to you.

Itiel Shwartz: Okay, that sounds good. If you guys like both can do like a quick brief like each one on on Helmself around like what led you to the tech industry in a short and maybe a bit on like the first time you started to work with Kubernetes basically like how why and then we’ll move on into like more recent times. But I always love to understand like the story behind people.

Matt Butcher: Ari, you want to start?

Ari Weil: No, you should start, please. [laughter]

Matt Butcher: I have a tendency to tell long stories, so buckle in. I got into tech on accident. I was hired at the same time as another guy named Matt and they accidentally swapped our jobs and I was supposed to mow the lawn and he was supposed to do computer programming stuff and he never showed up for his job and I ended up learning how to write code. and that was back in the late 90s. I really got into cloud native around the time HP I was working at HP cloud and they were were really really invested in OpenStack. And so I started working on that went from there into containers. Got really excited about containers. began working on platform as a service stuff using containers. that got me to Kubernetes. got really invested in Kubernetes and and it just kind of taken off from there. I think first version of Kubernetes I ran was maybe either the very tail end of 1.0 or the very right after 1.1 was released. So I’ve been around for for a very long time since it was pretty rough around the edges. And it never ceases to amaze me how rapidly and how deeply that ecosystem has evolved over the last 10 11 years.

Ari Weil: Well, from so similar I think origin story for me for for technology. I graduated from from college and had gone to school for English. which got me ready to be a writer and not do a hell of a lot else. and so when I got my first job to actually pay the bills, I ended up in sales at a startup. Um, and I didn’t realize it was a startup at the time, but thankfully, as much as I was was not great at and didn’t enjoy selling, they needed somebody to help them write the website, write a bunch of emails out to customers. Google was just getting started. And so, it was by figuring out how to create a website, work on a bidding strategy, and work on enabling the marketing side of a startup that I got involved in technology because from there it was you really need to understand what the product is, what we’re trying to build, how it works. and I transitioned in actually working on the product instead of working on the copy. Um, and that, you know, kind of led to database administration and systems design, systems engineering, scaling. Um, and it led to a a pretty interesting career where I’ve been working sort of all across the go to market engine, primarily on the vendor side after that initial uh, startup. But, you know, what really got me interested in containers in the beginning was I was working at a startup um, in the web performance space when Kubernetes 1.0 was released. And that was something that a lot of our customers got super excited about and they started talking about how this was really going to change the way that they were going to architect their applications to take advantage of the cloud. Um, and it was something from a front-end optimization perspective and working with e-commerce travel companies and a lot of financial services businesses that we really understood the power that Kubernetes represented to the enterprise. By then I wasn’t coding anymore. I was working on sort of product management, product marketing. But that that visceral reaction that people had I think from a customer perspective really got me excited right from you know version w1.0

Itiel Shwartz: it’s like a it’s like one of the best stories I heard like going from a salesperson in a startup to to that like it’s not the it’s not like the most traditional like crowd that I ever heard and that’s that’s quite cool. What happened to the startup? Did they succeed? Did they close? [laughter]

Ari Weil: They did. So the very first online insurance quoting engine ended up buying by was bought by First Union Bank and then Wakovia. yeah, so that was that was interesting and it led to Geico and Progressive and sort of everybody else that can get you a quote in 15 minutes or less. But it’s crazy that in 1999 that didn’t exist.

Itiel Shwartz: No, that’s even like a happier story than I thought. okay, like there’s I feel there’s a lot to cover and I know also like Akami is very strong with like the Kubernetes and Linode, right? And like there’s a lot of history there. But before that, I have to ask like Helm illustration guide maybe like it’s it’s more for you Matt like maybe share with us like both both those stories because it was a interesting start.

Matt Butcher: Sure. the fun fact the illustrated children guide and Helm were both sort of born on the same day. so I was working for a small startup called Deis. I left HP and and joined Deis because I was so interested in and invested in the potential of containers and and while HP was chugging along on the infrastructure layer pre you know and Docker was really young I I kind of wanted to jump in there right away and be part of this kind of wave of cloud native computing and so I was working for Deis and we were building a PaaS system that was based on containers one of the first PaaS systems based on containers we were trying to provide an open-source alternative to something like Heroku. and we had an all company meeting and part of the goal of this meeting was to introduce the entire company to Kubernetes. And so I had been on the early research team working on Kubernetes and the CTO of Deas came to me and said okay I’d really like you to give an introduction to the entire company on Kubernetes and thiSRE is Engine Yard by the way was the parent company of of Deis. So thiSRE is like bunch of people had done Ruby on rails for a long time and stuff like that. And I said, “You want me to give a presentation on Kubernetes to like all the engineers?” And he laughed. He goes, “No, no, no, no, no. I mean everybody. Finance is going to be there. Marketing is going to be there. Everybody is going to be there.” And I went, “I have to explain Kubernetes to everybody.” Um, okay. So, I went home that night and and and sat around trying to figure this out and was kind of looking at my kids stuffed animals and thought, “Oh, it’d be really funny if I posed some stuffed animals and did some, you know, like a slideshow and and called it like, I don’t know, the children’s guide to Kubernetes or something like that.” So, I put together this really cheesy slideshow called the illustrated children’s guide to Kubernetes, which was not illustrated at all. It had pictures of my kids stuffed animals. And I gave this talk to the to the whole company. I don’t I don’t know how well the talk landed with everybody but certainly it got a lot of people sort of engaged including the Deis marketing department who was like thiSRE is fantastic like I understand Kubernetes we should turn this into a real thing. And so they hired an actual illustrator who illustrated all of that up and we you know handed them out at the first KubeCon. Uh, and then we ended up donating both the book and the and the rights to the characters over to the CNCF. Uh, and so they have continued to use those uh, characters ever since, which is really weird because they’re based on my kids stuffed animals. My kids are now grown and I walk into a conference and it’s like a flashback to when my daughter was five. You know, that kind of thing.

Itiel Shwartz: Your kids are getting nothing. like only the CNCF like [laughter] from this

Matt Butcher: I don’t I don’t think anybody’s making money off the illustrated children’s guide but I am happy to to know that people continue to find it helpful and I know there are a whole bunch of books in that series now I did the first couple together with Karen Chu the one that that that we co co-worked on that initially but I think at this point it’s kind of outgrown our original vision and it’s really fun to watch u but on that same day in order to get people doing work with Kubernetes. We had an all company or a big hackathon project and my team we got together and went we know exactly what’s missing from the Kubernetes ecosystem. It’s it’s a package manager. And so we sat down and and and hacked out what what might be the world’s worst package manager. we called it Kate’s Place, K8s Place. and had this whole coffee shop motif. And so I think like you know installation packages were called shots or something like that. This whole coffee theme and at the end of the hackathon we presented this to the to the company. at stake was a $75 gift card. There were three of us on my team Reis, Jack and I. we won the hackathon and split our $75 gift card three ways. I spent mine on coffee. Um, and the day after, um, I was back in my office and, uh, and my phone rang and the CEO and CTO were both on the other end of the line and I went, “Oh no, you know what? What’ I do? [laughter] Why why are the CEO and the CTO both calling me?” And they said, “So, uh, so we we we were thinking about that hackathon project and and you know, the idea of a package manager for KuberneteSRE is a really good idea.” And I said, “Oh, yeah. I thought I thought so too. And they said, “We want you to keep working on that. In fact, we want to give you a team to to go build this.” And I went, “Really? That’s fantastic.” And they said, “There’s just one condition.” And I said, “Sure, what is it?” And they said, “It cannot be coffee themed. You have to come up with a different name.” Uh, so I sat down and tried to find with with Jack, one of the co-creators. We sat down with a nautical dictionary and just flipped through a nautical dictionary reading words back and forth to each other trying to come up with a good sort of like Kubernetes adjacent name and and landed on Helm. And once we had Helm, then all the rest of the sort of idioms, charts, and all of that kind of came along from there. But it was one really, you know, one day that, you know, I look back on it now and think I had I was completely oblivious at that time that anything I was doing would have much impact beyond that day, right? beyond getting me a $75 gift card and educating the team on on Kubernetes. But in hindsight, you know, it’s funny how those kinds of things can all sort of bunch up into one moment and and have so much impact.

Itiel Shwartz: No, it’s like I’ll be honest here that like on one side I really like Helm Commodore contributed like Helm dashboard which is an open-source dashboard for view and Helm a couple of years ago and became like super popular. But on the other side, Helm always looked to me like one of the weirdest project that I saw. It’s like it’s it has this like package management solution which is great, right? Like pip install uninstall makes sense. It has a template and engine which doesn’t make any sense because pep doesn’t have again in this example like the ability to template and inject variables and it also manage your production or like the actual life cycle of the release. It’s like three completely different things that somehow works. But again, like the first time when I heard about Helm, I think it was like n I don’t know like was like super new. I was like is it like I feel that I just don’t understand like the story here.

Matt Butcher: It it was it was weird in that you know the very original version of Helm was nothing more than a straight-up package manager. No templating system, no operational characteristics whatsoever. it was just you know package up some YAML and ship it. and then we merged with Google deployment manager which was really more of a management feature and and had templating in it. and that that kind of introduced some things there and then we tried to kind of pull back and say okay we’re going to go back to package manager plus templating. And it was like that for for quite a while. but the management stuff just keeps creeping in. and it’s a good indication. I mean if you look deeply at npm or rust cargo or or other package manager systems you see this very similar trajectory where you start out with something very basic. Let’s just figure out a way to manage you know the versioning of packages and and and it seems like inexorably you get kind of pulled more and more toward solving other problems. So you know npm’s got this very sophisticated mechanism for executing commands and managing the life cycle of a of a local instance of an npm node server and you know it seems unavoidable and I think constantly and Helm 4 which came out just a few months ago has once more tried to pull back more toward core package manager but it represents you know sort of the software conundrum that I think we all deal with over time which is that any any successful piece of software gets pulled in so many different directions and it it becomes incumbent on the creators or maintainers of that software to kind of keep true to the vision. And in many of our in many of these cases, we all learn that lesson just a little bit too late and gone just one step beyond where we should have gone before realizing we need to kind of rein everything back in. And so, uh, while I was never terribly happy with starting to introduce some life cycle management features into Helm, I’ve been happy to see most recently that the that the core Helm team has has sort of said we’re we’re going to maintain what we’ve got with Helm 4. We peeled back a little bit of it and said we’re not going to try and do that much anymore. And I think that’s about where we’ll stay. Uh, you know, Helm 4 to me the iSRE is a true mark of a of a very mature piece of software that no longer is playing the game of we just need to release more features, release more features, but is now just saying look, we’re a foundational piece of software. We have to be reliable, we have to be consistent, we have to have long-term maintenance and and that’s it’s been a long it was a long run to learn that lesson, but I think we as a project as a whole have kind of done that now. in the Helm 4 release is a good indication of that.

Itiel Shwartz: Okay, that that’s quite cool. Yeah, in my feeling, Helm was like a foundational piece of software eight years ago or something like that. I’ll be honest here and I was surprised by you know like the startup mode that it had back then between Helm one and two, right? Like breaking changes and so on. but that’s super cool and now Ari maybe like you you share a bit about yourself, right? Or like some cool interesting project that you have done. Did you create Isto or like do you want to have something better than Helm H to to to continue that?

Ari Weil: Well, I’m going to leave Helm. I’ve kind of sat on the sidelines and watch people talk about customizing Helm and what they think, you know, the right or the least amount of pain or the right amount of of standardization is. Um, the things I’m I’m fairly excited about at the moment. AI has been really focusing on distribution lately in enabling agents and AI. You know, we’ve been talking about the new AIF and pretty excited about driving standards for agents and how people are going to be able to identify different types of agents and their intent. And when I think about that around Kubernetes and sort of the open stack, I think it’s interesting to think about things like K agent. So if I want to think about how I’m going to be releasing my agents and managing those, I think it’s interesting to consider the potential of vertical scaling if we actually look at like timberes and the latest release there. And so what what I’m like what we’ve been working on as a company is how to take a foundation of Kubernetes and an open approach to scaling cloud where we realize that distribution is going to get more and more important and centralized sort of the factories that built the AI initial LLMs the initial sort of AI factories are not going to be what is going to help us with real time response embedding AI intelligence inside of our applications and really creating something that feels like that almost magical conversational interface that we all thought Alexa was going to be back in 2012 13. Um, and now, you know, it’s like it’s cute when I talk to Claude and I talked to Claude a lot and I like to call Claude Claude, but like I like to talk to it and I like that it kind of spools things out for me as I’m reading because I can consume it and kind of think about it as it’s happening. But more practically, if I start thinking about like how do I need to enable an e-commerce customer regardless of how they’ve built their application really do a better job of recommending things. And if I look at, you know, sort of best-in-class at the moment and what Amazon is doing with their application, having things that can actually give you recommendations based on your order history and what you currently have been browsing and what they want to promote finally feels like it works versus, yeah, that kind of looks like the demo that I was expecting. And so I’m wondering how the future of the Kubernetes stack is going to allow us to really let developers choose where their data should be, where their model should be, where they should deploy their agents based on their use case. And I think that you know the standardization that we’ve been afforded with tools like Helm, yes there’s pain to it, but like now if we just start thinking about how many places do I need to deploy this code? How many different people are going to run it? How many cloudSRE is it going to be distributed across? I would fall more sort of into the pro-Helm because I’m more interested in like how am I going to scale up and how am I going to start managing my agents without forcing the lock in. So like I don’t I don’t know if that problem is as important to me to solve as how am I going to scale because I feel like that’s where all the developers are going to be pushed to really deploy and accelerate their development in the next 18 months.

Itiel Shwartz: Can you maybe like give a bit more like context here just so for like the audience to understand? I know Akamai as a CDN. Maybe I’m like downgrading you guys. But you know it’s like a CDN. Maybe like a smart CDN, a good CDN. like where is all of this like Kubernetes and AI fits in? You know like overall like when I think about CDN I’m thinking about I don’t know like serving WordPress faster or something like that. How did we transition here? And obviously I’m exaggerating right like of course but

Ari Weil: I mean so think about it like this and I’ll try to draw two parallels right. So, Akamai going from a CDN to what we are today and what’s been going on sort of with the internet and internet enabled businesses because I think Akamai’s followed a very similar trajectory. So, yes, Akamai pioneered CDN back in 1999. Um, the whole idea then was websites basically were static collections of images and fonts and you know other really basic elements that we could distribute multiple copies of and make it faster for people to download. But as websites became distributed web applications, it was necessary to start running more code. So we’ve done versions of Java running at the edge. That gave way to JavaScript. Now we start looking at all sorts of different ways that people are running serverless. There’s a whole metadata language and XML that we had to use in order to configure the way that we would manage caching and distribution and aging content out and figuring out, you know, who should get what version of what content based on keys and authorization and things like that. So the problem has become not so much content distribution as it’s been application distribution and Akamai evolved as that occurred bringing things like edge computing to market and serverless functions and cloudlets and not being as developer friendly as companies like Cloudflare have been since their inception. But you know if we sort of look at what we’ve both been doing as two companies that have edge platforms that really started in the CDN space it was important to secure and it was important to have intelligence about what the application was doing to secure it correctly. So everybody in the CDN space started adding security capabilities. Akamai invested very heavily there, especially from a real-time application perspective because that’s what we’re good at. And we were able to say if you’re a human or a bot, we can make sure that you’re getting the best experience, whether that’s a fast experience, a secure experience, or both. And then as the sort of apps continued to evolve and as as engineers were telling us more about how they were using the centralized public clouds and where they needed to start migrating their workloads, it became clear that we needed infrastructure as a service in addition to serverless functions and our own sort of proprietary XML language. The interesting thing about Akamai is when we did this about four years ago when we acquired Linode, we really were were curious about what would it mean to build a cloud into a high performance network versus having a cloud that was connected for availability because that’s sort of two different ways that you think about the CDN space versus the public cloud market. Public cloud was all about architecting your applications so that you can use distributed you know locations for failover and availability. Some application load balancing but not really because you would typically shunt that off to the CDN provider to accelerate for you. And so for the CDN space we always have architected for high availability failing out systems and and machines and servers and other things sort of as we needed to realize the scale that we needed. So you needed some ability to constantly balance where were people being connected to, where were we serving content from, where were we executing code, both security code and application code. And then it became okay with an infrastructure as a service cloud built into this network with the ability to secure and distribute your application. Now you can build with a continuum of compute whether you want the core primitives whether you want abstractions away with like managed Kubernetes platforms to serverless functions we can have access to CPU GPU or virtual sort of compute resources wherever you need to reach your customers we can integrate with all of the public cloud so that we can accelerate getting your data to the platform and then distributing it out but as sort of the web now is moving from human centric to more you know agents were assisting us Soon agents are going to be doing most of the work for us. Now that trajectory for Akamai for distributing application code is going to be distributing intelligence. So the thing that we did over the last 25 years will now be doing over the next five years as we take agents and a lot of autonomous processes and we provide them with the best ways to get to the right models to manage and and and maintain any data any fine-tuning any sort of weights that they need and then distribute the application logic closer to the users. And where we’re starting to see the most interest, just like in the first version of the internet, is with commerce companies, publishing companies, manufacturer and travel companies, people that monetize via applications primarily and via the web. And what we’re seeing is their users were going from wanting recommendations to wanting conversations to now just wanting answers. And so if we’re going to give you an answer, that means we need to get the model and the code and the context right where the user asking for it because they’re not going to want to talk to Claude. They’re just going to want to say, “What should I buy?” and get the answer.

Matt Butcher: Okay, that if I can riff on what Ari said for a second, I think what really was interesting to me about the direction that Akamai has gone versus others. I so I was at Azure for a while, right? And and very much the way we were building Azure was saying we’re going to drop down a region. This region sort of lives by itself. We’re going to drop another region, this region’s going to live by itself. you as an S SRE are required to figure out how to manage your different deployments in multiple regions and it’s always sort of like a a an exercise in saying do this here this here this here this here and and trying to figure out how to manage all of these in as as separate things right so look at the way okami and CDN developed right the idea with CDN was I as a developer push out my assets and they just get deployed all over the world so if you’re starting from this idea a that the the responsibility of the infrastructure developer is just to get their data uploaded somewhere and it goes everywhere then when you tackle the compute side of that problem right when okami introduced edge workers you know that kind of thing the idea was well I deploy code and it’s just everywhere and I don’t think at all about infrastructure so the perspective that okami has taken as they have undertaken this sort of like journey into infrastructure as as not I shouldn’t say infrastructure because that sounds like all the wasn’t and and core infrastructure right I as you know the Linode acquisition things like that is to turn this all into one big continuum of computing where I as an infrastructure administrator don’t have to think about little silos and how to manage each of those little silos but can think rather holistically about it and say I just need this application to be available everywhere in North America everywhere in Europe everywhere in Israel you know everywhere wherever right and just have it go and I it has been for me personally sort of like a fundamental shift in perspective to go from thinking about things in terms of individual regions to thinking about this sort of compute continuum. and that was the kind of runtime that that my team really wanted to build was a super high performance serverless runtime that would run on top of that. But I think and and here’s where I wouldn’t want Ari’s story to get lost. ThiSRE is what enables getting AI toward where we need it to be. Right? We know that inferencing is incredibly expensive to do. Yep. and and the more we centralize it, the more we’re just crushing one particular region under load, which means we need increasing amounts of power and increasingly large GPUs. When you can distribute it, then suddenly you can do inferencing closer to the user, which means it’s both cheaper for you and it’s easier for it’s it appears much faster for the user. So I’m very excited to be part of this story where we’re talking about AI as just fundamentally distributed not centralized but fundamentally distributed and I think that is basically it it takes a history like Akamaise where you start with something like CDN before you can get to this idea of a continuum computing of computing that then opens up even the option to be able to do AI this way. Right? everybody else is going to be working to try and cobble together those regions to behave this way. the vision that we’ve got is that should just be the default, right? That should be where you start from. And because of that, you’re going to give people better experiences. You’re going to be able to develop far more sophisticated code. And the kinds of applications that you can build are really going to target what people actually care about, which is has already boiled down to one word, right? What they care about is getting answers. [laughter]

Itiel Shwartz: No, that that makes total sense. So we almost ran out of time. We have like less than two minutes. like so like 30 seconds for each one of you. A SRE is something quite popular that we keep on hearing in the industry. Your take if any. So I’ll start with Ari just to to blend things. Yeah. So your take here on like the future maybe of like sorry in like 30 seconds or less. Go.

Ari Weil: So I think that the SRE fun function has never been more important especially with how complex our systems have become. I also think that it’s one where the assistive capability of AI is going to be really really critical because I just don’t see how a humanly approach iSRE is ever going to be able to scale with the systems or that anybody’s going to try to tackle the entirety of an SRE’s role. Um, so instead of seeing more specialization of SREs, I think we’ll have SR experts that are really going to be helped by some pretty sophisticated AI. And it’s fun to see some of the startups looking at security sides of this, deployment sides of this, and sort of runtime scaling considerations because I think they’ll all have to come together.

Itiel Shwartz: Okay, that’s cool. And Matt,

Matt Butcher: I’ll go I’ll go a different direction. Um, you know, uh, I’m a compute nerd, right? Compute is sort of what I where I’ve lived. when big data started what I observed happening was there was a whole bunch of innovation in big data while the rest of us were still doing things different ways and big data innovated a lot but then at some point had the operational aspect of all the all the big data stuff had to migrate back over to the best practices for cloud and cloud native. AI I think had a very similar trajectory. it it it exploded and and experienced all this innovation very rapidly but it was innovation happening sort of in the in the name of experimentation now it’s gone mainstream which means I think that this that what we’ll see specifically in 2026 and 2027 is a big effort to re-examine how we operationalize AI that is how SREs are going to wrap their hands around these things and say now I understand these are the best practices for managing this these are these are the kinds of runbooks we need to shoot for. these are the security constraints we need to have at top of mind. ThiSRE is how we allocate enough compute resources. So I think that’s what we’re going to see on the SRE side over the next two yearSRE is this sort of like operationalization and expert expertization that’s not really a word formalization maybe of how we manage AI applications.

Itiel Shwartz: Okay, I think like with that we kind of ran out of time and Matt, it’s been a pleasure and hope to meet you guys maybe in like KubeCon, Amsterdam, something like that.

Matt Butcher: Sounds great.

Ari Weil: We’ll be there. It’ll be great to see you. Yeah.

Itiel Shwartz: Yeah. So, we’ll meet in a couple of months. Thank you very much.

Matt Butcher: Sounds good.

Ari Weil: Thank you. Thank you. Bye. Take care.

[Music] Kubernetes for Humans.

This is an AI generated transcript of the conversation