Episode #36 39:53 2025-02-19

#036 – Beyond Kubernetes: A Radical Vision for the Future of Infrastructure with Adam Jacob (System Initiative)

Adam Jacob
CEO, System Initiative

Listen to the Podcast

Episode Overview

In the first episode of 2025, host Itiel Shwartz sits down with Adam Jacob, CEO of System Initiative and original author of Chef, for a wide-ranging conversation about the past, present, and future of infrastructure automation. Adam traces the lineage of configuration management from Mark Burgess and CFEngine through Puppet, Chef, and Ansible, explains why the rise of EC2 made Chef's flexibility necessary, and why, after 15 years, he became convinced that the core primitive behind infrastructure-as-code is fundamentally wrong. The conversation digs into System Initiative's bet that infrastructure should be modeled like a video game engine rather than packaged like an application, why abstractions over Kubernetes, Crossplane, and Helm inevitably leak, and what a multiplayer, simulation-based approach to automation looks like in practice.

In this episode we discuss:

  • The history of configuration management from CFEngine to Puppet, Chef, and Ansible, and how cloud changed the game
  • Why Adam stepped away from Chef after seeing enterprises repeatedly fail to reach the outcomes they wanted
  • System Initiative's core thesis: infrastructure is not an application and shouldn't be modeled as code
  • The '200% problem' with abstractions like Helm, Crossplane, and platform layers built on top of Kubernetes
  • Why the next era of infrastructure tooling will require wild swings, not incremental improvements on the status quo

Key Takeaways

1
Configuration management as we know it descends from Mark Burgess and CFEngine; Chef's innovation was using a real programming language (Ruby) to handle the integration complexity that DSLs couldn't.
2
DORA metrics show the industry hasn't actually improved deployment outcomes in roughly a decade — the tooling stack has grown but the end results for most teams have not.
3
Abstractions over complex infrastructure tend to leak: any wrapper that tries to fully cover Helm, Crossplane, or Kubernetes ends up roughly as complex as the thing it abstracts.
4
System Initiative models infrastructure as high-fidelity digital twins inside a multiplayer, simulation-style workspace — closer to how Unity models a game than how Terraform models a stack.
5
Adam's prediction: expect more radically different automation primitives to emerge, because the status quo can't be incrementally fixed into a better outcome.

Itiel Shwartz: Hello everyone and welcome to another episodes of the Kubernetes for Humans podcast today with me in the show I have Adam Adam pleasure to have you

Adam Jacob: Hi thanks

Adam Jacob: Yeah pleasure to be here

Itiel Shwartz: H so it’s our first episode for like 2025 I hope we will have a nice episode today without fur ad go and introduce yourself who are you what do you do and why do you care about Kubernetes if at all

Adam Jacob: Yeah well I’m Adam Jacob I’m the I’m the CEO of System Initiative in the previous life I was the CTO and I was the original author of a thing called Chef which did configuration management and you know if you I’m a CEO now I was a CEO I was a CTO for a long time in reality I still feel like what I am is like a really really good systems administrator who’s just sort of continued on that Trend and yeah why do I care about kubernetes I mean I care about kubernetes because I care about infrastructure and have always cared about infrastructure because that’s the thing I think is the most fun and have thought was the most fun sort of my whole career and so you know you sort of can’t at this point you can’t sort of also care about infrastructure and not care about kubernetes

Itiel Shwartz: Yeah so you know it’s I have to you know I have to ask about the beginning and like Chef maybe if you can give a bit of details and I’ll be honest here you know I can say as someone who worked with Chef I admire Chef like it was a tool to do a task but it was never pump for me at least so maybe you can share a bit about the history of Chef what why how and I know that a lot of our listeners are using to chef but they are using Chef so you know I’ll be happy to hear about the beginning and the current state of Chef

Adam Jacob: Yeah so wait you did admire Chef or you did not admire

Itiel Shwartz: I didn’t I didn’t I did not sorry I did not admire I’ll be honest it was in the time when most of I did what I did was like Jenkins and Chef which is not the best combination I feel for like a a different era for sure no no for for like a yeah yeah like 10 years ago something like that yeah and and Chef maybe it’s because the Ruby part and I’m like a python guy something never never clicked something

Adam Jacob: Fair enough again like I did use it and it was much better than what we had prer today which was a bunch of BS right like it’s not that again like I don’t come to to this I get it yeah so so let’s talk about the beginning of Chef and you were here for aot time I mean to to think about sort of the beginning of Chef and sort of how it evolved I think you know you sort of have to put it in its context in time so like you know when we when we think about how sort of automation has evolved and in particular infrastructure automation like my career started as the internet started in the United States and so you know as we figured out both how to just get everybody on the internet and then we figured out how to like serve people content on the internet like once everybody was on the internet they needed stuff to do and so we started like starting companies where what we were doing was like giving people stuff to do on the internet you know and all of that didn’t make sense in terms of how people understood how to build businesses how to think about you know it like when you thought about like how do I run servers what they what they what that meant was like how do I run the computers in a data center for my company to run the applic my workers needed to do their job which is very different than saying like what am I doing how am I how am I going to serve a 100,000 users in in an hour which just like nobody had ever done that before like that wasn’t there was no company big enough or with the demand enough to ever even see that problem and so like we sort of had this this time where everybody who was working in that industry we were just making it up as we went along you know like you nobody knew how to do it so we had had to invent how to do it and you know when you think about the history of automation it’s on the internet anyway and and sort of it’s it’s it’s very c-comm with the history of of how the technology evolved like how the external technology evolved and so as we figured out how to build stuff on the internet we also figured out how to automate it because often the problem we were seeing were problems of scale that didn’t exist in like Legacy it you know Legacy like by which we mean it ran the world right so like it wasn’t bad it was just did what it did so you know where Chef came from was this long relatively at the time it felt long but it was actually pretty short you know history of how do we think about running computers at large scale and keeping them stable and keeping them configured where you know a really common metric was like how many servers could a single systems administrator manage and you know if you were doing it all by hand and you were really good at it you know the ratio was like 50 to one 100 to one right 10 and one’s a lot you had to you had to be good at it and the way you and the way you were good at 100 to one was like you started to write programs right and and so like as soon as you started to write programs what we started to do was try to come up with ways that you think about how do I manage all these systems in a way that like allows me to do it and so that’s where configuration management comes from is in is this idea that says hey we need a method to think about how do we manage all of these computers in a more efficient more capable way and it starts with Mark Burgess and CFEngine right and the idea of like what I need is this like convergent system where the state of the computers is unknown but I can declare this state I want to see them in and then over time they’ll Converge on the correct condition whether or not they’re you know regardless of the state they’re in so you know if the computer’s turned off and you turn it on and it runs CFEngine what’s going to happen is eventually CFEngine’s going to do all the work it needs to do to like make that thing do what it’s supposed to do and so you know then there was a thing called Puppet Puppet added in this idea of kind of a resource abstraction or what you wanted to do was specify like the specific resources you wanted to manage and then those things would get mapped to these like platform level providers that would then sort of do this convergent Loop what Chef’s Innovation was was was really the first the first configuration management system that used like a regular programming language right it used Ruby partly because the complexity of integration was so much bigger so especially when you became really large if you were a global bank the number of systems you needed to integrate with and the things you had to do was just more complex than you could express easily sort of in in a more constrained DSL and so the it had a couple of other things in there about order and a few other things but like that’s pretty deep configuration management Foo but all of that kind of leads you to to this idea where as the cloud happened right so Chef was written roughly at the same released roughly at the same time as like EC2 was released it was two days old right and so I was a consultant who was building automation for startups Facebook apps people forget about Facebook apps but those were the first time you could like launch a business on the internet and and instantly go viral in like a in the way that we think of it now and so I had a lot of customers who were faced with this problem which was they just couldn’t get enough compute fast enough like you literally you know we were career systems administrators I had friends who were you know literally sending emails to everyone they knew being like can I just buy spare I’ll come pick it up in a truck take it out of your data center and put it in mind because they just couldn’t grow fast enough and so as as you got to this moment where suddenly compute was available on demand you needed a way even more importantly to think about configuration management right because you had gone from the limiting factor being the ability of a systems person to rack and stack compute or to think about building VMs but even building VMs was sort of constrained on on racking and stacking compute and suddenly it wasn’t that as your constraint it wasn’t could I order Hardware fast enough it was I could just order it off EC2 and it would just give me compute and so what you need now even more is a complex configuration management system that understands how to make the operating system do the things it needs to do bootstrap applications get everything set up and make it correct and so that’s what Chef did and that was really sort of where it shined

Itiel Shwartz: So you know like super super interesting story and I feel it’s a bit like the GPUs today like I see comp struggle to buy like you know enough enough GPUs and like especially now with with the rise of likei everyone was trying to utilize the GPUs better or to get like near best GPUs but walking me through like back in the day you know like at least for me there were always like the chef Puppet and Ansible right like it’s like the three main players in the configuration management like was it like were you friends was it War like go go you know you put a problem right then you started like to build a solution you call the cookbooks and so on but company right so maybe go

Adam Jacob: I mean the people were friends in to large in in large degrees you know H because look there’s just not that many people who care as deeply as like I care about this topic there just aren’t and if what you decide is that like every other person who who loves what you love is is a jerk then like it’s a very lonely life you know like it’s just you’re going to be alone with a thing you care about and it’s way more fun to care about that thing with other people you know and so yeah the people were friends and we all influenced each other to varying degrees right so like you know I had I was a very prolific early Puppet user Michael DeHaan worked for Puppet for a little while who wrote Ansible right we all sort of worship at the altar of Mark Burgess to varying degrees I probably am the like the like leading Mark Burgess worshipper but like all of us all of us descend from that tree in a way that like you can’t avoid and I think you know he doesn’t get enough credit I think kubernetes wouldn’t exist without Mark Burgess for example and it’s easy to forget people you know because it’s just people don’t know but like the ideas and those Concepts and how they come together like they’re in the water and in the air because Mark Burgess invented them in like these incredible notebooks with like perfect handwriting and you know you I think you so like that Legacy of it was very real that said like yeah we were building we were building companies and we were very competitive right like like we all wanted to win we all we all wanted it to wanted to be the best in that space and we and we did you know we all built you know very successful companies that that we’re all you know really proud of so

Itiel Shwartz: So what happened like how much time were you like a chef right like you guys were acquired right like that’s at least like that’s what I remember but maybe I’m line here so go go with me the history you know like you’ve been there you founded a company people are you like you are becoming I want to say like the leading like like the winner in that space at least that was my feeling back then

Adam Jacob: Yeah it depends on your point of view I think we won in the like in the places where Chef was most in the places where the complexity was highest we tended to win I think I think by dollars like by Revenue I think Puppet was the leader Puppet had more Revenue than chef and and certainly more than Ansible so I think the stack rank was probably Puppet Chef Ansible in terms of like Revenue I think in terms of eventual share you know like what’s the what’s the share of market like I think in the end Ansible one right by just pure market share right I think yeah so my that Journey for Chef roughly 15 years of my life so it started with this consulting company we ran and then you know I was for the last couple years that Chef was an independent company I was last year and a half or so I was on the board but I wasn’t sort of involved in the daily operations because I had sort of spent so much of my time going into these large Enterprises you know and and helping them do what was then called digital transformation you call it devops you call it whatever but I would basically take over teams in huge Enterprises for a couple of weeks and teach them how to like work basically how do you put all these tools together how do you think about them how do you how do you do that kind of work and what I learned from doing it was that you know the path to success where success was measured by your ability to actually like support your applications and you know and be able to do the work you need to do kind of on demand and and efficiently and effectively was so fraught with Peril that almost no one was going to succeed and it didn’t mean that I couldn’t teach them how I could but they were always going to have to make adjustments they were always going to have to make concessions for sort of the status quo and the way that the tooling worked was so fragile that you just I couldn’t actually get people to the outcome that I wanted them to get to and that they wanted to get to no matter what I did and that was kind of heartbreaking honestly and so I you know I didn’t want to play myself on TV I didn’t want to like keep showing up and teaching people how to do it knowing that they weren’t going to get there does that make sense like I just wasn’t you know like if if if we’re not going to get there then and I’ll be honest like I don’t think we’ve gotten there now like I think the kubernetes folks have done a great job but like their outcomes aren’t better you know like if you look at the DORA report and you look at like what’s the rate at which we can actually deploy our applications you know and who’s who’s being successful and who’s in the middle like we haven’t moved the needle in 10 years more right if anything we gone backwards so like the reality is we’ve been stuck in this sort of same pattern I just kind of saw it first and it broke my heart and so I spent a few years I spent a few years or not a few years I spent a while thinking about what I wanted to do and then I started another company called System Initiative which is what I’ve been doing for the Last 5 Years where what we’ve been doing is the R&D to figure out if we actually wanted to change those those end outcomes for everyone like what would a new system look like I sort of became convinced that what was what was missing was that the core primitive we thought about when we thought about how we build automation that that primitive was wrong that we’ve been treating the infrastructure problem as a code problem in the same way that we think about applications and that that perspective was ultimately flawed and and we needed to rethink what it meant to to imagine that we were automating infrastructure and and how those systems interacted

Itiel Shwartz: So so so maybe you know like so many questions maybe let’s start with the beginning and then I’ll go with the with the time what happened to Chef like like what’s the current status like what Chef is still Chef is great chef is still used by the huge companies it’s growing it’s always been growing it continues to get bigger it’s owned by a company called progress now yeah Chef’s going great okay so I have nothing to do with it but that doesn’t mean it’s not successful yeah yeah okay so the premise maybe of the current company I’m not sure that I completely yeah so walk with me again like I understand the problem but are you guys like doing product Consulting combination yeah

Adam Jacob: Yeah so look what System Initiative is is basically if you if you look at the current work way people work so if you say what is it we do you know like maybe you let’s talk about Terraform and talk about like infrastructure as code and then we can move to kubernetes but let’s just let’s just stick with infrastructure as code for a minute right sure you think about I want to I want some infrastructure so I write some infrastructure as code and then what do I have to do so I have to figure out like how do I plug that together into a pipeline how do I what’s the GitOps flow look like how do I secure it how do I relate to it and then how do I teach other people to use these abstractions I’m building because what we’re trying to do is sort of get to a spot where somebody can be like I can deploy my application and the infrastructur is on top of it and I understand how it all works and it’s kind of a mess and if you actually ask people how they feel about it they’re really pleased that it works you know what I mean like if you have it you’re stoked because it’s better to have it than not to have it and if you go well how did you feel about your day when you did it they’re like oh it was miserable sucks like everything about it is painful right the the it’s just it’s bad the feedback loops are terrible it’s hard to compose it’s difficult to reason about it’s very difficult to share the abstractions get weird like there’s just a lot in the way all of that adds up to slowing down in the end like we just wind up not being able to do the things we want to do because there’s too much complexity in between us a great example here was I was talking to some folks who are incredibly great Terraform users like world class Terraform engineers and you know part of what they do is like set upat forms for other teams so other teams SP up a project they come ask those people what to do and like you know what happens is they have this long pull request thread where people like try to build it and then fail and then they go back and forth about all the ways that they messed it up and you know hours or days or weeks later eventually that thing merges and then maybe it builds you some infrastructure that works and so what System Initiative does is say hey the problem here is actually that infrastructure is not like an application right the issue here is that we we started writing code and we started packaging up like applications but it’s not an application and instead what we need is a primitive that allows us to work with this system more like for example the way game developers work in unity right like infrastructure is a thing we can model it’s a thing that we can understand and so what we do is we build digital twins of digital things so we build these really high fidelity model of your infrastructure so like EC2 or vpcs or whatever it is you want to model we model it one to one and then we allow you to then use that model in different ways so you can program the model right so you can extend how it works you can also then compose the model visually so you can like you know basically put you know what looks like a living architecture diagram together on this canvas and what it’s actually doing is composing together all of the different pieces that you want to use in a simulation and that simulation can then tell you hey this looks like it’ll work it looks like it won’t it can you know automatically update values for you behind the scenes it’s this big hypergraph of all of these functions so the net effect is that like rather than thinking about your automation solely as a piece of code pipelining instead what you do is work in what feels like an IDE that puts together a model of how infrastructure works and then you program the model and then the model keeps track of what you’re doing and then knows how to apply that to the real world and it keeps track of both kinds of date right it knows like hey this real world thing that you’re modeling looks like this right now your model looks like this it’s all multiplayer so when we collaborate together you see updates happen in real time right as you merge things they show up in other people’s change that’ss automatically like System Initiative is super cool and it’s really the First new automation primitive like conceptually right where the tree it comes from is not the same that’s been built in the last like 20 years

Itiel Shwartz: Now that that sounds very ambitious like super ambitious how are like big org crazy ambitious yeah how organization like reacting to that like are you like

Adam Jacob: Yeah people are reacting pretty good to it you know honestly better than I thought that they would in some ways like yeah so I mean look the we’re very I’m very right about the problem I spent my whole career sort of in this I’m sure you are problem is like 100% it just is what it is and you know when you think about Solutions it’s just very very hard you know to think about what would a solution be that would be better than the status quo especially when you’re when when you know that in the end you have to automate the most complicated organizations in the world and so you know what we’re seeing is that people who have really deep expertise in the status quo they tend to resonate very strongly with System Initiative and so you know the difficulty the only real difficulty in System Initiative is that the amount of stuff that we need to do in order to get people comfortable with the coverage so essentially to replace their existing tooling is just a big lift because you got to think about sort of how do I map all the concepts and so you know where we’re at now is relatively early where there’s a bunch of you know really smart people who are really enthusiastic about what we’re doing and we’re fig figuring out sort of how to map all of those Concepts into this new paradigm and then you know every day it gets closer and closer to a thing that everybody just shows up and uses and and you know it’s the Flywheel spinning

Itiel Shwartz: No again like super super like you know ambitious and interesting you know you talked about HashiCorp which also did like a revolution right in how people are defining stuff gr way even in in its form they try to stick very closely to actual objects right like EC2 load balancer it’s not that they actually created object at least not the I feel like the beginning but more on how can I take existing INF and model them right yeah but how how do you see system like Crossplane which I see is gaining more popularity is yeah to you guys

Adam Jacob: Yeah I mean one of the core mistakes that we’ve made is that we started abstracting these systems away so you know we got to talk about abstractions for a minute because this is a little bit of heresy but like when we talk about abstractions in software what we mean usually is that there’s some complex interface under the hood of which my end user needs a fraction of its capability and so I build a new interface that allows them to like operate only at the high level for only the thing that they need we call abstractions leaky when what the complexity is of the thing under hood under the hood it becomes the surface area of the thing at the top right and at that point the abstraction starts to break down because it’s not really an abstraction it’s really just it’s really just a restatement of what it is that’s under the underhood so you can see this a lot if you look at like a Helm chart that’s goal is to fully automate every capability of the thing that it deploys the helm chart then grows to be roughly the complexity of the thing that it’s automating only you have to understand both Helm and the clean you’re automating and the way that the person who put the chart together put it together I call this the 200% problem Chef had this problem Puppet had this problem Helm has this problem everybody had Terraform has this problem everybody has this problem right it’s just it’s inherent in the way the system works and part of the reason that this happens this way is that when we’re thinking about infrastructure we’re not thinking of it as a one-to-one model with its actual capabilities we’re always trying to twist it into the shape of the program’s understanding of what it is so Crossplane for example looks at infrastructure and moves it into the shape of a kubernetes CRD right it’s like I don’t know what your infrastructure does I don’t know what the capabilities are I don’t know anything about any of that what I’m going to do is turn it into a CRD so that you don’t have to think about it well except when you do have to think about it except when that CRD doesn’t quite do everything you need it to do except when it doesn’t come together exactly the way you need it to do and then what happens is the abstractions we try to build on top of that where we’re like well don’t worry about it Crossplane nobody should ever have to touch Crossplane right shouldn’t have to actually understand how that works instead you should use this higher level platform abstraction and then you never have to think about it and the truth is can I swear the swearing okay just a little bit it’s [expletive] it’s [expletive] because what happens is what happens is the platform abstraction always winds up leaking all the way to the bottom and you can see this with everyone who’s ever tried to extract kubernetes right like kubernetes is such an incredibly strong point of view about what the interaction model is with the API surface that when you try to abstract it you just wind up leaking kubernetes point of view because you can’t not do you know what I mean like it’s so it’s got such a strong perspective which is why it’s so successful so like you know it giveth and it taketh away right so like it’s incredible and its point of view is so strong that there’s you can’t really build on top of it without leaking everything about how it behaves so when you think about how infrastructure evolves and how models evolve changing flipping from saying hey what we’re going to do is try to build a program that understands how to abstract this low level infrastructure and its capabilities to a program that says nah we’re just going to build you a model of how they actually behave and and if you think about building a model of how things actually behave that and then what you can do is build models that rely on those behaviors and drive them compositionally that’s very different than abstraction right and that’s the kind of thing that System Initiative is doing and it’s why what you tend to see when we glue these systems together is that you can’t they don’t scale very well semantically right they scale okay for a developer but if I you know if I try to say to like to a to a software developer hey here’s how your application works and then let them under the hood go drive deeper and be like well how do the AWS resources that run I know behind the scenes there’s load balancers I know behind the scenes there’s a kubernetes cluster how do I how do I understand what that is that you just the answer is you just get punched in the face you know you’re like well go read whatever the definition try to figure it out and like you just that level of mapping is so brutal and so with System Initiative that mapping kind of goes away because instead what you say is hey model your application as a thing and it has Behavior so for example we have a model of System Initiative we deploy System Initiative with ourselves and one thing you can do with the application is dynamically change its tracing level and so the high level abstraction of the application has an action on it which changes the tracing level across the fleet and that thing has relationship sh that go down to the lowlevel infrastructure you can also see that there’s a load balancer do you know what I mean like it’s sort of the way it’s composed is different and that composition is critical to letting people scale organizationally because you can think about saying hey I only want to look at my application but if I want to understand what the infrastructure looks like I can dig into that right I can dive down into it nothing’s hidden from me in the same way that if I’m a game developer and I’m and we’re building Grand Theft Auto it’s not like I couldn’t look at the code that gets generated for a Shader or write it by hand I could but I don’t you know instead what I use are all of this incredible tooling that like you know allows me to see a scene that gets rendered and place a character in something and do those sorts of things and that’s that’s that’s what I’m talking about and like and that’s the problem with our tooling because we’ve we’ve had this point of view all along that says let’s pretend infrastructure isn’t infrastructure let’s pretend it works like software this bull buit it doesn’t [expletive] work like software it’s different and we need to model it like it is

Itiel Shwartz: Let let me know like first of all it’s a super interesting take on like a a real pain painful like problem but you know in end of the day my skeptical side will say something like you know what you s you say sounds magical what happens once I try to change the tracing level right of my application and for some reason it doesn’t work as expected like when you force me to go down to you know the rabbit hole in unity or I think I never program like computer like games but I think the engine is super solid it works on all all of the time and that’s great like as long as you can trust the engine it’s great but it’s a bit like you know I’ll give like an example that I know a bit better like Python and C you almost never need to troubleshoot or debug some NumPy or whatever the actual C extension because it works well most of the time so you the question is are you at the on pair with that level of like you know like trouble shooting and debugging like once you stop working then all of the magic goes away right like I’m a python developer that is now forced to know both Python and C in your example I need to know your point of view but also to go down to the actual

Adam Jacob: You don’t need to because the truth is what we’ve always done is collaborate with other people and like and the problem is that we can’t do it because we don’t have a model of how things work when you talk about the engine being super solid that’s what I mean the engine has a conception of what a video game is and we have a conception of what infrastructure is and how it behaves and how it relates to other pieces of infrastructure and allow you to model those things in that way and it turns out that that conception also extends up into things like applications and scale semantically but that core primitive that says no actually I don’t know what it is that you’re trying to automate what I know is that we can model it and work on it in this simulation and track Its Behavior over time and if we program it that way suddenly it unlocks all of this collaboration for example when people when we built our own SaaS so when you use System Initiative you just come to the website and sign up and you just wind up in this workspace and it just happens for you it’s super easy and that SaaS deployment we built it together so our team in the same workspace in real time were just collaborating on different parts of the infrastructure like some people were doing low-level stuff they understood other people were doing higher level pieces like you’re not you’re not forced to stare at all the details right what happens is that you collaboratively understand the pieces the same way that like if we’re building a complex video game it’s not one developer who does it alone right these are incredibly huge teams you ever finished a video game and watched the credits roll holy crap and like it’s roughly the same level of complexity as the number of people involved in running a giant application for a global company yeah and like we got to give them a platform in which they can collaborate in that way and they don’t have one instead what we give them is this idea that they should all be software developers who all understand every piece of the stack which is why you’re sort of forced to do it and so that like entire the entire frame of your question where it’s like well how can I deal with this abstraction like am I forced to understand how how you know the C extension for numpy works and the answer is no but someone does right like the people who wrote it do and what I need to be able to do is move between them only if I need to move between them and yeah it’s going to be vanishingly rare that a random application developer sitting at a global bank who’s running on top of a a stable platform needs to go all the way to the bottom to understand like how did the you know how did the how did the IAM rules get set that give them permission for their application domain they won’t need to do that they’ll probably never look but there’s someone who does right and like and that and all of that information needs to live in the same place right because it’s all actually the same piece it’s all of piece does that make sense

Itiel Shwartz: Yeah it does it does I think it’s you know it it comes down to the promise that I W need to look under the hood most of the time like that is where if you are able to provide that level of you know an I think it’s like you know most of the people never see any c Cod like most of the Python developers I worked with like Apache Storm that was written in Clojure and after like three years I only needed to debug Clojure once and we had some very hardcore things there so sure because you just use the program

Adam Jacob: Yeah yeah yeah as long as I don’t think that’s a particularly hard problem right if what you do is think about it you know think about the how that works today if you’re an application developer and the infrastructure your application runs on was deployed with Terraform or deployed with kubernetes and I tell you to go troubleshoot it I just told you to go [expletive] yourself you don’t understand it at all and the and the amount of stuff you have to climb up in order to understand it brutal crazy right think about Crossplane you’re like okay I know I like can I understand that there’s a load balancer in in AWS yes can I understand that that load balancer got provisioned because it got translated into a CRD which then gets translated into the higher-level YAML that I did which then went through our GitOps pipeline which ran from this other thing which ran from that thing which ran from this thing and then don’t worry about it we stuck it behind Backstage no no like a normal person no no in the same way that you didn’t want to have to troubleshoot Clojure right and so like thinking about how does the system relate like what is the Primitive we use to model it that’s my point like we don’t have one and that’s why we’re suffering because if we had a stronger conception of the engine if we actually had a conception of like well I actually I do know what the load balancer is and I do know how that load balancer relates to this other thing and I can understand if I need to what the code is that causes this to happen suddenly those semantics become elegant right suddenly the ability to move between them becomes a question of of whether you understand the domain and then because you make the whole application multiplayer it becomes a question of whether or not the right people can come help you understand it right

Itiel Shwartz: No but that that makes total sense really total sense but it is crazy ambitious right it’s it’s like a complete you know like I said it is the First new automation primitive really like foundationally different in the last agre long time

Adam Jacob: I agree

Itiel Shwartz: So we are to be honest like we’re like already out of time but I will give you like two minutes to talk about prediction and and else you would like to talk about

Adam Jacob: Oh what’s my prediction I guess my prediction is that you know I think that what we’ll start to see is not only System Initiative but multiple new attempts at thinking about how how to approach infrastructure automation I don’t know what they’ll be or what their takes will be but I think in the oncoming years the the shape of our current approaches and the idea that sort of platform is going to evolve in the way that people have said that it will like you’re starting to see the people who sort of understand infrastructure best and who understand the system the most they’re not rejecting that idea but they’re they’re pretty clear that that that it’s that it’s not working the way they need it to work and so I think we’re going to see more and more new attempts and the ones I’m most excited about are the ones that are taking really wild swings right like you know I want to see more System Initiatives I want to see more wings I want to see more like you know I don’t know what Brian Grant’s going to build but I want to see what Brian Grant builds you know what I mean like people got to get a little crazy because the status quo is not working and the idea that what we’re going to do is leverage the status quo in order to get a better outcome in some dramatic way is is false and so like if if we’re going to do better we have to actually build and really excited for that era of building that’s my prediction is we’re going to see more people build

Itiel Shwartz: Okay okay sounds sounds you know like fair enough fair enough like to be honest he’s also building something in like the config space right I’m not mistaken with the Alexis from like a weave Weaveworks right yeah yeah looking and Jasper as well like a really like a really strong team of people very strong team right one of them I think is like the you know I know like a top eight player in the world pretty much so yeah okay okay interesting maybe I’ll bring Alexis to to to an episode as well

Adam Jacob: You should and anyway any last words Adam that was it thanks for having me I appreciate you

Itiel Shwartz: No sure I think it was one of the most like interesting like episodes I’m sure that a lot of people like me that have done like infrastructure-as-code automation it’s like a very cool and weird concept that I need to think more about and I really like the analogy of like basically games compared to infrastructure code because I do feel there’s like some great overlapping in how you describe it and how I think most like most of the hard hor like you know Engineers feel around that like infrastructure Engineers yeah

Itiel Shwartz: H so pleasure having you are you going to go to KubeCon London I don’t know if I’ll be in London I was in North America so we’ll see if I wind up in London but Okay cool so pleasure pleasure

Adam Jacob: Thank you

[Music] Kubernetes for Humans.

This is an AI generated transcript of the conversation

About the Guest

Adam Jacob
CEO, System Initiative
Adam Jacob is the CEO of System Initiative and the original author of Chef, the configuration management tool he built and led for roughly 15 years before stepping away from daily operations. A self-described career systems administrator, Adam has spent his entire career working on infrastructure automation, from helping early internet startups scale, through teaching digital transformation inside the world's largest enterprises, to founding System Initiative in 2019 to rethink the core primitive of how infrastructure is automated.