#026 – Kubernetes for Humans Podcast with BJ Badyk (Nexxen)

Itiel Shwartz: Hello everyone and welcome to another episode of Kubernetes for Humans. My name is Itiel Shwartz, and today on the show we have BJ Badyk. BJ, how are you?

BJ Badyk: I’m doing well, thanks for having me on. I really appreciate it.

Itiel Shwartz: Sure, we have a lot to discuss today. For those just tuning in, we’re going to talk about migration, acquisition, and managing environments that aren’t greenfield. But before we jump into all the details, can you share a bit about your background? Where did you start, what did you do, and how were you introduced to Kubernetes?

BJ Badyk: Oh boy, I started as a kid, back in the early ’80s with things like the TI-99, TRS-80s, Commodore 64s, and 8086s. I was on a computer as far back as I can remember. Back then, it wasn’t really cool to be into computers, but I didn’t care—I was happy whenever I could get in front of one. So, throughout school, I did whatever I could to get on computers and play with them.

Towards the end of high school, I had the opportunity to start working part-time at an ISP in my hometown in Maine. This was back in 1996. We started with Solaris on old Sun pizza boxes, Spark 20s, Spark 10s, modem chassis—stuff like that. I was in heaven, getting access to all of that as a kid. It was a great time; I learned a lot and was mentored by some really great people.

That experience pushed me forward, and I moved out to California, to the Silicon Valley area in the late ’90s.

Itiel Shwartz: Right during the dot-com boom?

BJ Badyk: Oh yeah, it was August of ’99 when I moved out to San Jose and got a job with Concentric Network. Some might remember them for their IRC server, IRC.concentric.net. I was 20 years old, and it was quite the experience moving from a small-town ISP to a major web hosting platform. We went from using recycled Sun hardware to brand-new Ultra-class boxes. It was a significant learning experience.

Itiel Shwartz: How did the dot-com bubble burst affect you?

BJ Badyk: I was at Concentric for about a year and a half and started looking for something else. I wanted to branch out into more open-source projects. I took a job with a smaller company called First IP. They aimed to mirror whatever you were running in the data center onto their platform—kind of a precursor to the cloud. Unfortunately, after about two months, I was laid off when the entire company, around 20 people, was let go. That was tough, especially since we had just moved to the Santa Cruz area.

Itiel Shwartz: That’s a bummer. What happened next?

BJ Badyk: We ended up moving back home, and I got a gig with the gentleman who had started the ISP where I’d previously worked. After a couple of years, I decided to take a break from tech and moved to Richmond, Virginia, to get into the punk rock and hardcore music scene. I worked at a natural food store and a scooter shop, but eventually, I realized I needed to get back into tech.

I took a job at Bank of America in a large data center, doing the night shift. It was a bit procedural and slow-paced for my liking, so I started looking around and found a job with Playboy magazine.

Itiel Shwartz: Playboy had a lot of traffic, right? What was the business like, other than the magazine?

BJ Badyk: Yes, the website received a lot of traffic, and we had to learn how to handle load balancing effectively. We had some really talented people there, and it was a great learning experience in terms of engineering solutions, communication, and workload sharing.

Itiel Shwartz: For those who haven’t worked on a high-traffic website, the response time, latency, and handling large spikes in load are crucial. It’s a different ball game compared to B2B solutions.

BJ Badyk: Absolutely. The traffic itself costs money, both to purchase and to utilize, so it’s essential to manage it effectively. It was a great school for anyone interested in hardcore ops and infrastructure.

Itiel Shwartz: How long were you at Playboy?

BJ Badyk: I was there for about three years. We dealt with a lot of challenges, including acquisitions, and towards the end, I wanted to move on to something different. I joined a company called GoHealth, which was in the insurance industry. It was a small company when I started, so I had a lot more responsibility, managing the entire dev and prod environments.

Itiel Shwartz: Let’s jump to Kubernetes. When and why did you start working with it?

BJ Badyk: Kubernetes presented a solution for what we used to try to accomplish with config management. After leaving GoHealth and moving back to New England, I found a job at Tremor, an ad tech company that runs an online video advertising platform. We deal with millions of requests per second, so monitoring and managing our infrastructure was crucial. 

We started with Puppet but found it cumbersome, so we switched to Ansible. Eventually, we began exploring Docker and Docker Swarm, and that led us to Kubernetes. We started with Kubernetes the hard way, following Kelsey Hightower’s guide, and that allowed us to move from bare metal to Kubernetes more effectively.

Itiel Shwartz: How did the migration process go? You weren’t working with a greenfield environment, so how did you handle the transition?

BJ Badyk: The migration process depended on the app’s architecture. In some cases, we had to completely re-architect the app, while in others, we could move it as-is. We went through a lot of benchmarking and figured out how to optimize performance. We also had to drive a mindset change among developers, helping them understand resource consumption and how to optimize their apps for Kubernetes.

Itiel Shwartz: It sounds like you focused on showing developers the benefits of Kubernetes, rather than forcing them to migrate.

BJ Badyk: Exactly. We found a few key people willing to take a chance and try it out, and that helped us get others on board. Once they saw how Kubernetes made their lives easier, it sold itself.

Itiel Shwartz: How much time did the migration take, and are you currently running on Kubernetes?

BJ Badyk: We were able to move the entire Tremor DSP platform to Kubernetes. After we acquired Amobee, we started doing the same thing for their platform. We’re running quite a few clusters now, and most of our Kubernetes deployments are on Talos Linux, which has been a game-changer for us.

Itiel Shwartz: How many clusters do you have?

BJ Badyk: We’re into the tens of clusters, with some having a couple hundred nodes. It depends on the needs, and we size them accordingly.

Itiel Shwartz: What do you see as the future of Kubernetes, both for your company and in general?

BJ Badyk: I think AI will play a big role in predictive analysis of cluster behaviors. Tools like Talos Linux, which create an appliance-like experience for each node, will become more popular. It makes running Kubernetes much easier, and I see that as the future, especially for bare metal environments.

Itiel Shwartz: That’s super interesting. Do you have anything else you’d like to share before we wrap up?

BJ Badyk: Thanks for having me on. I just want to say that ham radio is one of my hobbies, and there are some really cool things happening with AI analysis in that space. It’s a lot of fun and gets you outside and active.

Itiel Shwartz: That’s unique! Thanks for sharing, and thanks for being on the show, BJ.

BJ Badyk: Thank you so much, Itiel. I appreciate it.

[Music]

BJ Badyk is a human who desires an easier life. Nerd from birth, his curiosity led him down a path through the start of ISPs, Silicon Valley during the dot-com bubble, the last few years of the Playboy brand, and into the world of Adtech.

He currently runs the platform engineering team at Nexxen, where they work on unique ways of handling millions of requests per second with Kubernetes. The team was an early adopter of Talos Linux, which they now run at scale. He presented at TalosCon 2023 and continues to pursue simple solutions to complex problems.

In his free time, he can be found listening to synth-wave, motorcycling, or roaming around the woods with his ham radios.

Itiel Shwartz is CTO and co-founder of Komodor, a company building the next-gen Kubernetes management platform for Engineers.

Worked at eBay, Forter, and Rookout as the first developer.

Backend & Infra developer turned ‘DevOps’, an avid public speaker who loves talking about infrastructure, Kubernetes, Python observability, and the evolution of R&D culture.  He is also the host of the Kubernetes for Humans Podcast. 

Please note: This transcript was generated using automatic transcription software. While we strive for accuracy, there may be slight discrepancies between the text and the audio. For the most precise understanding, we recommend listening to the podcast episode