#029 – Kubernetes For Humans Podcast with Elizabeth K. Joseph & Holger Wolf (IBM)

Itiel Shwartz: Hello everyone and welcome to another episodes of Kubernetes for Humans podcast today with me in the show we have two special guests so if you can please introduce yourself

Elizabeth K. Joseph: Yeah hi I’m Elizabeth K Joseph I work with over at IBM on running the open source program office for IBM Z and LinuxONE main frames

Holger Wolf: So my name is Holger Wolf I’m the chief product owner for Red Hat OpenShift on the IBM Z and LinuxONE platforms and taking careful getting things bed to to the platform with the team

Itiel Shwartz: Okay usually I start with like a short background and ur2 so try to keep it brief but just give me like a bit what did you do how did you a bit of what did you do how did you came

Elizabeth K. Joseph: I’ve been involved in open source software for about over 20 years now and my first job was doing Linux systems Administration so I did that and I worked on you know I was racking servers and data centers back in the day and like you know building up my career through that and I worked on distributed systems so I worked on OpenStack for a time I worked on Apache Mesos for a little while and then started working on kubernetes a bit and then I was kind of looking for a change and my I was put into contact with someone at IBM and they’re about five years ago and they’re like hey you want to work on mainframes and I’m like I have no idea what those are but I looked into the tech and I spoke with some of their technical team and I actually got really excited about it so I joined IBM five years ago to work on mainframes and in in the course of that in 2019 I spoke at cucon on the state of of kubernetes on the mainframe and that was a really fun talk I should do a refresh sometime soon lck change in like main frame on kubernetes oh yes yes we’ll go back to it like in a second and hear about about so that’s that’s what I work on now so running the open source program office obviously kubernetes is in our portfolio of projects that we sort of keep an eye on and make sure that it’s still running well on the platform

Holger Wolf: Good and so in my my career was maybe after the university maybe I was always interested in Linux and and all those stuff but there was no company really taking care for that and therefore after my studies I was interested in IBM because I was making a internship semester there as a student it was really fun because of a lot of Technology then I started and then I started with IBM Z mainframe really in 1998 so it’s quite a while ago and then I had really the chance when Linux came to the platform in 2000 that I jumped to this team as a performance engineer and really this was like you should IM there was no open source at IBM at this time at all so this was kind of the first project open source and management was not happy to see this in the first moment because I mean this is what is this what what are we doing there and therefore we had a lot of things we had to to explore and to understand here and then it was progressing and I was going through several kind of other projects like looking into KVM looking into into Cloud resources when this emerged with with containers and then then when when OpenShift came to the platform or was wanted to get on the platform I jumped on this because of of the interesting technology behind this based on Linux based on virtualization which is playing a role and then and then was looking into the testing areas and then getting more and more involved into the project so that that I’m I’m getting the lead to it and and was looking into all those technology kind of environment and how this is sted how this is operated what what it means for the Linux

Itiel Shwartz: Okay okay also like a super interesting like like story for both of you so Elizabeth maybe like let’s start with you and feel free to to jump in but if you can like expain a bit like what is a main frame or like you know you said that like five years ago they told you hey can can you can I should do you do you want to join the mainframe team and he weren’t sure so if you can like explain a bit I will say someone that never work like on real main frame like my answer would probably be be no because it has like terrible PR main frame but maybe you can Sho some light on why mainframe is not that bad right and what does it mean like mainframe on kubernetes

Elizabeth K. Joseph: Yeah so you know when actually my my first answer to IBM when they asked me if I wanted to work on mainframes was no yeah that’s with yeah yeah and I mean part of it was that I didn’t know anything about them and and you’re right I mean the pr historically has been not great because like one they’re they’re they’re not real right like in in movies you see them and like everyone wants to hack the main frame there’s always like a climax in the movie for someone like and it’s it’s all ridiculous and has nothing to do with that so there’s like the Fantasy main frames and then there’s actually the real ones and it turns out the real ones as I said like they’re actually super cool and and there’s a bunch of like marketing numbers that are thrown out out there like you know percentage you know like X percentage of Banks and X percentage of Airlines and which is actually a huge chunk like most of them in the world are using main frames and the one thing they say is you know when you swipe your credit card it’s going to go through a main frame at some point so these things are turns out everywhere and still powering our global economy but what really and that that’s great that’s cool but what really got me is like the hardware so when I started diving into it even before I joined IBM I was looking into it just to make sure I wanted this job and the processors they’re designed by IBM and they have lots of innovative interesting things in them they have like large caches because most of the workloads on the main frame are data driven and if you think historically the companies that needed mainframes back in the day I mean they they were the Airlines and Healthcare and government and they were organizations that were leveraging data most people didn’t need to work with data but these ones did and it turns out that that really aged well it turns out today everyone is using data like my cousin who’s an artist he’s like you know going through analytics on how to promote his art better by you know Ling all kinds of things so even artists are using data machines and that’s what these machines were built for from the beginning so the architecture and Hardware inside of them are really built for that they don’t have storage on the machine so the machine is basically like a Computing monster it’s got a ton of ram ton of CPU everything is really fast and connected tightly in these drawers and then there’s a bunch of cards that do very specific things like you know connecting to the storage there’s like an eight dot Hardware security module built in there is on chip compression and decompression which works on Linux so like if you’re using gzip on the main frame that’s going to be using Hardware driven stuff with crypto a bunch of crypto algorithms are interpRed Hated on the CPU itself with the latest release there’s an AI accelerator which makes inferencing really fast so you train your model somewhere and then you bring it over to the main frame to execute so things like fraud detection happen very very fast because it’s built to the hardware and you know digging into this technical it’s just so cool to be working on and yeah

Holger Wolf: You’ve got you forgot the LPAR partitioning and so on I mean it’s it’s a kind of a cloud in the box and has the capacity to really run a lot of things this was was always things which is fantastic because it’s even I’m 20 years on or more than 20 years on that platform it’s renewing itself but keeps what is good for and this is this is really amazing I’ve never seen such a platform which is kind of that stable and and scalable

Itiel Shwartz: So so let me ask you like two question and then we’ll go to you like first of all it’s main frame like a growing thing like is the usage of like mainframe is growing like with each day are more people using mainframe or less people using mainframe

Holger Wolf: I would say it’s stable used and it’s important I mean a mainframe is nothing you put on under your table I mean now maybe nowadays there are there are options you can you can have that and there’s some grow but it’s it’s kind of a a data monster and and depending on your use case and and I think the use case defines the set of people which might use the main frame of course we see a growing customer set of that and and and and the growing need to get Engineers on board so I see growth here to work more and more on on those topics because we have more and more things to deliver but it still kind of offers specialized edge case because not everybody can afford this one and needs that kind of is it’s a back end classical backend environment right

Elizabeth K. Joseph: Good okay yeah speaking of the form factor as well so they they these days they’re they don’t take up a whole room they sit fit in like a 19-inch rack spot last year we even announced a rack mountable version which I think starts at like 20U so okay okay

Itiel Shwartz: Super B I don’t remember like I don’t remember what maybe like you know maybe in a brief like no later later we’ll do like kubernetes and mainframe and how does those two like live together butar maybe you can share a bit about what you’re working on and yeah

Holger Wolf: I mean yeah what what we are working on I mean mainly I’m responsible very concRed Hate to to get the OpenShift platform working on the main frame and working very closely with Red Hat in the community I mean depending on when you’re looking at the overall stack there are a lot of projects which are we have first to identify in the either it’s already ported from teams or or it’s or we have to take care that we are porting it and and get it in integrated and then get it on the main frame because it’s a special architecture it’s it’s something which you’re not getting for free because the hardware is not available there’s IBM Cloud wise you have the possibility to to get this kind of hardware and there are Services which are for free for developers and you might have builds for s390x but it’s if you want to test it and so on it’s it’s kind of where we are getting integrated in and working out what’s available the downside of it maybe not everything is there because not all communities are on boarded yet and and not all is ported but when we are talking about porting it’s not like like an effort like an amazing horrible effort it’s sometimes a recompile only sometimes it’s a little bit more biggest problems are little or big-endian issues because we are the only big-endian that’s on at the planet right now potentially or or commercial one and and but that’s that’s fine problem there so the main thing is really getting things being being in EN so the I said the platform itself is a very modern one so it’s it’s has very is very capable from from CPU kind of speed things and that’s that makes it very interesting so because you have something which is which is really modern and and and this thrills me a lot and also the people when when they are starting when when I have new team members so so first of all oh what’s the mainframe then they see the Linux console or the OpenShifting and then oh that’s not very different to to all of it which which is kind of a cool because you have hundreds of CPUs available ter bites of storage if you want and and all this is SE seamlessly integrated so

Itiel Shwartz: So so maybe work with me like you know from what I know about like mainframe and and you said to yourself it has a lot of CPU it has a lot of ram in my head it’s great for I don’t know like monolithic application because you can put everything very close to you everything is in the RAM and everything is very very fast how does it work like in a kubernetes like distributed microservice architecture why should I combine the two

Holger Wolf: Yeah maybe maybe one thing the the main frame is architect not just for monolithic things it’s architected for transactional workloads as well and also to scale with a lot of processes so if you think of of kubernetes then you think of a lot of processes which are running in parallel on the platform many many small things which are coming up going down so the scheduling and so and and the system is really designed to to cope with that very easy and to handle this much more and I would I would say since I was in performance working on that a lot and we looked into to the scalability and to those issues starting a lot of process of scaling up we are scaling really really really like like like a flat line and get 100% of the CPU being being being utilized with a lot of processes and it’s not a problem context switching and so on it’s not a problem for it’s it’s even even much better design on theform so therefore for me it’s the perfect match to get kubernetes or something like this ons workloads on the platform additionally to of course there there are a lot of traditional workloads also transactional system like CICS where also some some kind of those those patterns are available but therefore it’s very fitting very well to to this modern world of computing because it’s it’s designed for this well and Kubernetes Elizabeth you want also yeah so a couple a couple of things I can I can interject here so one of the things we have to do with open source communities when we start working with them is just first education so when you know someone on Holger’s team you know submits a patch and says hey add s390x support the first question is usually what is that or does anyone use that and then we kind of have to put on the show of yes people use it there we have you know all that also it’s really cool so in in that in that case we you know I do a lot of my my team is the ecosystem team so I do a lot of conferences where I just give generic talks about Linux on the main frame or porting your open source application actually this year I did a couple on porting to various architectures so I gave examples on on RISC-V and Arm along with s390x kind of just educating developers that there are other architectures out there Arm has actually helped us a lot because they people know what an Arm is you know they know their Raspberry Pi or their watch or their phone it’s all Arm and that has kind of been an inroad for the work that I do in just say like okay you support Arm now now you know you can just add another architecture and to help with that I we mentioned like access to resources my team runs an open source Cloud well out of a university in New York so there’s an application folks can fill out if they are running an open source project and can get a VM we you know you can you can throw out some specs and I’ll say yes or no or you know I’ll work with you on on how much we can we can give out but that that program is specifically so folks can test on the platform but again like a lot of people just use QEMU I think the binaries from the kubernetes project directly are cross compiled so they’re compiled on x86 through emulation and you know that that’s okay but then like teams like hoggar team like they will run through make sure that those are actually working once they’re pulled into something like OpenShift because emulation not perfect and it’s a little slow so

Itiel Shwartz: Okay okay so so one of the things that you know you mentioned is that you gave a Talk At KubeCon I think you said like 2019 right yep and if you would to give the same lecture today a lot of change so have change in like the mainframe kubernetes space like what are like what change and also because I’m really clueless around that like where are we going like in in like what’s the vision like in a couple of years like what’s on the road map in this area so the

Elizabeth K. Joseph: The big thing that changed was there was basically no commercial support back in 2019 the OpenShift work had begun but there hadn’t been like a production release and so it was kind of experimental I believe at the time so my talk was very much like hey you can run Minikube and you know ranchers kind of working on this thing since then I mean OpenShift is is a mature product now on the platform with the latest release you can actually run mixed so like your host can be x86 and you can have s390x workers and the opposite is true as well which is really exciting because if you have a hybrid environment and you have the OpenShift you know shift running there you can sort of mix and match so companies can Leverage The Cloud as well because you know they’re already invested in that with their microservices there and whatever usually like things that are like don’t take advantage of the mainframe Technologies you can just you know toss them on the cloud and have that sort of mixed environment and that’s kind of the dream right especially for for IBM and just this hybrid Cloud Vision of getting our customers you know put your data and put your applications and put everything where it makes sense instead of being you know pulled into something just because you know you feel like you have to do that and so that’s that’s I mean that’s some of the work that I’m most proud of from you know OpenShift side Rancher briefly supported the platform but I think they they’ve put that on pause for now and so that that was a big deal and then now it’s a little bit less so you know we still work very closely with SUSE but I think that’s just it’s just not happening at the moment but there’s definitely you know there’s interest there’s there’s clients I think hogart can can chime in a lot more concRed Hate examples this as well

Holger Wolf: Yeah but but your example was the seamlessly running it where it’s where it’s fitting best is I think a good example for the customers because they they they have needs to run things on on x86 but they have also the need to run it on on the LinuxONE IBM Z platform and and if they could combine this in one single cluster this is this is a it’s a cool thing but even have have the ability to to to install it on on the on the same platform but having other clusters so so the flexibility is is a big thing I would say and also that the platform I mean what how I see it is that the platform combines a lot because you have no dis differences on on that you operate always the same so you do not need to have the platform specifics being very well known so you have to set this up but developers are really H doesn’t matter more or less I mean yes it’s another architecture you might need to be aware of it but to have that being in place this is pRed Hatty pRed Hatty cool and therefore they can start up to make their build process accordingly and and so on so potentially see more of of that those

Elizabeth K. Joseph: It’s actually it’s actually funny I we have we also so we give out VMS to open source communities but we also have a free version of the OpenShift environment that’s I think you get like two weeks of access to it just to test it out and and the funniest thing about that is you my you know my boss is like hey write write a write a a tutorial for that so people can learn how to use it and I’m like it’s the same as on x86 I could just pull down a random tutorial on x86 and tell them to run it so but any I got the pressure and I wrote one about NGINX but it is it is identical the only difference is you pull down a different container image because you have to pull down the s390x one instead of you know to the x86 so that’s the only thing I had to change really in the tutorial and it just goes you know went to show that like it’s the same thing developers probably won’t notice y

Itiel Shwartz: So so maybe like you know I’ll like maybe again like two questions H because I’m really out of my like game here right like I don’t I didn’t work that like never on Main Fram so why and how big is like the open source M frame Community like again if I would need to guess prior to this call I will say a lot of like the main frame is like you said like you know the big banks and telcos or like data companies fraud it’s not like the first thing that comes to mind when you are thinking about open source so how big is the open source Community around main frame and who are those people like are there normal people that love like this or they like is it big companies that are contributing as an open source to the greater good so maybe if you can share a bit about who is the community and yeah

Elizabeth K. Joseph: Yeah so the community it’s it’s growing and one of one of the sort of cornerstones of the open source community on mainframe is the open mainframe project which is part of the Linux Foundation so that that’s like the home of a bunch of different projects a lot of them are focused on z/OS and so the Linux I mean in the mainframe world you kind of have the z/OS folks and you have Linux folks and there’s other you know proprietary operating systems in there but broadly speaking you have these two groups and one of the really fascinating things that’s happening not like so I work mostly on Linux porting just because my background I still oversee some of the z/OS stuff but this was not as familiar to me but one of the things that’s been so fascinating on the z/OS side is there’s this project called z/OS open tools and what they’ve been doing is they’ve been porting traditional tools that are traditionally Linux tools over to UNIX System Services which is a Unix like environment on like sort of bolted onto the main frame but now everyone uses it but the thing is when you log into Unix system services for the first time you’re like oh this looks familiar and then you start running commands and they’re just different enough to be irritating but one of really cool things that this project has been doing is they they’re porting bash they’re porting git they’re porting sudo all these traditional tools and they’re contributing the patches Upstream so now if you download bash it’s got z/OS support inside of it just you know you just you know tra you know do the flag and run it on z/OS and so they they have over a 100 projects now and so you really can change that UNIX system environment to be something that developers may be more familiar with like in a Linux environment and I just thought so cool that they’re doing so much Upstream work and just going into these projects that were traditionally Linux only so Podman is now ported to z/OS I was like wow that’s that’s really something so you know even the z/OS side of the open source Community is growing and there’s a lot of interest there as far as who’s who’s there I mean obviously a lot of IBMers but there are other companies in the FR space and so they’re they’re all involved as well a lot of the developers also come from clients so people who get permission to work on open source but I guess because the big the big thing is you know first you have to have familiarity with the main frame and secondly you have to have access and a reason to do it so I have worked with a few hobbyists but honestly there’s not very many of them because it’s kind of hard to pick up this as a yeah like I feel like like expensive hobby or not the one that you know like I would go and buy like a Raspberry Pi at like I know like whatever at like $20 and like that’s like whole side project that is very easy easy to get started

Itiel Shwartz: And do you see something else on on your side like of the community

Holger Wolf: But the community Wise It’s I would say it’s a lot of IBMers but also I think the chance as a customer working on open source Max is attractive because you can resolve or solve problems you you might not be solving with a with a another operating system here or it’s the same as on other platforms that’s what I see but the contribution is huge here by by the IBM team so I’m here in an in an area where almost everybody is working open source and or contributes to it because this is the business we are in here

Itiel Shwartz: Is the Red Hat that like acquisition change something for you guys like the IBM like HashiCorp predit acquisition does it affect the mainframe or not really like the day-to-day

Elizabeth K. Joseph: I mean anything with open source will I can’t I don’t know anything about the HashiCorp stuff but like I you know anything anything that happens in this space eventually will trickle down to us because we do run these tools and we you know we want to Port everything to the platform so so as soon as something comes onto our platform usually it’s a it’s a client need or it’s it’s something that’s integrated in one of our products where you’re like oh and then we need support we’re like all right we’re on it

Holger Wolf: Yeah I mean with Red Hat I mean I’m I’m I’m also a Red Hat PE so I two places here working very with folks I mean unfor unfortunately I’m getting only paid by IBM and not by Red Hat as well so with the people at IBM you know yeah but but the point is I guess on one side Red Hat was already on the platform so far it’s it’s not like they as there was much change then I mean this was kind and rits is still kind of an independent company we are working very close with them and and but it’s also kind of we were used to work with them on the community base and just to the title integration we want to have that on the platform we might need to spend some those efforts which are important to get this on the platform but but what we are doing and that’s that’s maybe that would would not have been done before but therefore we are integrated more or less in the red head processes and IBM is really providing all those kind of services to get it on the which sometimes is very interesting because you’re Bridging the kind of of different work environments and and and that’s that’s also very thrilling I would say but on the other side we have one goal in in getting the products being supported for I on one side you have this community work which is always kind of yeah independent of because you cannot you cannot plan to say okay now I want to have it because the community might see something different and you have to discuss this through and you have to think out out of the box so this is one side which is maybe kind of we are used to it already and this is what what brings us together and then on the other side we are working towards to get when we have something Upstream being working on the downstream stuff concentrating on getting this done support for the customer so that they can have their solution being in a supported way and and and and and then yeah we we have to to bridge those gaps sometimes with with with a lot of communication but that that makes it really interesting to be honest and I would say it’s a very I like it because I think this to to keep it like this makes it very powerful for me course I mean still this this agile kind of work environment from a smaller company but dedicated to fully open source I mean it’s not said that IBM is not dedicated to open source best example is here Elizabeth and her efforts or what we are doing here it’s it’s but we still working as we are a hardware company providing that hardware and working around of this and and providing the best solution for the customer for it and therefore it’s a very good match to get things here together

Itiel Shwartz: Okay I think that we almost ran out of time any last word Elizabeth either

Elizabeth K. Joseph: I just I want to say to anyone listening if say this is the first time they’ve heard of main frames like don’t let it be the last these are really cool things I’ve I’ve put out some articles and other things just kind of giving brief overviews of the technical stuff it’s just so cool just just take a look well we let it to the notes in the in the show great right and

Holger Wolf: It’s fascinating it just can’t not say anything

Itiel Shwartz: I’ll be honest like again for me not a main frame guy like it was super interesting first of all like hearing about like main frame General and more than that like both main frame in open source and main frame in kubernetes like two things that I’ll be honest for to this call I didn’t really knew existed and so it was like super interesting and like a a bit different than a lot of the things that you know at least I warped on like a day-to-day day to day so it was a pleasure having you here and I will meet maybe in KubeCon Elizabeth like if you are going happy to coffee or beer

Elizabeth K. Joseph: That would be great thanks

Holger Wolf: Be cool thank you

[Music] Kubernetes for Humans.

This is an AI generated transcript of the conversation