#033 – Kubernetes For Humans Podcast with Mirko Novakovic (Dash0)

Itiel Shwartz: Hello everyone and welcome to another episode of Kubernetes for Humans podcast today with me in the show we have Mirko Mirko do you want to introduce yourself

Mirko Novakovic: Yeah hi my name is Mirko I’m co-founder and Co of Dash0 before that I was co-founder and Co of Instana and both are observability companiesanies where Dash0 is now very much focused on OpenTelemetry and Open Standards

Itiel Shwartz: Yeah so happy to have you on the show to be honest we also recorded an episode live on KubeCon but because of technical Giles the episode like disappeared and we’re doing like a re-recording so it’s going to be like a very nice and special episode and super interesting it was very important for me to do this recording because I think you have a very interesting story so maybe let’s start with the beginning right like maybe share a bit about your background where did you came and maybe a bit about the start of Instana

Mirko Novakovic: Yeah sure I mean I’m I’m basically a classical software engineer right studied companiesuter science programmed I loved programming then I founded my first companiesany which was a consultant business in Germany codecentric grew it to 500 people over 10 years and then then decided that I want to build software and found it Instana in 2015 which was at that time APM for microservices because this is about kubernetes we essentially at the beginning of kubernetes around 2015 saw that there is a demand for a different type of application Performance Management today we say observability solution which deals better with the containers with microservices the interactions has a different set of understanding of events and so we did that I did it for five years Venture Capital backed and then we sold to IBM and yeah then I said I will never work again I will never do a startup again I will never do anything in observability again and had a threeyear break yeah and then and we can talk about that but a year and a half ago I really got bored and I got excited about this space again and and I saw what OpenTelemetry has done in the in the years and so I decided to start with some colleagues Dash0 an OpenTelemetry native observability solution and yeah we released the product basically at KubeCon where we met and now we are month in

Itiel Shwartz: Okay no like a super interesting story so let’s maybe do like a bit of a focus on things St phase then let’s talk about your like three years like resting and then and then we’ll move to the present and I will say that when Komodor like was founded like five years ago we took a lot of inspiration from you guys because Instana unlike most APM tools it felt that it was built for a micros service like world like in in its UI and the easiness of Instanaallation while traditional APM tools AppDynamics and so on were very heavy and felt very Enterprises right like and that is the customer based in end of the day in felt like the new kid on the Block so we took like a lot of inspiration internally in Komodor from a lot of your stuff maybe you can share a bit about maybe like both the first day of Instana and then you can you know go over like the journey until you were like bought by Instana which I will say we also had like Chris Bailey which is now the CTO of Instana in like one of our episodes so like it’s a full circle here so yeah

Mirko Novakovic: Yeah absolutely Chris is a great guy and yeah let’s start by what’s different right it’s exactly what you said the classical APM vendors the agent that you Instanaall to get the data were pretty heavyweight right which didn’t really matter if you have a monolithic application in large I don’t know WebSphasere application server with which already is is is super big but when you run I don’t know in in the Java world little spring boot applications and you start them up in containers then I don’t know for for a JVM with with 100 megabytes of of memory or something you don’t want to put in an agent that requires 500 megabytes right so so and we essentially developed an agent that is super small footprint we called it sensors so basically decoupled the communication layer from the in ingestion layer and we had autodiscovery because containers also come and go as you know right kubernetes can just say oh I spin up a new container and then you want that to be Instanarumented automatically and detected in real time so we implemented real-time event streaming we in implemented autodiscovery to inject the things automatically into the containers and that was the core technology behind Instana right and we and and we built a UI if you know the little boxes right on the infrastructure map that basically helped you understand how your kubernetes environment how your application stack looks like in real time and how are connected and gave you insights into into problems yeah that that was the basic idea and that’s what we built and it’s it’s technically challenging especially on the agent side and of course also processing a lot of data and stuff but that’s kind of a fixed problem and yeah that’s what we built and we started and we were focused on kubernetes customers had really good traction at the beginning so I don’t know we it took a year to build the first initial product then we started going out in 2016 and we were really on that high growth track Right double triple double double triple right thing and and and yeah then in during covid IBM reached out and yeah the rest is history we sold

Itiel Shwartz: How was you know like I’m interesting because APM always feels to me like a very like maybe red ocean maybe now it’s more than it used to be but in the end of the day you have a lot of vendors right like Datadogg is the most dominant and successful one by far right now even I’m not sure if that was the case back then and everyone does claim at least to do pretty much the same right like the traces the logs and and and the metrics so how like how did you guys different like between one another and is it really that hard like does existing customers moved to Instana back then or it was mainly like people who didn’t had any APM like bought Instana as the first APM maybe you can TR bit about that

Mirko Novakovic: Yeah I mean we have to go back to first to 2015 and in 2015 Datadog did not have APM Datadog started basically as a metric solution right so they did metrics for for the cloud for AWS mainly and they had nice dashboards then they bought a companiesany I don’t know which year a Logmatic a French companiesany which and and they added logs to the stack and then they started building tracing right so they got into the APM space but when we found it in St the world was still pretty divided right you had the Splunk of the world they did logging you had the Datadoggs which did infrastructure monitoring and then you had the APM vendors like AppDynamics at the time the leading two APM vendors and then us Instana are focusing on kubernetes what then happened and that was also a challenge for us is that those categories merged right Splunk bought SignalFx they bought Omnition to get metrics and tracing to their logs so Splunk build a platform as I said Datadog bought Logmatic and then they built out metrics Dynatrace added logs and and and other things so the vendor started merging these categories together and at the same time this observability topic came out and I know that the honeycomb people don’t like it if you say it’s logs metrics and traces but essentially that’s what happened right observability is now the new word for those categories in one tool right that’s that’s kind of what happened and it was challenging for us as a startup because when we started we had a clear differentiator we were APM for microservices and kubernetes and we were companieseting with AppDynamics and Dynatrace and they they had This heavyweight agent but then we also had to build the other tools right we we at the beginning we partnered you probably know the Coralogix guys also Israel so we partnered with them because they only have loging at the time and we took them as a logging tool but then today they also added everything else right yeah they have everything as well like everyone is doing everything you know every everyone is doing everything and so back to your question yes it’s definitely red ocean and when I started zero I was asking myself oh do I want to get into that companiesetitive market but now what I’ve learned in the meanwhile I’ve done so many investments in different categories there are pros and cons about getting into a red-ocean Market the pro is there is a market yeah yeah and people and people know what it is and and there is a category there are budgets already and it’s more companiesetition but it’s also a huge market like $50 billion market so there’s always a chance I always say to my team if we get 1% of the $50 billion Market we have 500 million Revenue right that’s not bad that’s not bad so so the thing is that in a large red-ocean Market you still have chances to get into the niche make a lot of money where when you have a blue-ocean Market you have to evangelize right there’s no budgets people don’t know the category I mean probably with with platform engineering the same you have you have seen that right that

Itiel Shwartz: No no a budget plan from a customer who didn’t like he’s already paying Datadogg $2 million right and then for example Commodore which really helps to simplify kubernetes but in the end of the day he doesn’t have a budget line Commodore or kubernetes management or kubernetes Trav shooting and then he needs to invent it and people don’t like to pay money in the end of the day like it’s something that like in order to create a new category you need to work really hard on on the evangelism so so let’s talk a bit about you know you sold in right like you can say the amount you can you can not say the amount but then you are free to do whatever you want right so what did you do like between yeah between now and then

Mirko Novakovic: There are some there are some press reviews that say $500 million was the sales price and it’s not that wrong right I can’t say the the clear amount but it’s somewhere around that so for us it was a really good exit and I stayed for a few months then I left and and then I first did nothing right I was tired 5 six years of building a startup is really like you say oh now I can lie at the beach right and that’s what I did but then you also figure out oh it’s kind of boring right you can’t or or you start to drink too much alcohol right so so you need to do something right and also I have a family with three kids they go to school and you can do whatever you want right yeah so I started different things I started a restaurant I started wine bars I a brewery a co-working space in my city so I did different things yeah and I also did a lot of Angel investment I did around 80 Angel Investments all in basically the category of infrastructure software SAS business I helped a few Founders got on board yeah that’s who is your like top three investment top three Investments I had a really good one because and I think they they also out of Tel Aviv it’s called Granulate yeah and it was one of my first Investments and I got in and and and they sold for a really good price pretty quickly to Intel which yeah as an angel investor that’s that’s a ni thing to do right but I have a lot of really interesting ones that that I don’t want to take out someone out of this but there are very different very different paths but what I learned if I want can give you a learning of doing at early stage Investments it’s really about the founders it’s not so much about the product the category if it’s very early stage because you can’t really judge I’ve seen really exciting product with bad teams and that will never go anywhere but I have seen like products that you think hm I’m not sure but if it’s a world-class team they will figure it out right they will pivot they will change they will listen to customers they they they they will do the right things so my learning is invest into the right people right then especially early stage that is what works what what did you learn about wine bars and restaurants what can you share like the audience on those those topics

Mirko Novakovic: The first thing I learned is that you cannot apply anything you learned in startups and our business to rest because the main difference is it’s very low margin yeah the margin it’s very different business right so we have when you have Venture Capital right we go to a KubeCon show and we build a booth and it costs tens of thousands of dollars and you can do that in a restaurant right I mean you will you will never get your money back right so you have to really calculate you have to make sure and the other thing is that you have a very short feedback loop which I love I mean your your guests basically if you do a bad job if the food is not good or the service is not good they will tell you immediately right or you get a one star on Google or whatever it’s a really direct feedback it’s a hard job Saturday Sunday evenings you work pretty hard but it’s also rewarding right it’s it’s it’s it’s a it’s a nice category but it’s especially in Germany it’s it’s it’s very hard to make money with it right it’s it’s not something you will become rich if you do a restaurant right it’s it’s it’s it’s it’s more a lifestyle thing

Itiel Shwartz: So for like all of our listener that are thinking about opening one like probably not the best unless you already sold your companiesany for a couple of hundreds of million of dollars so that’s like a a good tip of advice and maybe like share about like why opening a new startup maybe let now now let’s focus a bit on what you’re are currently doing why how who and

Mirko Novakovic: Yeah I mean the first thing is probably coming to the conclusion that you want to do another startup right and and that for me was a process over the three years right I figured out to be honest that I’m not really good at being an adviser especially not for myself because I get very nervous pretty quickly and then I mean as an adviser you’re outside right you’re like the coach you’re not on the field and I’m more a player right I love to do things and so I figured out that advisor role is not what really satisfy me and what I want to do and also investing is pretty hard because in in our business when you do something you said hey you were at cucon then you get leads and you can follow up the leads so you see the result pretty quickly m in in the business of investing you see the results five seven years later data right and that’s also very hard for me because I like like this quick feedback cycle so I figured out that investing is not what I want to do so I knew I want to do be operational again I want to build a team I want to build a product so then it was about what I want to build yeah and I saw that I don’t know and I love the space right I love observability I get excited I know the space pretty well I know a really I had a really team so I want to bring the team together again so I decided to go into observability again and the chance that I’ve seen is that this new standard OpenTelemetry which is standardizing the way you get the data right what I said the agents basically all standardized there’s an API there’s a standard for the data format so that’s pretty cool because it’s democratizing the data part and so I saw that there’s an opportunity to build something that’s really made L supporting OpenTelemetry and not only that we also support things like PromQL for querying or Perses for dashboard so we want to be as standardized as possible so that the customer does not have the lockin problem anymore right today if you use Datadog you have the Datadog agents you have maybe configured 100 dashboards in Datadog you have configured 100 alerts it is really hard to to move right because you can’t take the dashboards and run them in in another tool but if you are based on standards like you can take your alerts with you because they are in standard format you can take the dashboards with you because they are in standard format you can leave the agents you just have to pinpoint the OpenTelemetry collector to a different tool and it will work right away right so it’s your investment is just safe and also your learnings and and so I thought this is really a great opportunity to build something that’s 100% companiesatible with those standards

Itiel Shwartz: No it’s interesting I know that you know like Datadog for example released I think V Vector.dev right victor. Dev Vector so like how do they play along like in the end of the day APM unlike seven eight years ago is like a very like like well understood category with existing players and like you said $50 billion like market and so how are the other players like playing with OpenTelemetry how do you different from them like

Mirko Novakovic: Yeah I think Vector was also an acquisition and there is a whole C I think there is a whole category which is called these observability pipelines yeah basically the big the big gorilla there is Cribl right and and what those vendors did is they basically tried to solve the problem that I just described that there are multiple agents and multiple backends and you can’t really send data from I don’t know Splunk to I don’t know Prometheus or something like that so you put something in the middle but just like a queuing system and then you can transform the data you can send it to different backends you can remove data you can enrich data and that’s these pipelines that’s what Vector also does right and I think the product is now called observability pipelines and Datadog and but on on the OpenTelemetry side I think the main difference between some somebody who’s OpenTelemetry native and somebody who’s supporting OpenTelemetry is that if you supported we did the same at Instana that means you integrate the data you take OpenTelemetry probably with your agent and then you convert it into your internal data model because that’s how your tool work right you have your data model you have your semantic convention Etc and then you basically lose the OpenTelemetry part of the data so what we said is no we will not do that our whole system is based on OpenTelemetry so the back end index zero is based on open telemetry our semantic convention is based on the OpenTelemetry semantic convention the tagging is the same so when you set OpenTelemetry in natively you can also get it out everything you see in the tool is OpenTelemetry based so it’s not it it’s it’s a different approach right on and it is hard for a vendor again I was in that shoe if you have already done all the work you have your data model you have your UI you have your things maybe you have even more than the current standard supports it is very hard to support it you can just integrate the data but it’s hard to supported natively and that’s the main difference

Itiel Shwartz: So so let me ask you like two two two follow-ups based on what you are saying everything is standard right like and that’s what you guys are building so why should they choose you guys right like let’s say that there’s a lot of like OpenTelemetry you know standards everyone is like collecting the same data you don’t invent the data you’re saying you’re are not the one doing the pipelines right and even the dashboard you’re saying are quite standard so what is your Edge in like this kind of phase or like who is going to win

Mirko Novakovic: Yeah good question I think and that’s kind of the very interesting part if it’s not anymore about the data I think the winning factors will be what you do with the data right and one of the challenges of of observability is that you get I mean especially in these kubernetes environments you get more and more data metrics more spans more logs and so I think the vendors will companiesete on who does the best job for the developer for the SRE for the platform engineer to give them the insights that they want in an easy way right so you will companiesete on the user experience and by the way for me AI is part of the user experience right I think really good AI I I always like to companiesare it with Apple in Apple products the AI is somewhere right you make a photo but you can copy the text out of the photo easily that’s actually AI but they don’t name it AI it’s just a feature that’s pretty cool and it works and so this this is how we see it right AI should make it very easy to correlate things to find problems to visualize them and to to to make it easy for you to get the data right group them reduce them and that’s user experience also make like we have full key keyboard support it’s super fast super slick UI so we try to make the user experience as easy and slick as possible and then there’s another point and and I think if you talk to Datadog customers they will tell you as an example that’s the cost part yeah and the co and the cost part has two things one is is it expensive or not and that’s a discussion I don’t even want to I don’t want to blame anyone and say this is too expensive I mean if if if somebody pays as you said $2 million for Datadog kudos to Datadog because that also means they provide so much value that somebody pays $2 million for it right so that’s I wouldn’t say that’s that’s something bad if you if you can pay if you pay a lot of dollars for a product right that just means they did a good job what I don’t like and what customers don’t like is the transparency so what happens with most vendors is they say oh I tried you for I don’t know one gigabyte of logs $22 and then you send something in and at the end of the months you say oh you got $40,000 for so many but you have no idea where the data came from and why right I don’t think I met one that do customers that understand is buing including we like in Comm to ourself we really like Dat Dog and we also paid him quite a lot of money but I never know what’s going to be my billing in the end of the month and it’s one of the trickiest thing they have so many section and so many different like they’re very notorious known for I’m just opening a new opportunity in my CRM right

Itiel Shwartz: No that it’s exactly the issue we hear a lot right it’s like the transparency I think it’s a it’s a real pain point the thing about Dad do is that the product is good you know like we are paying dat do like you said because you are getting a lot of value in return and it is hard for organization to do the switch because your developers are already used to it right and the alerts are already like that and then you need to do a migration and I think I had an interesting conversation once with the folks in Coralogix and they told me that they you know when sometimes like that the customers move to them it take it can it can take like a lot of time right to do the migration it can be like six months just to do the migration and it’s also money right like it’s not that like a like a little dwarf is doing it for you like you need to work super hard on making it happen

Mirko Novakovic: Yeah so no absolutely I totally agree that’s why I say it’s not it’s not that easy right people it’s a good Tool but the frustration on the transparency is basically there right and that’s also something we want to basically fix not necessarily say you have to switch or something but the market is just so big that there is a lot of opportunity in there there is

Itiel Shwartz: Well one question you know I talked in the conference in KubeCon with the Morgan McLean from like Splunk who is one of the OpenTelemetry like contributor or creators even and he told me a very a very similar story to you how spank is like leveraging OpenTelemetry do you see like do you see big companiesanies starting to say things that you’re saying like that everything should be OpenTelemetry like is it a trend like in couple of years is everyone going to be super OpenTelemetry like adopted for companies

Mirko Novakovic: Yeah I mean I like that we basically started as the first we said we are OpenTelemetry native observability and when I K companies I walk past the spun booth and it said big OpenTelemetry native observability then I walked by Elastic booth and said OpenTelemetry first observability right so it seemed like the message is resonating and I see that as a really good sign because what does it mean nobody changes it if there is no demand and they have thousands of customers probably that means that they get those requests from the market and I think that’s the answer to your question I really think that the market will shift to OpenTelemetry because the benefits are just too obvious right no lockin also very transparent good agents which are developed by the community more secure right and and so I don’t think that and also it’s integrated in a lot of things right it’s integrated in kubernetes it’s integrated in in in platforms like Vercel or or in the clouds like it’s won like I feel that now it’s only a question of like how how much time will it take to be like exactly so I see it the same it already won it will take probably a lot of time like always with standards right it takes time until it’s adopted but I think it’s it’s the winner and it’s we always say it’s the last agent you will Instanaall right because it’s now the standard agent and and that’s a benefit and so that’s why they are jumping on that right and yeah exactly I I can’t tell you if if Splunk and and how good Splunk and elastic will support in the future I can tell you what they do today and that’s not OpenTelemetry native but they probably will do a good job it’s the same when we started with Instana 2015 we were basically the only one who could really do kubernetes APM but the other vendors catched up right it takes two three years and then yeah everyone could do kubernetes right and I think that’s the same here I think over the next years you will see more and more OpenTelemetry native Solutions really really OpenTelemetry native Solutions but then again it’s with every startup right we have to be faster we have to be better we have to support it quicker and and in in a nice way so it’s it’s a good companiesetitive market I I actually like it

Itiel Shwartz: Okay no that that that is super super cool okay like I think we’re out of time and like maybe like a prediction what does the future have in store what do you want to tell us and and to our like listeners

Mirko Novakovic: I think I mean this very obvious right I I after KubeCon we also internally spoke what are the two main topics At KubeCon I think for me there were two main topics one is platform engineering yeah like if you look back there was devops first then there was like SREs and now everybody talks about platform engineering and I think platform engineering will also have an impact on observability right how you do it how it’s integrated with platforms how people will use it how it’s provided to the end users selfservice Etc so that’s one Trend which will impact and the other is a right yeah and and for us that means and we already see that with our customers every application will have some an LLM somewhere and LLMs will be observed differently right and so you have to build observability for LLMs but you also have to integrate them into your tool could provide better user experience right that’s that’s the trends that I’m seeing I think it’s nothing revolutionary here or new that if you are in this space you can see it but that’s that’s actually the trend yeah

Itiel Shwartz: Okay sounds super cool so thank you a lot for doing this episode the second time I hope this recording won’t get lost in the mail and pleasure I know are are we going to see you in the next KubeCon over London

Mirko Novakovic: Absolutely we are Platinum sponsors this time

Itiel Shwartz: Ah budget in restaurant world put like all thank you

[Music] Kubernetes for Humans.

This is an AI generated transcript of the conversation