History has a tendency to repeat itself. This is because bad habits and anti-patterns are hard to break.
And this remains the case with the latest sought-after engineering unicorn––the “Kubernetes expert”.
These days, there is a veritable gold rush to hire the best and brightest Kubernetes wizards. Like all forms of expertise––this gold is rare, and as a result––is also costly. But this isn’t a new phenomenon in the technology world.
The discussion about whether there is a talent shortage goes as far back as 2013, with Andrew Clay Shafer’s iconic talk at Velocity NYC, “There is No Talent Shortage”, which has been referenced many times over the years in different contexts. This was true when OpenStack was an emerging technology, DevOps and config management were novel and its experts rare, all the way through to DBAs, and even once upon a time software engineers being hired based on their deep expertise in a specific programming language (which is an all but obsolete practice).
This is also why the idea of “shift-left” in nearly every domain has gained such widespread buy-in and consensus. Having specific domain expertise concentrated in a single person or group has been proven as un-scalable with modern cloud-native operations.
Yet, the demand for skilled Kubernetes experts is only growing.
I recently wrote a LinkedIn post (that got folks riled up) about why I think this is all wrong, and I’m going to double down on this perspective and share why I still believe this whole mindset needs a rethink.
The Anti-Patterns with Seeking Engineering Gold
Below I’ll dive into some of the anti-patterns that develop in organizations when chasing the best and the brightest in specific domains, and what companies can do in order to hard-reset these practices.
Companies as Recruitment Agencies
One of the common phenomena that arises when companies chase scarce talent, is that the entire organization becomes deeply invested in this very particular task and cause, diverting attention from the real and actual goals of the company. Engineers, HR, CTOs, and many others find themselves on the continuous hunt for rare talent, clearing schedules, interviewing candidates and assignments, and investing significant time and energy to find that one perfect candidate.
But, like dating, perhaps there are many potential and capable candidates, if we open our minds enough, and are a little bit more flexible about the skill set we need. By investing so much company attention in hiring, you find the organization losing sight of its actual goals! What’s more, you end up only enabling and fostering this anti-pattern that shouldn’t exist in the first place.
Anti-Trust in Technology Disciplines
Another anti-pattern that arises from this flawed mindset, is that you continue to invest in siloed and very specific domain expertise, creating organizational bottlenecks and friction concentrated in single individuals or teams. There is a dependency in the workflow, depending on the size of your organization that eventually doesn’t scale––which will lead to either the need to increase the number of instances (i.e. hire more such individuals), which also isn’t economically or linearly scalable. This renders you in a boot loop of the first anti-pattern––and then the company goals and vision get derailed.
This is where the whole concept of shift-left or DevOps originated from to begin with. The original premise behind the DevOps culture was intended to break down the silos between those writing the applications, deploying them, and then maintaining them in production. Basically, it holds together all the critical pieces of application delivery and operations that can’t be decoupled––while still managing to do continuous delivery successfully.
All this tells us is that modern, infinitely scalable, microservices and cloud-native operations are complex. The worst solution to complexity is to concentrate on specific expertise––the exact opposite approach is needed.
Democratizing Technology and the Big Tent Approach
Just as shift left is based on the premise of empowering developers to own parts of the stack that weren’t formerly their responsibility – from operations to security, and even testing, the same goes for specific tools that are now a big part of how their applications are deployed, whether its Kubernetes or just the regular ol’ cloud.
The better, more inclusive approach for the greater company good, is to work on demystifying tools perceived as complex, like Kubernetes. In open source communities, this is called the big tent approach, where all vendors, competing technologies, expertise from noob to ninja, and even peripheral skills are considered an integral and even important part of what makes the community successful. A hodgepodge of capabilities that complement each other.
When it comes to complex tooling adoption in organizations, this starts with breaking it down, and making its underlying concepts and fundamentals more approachable so that more of the team members can use it effectively. This doesn’t mean diluting its power or limiting its use cases or capabilities, but rather unlocking it for a broader audience by democratizing the knowledge and know-how and increasing its adoption across your engineering organization.
In order for this to happen, the Kubernetes ecosystem and culture itself should rethink its mindset, and transition from a system that is very focused around domain expertise, to a system that empowers its end-users (developers). This approach has been successfully implemented in many other engineering domains, from frontend/backend engineering to full-stack engineering, likewise the dichotomy between development and operations, and even security that is being embedded much earlier in engineering processes.
Below are a few tips to make this practically possible:
- Simplify Access: Develop user-friendly interfaces and tools that make Kubernetes approachable for developers of all skill levels. Use automation to handle the complex parts of setup and deployment, allowing developers to focus on building their applications.
- Enhance Education: Create concise, clear, and practical tutorials and documentation. Offer workshops and online courses that cater to different levels of expertise, from beginners to advanced users, focusing on real-world applications and problem-solving.
- Build Community Support: Foster a strong community support system where developers can easily find help, share knowledge, and exchange ideas. Encourage contributions from diverse backgrounds.
It is possible to empower more developers to understand and manage their Kubernetes operations, dispersing the knowledge among many and removing the friction and bottleneck that hiring x-technology experts require.
Fullstack Kubernetes FTW!
Kubernetes is a technology that has been around now for more than a decade. It has reached maturity and adoption while having a robust community and ecosystem that powers it. Like Linux, cloud, or other technologies that are commoditized––and the domain expertise distributed among many types of engineers, the same should go for Kubernetes. Kubernetes should become just another tool in the engineering stack that modern engineers work with (like infrastructure as code or cloud).
While this is easier said than done, in the same thinking as DevOps as a culture or the shift left mindset, this begins with building bridges between complexity and usability. By shifting our focus to simplifying Kubernetes for everyone, we can harness its true potential without being held back by its steep learning curve. We empower our teams, foster innovation, and move from being talent hunters to technology leaders.
