Day 2 Challenges – Why Hiring a Platform Team is Not Enough

If you’ve been anywhere in the DevOpsphere in recent times, you have certainly encountered the Platform Engineering vs. DevOps vs. SRE debates that are all the rage.

Is DevOps truly dead?! Is Platform Engineering all I need?! Have I been doing it wrong all along? These have become more popular than the mono vs. multi-repo flame wars from a few years back.

The core of the debate isn’t around the terminology, it’s about what the practice actually involves––because honestly if you haven’t been doing platform engineering by now, you haven’t been doing DevOps right. If you haven’t unlocked the promise of DevOps yet, then we’ll show you how to ensure your platform team isn’t completely useless, and your DevOps isn’t floundering.

In this post I am going to unpack why I think building a platform engineering team isn’t the panacea you might expect, and what challenges organizations face with actually getting these teams to deliver the value we demand in this brave new DevOps-driven world.

Understanding the Role of a Platform Team

Let’s start by understanding what we are referring to when we talk about platform engineering. Many think that platform engineering is a drag-and-drop replacement for DevOps, but really just like other well-established disciplines such as SRE (site reliability engineering), it is a subset of DevOps and a domain expertise unto itself.

No need to state the obvious, but cloud and even cloud-native architecture (e.g. Kubernetes/ Serverless patterns) – have become the leading way for software-based companies to build their products. Like most engineering organizations, there comes a point where centralized DevOps teams become a bottleneck and a new paradigm needs to be explored to reduce the friction and enable greater autonomy for developers to move at the pace that modern businesses require. Enter platform engineering.

Platform engineering is not new. Historically it was called release engineering, similar to SRE being called production engineering. The role itself has just been rebranded and has gained widespread popularity. The challenges it comes to solve remain the same though. 

So how does a platform engineer differ from your regular DevOps engineer?

Platform engineers are oftentimes senior engineers with deep expertise in both development and ops. The main objective that platform teams are trying to achieve is reducing the friction, cognitive load, and complexity for developers when managing cloud ops at the scale we see today.  Basically, when done right, platform engineering becomes the oil in the machine that keeps everything running smoothly.

This is achieved by building the tools and repeatable infrastructure that developers need, with the goal of reaching self-service, for developers to have greater autonomy with guardrails to build the environments they require to power their day-to-day work.

Why is Platform Engineering So Hard?!

This is why it’s important, before setting out to dive right in and build the tools, to actually work closely with your “customers” – the internal developers, to ensure that the tools that you build are the ones they actually need and will want to use. This is because there is no one-size-fits-all for engineering organizations, each organization has methods and practices, flows, tooling, and stacks, and in large-scale engineering organizations, this is true down to the individual team––where the tools, methods, and practice differ significantly from team to team.

If we take a reference from the observability world––platform engineering is a high cardinality and high dimensionality practice that requires much customization (remember that word “customer”––that’s because what is being built is highly customized to their needs). The platform being built needs to be tailored to the internal processes, skill set, stack, and workflows to be successful and adopted.  

This means that the same level of productization you invest in your outbound customers needs to be invested in your internal customers, including robust documentation, proper onboarding and training sessions, and other education as needed, as well as incorporating feedback, and continuously iterating until you reach product-market fit.  That is why much of the DevOps practice is now evolving into the Platform as a Product mindset.

Where Does This Break Down?

In the context of systems thinking, many times internal developer portals (IDPs)––the place where all of the dev tools the platform team builds are usually housed––aren’t optimized for all the parts in the system.  A system is comprised of humans, machines, and the culture and practices that power them.  Good day 2 operations are optimized for the humans and machines in the system, and IDPs are essentially your portal to manage the day after tomorrow as well as possible.

When not properly designed and built for the end-user developer, IDPs end up being costly abandoned, beached whales, instead of being the king of the ocean (your systems).  This inherent developer experience is what will be the differentiator between platform engineering tools that are first-class citizens, and natively embedded in your SDLC and those that are completely useless.

What happens many times, is platform engineering teams invest heavily in developing these tools( they are a costly team for the company comprised of seasoned engineers) with a significant time sink until the team begins delivering value to the developers. This is OK though, if they are set on building the right set of tools and productizing them. This is an initial productivity trade-off that management needs to be aware of when hiring platform teams. This is also where fostering trust is a critical piece in platform engineering success. Management tolerance and support will define a platform team’s success, especially considering the known initial productivity dip. The reality is that a platform team may initially consume more development time than it ultimately saves.

Without this deep understanding and sufficient patience to enable the platform team to have the resources and time they need to succeed, this may result in frustration and even potentially set the team up for failure.

The Importance of Fostering Collaboration and Trust

On the other hand, the platform team needs to constantly be aware that their end goal is to better serve the developers, and they need to be careful not to fall into known patterns of “handouts” vs. “hands up”.  In a “handout” culture, platform engineers will become the firefighters of the organization and provide escalation support, or at worst, become internal Kubernetes maintainers. With a “hand up” mentality, platform engineers provide the tools and the education to enable developers to succeed autonomously, a culture of learning and empowerment.  (Basically teaching the teams how to fish, and not fishing for them).

Trust is the backbone that powers this collaboration. This level of trust can only be built upon close alignment on shared business goals of improving the velocity, reliability, and security of products.  This is why superficial separations only hurt organizations––by creating unnecessary silos, bottlenecks, and friction, that ultimately lead to burnout and really just overall toxic organizational culture for everyone in the process.  

And like all things, there is no magical way to foster trust.  This can only be created by consistency and quality over time, through a platform team that is open and collaborative, with the end goal of building tools that the developers will love.  Developer trust shouldn’t be blindly given, it is fostered by creating good practices and a culture of transparency, and visibility alongside the required audit trails and guardrails, to ensure that whatever they do autonomously they can do safely.  Essentially the equivalent of least privilege for trust, least trust––create the minimum viable amount of trust in the system to get started.

Wrap Up

Platform Engineering, whether a new discipline or just a rebrand of an established practice, certainly has become more acute with the growth and scale of modern cloud-native engineering. The need to create an additional bridge and golden path between the world of development and engineering makes platform teams a central piece to a modern company’s success.

When given the proper resources, fostering the trust and patience required to build and productize the right tooling, and ensuring open communication and collaboration–platform teams can be what creates the well-oiled machine of an engineering organization. The future of software delivery, and even DevOps, will be where internal processes intersect with shared responsibility and goals. This is how you unleash the power of platform engineering! 

The humans in the system should never be the barriers or bottlenecks––but rather the enablers. By empowering developers and elevating the developer experience, we can really disrupt the status quo and reach the next phase in the evolution of engineering, where every engineer has DevOps superpowers.

Conclusion

Kubernetes Day2 Operations can be complex and challenging. However, the benefits of Kubernetes, such as scalability, portability, automation, resiliency, and cost savings, have made it a compelling choice for modern application development. 

By understanding the challenges and leveraging tools like Komodor, you can successfully navigate the ‘day-after’ process and take full advantage of the opportunities for innovation that Kubernetes provides.

To make your Kubernetes migration journey a success: 

• Focus on proper planning and strategy by conducting thorough assessments of your current infrastructure and applications.
• Implement tools like Komodor to simplify dev onboarding process; they provide valuable insights and capabilities to help manage, monitor, and troubleshoot Kubernetes resources.
• Invest in continuous education and collaboration so your teams stay up-to-date with best practices and new developments in the Kubernetes ecosystem. 

Enabling Kubernetes is a significant undertaking. Still, with careful planning, the right tools, and a focus on staying ahead of the curve, you can successfully complete the journey and unlock the full potential of Kubernetes for your organization.

You’re invited to join us and get started today!