Are your developers on PagerDuty? That’s the core question, and for most teams the answer is emphatically “yes.” This is a huge change from a few years ago when, unless you did not have DevOps or SRE teams, the answer was a resounding “no.”
So, what’s changed?
A long-term trend is happening across large and small companies, and that is the convergence of developers, those who code apps, and DevOps, those who maintain the systems on which apps run and developers code. There are three core reasons for this shift – (1) transformation to the cloud, (2) a shift to a single store of observability data, and (3) a focus of technical work efforts on business KPIs.
The impending impact on DevOps in terms of role, workflow, and alignment to the business will be profound. Before diving into the three reasons shortly, first, why should business leaders care?
The role of DevOps and team dynamics – The lines are blurring between traditionally separate teams as developers, DevOps, and SREs increasingly collide. The best organizations will adjust team roles and skills, and they will change workflows to more cohesive approaches. One key way is via communicating around commingled data sets as opposed to distinct and separate vendors built and isolated around roles. While every technical role will be impacted, the largest change will be felt by DevOps as companies redefine its role and the mentalities that are required by its team members going forward.
Cost efficiency – As organizations adjust to the new paradigm, their team makeup must adjust accordingly. Different skills will be needed, different vendors will be used, and costs will consolidate.
Culture and expectations adaptation – Who will you be on call with PagerDuty? How will the roles of DevOps and SREs change when developers can directly monitor, alert, and resolve their own questions? What will the expectation of triage be when teams are working closer together and focused on business outcomes rather than uptime? DevOps will not just be setting up vendors, maintaining developer tools, and monitoring cloud costs.
Transformation to the cloud
This is a well-trodden topic, so the short story is… Vendors would love to eliminate roles on your teams entirely, especially DevOps and SREs. Transformation to the cloud means everything is virtual. While the cloud is arguably more immense in complexity, teams no longer deal with physical equipment that literally requires someone onsite or in an office. With virtual environments, cloud and cloud-related vendors manage your infrastructure, vendor setup, developer tooling, and cost measures… all of which have the goals of less setup and zero ongoing maintenance.
The role of DevOps won’t be eliminated… at least not any time soon, but it must flex and align. As cloud vendors make it so easy for developers to run and maintain their applications, DevOps in its current incarnation is not needed. Vendors and developers themselves can support the infrastructure and applications respectively.
Instead, DevOps will need to justify their work efforts according to business KPIs such as revenue and churn. A small subset of the current DevOps team will have KPIs around developer efficiency, becoming the internal gatekeeper to enforce standardization across your developers and the entire software lifecycle, including how apps are built, tested, deployed, and monitored. Developers can then be accountable for the effectiveness and efficiency of their apps (and underlying infrastructure) from end-to-end. This means developers – not DevOps – are on PagerDuty, monitor issues across the full stack, and respond to incidents.
Single store of observability data
Vendors and tools are converging on a single set of data types. Looking at the actions of different engineering teams, efforts can easily be bucketed into analytics (e.g., product, experience, engineering), monitoring (e.g., user, application, infrastructure), and security. What’s interesting is that these buckets currently use different vendors built for specific roles, but the underlying datasets are quickly becoming the same. This was not true just a few years ago.
The definition of observability data is to collect *all* the unstructured data that’s created within applications (whether server-side or client-side) and the surrounding infrastructure. While the structure of this data varies by discipline, it is always transformed into four forms – metrics, logs, traces, and, more recently, events.
Current vendors generally think of these four types separately, with one used for logs, another for traces, a third for metrics, and yet another for analytics. However, when you combine these four types, you create the underpinnings of a common data store. The use cases of these common data types become immense because analytics, monitoring, and security all use the same underlying data types and thus should leverage the same store. The question is then less about how to collect and store the data (which is often the source of vendor lock-in), and more about how to use the combined data to create analysis that best informs and protects the business.
The convergence between developers and DevOps teams – and in this case eventually product as well – is that the same data is needed for all their use cases. With the same data, teams can increasingly speak the same language. Workflows that were painful before now become possible. (There’s no more finger-pointing between DevOps and developers.) The work efforts become more aligned around what drives the business and less about what each separate vendor tells you is most important. The roles then become blurred instead of having previously clean dividing lines.
Focus of work efforts on business KPIs
Teams are increasingly driven by business goals and the top line. For DevOps, the focus is shifting from the current low bar of uptime and SLAs to those KPIs that correlate to revenue, churn, and user experience. And with business alignment, developers and DevOps are being asked to report differently and to justify their work efforts and prioritization.
For example, one large Fortune 500 retailer has monthly meetings across their engineering groups (no product managers included). They review the KPIs on which business leaders are focused, especially top-line revenue loss. The developers (not DevOps) select specific metrics and errors as leading indicators of revenue loss and break them down by type (e.g., crashes, error logs, ANRs), user impact (e.g., abandonment rate), and area of the app affected (e.g., startup, purchase flow).
Notice there’s no mention of DevOps metrics. The group does not review the historically used metrics around uptime and SLAs because those are assumed… and are not actionable to prioritize work and better grow the business.
The goal is to prioritize developer and DevOps efforts to push business goals. This means engineering teams must now justify work, which requires total team investment into this new approach. In many ways, this is easier than the previous methodology of separately driving technical KPIs.
DevOps must flex and align
DevOps is not disappearing altogether, but it must evolve alongside the changing technology and business landscapes of today’s business KPI-driven world. Those in DevOps adapted to the rapid adoption of the cloud, and must adapt again to the fact that technological advancements and consolidation of data sources will impact them.
As cloud infrastructures become more modular and easier to maintain, vendors will further force a shift in the roles and responsibilities of DevOps. And as observability, analytics, and security data consolidates, a set of vendors will emerge – looking at Databricks, Confluent, and Snowflake – to manage this complexity. Thus, the data will become more accessible and easier to leverage, allowing developers and business leaders to connect the data to the true value – aligning work efforts to business impact.
DevOps must follow suit, aligning their efforts to goals that have the greatest impact on the business.