Effective DevOps Requires a Clear Way to Measure Progress

Organizations often embark on a DevOps transformation without a clear picture of what they want to improve or otherwise get out of the process. Generally, DevOps serves to help organizations deliver better software more rapidly. Many measurements, however, could indicate how well or how poorly a team is achieving this objective. 

A more focused model — and one that I often recommend — comes from DevOps Research and Assessment. DORA collects and analyzes data from many organizations and publishes several resources, including an annual “Accelerate State of DevOps” report. This year’s report has identified four keys to measure DevOps performance: deployment frequency, lead time for changes, change failure rate and time to restore service.  

Within these keys, DORA provides Elite, High, Medium and Low tiers that organizations can use to measure their progress. It’s important to note that an Elite ranking may not be feasible for every organization. Often, that’s reserved for major cloud-native services with millions of users and massive budgets. 

Every organization should, however, establish a baseline so it can identify growth when it happens — especially the early phases of growth, which can be especially meaningful. Start measuring now so that you can see the impact of changes as they happen.

We often describe DevOps as a culture: a set of guiding principles that transcends a mere collection of processes. In a perfect world, if an organization is adopting DevOps successfully and its culture is improving, the metrics will also improve. It’s very rare that metrics will improve without an organization’s culture improving. That said, organizations may want to augment process-oriented metrics with tools designed to gauge sentiment and opinion related to culture. 

Democratization of information within and among teams is another key concept. This applies to both metrics and to DevOps processes themselves. Openness requires a level of trust that leads to better outcomes. The alternative — distrust and the siloing of information — is destructive, and it ultimately degrades culture and product quality. 

Measurements and dashboards should be visible at every level. When scores improve, everyone can see it. Everyone feels a sense of ownership and motivation. When teams lack that visibility, it’s easy for members to feel disconnected and to believe they don’t need to contribute to improvements.

The pace of DevOps transformation is less important than continuous improvement. The scope of each increment may depend on an organization’s capabilities: how many people can support the effort, how much bandwidth they have and how much leadership support there is to temporarily disrupt some priorities to benefit others in the long run. 

Changing habits is disruptive, and many teams and organizations may resist that disruption for fear of losing what they have already accomplished. In response to that concern, I usually point out that there are also risks for failing to measure DevOps progress. As with most endeavors, lack of measurement often means lack of results. Organizational leaders should also remember that although their organizations may not be doing this, their competitors are. 

Outpacing the competition — and making your own organization stronger — requires a clear sense of direction and the ability to mark progress along the way.

Story by Mark Yorko, a senior field solutions architect for CDW's DevOps practice.