The Right KPIs for Mainframe DevOps

To compete in fast-moving, disruption-prone digital markets, enterprises must aggressively embrace DevOps on the mainframe. DevOps is becoming increasingly important, and without it—along with associated agile best practices—delivery on these digital initiatives will be too slow and infrequent to keep pace with escalating customer expectations. To successfully drive adoption of DevOps on the mainframe, IT leaders need the right metrics. Good metrics ensure that decisions are fact-based, that changes in processes and tooling yield quantifiable results, and that mistakes can be quickly corrected. The right metrics also enable DevOps leaders to inspire continuous learning and improvement across organizations, teams and individuals—which benefits the business while creating a more engaging workplace culture. Unfortunately, many enterprises can still fail to measure software delivery on the mainframe even with the right KPIs. Quality-centric KPIs such as number of defects and mean time to defect resolution are critically important for many organizations. And that’s understandable, because the stability and reliability of mainframe applications are central to the platform’s long-established value proposition. But measuring quality alone isn't enough. Enterprise IT leaders must be highly intentional about complementing their quality-centric KPIs with those that measure two other critical attributes of modern software delivery: velocity and efficiency.

The DevOps Trifecta

With a balanced approach to DevOps KPIs that includes quality, velocity and efficiency, organizations can better leverage mainframe applications in support of broader digital transformation. Here’s why each element of this DevOps trifecta is important: Quality. As noted earlier, quality should never be sacrificed for speed. Mainframe applications must be rock solid because they deliver the core logic of the business. Defects in mainframe applications can also generate bad customer experiences. Quality isn’t just about finding and fixing defects, though. It’s about consistently finding defects earlier in the software delivery life cycle. Anyone leading a mainframe DevOps effort should focus on unit testing that empowers developers to pinpoint coding issues immediately. This shift left is a staple of DevOps and agile methodologies. It increases the number of trapped defects, while decreasing the number of escaped defects—resulting in safer, more efficient code delivery. Unit testing also promotes richer, accelerated developer learning about good coding practices and the applications they work with. Velocity. Speed is perhaps the single greatest deficit in mainframe software delivery. Slow delivery of new mainframe-dependent digital capabilities can threaten large enterprises as they compete against highly nimble greenfield market disruptors. That’s why mainframe DevOps leaders must elevate velocity KPIs. Some of these velocity KPIs should measure individual steps in the software life cycle, such as the mean and maximum amount of time it takes to get user stories logged and into the coding pipeline. Others will be more “macro”—measuring the end-to-end time it takes to get code that has made it through unit testing, integration, quality assurance and promotion into the production environment. Regardless of which KPIs are chosen, the main imperative is to prioritize substantial, measurable increases in the speed with which ideas become digital realities. Efficiency. With more work to do and fewer people to do it, mainframe leaders must drive quantifiable increases in DevOps efficiency. This neglect of efficiency cannot stand, especially as IT organizations lose their most experienced mainframe professionals, and are only filling about a third of those open positions. Mainframe leaders have to take a cold, hard look at how much of the mainframe software life cycle they’re automating. The good news is that there are growing opportunities to bring automation into the mainframe DevOps pipeline in areas such as unit testing, creation of appropriately obfuscated test data and integration with non-mainframe or cross-platform solutions such as Jenkins, SonarSource, and XebiaLabs. Efficiency gains can be measured with KPIs, such as the number of story points per sprint and the number of epics per deployment. Pipeline efficiency improvements will also result in more frequent code drops, which contributes greatly to business agility.

What’s My Motivation?

The move to mainframe DevOps requires time, effort, cultural change and budget. It’s natural to ask why you should invest in it at all. The answers are pretty compelling. Enterprises that measurably improve quality, velocity and efficiency across the mainframe software life cycle deliver better customer experiences, respond more nimbly to changing market conditions, and achieve higher revenue across both digital and traditional lines of business. They’re also better able to navigate the generational shift in mainframe stewardship from platform veterans with multiple decades of hands-on experience to next-gen DevOps artisans who play such a pivotal role in enterprise digital transformations. So if you weren’t planning to substantially change your KPIs as you seek to substantially improve your mainframe software delivery practices, think again. You have to manage to the right metrics to get the right results, and that means being intentional and precise about adding velocity and efficiency KPIs to a set of modified quality KPIs. You may not change all your KPIs at once, and you may at times abandon some KPIs for others that seem more appropriate for your goals. But your metrics will be key to your success—so choose wisely, and never stop using metrics to challenge your organization to achieve higher levels of mainframe DevOps excellence.

---