It is said that what cannot be measured cannot be managed. But how do you measure behavior or mindset? Therein lies the challenge with measuring DevOps. Much more than a buzzword, DevOps is about a mindset – a culture that is made up of processes and practices that bring development and operations teams together within a company to develop and deploy software, with maximum efficiency and minimal interruptions. So how then can we define and measure the benefits and importance of DevOps?
First, it’s important to understand what DevOps is not, so a better contrast can be made around how to understand its benefits. DevOps is not about one tool or another. It is, however, about behavior – a mindset to leverage and use the correct tools that shorten and improve software development life cycles. DevOps doesn’t seek to erase the differences between the two disciplines of software development and IT operations, but instead builds a bridge to make them work better together while continuing to follow traditional processes in each discipline independently.
DevOps is about business agility and continuous delivery, a discipline created through understanding both development and operations. A move to DevOps engineering culture is relatively easier for a smaller company (including startups) than one that is large and established. For larger and well-established companies, it is advisable to take an incremental change approach, introducing DevOps slowly to change people behavior and maximize adoption.
While measuring and quantifying people behavior, in regards to DevOps, may be challenging, it does not mean that the benefits of DevOps cannot be measured. One tangible benefit of DevOps is an observed decrease in development and operations cost. Other measurable benefits of DevOps include:
- Shorter Development Cycle
- Increased Release Velocity
- Improved Defect Detection
- Reduced Deployment Failures and Rollbacks
- Reduced Time to Recover upon Failure.
These benefits can, in fact, be turned into key performance indicators (KPIs) for your company to track. Companies can track the time and cost it takes for project that develop software applications using a traditional non-DevOps approach and compare it against using DevOps to better appreciate the quantifiable benefits of DevOps.
Shorter Development Cycle – DevOps promotes a culture of increased collaboration and communication between the development and operations teams. This translates into shorter timeframes to move from engineering code into executable production code.
Increased Release Velocity – The shorter development cycle in turn creates increased frequency for release of code into production (also known as application onboarding). What conventionally used to take 3-6 months from requirements to release is now reduced to a daily, if not hourly, release build cycle. This fosters continuous development and deployment, subsequently increasing the value of IT to the business. Increased release velocity also provides companies that successfully manage DevOps a competitive advantage due to their quickness to market features that their customers need.
Improved Defect Detection – DevOps builds on top of the agile programming methodology and, in a sense, it can be considered as extending agile programming. It prescribes to several agile principles such as collaboration, iterative development, and modular programming, breaking larger codebases into smaller manageable features. This makes it easier to detect code defects.
Reduced Deployment Failures and Rollbacks – The benefits gained from faster development and deployment can be nullified by failed deployment. But software when developed using a DevOps mindset takes into account an operational viewpoint as well. This, when combined with improved defect detection, significantly decreases the number of pre- and post-deployment issues and therefore fewer rollbacks.
Reduced Time-to-Recovery – Even if the likelihood of failures is minimized, failures as a rule are inevitable. When failures do occur, the time to recover to operational efficiency is relatively reduced in a DevOps environment compared to a traditional IT environment. This is primarily due to the efficiencies gained by development team members understanding how operations teams work and vice-versa. Combined with a robust versioning process, rollbacks are made easier.
Not all of the benefits of DevOps can be directly measured and tracked as KPIs. In fact, one could argue that the intangible benefits of DevOps hold equal, if not more weight, than the manifest outcomes of the discipline. Some of these important benefits of DevOps include:
- Increased Communication and Collaboration,
- Improved ability to Research and Innovate
- Performance Oriented Culture
Increased Communication and Collaboration – Since DevOps takes two disciplines, development and operations, which were traditionally siloed into one discipline, in inculcates a culture that is characterized by increased communication and collaboration.
Improved Ability to Research and Innovate – By fostering a culture of high trust between team members and sharing of risks, team members are encouraged to experiment to continuously improve the company’s products and services, making it possible for them to research newer customer needs and innovate accordingly to address those needs. A metric to track here would be to see the number of innovations that have resulted, before and after, moving to DevOps.
Performance-Oriented Culture – DevOps shifts the company culture to be more performance-based than being rule-based or power-based. This reduces the bureaucratic ‘red tape’ and fosters sharing of risks, leading to a more satisfied and therefore productive workforce, which has a direct impact on proportional to business performance. Job retention and turnover statistics, pre- and post-DevOps can be helpful in determining this cultural trend.
The goal of DevOps is to increase business value by making it agile through continuously delivery of products and services that satisfy customer needs. There are several tangible and intangible, yet very important benefits to DevOps that require measurement to help an organization better appreciate the impact of a DevOps approach. As the saying goes, what cannot be measured cannot be managed.