How do you measure DevOps success ?
You may not know if you are achieving success with your DevOps implementation unless you measure it and also manage it continuously to keep the key performance indicators (KPI) in right balance.
Following are the few KPIs that help in measure DevOps success.
- KPI 1 Quality
- KPI 2 Speed
- KPI 3 Application Performance
- KPI 4 Customer experience
- KPI 5 Business Success
Underline principle of DevOps is faster and better software delivery.
Shorter development cycles and close collaboration under a DevOps approach mean you catch problems before they go to production, and so spend less time fixing them.
Software quality is directly correlated to customer- impacting issues, so it pays to invest time and energy into addressing quality issues earlier in the cycles. Anyways you do not want customers to be the ones discovering your issues.
Following metrics can help track and quantify the quality of your software’s overall continuous improvement efforts:
- Deployment success/failure rate
- Application error rates
- Issue severity
- Outstanding bugs
Since many quality issues do not appear until the software hits production, the ability to compare all metrics pre/post deploy is extremely helpful in understanding the impact of a specific release. This approach also allows teams to respond quickly and rollback a release or provide a quick resolution to any incidents that have occurred.
DevOps teams need to focus on the speed of development, delivery, and response to issues that occur in production. Metrics to consider for helping to track progress and success in this area include:
- Lead time for changes
- Frequency of code releases
- Mean time to resolution
Catching performance problems before they manifest requires tracking number of performance metrics like
- Up-time (availability)
- App response time
- Database response time
- % of transaction time spent in database
- Slow SQL queries
- Resource usage
A poor application performance will result in bad customer experience. Customer experience KPI is directly related to Application performance. Following are the few metrics that can help measure customer experience.
- Response time of key transactions
- Frequency of key transactions
- Number of visits per user/per week
- User growth rates
- Amount of time spent in app
Business success can be measured by measuring goals that are important for the company. For example a software can be valuable to the company when it helps grow the customer base, increase revenue, reduce customer service costs, or some other objectives.
These objectives can be identified by talking to the business stakeholders and understand their business goals and how they will be impacted by delivering the software quickly.
In a nutshell, all the KPIs are interrelated to each other and impact each other. If we have quality and or Speed issues on production it will result in bad customer experience which in turn will lead to reduced business success. So, if we measure the 5 KPIs we can find out if our DevOps implementation is resulting in Business success.
|Quality||1. Deployment success/failure rate
2. Application error rates
3. Issue severity
|Speed||1. Lead time for changes
2. Frequency of code releases
3. Mean time to resolution
|Application Performance||1. Uptime (availability)
2. App response time
3. Database response time
4. % of transaction time spent in database
5. Slow SQL queries
6. Resource usage
|Customer Experience||1. Response time of key transactions
2. Frequency of key transactions
3. Number of visits per user/per week
4. User growth rates
Amount of time spent in app
|Business Success||Depending on Business Goals.|