DevOps: Metrics For Identifying Software Development Problems

Organisations often fail to use effective metrics during analysis. Using poor quality metrics may lead to unactionable data or, worse, inspire changes that harm productivity rather than improve it.

This article reviews the importance of using appropriate metrics during developer analysis. We highlight why some of the most commonly used KPI metrics for software development might not provide accurate or relevant information to managers.

The Importance of Using the Right Metrics

By utilising high-quality and actionable performance data, everyone benefits:

• Recruiting Managers: Able to highlight skill gaps within an organisation when hiring new personnel.
• DevOps Teams: Gain better information when trying to optimise workflows.
• Senior Managers: Obtain a clearer picture of software delivery performance.

Reliable, actionable metrics are critical for identifying where problems lie in your software development processes.

Key Features of Effective Metrics

• Reliable: Data must be consistent and not open to interpretation. Leaders must trust the accuracy of the metrics to ensure changes are effective.
• Objective: Performance data should not be influenced by a specific hypothesis. Automating analysis removes the risk of human bias.
• Actionable: Metrics must help make real, concrete changes to how a software development team works rather than just producing "interesting" data.

Metrics to Avoid When Identifying Software Development Problems

Without the correct metrics, managers struggle to get an accurate picture of the problems blocking their team’s productivity.

1. Counting Lines of Code (LOC)

Counting lines of code (also known as LOC or SLOC) is one of the oldest KPI metrics, but it is deeply flawed.

• Advantage: Simple and easily automated.
• Disadvantages: Offers no strategic value and cannot measure quality. It only shows lines added, not the intellectual effort required, and is rarely applicable across different programming languages.
• Use Case: Indicates if members are contributing, but fails at managing technical debt.

2. Function Points

This involves dividing a project based on the specific tasks or functions it must complete.

• Disadvantages: It does not account for the time or resources necessary for complex functions. Furthermore, defining these points is time-consuming and risks being subjective because it involves human judgment.

3. Sprint Burndown Chart

These charts compare remaining workload to the time left to finish a project.

• Advantage: Allows managers to quickly gauge progress on specific development stages.
• Disadvantage: Often distracts teams from underlying workflow problems, as significant focus is shifted toward organising and improving the chart itself.

4. Code Churn

Code churn measures code that is rewritten or deleted shortly after being written.

• Advantage: Helps in making staffing decisions by identifying if a developer is unsuited for a current project.
• Disadvantage: It cannot accurately identify the cause of the churn, leading to potential subjectivity during manager-developer discussions.

A More Effective Metric: Actual Coding Effort (ACE)

Relying on a single flawed metric is insufficient for understanding team dynamics. As an efficient alternative, BlueOptima utilises a metric called ACE (Actual Coding Effort).

ACE combines up to 36 separate metrics to define developer productivity as accurately as possible. These are categorised into:

• Volume: How much code a developer creates.
• Complexity: How difficult the code was to create.
• Interrelatedness: Whether the different components a developer is working on are related.

Coding Effort provides a meaningful foundation for decisions because it is reliable, objective, and fair across different teams and individuals.

Customer Success Story: Project Resource Allocation

A low-performing project with a high allocation of resources

• Company Type: Bank
• Developers Analyzed: 9,000 to 11,000
• Product Used: Developer Analytics (DA)

The BackgroundThe client had a large project, "Project X," that was ~25% less productive than the average project across the estate, despite being allocated ~50% of the company's resources. The client needed to understand why these bottlenecks were occurring.

The ResultsBlueOptima uncovered that ~40% of the coding effort was delivered by long-term staff, who represented only ~10% of the developers on the project. Additionally, ~40% of the developers were found to be inactive, significantly lowering team averages.

Customer Success Story: DevOps Transformation

Identifying and implementing best practices

• Company Type: Bank
• Developers Analyzed: 7,000 to 9,000
• Product Used: Developer Analytics (DA)

The BackgroundThe client wanted to justify investment in DevOps by proving it would increase productivity. They aimed to identify high-performing teams and replicate their practices across the entire estate.

The ResultsBlueOptima confirmed the highest-performing team was 31% more productive than others. By replicating these best practices, the client achieved:

• Productivity Increase: 30% overall (from 1.5 to 2.0 BCE/Day).
• Capacity Saving: £63 Million.
• Quality Increase: 4% (reduction from 7% to 6% aberrant BCE %).

Who are BlueOptima?

We provide a SaaS technology that objectively measures software development efficiency. Our core metrics for productivity and code maintainability allow executives to make data-driven decisions related to talent optimization, vendor management, location strategy, and much more.

Contact Us

To discover powerful insights and determine areas of improvement specific to your organisation, reach out to our team:

• Phone: +44 207 100 8740
• Email: [email protected]
• Website: www.blueoptima.com

‍