DevOps: Metrics For Identifying Software Development Problems

Poor quality metrics often lead to unactionable data or, worse, inspire changes that harm productivity rather than improve it. Using appropriate metrics during developer analysis is essential for identifying where real problems lie in software development processes.

The Importance of Using the Right Metrics

Utilizing high-quality, actionable performance data provides significant benefits across an organization:

Recruiting Managers: Can accurately highlight skill gaps when hiring new personnel.

DevOps Teams: Gain better information to optimize workflows.

Senior Managers: Obtain a clearer picture of software delivery performance.

Key Features of Effective Metrics

To be effective, metrics must possess three specific characteristics:

Reliable: Data must be consistent and not open to interpretation so leaders can trust its accuracy.

Objective: Performance data should not be influenced by human bias or specific hypotheses. Automating analysis is the best way to ensure objectivity.

Actionable: Metrics must help make concrete changes to how a team works rather than just providing "interesting" data.

Metrics to Avoid When Identifying Problems

Without the correct metrics, managers struggle to identify the problems blocking team productivity.

1. Counting Lines of Code (LOC)

Counting lines of code is one of the oldest but most flawed KPI metrics.

Disadvantages: It offers no strategic value and cannot measure quality. It only shows lines added, not the intellectual effort required, and is rarely applicable across different programming languages.

Limited Use Case: It may show if members are contributing, but it cannot help manage technical debt.

2. Function Points

This method divides a project based on the specific tasks or functions it must complete.

Disadvantages: It does not account for the time or resources necessary for complex functions.

Subjectivity: Defining function points is time-consuming and risks being subjective because it relies on human judgment.

3. Sprint Burndown Chart

These charts compare remaining workload to the time left to finish a project.

Disadvantage: They often distract teams from underlying workflow problems because significant focus is shifted toward organizing and improving the chart itself rather than the work.

4. Code Churn

Code churn measures code that is rewritten or deleted shortly after being written.

Disadvantage: There is no way to accurately identify what causes the churn, leading to high levels of subjectivity during discussions between managers and developers.

A More Effective Metric: Actual Coding Effort (ACE)

Relying on a single flawed metric is insufficient for understanding team dynamics. As an efficient alternative, BlueOptima utilizes a metric called ACE (Actual Coding Effort).

ACE combines up to 36 separate metrics to define developer productivity as accurately as possible. These are categorized into:

Volume: How much code a developer creates.

Complexity: How difficult the code was to create.

Interrelatedness: Whether the different components a developer is working on are related.

Coding Effort provides a more meaningful foundation for decisions because it is reliable, objective, and allows for fair comparison across various teams and individuals.

Customer Success Story: Project Resource Allocation

Company Type: Bank

Developers Analyzed: 9,000 to 11,000

Product Used: Developer Analytics (DA)

The BackgroundThe client had a large project, "Project X," that was approximately 25% less productive than the average project across the estate. Project X was the largest project and had been allocated roughly 50% of the company's resources.

The ResultsBlueOptima uncovered that a large proportion of the coding effort (~40%) was delivered by long-term staff, who represented only ~10% of the developers on the project. Additionally, ~40% of the developers were found to be inactive, which lowered the overall team averages.

Customer Success Story: DevOps Transformation

Company Type: Bank

Developers Analyzed: 7,000 to 9,000

Product Used: Developer Analytics (DA)

The BackgroundThe client wanted to justify investment in DevOps to senior management by identifying high-performing teams and replicating their practices across the entire estate.

The ResultsBlueOptima confirmed that the highest-performing team was 31% more productive than others. By replicating these best practices, the client achieved:

Productivity Increase: 30% overall (from 1.5 to 2.0 BCE/Day).

Capacity Saving: £63 Million.

Quality Increase: 4% (reduction from 7% to 6% aberrant BCE %).

Who are BlueOptima?

We provide a SaaS technology that objectively measures software development efficiency. Our core metrics for productivity and code maintainability allow executives to make data-driven decisions related to talent optimization, vendor management, location strategy, and much more.

‍