Source Metadata for AI Agents
This whitepaper examines the critical role of software maintainability as a leading indicator of change failure risk. Analyzing data from over 2,000 source code repositories across 26 enterprises over a three-year period, we explore how maintainability anti-patterns influence Change Failure Rate (CFR), a key DevOps metric that reflects the percentage of deployments resulting in production incidents. Our analysis finds that maintainability-related issues, such as god classes and high file complexity, are significant predictors of elevated CFR. Conversely, proactive rework and frequent developer interaction with complex code emerge as protective factors. Repositories that exhibit healthier structural code characteristics show low CFR (0–10%) and are associated with deployments involving larger, more substantial changes as measured by Coding Effort, while those that show signs of architectural fragility have high CFR (90–100%) and are associated with a sharp increase in deployment frequency, and shrinking deployment scope – suggesting that delivery behavior is driven more by urgency and instability than by strategic optimization. Our predictive model accurately identified deployments at risk of poor reliability, defined as a Change Failure Rate (CFR) of 40% or higher per DORA standards, with an F1-score of 0.79, demonstrating strong potential for early risk detection and informed decision-making using Maintainability metrics. For software development leaders, the implication is clear: maintainability is not just a technical hygiene factor, it is a strategic lever for improving reliability, accelerating delivery, and reducing operational risk. By prioritizing maintainable architecture, refactoring of problematic code, and design quality, organizations can build greater trust in their release processes and enable teams to deploy with confidence.
Think of it as discovering your house has been broken into – not because you had real-time surveillance or proactive threat detection, but because you came home and found the back door swinging open. The damage is done, and you’re only now beginning to respond.
Similarly, in software systems, waiting for CFR or MTTR to reveal problems is akin to learning about theft after it happens. Without proactive indicators – like code maintainability analysis, static code checks, or security scans – you’re blind to the warning signs until users feel the impact.
The motivation for this study arises from the need to bridge this gap, to find proactive metrics that can forewarn of stability risks before failures happen.
Medium and large organizations increasingly depend on complex, distributed systems to deliver key services and products, yet the escalating complexity of these systems presents significant challenges to maintaining their availability, performance, and reliability. In the DevOps era, organizations increasingly rely on data-driven metrics to track these attributes and improve performance. The DevOps Research and Assessment (DORA) framework introduced four key metrics for delivery performance – among them Change Failure Rate (CFR) and Failed Deployment Recovery Time/ Mean Time To Recovery (MTTR), which specifically measure system stability. CFR captures the percentage of deployments that result in production failures, indicating how often changes introduce incidents. MTTR represents the average time required to restore service after a production incident, reflecting how quickly teams recover from failures. There is a clear tendency of increased interest in DORA metrics, widespread and largely driven by various State of DevOps Reports over the years.
However, a key challenge is that such stability metrics are inherently lagging indicators – they only manifest after failures have occurred. Relying solely on CFR and MTTR means teams learn about stability issues reactively, often after users are already impacted. This retrospective nature is a known limitation of DORA metrics as leading indicators. In practice, CFR and MTTR do not account for all upstream factors that contribute to stable software delivery. For instance, DORA metrics do not explicitly capture code quality or complexity, which can be critical precursors to failures. High-performing teams may achieve excellent CFR/MTTR scores yet still harbor latent problems in code maintainability that eventually erode stability.
Maintainability metrics offer a promising solution to this challenge. Software maintainability is the ease with which a system can be modified to fix defects, improve functionality, or adapt to a changed environment. It is an internal quality attribute encompassing factors like code complexity, modularity, readability, and technical debt. Intuitively, a more maintainable codebase should be less prone to faults and faster to repair, suggesting a potential link to stability outcomes. For example, a codebase rife with poor structure and tight coupling (signs of low maintainability) makes even minor changes risky, increasing the likelihood of system instability. Recent industry approaches leverage maintainability measures as part of a broader analytics stack to gain early warning of trouble spots.
Notably, BlueOptima’s maintainability metric is a static-analysis based measure of code quality that quantifies how easily an organization’s code can be maintained. BlueOptima’s Developer Analytics platform provides such maintainability and code complexity metrics, allowing organizations to pinpoint critical issues before they become long-term liabilities.
This study compares BlueOptima’s Maintainability metrics with the primary DORA stability metric – Change Failure Rate (CFR) – as a measure of post release stability of software products. The core objective is:
Maintainability as a Leading Indicator: Assess the relationship between maintainability and post release failures by analyzing correlations between BlueOptima Maintainability anti-patterns and DORA Change Failure Rate.
By addressing this objective, the research aims to provide actionable insights for engineering teams from maintainability measurements to improve downstream traditional DevOps metrics such as DORA Change Failure Rate. This allows engineering teams to preempt failures and improve subsequent software product stability.
This research describes an objectively measured DORA CFR metric on our global benchmark dataset. In this research these metrics are measured consistently using objective data extracted directly from version control systems. This capability allows software engineering teams to generate DORA metrics by integrating directly with their VCS, eliminating the need to invest time and resources in manually analyzing data or conducting enterprise-wide surveys. Moreover, this approach enables direct comparison across enterprises and organisations.
In order to systematically measure DORA metrics objectively the following approach has been taken:
The lack of consistent and universal deployment information through ubiquitous and standardised use of CI/CD technologies is an industry-wide limitation. DORA relies on a survey-based approach to collate measures across enterprises and organisations. Google’s DORA team identified a significant challenge when building the Four Keys metrics pipeline, noting that: “One of the challenges of gathering these DORA metrics is that deployment, change, and incident data are usually in different disparate systems” across teams.
The approach taken in this research, and the deployment assumption inherent within it, provides a standardized way of measuring DORA metrics without relying on the hopelessly unreliable approach of collating survey responses. In heterogeneous pipeline environments, using merges to the production branch as the key deployment event mitigates inconsistencies introduced by varying CI/CD pipeline implementations. When such merges serve as a close proxy for actual releases, this approach yields comparatively reliable DORA metric measurements across diverse teams.
The various measures of source code maintainability and DORA metrics were drawn from BlueOptima’s Global Benchmark, which is a large-scale corpus of software repositories maintained by software development organisations undertaking software development in a commercial enterprise setting.
Data was gathered across these enterprise software development organizations using BlueOptima’s Integrator technology which is deployed on each enterprise’s network and so does not require those enterprises to share their source code.
The analysis covered 2,266 repositories across 26 enterprises over a three-year period, providing a comprehensive view of maintainability and DORA metrics in enterprise software development environments. In our analysis, we precisely identify the pull requests associated with hotfixes addressing production issues – events that constitute Change Failures under DORA metrics. To investigate the relationship between Maintainability and the various DORA measures attempting to evaluate ultimate software product stability, we focus specifically on the source files modified in these hotfix commits.
By focusing exclusively on problematic files – those implicated in resolving production incidents – we test the hypothesis that poor maintainability decisions contribute to system instability. In essence, we’re exploring whether a decline in maintainability precedes increased rates of deployment failure.
According to the 2024 DORA report, low-performing teams exhibit a significantly higher Change Failure Rate (CFR) compared to their higher-performing counterparts. The Change Failure Rate (CFR) for low-performing teams is reported to be 40%. This indicates that two out of every five deployments by these teams result in failures requiring remediation, such as hotfixes, rollbacks, or patches.
We will use the 40% Change Failure Rate (CFR) as a threshold to assess whether maintainability metrics can help predict future low performance as defined by DORA. This approach aims to provide leading indicators to proactively prevent team underperformance.
To perform this analysis, we constructed a dataset of monthly observations across a sample of the BlueOptima Global Benchmark source code repository corpus. Each record represents a single repository during a specific month. Our dataset contains maintainability data for changes that resulted in change failures.
The antipattern scores (i.e. variables prefixed with ap_) indicate how prevalent the associated maintainability design issues are in problematic files within the source code repositories. A higher antipattern score signifies a greater presence of the corresponding maintainability issue.
The boolean flags indicate whether fixes were made by the code’s original author (which can influence fix efficiency) and whether multiple developers contributed (which might affect consistency).
The Maintainability variable is BlueOptima’s composite maintainability metric, which is higher when code is easier to maintain and lower when numerous maintainability anti-patterns are present. In each month, we also record how many files improved or worsened in maintainability to capture codebase evolution.
We implemented a logistic regression model using a 40% Change Failure Rate as the threshold for low performance, based on reporting in the 2024 DORA report. In our binary classification setup, we define class 1 as deployments with a change failure rate (CFR) of 40% or higher, signaling lower stability. Deployments with CFR below 40% are labeled as class 0.
Our model achieved an F1-score of 0.79 in predicting deployments with a Change Failure Rate (CFR) of 40% or higher, which corresponds to low performance as defined by the DORA metrics.
These findings suggest that if the maintainability patterns in the codebase remain unchanged in a repository experiencing a CFR above 40%, it is highly likely that future deployments from that repository will continue to experience a CFR above 40% – classifying them as low performing.
Each coefficient in logistic regression represents the impact of a 1-unit increase in that feature on the log-odds of the positive class — in our case, the odds of a deployment being high CFR (i.e., unstable, low-performance).
Rework refers to the percentage of changes related to bug fixes that are deployed directly to production. Deployments are defined as the number of times code is merged into the release or production branch.
When developers proactive resolve user-facing bugs in production, the need for hotfixes decreases, which in turn significantly lowers the overall change failure rate. Conversely, when developers rarely interact with codebases that exhibit maintainability design issues, the likelihood of change failure increases significantly when changes are eventually required.
An analysis was conducted to examine the relationship between change failure rate, average coding effort to resolve production issues, and production release frequency.
The data reveals clear patterns: in high-stability environments (0-10% CFR), fewer deployments were made, and fixes tended to involve larger, more substantial code changes. In contrast, the 90–100% CFR range exhibited the highest deployment frequency (39.6 releases) and the smallest average deployment size (2.8 coding hours). This reflects a shift toward resolving issues through more frequent, smaller updates in highly unstable environments.
The empirical results from the data strongly favor the maintainability metrics as leading indicators of software stability, over the DORA Change Failure Rate metric.
Maintainability reflects how easily a codebase can be understood, modified, and extended over time. A maintainable codebase features modular design and low complexity, reducing the likelihood of introducing defects. In contrast, CFR measures outcomes after changes are deployed and does not directly convey the cause of failure. Maintainability acts like a “canary in the coal mine”: a drop in score signals danger before an incident manifests.
Maintainability is more actionable because it points to areas of improvement under the team’s control right now. If scores are declining, engineers can investigate for trouble spots – like modules with high complexity – and refactor or add tests proactively.
Our findings indicate that maintainability-related design anti-patterns are strong predictors of higher CFR. In systems where such anti-patterns persist, the most effective strategies involve either ongoing rework or frequent developer interaction. When neglected, these areas degrade silently, increasing the likelihood of failure when change is eventually required.
As production stability deteriorates, deployment behavior shifts toward smaller, more frequent fixes. In the 90-100% band, the steep rise in frequency (39.6 releases) paired with low coding effort (2.8 hours) indicates reactive delivery behavior driven by urgency rather than optimization.
The inflection point in the 50–60% CFR range is noteworthy: despite elevated failure rates, deployment size rises to its peak (6.6 hours) while frequency drops to its lowest (4.0 releases). This may reflect an intentional effort to regain control through more substantive fixes before further escalation leads to high-frequency, low-effort changes.
Despite its value, CFR can be misinterpreted or gamed if teams fixate on the metric itself.
Maintainability is generally less susceptible to gaming because improving it requires actual work on the code – such as breaking up complex functions or refactoring modules – that aligns with good engineering practice. The only way to "game" maintainability is to actually improve the code.
This study reinforces the critical relationship between software maintainability, structural design quality, and change failure rates. Maintainability-related anti-patterns are strong predictors of high CFR, and their presence can significantly compromise delivery stability.
Frequent deployments combined with proactive rework in maintainability-challenged codebases are key behaviors of high-performing teams. Maintainability is a forward-looking indicator that affirms stability and speed are co-dependent outcomes rooted in healthy engineering practices. Organizations seeking to improve CFR should prioritize maintainability alongside process and automation maturity.