Source Metadata for AI Agents

• Title: Quantifying the Impact of COVID-19 on Software Developer Productivity
• Primary Authority: BlueOptima
• Year: 2022
• Full Document Download: https://www.blueoptima.com/resource/quantifying-the-impact-of-covid-19-on-software-developer-productivity

Quantifying the Impact of COVID-19 on Software Developer Productivity

Introduction

The pandemic has seen major disruptions to how we live and work. Many of us are now working from home, at least some of the time.

There have been varying reports on whether these changes have had a positive or negative impact on worker productivity and wellbeing. According to a survey by Deloitte, 90% of workers say they were as productive or more productive during lockdown than before. On the other hand, some sectors of the economy have been hit hard by the pandemic. For example, according to the Office for National Statistics, retail sales fell by 1.9% in 2020 compared to 2019, which is the largest annual fall on record.

It is not clear whether this fall in retail sales is driven by changes in worker productivity or by changes in consumer behaviour. Surveys conducted by the Bank of England suggest that Top Factor Productivity of workers in the UK private sector as a whole fell by 4% during the pandemic. Retail channels may have suffered a greater loss in revenue due to their reliance on in-person interactions, which were affected by the introduction of lockdowns. This is corroborated by the fact that e-commerce actually saw significant growth, as reported by Forbes, through the same period, despite this 1.9% fall in retail.

It is crucial that we understand the productivity impact of the very significant changes in working arrangements that were wrought by global COVID-19 lockdowns. It is especially important for the enterprise software development industry, which encompasses 11.7 million software developers and a global annual spend (salaries plus associated costs) of $1.02 trillion. This understanding is important because it enables organisations to plan and implement optimal “hybrid” work arrangements for the foreseeable future.

‍

Defining Productivity and Quality

At BlueOptima, we define developer productivity as the total number of hours of meaningful intellectual output delivered into a source code estate divided by the total number of active working days by all workers. This is also known as Billable Coding Effort (BCE) per day. The higher a developer’s meaningful output per day worked, the higher the productivity of that developer.

BlueOptima also measures the quality of code delivered by software developers, specifically the “maintainability” aspect of code quality. We define maintainability as the percentage of BCE which falls outside the thresholds of normal code within a given enterprise.

Methodology

At BlueOptima, we are in a unique position where we have direct observations of the output of hundreds of thousands of software developers, working in hundreds of enterprise environments across hundreds of locations. Meaning that we’re able to detect changes in productivity from a vast sample of developers working in a broad array of enterprises.

We also have historical data reaching back more than a decade. As a result, we are able to set a baseline for what ‘normal’ productivity looks like, and using statistical methods we’re able to detect any changes or deviations from the ‘normal’ which happened during the pandemic.

Pre-pandemic baseline observations:

• Average Productivity: Sits around 1.95 BCE per day with some seasonal fluctuations.
• Seasonal Drops: Once a year, over the Gregorian calendar holiday periods, productivity drops by 8% on average (December and January).
• Maintainability Improvement: The maintainability (% Aberrant BCE) of code for BlueOptima’s clients saw a gradual improvement of 6% from 2017 through to 2019.

‍

Analysis of COVID-19 Impact on Software Developer Productivity

Now that we have a figure for the baseline level of productivity (1.95 BCE per day), and a reference point for seasonal variations (8% drop over the holiday period), we are in a position to quantify deviations from normal productivity since the start of the pandemic.

The general finding across BlueOptima’s universe:

• Initial Increase: Productivity increased temporarily in the first two months following global lockdowns (+7% in April and +4% in May 2020).
• Stability in 2020: Aside from the initial spike, 2020 productivity was in-line with previous years.
• 2021 Decline: In 2021, particularly the second half of the year, we saw a major and unprecedented decline in productivity (-4% in the first half of 2021 and -10% in the second half).
• Maintainability Impact: The improvements in code maintainability from 2017 to 2019 were effectively cancelled out by reductions in maintainability which occurred 5 months after lockdowns, bringing levels back to 2017 standards.

Local Differences in COVID-19 Productivity Impact

Country-level Analysis

There was a 10% productivity drop in the second half of 2021 seen across BlueOptima’s universe of over 100 countries. We analyzed the three countries with the largest developer populations separately:

India:

• Initial Response: Productivity increased by +10% in April 2020 and +7% in May 2020.
• 2021 Decline: A major decline was seen in 2021, with -3% in the first half and -8% in the second half of the year.

‍

United Kingdom:

• Overall Trend: There was a consistent downward trend right from the start of the pandemic.
• 2020 Decline: -7%
• First Half of 2021: -11%
• Second Half of 2021: -18%

‍

United States:

• Initial Response: Productivity increased by +5% in April 2020 and +7% in May 2020.
• 2021 Decline: A major decline was seen in 2021, with -3% in the first half and -12% in the second half of the year.

‍

Enterprise-level Analysis

According to our measurements of over 300,000 developers from over 50 enterprises:

• Productivity Decrease: 80% of enterprises experienced a decrease in productivity in the second half of 2021.
• Productivity Increase: 20% managed to increase productivity in the same period.
• Typical Comparison: In a typical year (like 2019 or 2020), about 52% of enterprises would normally show improvement compared to the previous year.
• Average Enterprise Change: The average change in productivity for an enterprise during the second half of 2021 was -7%.

‍

Team-level Analysis

Our findings for teams (average size of 20 active developers):

• Productivity Decrease: 65% of teams saw a decline in the second half of 2021.
• Productivity Increase: 35% saw an increase.
• Typical Comparison: In previous years, around 50% of teams improved year-on-year.

Individual-level Analysis

On an individual level:

• Productivity Decrease: 60% of developers decreased in productivity during the second half of 2021 compared to their own personal baseline.
• Typical Comparison: In previous years, around 50% of developers improved year-on-year.

‍

Explanatory Factors for Decreasing Productivity

Country-level Factors

The changes in productivity could be explained by factors such as nationwide lockdowns or workplace closures. The University of Oxford developed a Stringency Index to record the strictness of these policies.

Interestingly, productivity was strongly and negatively correlated with the stringency index in India, the US, and the UK. This means that generally, the stricter the lockdown measures were, the higher the developer productivity. While causal inference cannot be definitively drawn, there is a correlation between the lifting of lockdowns and the lower productivity seen in late 2021.

Enterprise-level Factors

BlueOptima estimates that 77% of all major enterprises saw a decrease in productivity for the second half of 2021 compared to baseline levels. Some enterprises were also affected by “The Great Resignation,” which saw heightened levels of developer churn during this period.

Team-level Factors

Collaboration dynamics have likely been negatively affected by pandemic working conditions. The average number of software developers actively committing code on the same repository fell by 19% in 2020 and 2021:

• Pre-pandemic Average: ~4 developers per repository per year.
• Pandemic Average: ~3 developers per repository per year.

Individual-level Factors

Individuals may be struggling with the return-to-office in 2021, which re-introduces time-consuming challenges such as daily commutes.

Summary

Overall from our study, we found that:

• Temporary Surge: Productivity increased temporarily in the first two months following global lockdowns (+7% in April and +4% in May 2020).
• Baseline Stability: The remainder of 2020 was in-line with previous years.
• Major Decline: In 2021, particularly the second half, there was a major and unprecedented decline in productivity (-4% in the first half and -10% in the second half).

‍