Measure software developer productivity

Bug rates — average number of bugs that are generated as new features are being deployed. It can help you estimate whether you are delivering value or just deploying some half-baked code for the sake of frequent deployments. Additionally, you may want to gather some common application health statistics to gain more insight into how healthy it is and align your actions with that data.

Agile methodology welcomes and accounts for changes at any stage of the project. Your goal here is to make sure that your team can work at the consistent pace when presented with both static and dynamic requirements. The following metrics should help you gain some insights into that. It is a good indicator of incomplete or inconsistent requirements that you may want to investigate. Lead time, velocity, and development time — all measure how long it takes for your team to get tasks completed, considering the requirement change.

But at the end of the day, you should not get sidetracked from your main goal — deliver value and remain fair to your engineering team and clients. Every project will have unique complexities and difficulties. While you should establish some baseline software productivity metrics that you will track consistently, do leave them open for discussion with your team and key stakeholders.

All rights reserved. Sitemap Privacy Policy Cookies Imprint. By using this web site you accept our use of cookies. More information about cookies.

Mobile panel. Contact us. Privacy Terms. Your privacy is important to us. We will never share your data. Thank you! In the meantime, you might be interested in learning more about the following:. Send an email to Infopulse Please fill in this quick form to contact our expert directly. Download ebook Please fill in this quick form. Software Engineering Written by Infopulse. Originally published October 19, Updated April 18, More on this topic.

Manual vs. Share this blog article. Featured blog posts. Latest news. Intuitively, the benefits of measuring productivity should outweigh the cost of the measurement process but I think it is a good idea to be aware of the added cost. It may be hard to believe that hours worked would be a measure of software development productivity, but it is frequently used.

If you compare the productivity of two software development teams using this measure, and they work the same number of hours, you will conclude that the less expensive team is more productive i. This is often used as justification for moving software development offshore where labor rates are cheaper or the driver for a policy to hire the cheapest developers in a local market.

This is also used in some organizations as justification for encourage software developers to work more hours per week. Managers who use this productivity metric are focused on increasing the numerator hours worked and decreasing the denominator cost.

The problem with this metric is that it assumes that every software developer produces the same amount of quality code per hour. This far from the truth. Studies have found that there can be an order of magnitude 10x difference in productivity between programs and between teams. Bill Gates said that a great writer of code is worth 10, times the price of an average writer.

Robert C. Martin said that ninety percent of code is written by ten percent of the programmers. There is also a myth that the more time a developer spends in her seat, the more productive she will be the greater the hours-worked numerator will be and the more quality code she will produce. As Tom DeMarco and Timothy Lister pointed out in their book PeopleWare , overtime will often lead employees to take compensatory undertime whenever possible, either through suboptimal work, leaving early, getting sick and so on.

It gives you the illusion of higher productivity. Another measure of software development productivity that is used is counting the amount of code that has been written and dividing that by the cost to write it. There are different ways to count lines of code and a variety of code counting tools available. At the team level, measurement's greatest contribution lies in allowing relentless experimentation.

Step one is to establish the baseline Line Impact during a typical week. Step two is to try an experiment for a couple weeks and measure the change in Line Impact.

One of the first experiments we ran upon gaining access to developer measurement was to analyze the impact of working from home. Here were the results of our analysis , and here's the corresponding graph that exhibits our per-hour output over the past year:. At face value, it appears that working from home on Wednesday has zero impact on the team's output. But the real story is more nuanced. It's typical for our team to schedule all chores, errands, dental visits, etc.

How do they make up the lost time? Check out Wednesday evening -- the only day of the week with significant activity occurring after 5pm. Our developers are rewarding our trust by making up time after hours.

It would also let us spot if we hired a developer bent on abusing that trust since any stat can be scoped to the individual level. While working from home yields a neutral result on productivity, it has positive implications on morale, so this experiment is now ensconced as our company policy. Being able to run interesting experiments like this is a cornerstone of the well-measured company.

If you're considering paying for developer measurement, we'd highly recommend visiting this article to get acquainted with how each provider sources their data.

Now that we've seen how teams can benefit from making decisions using data, let's pivot to explore the different flavors of measurement offered by the best developer productivity tools in If this article were written five years ago, your options would lie somewhere between "shabby" and "pitiful.

In these past few years, the performance measurement space has blossomed to host three "high polish" options worthy of your consideration. Each brings a distinct set of strengths and its own measurement philosophy. Let's review them in chronological order of their launch date. To this end, GitClear built robust code review tools , including a graphical Directory Browser that pinpoints tech debt.

In the time since , GitClear has shifted to differentiating based on data source quality. Since competing products rely so heavily on commits, pull requests, and issues , GitClear spent doubling down to prove that data quality matters. To the extent GitClear succeeds in creating a single reliable metric to capture code activity, managers can unambiguously see how active each developer has been -- whether they're comparing between teams' location remote or on-site , domain front-end or back-end , seniority senior, mid-level, code school graduate , country outsourcing vs not , etc.

Plenty of companies claim to allow such broad comparisons. As of , a new one falls out of a VC incubator every few months.

But among competitors that back up their claims with detailed analysis of how signal is maximized and noise is minimized: GitClear stands alone in the fervor of this pursuit.

Here is a graph from another article that outlines how much more exhaustively GitClear has worked to refine Lines of Code into a consistent, reliable data source. The bonus for having a data source as consistent as Line Impact? You don't have to hide it from developers. They benefit from being able to review their teammates work with a set of tools that add clarity where even GitHub and GitLab fall short for browsing commits and directories.

The first company to plant their flag in the performance measurement space was GitPrime, debuting in In , GitPrime was acquired by Pluralsight, which subsequently rebranded the product as "Pluralsight Flow" home page , pricing in January The Pluralsight business model relies on cross-selling customers into their tutorial and instruction business "Pluralsight Skills" when customers seek out git analytics.

Work on Pluralsight Flow is interpreted via four key metrics: "active days," "commits per day," "impact," and "efficiency. In terms of data sources, Pluralsight sources fairly evenly between commits , issues , and lines of code. We have written about the perils of these data sources elsewhere. Like GitClear, Pluralsight offers an on-prem version for enterprise-sized customers, although prices are higher for on-prem installs.

A recent entrant in the performance metrics space is Code Climate, which launched Velocity homepage , pricing in As of early , the v2 launch of Velocity is now the focus of the Code Climate homepage, supplanting their long-popular code quality tools.

This placement suggests that Code Climate expects Velocity to be the primary focus of their company moving forward. Velocity recently published a robust comparison between their features and Pluralsight's. If you're considering Velocity, I recommend checking out their article. It illustrates how Velocity shares with Pluralsight a common thread of ideology, features, and design elements. For example, both offer a "commit activity log" that use shapes of various color and size to represent commit activity:.

In contrasting with Pluralsight Flow, the article points toward "Surfacing issues" as the area where the products diverge most. In their words, This is the category in which the two analytics tools differ most. Velocity, with PR-related metrics at the core of the product, does a better job drawing attention inside and outside of the app to actual artifacts of work that could be stuck or problematic. GitPrime, with mostly people-focused metrics, draws attention to contributors who could be stuck or problematic.

A scrappy product seeking to make waves in git analytics is waydev. Under the heading of "Why we built Waydev," the company states, "as a technology company working with developers, it is almost impossible to understand what your team is doing, and your intuition doesn't help.

So if your organization wants to measure individual performance just to be able to put some more rules in place based on this, then we are likely talking about a bureaucratic organization.

This type of organization will fare worse than a performance-oriented organization that is focused on overall outcomes. As a hands-on engineering manager, there a lot of things that you can measure to observe on the team. These go from the number of commits, average commit size, time to review, number and frequency of code reviews, time to merge pull requests to the main branch, and so on.

None of these metrics are helpful by themselves, but together, they can paint a picture, and draw attention to places that could be problematic. And all of them only make sense in the context of the team. For example, is one of the developers taking 3x as long to merge their changes, and do they get 3x as many code review comments as others? If they are a new joiner, this is expected. But if they are a veteran on the team, maybe there's something other going on there? Similarly, looking at these numbers can help identify top performers who are going over and beyond with code reviews, on top of their work.

You can spot brewing conflicts by reading through pull requests with lots of back-and-forths. There's a wealth of data to look at: but none of it is useful without context. Tools like Velocity or Pluralisght Flow and others expose much of this data: but they will never be able to deliver the context.

For example, you might see that the number of commits and code reviews has dramatically dropped from an engineer. But when digging in, it might turn out they were pairing with team members, speeding them up, mentoring them, and adding long-term value outside the sprint.

You won't be able to tell any of this until you look behind the curtain. There are a lot more things we can measure, though, at the team level. Like the output of the team. What has the team shipped? What was the business impact? How was the quality: how much rework was needed afterward? And, of course, let's not forget about the measurable things about the team. How is morale? What about attrition? How are people growing professionally: are they getting promoted? We've explored what you can and cannot measure.

So how to the likes of Google, Facebook, Stripe, Uber, Airbnb and other large or small, but innovative tech companies and startups measure software engineering performance?

All tech companies with good developer culture give performance feedback based on the competencies and levels they define for engineers. These companies have a career progression, or competencies framework, that clearly define expectations for each level. Some of these progression frameworks can be browsed on Progression. Engineering managers give feedback on an ongoing basis, and have more formal reviews every months. On these performance review sessions , they'll give clear feedback if the engineer is below, meeting, or above expectations.

So how does this tie to measuring performance?