Stop Measuring Developer Productivity…Do This Instead
Measuring productivity starts with the foundation of trust and flow
We know the importance of foundations when we see it. The Leaning Tower of Pisa is the most visible and famous of examples. The foundation of the tower was laid in 1173 and construction stopped in 1178 when the tower began to sink. It took two more stages and some engineering hacks to erect the rest of the tower and complete it by 1372.
You may have thought we would have learned our lessons about building on soft earth in the modern era. The Millennium Tower, a high-rise luxury residential building in San Francisco next to the Salesforce Tower, became subject of a lawsuit and a $100 million retrofit to fix a 14 inch lean caused by settling due to a weak foundation.
Building on the right foundation is everything. Foundations sit below the ground however, so we never see the value of a strong foundation. We only see the consequences when things go wrong.
The important thing is that you’ve got a strong foundation before you start to try to save the world or help other people.
- Richard Branson
I see IT organizations struggle to define this idea of developer productivity. Mangers use all sorts of activity based methods to determine how their engineering teams and employees are performing. By and large though, every IT leader I speak with agrees that this approach does not allow for the level of calibration needed to improve outcomes.
What do I mean by calibration? This is the mechanism where making and measuring a change in a process correlates directly to a meaningful change in an outcome. For example, if your metric is lines of code, what is the outcome? Other than more lines of code, there is no obvious change that results in a useful outcome such as faster cycle times or fewer defects.
The wrong foundation often starts with the incorrect mental model. In this case, building on an ideal of work that does not reflect reality. In the early years of the industrial revolution, the way you measured worker productivity was number of hours worked. The work itself was simple and repetitive, so time and output was a useful proxy for outcomes. Fast forward a century later, that way of thinking about work simply does not suffice.
Is designing and writing software a simple and repetitive task or does it require higher level thinking and experimentation? While there are some repetitive aspects, most of those are quickly getting automated away. Therefore, if programming is more akin to creative work, then does a model that measures work based on activity make sense?
Speak with any CTO or head of engineering and they will say they want better software released faster. This is the crux of digital transformation. The ability to serve the customer in ways that they desire ultimately comes down to faster cycles of releasing, learning, and iterating on software. However, if your foundation for improving software delivery is based on measuring developer productivity, you are building on the wrong foundation.
What is the right foundation for understanding software delivery performance and the elements that go into driving performance? There are three overarching principles:
- Strong Alignment to Vision
- The Four Metrics of Software Delivery Performance
- Investment in Developer Enablement
It is not uncommon for developers to feel their work has no connection to anything meaningful. Without understand why they are doing something, engineers write code based on their own limited understanding as Peter Varhol shares:
“If developers aren’t given the big picture, then the hundreds of decisions they make have no context.”
Alignment to vision helps connect the task at hand to the big picture. Strong alignment means this connection is understood by all members of the team and throughout the organization tied to incentives that reinforce behaviors that align to the vision.
The next principle connects specific tasks towards fulfillment of the vision. In the context of software development, we can use the Four Metrics of Software Delivery Performance: Lead Time, Deployment Frequency, Mean Time To Restore (MTTR), and Change Fail Percentage.
While useful, a common criticism of the Four Metrics is that they are still too high level and only provide a group level view of performance. The Three Level of Metrics framework is one way to bridge macro-level with detailed, task-level metrics. This enables the right level of calibration that surfaces strategic levers and enables impactful tactical decision making. Developer enablement is one of these levers to help tie strategy to tactics.
What is developer enablement? It is all the things that go into helping developers to do their job better. This is sometimes referred to as developer flow, where concentration is maximized and work seems effortless. Therefore, anything that creates friction in this flow are opportunities to improve through the process of enablement.
From the book Accelerate, survey data from the State of DevOps Report showed strong correlation between software delivery and three areas of team support and culture:
- Cross-Functional Collaboration — sharing of knowledge, open communications
- Climate for Learning — budgeting for formal training and time for informal learning
- Technology — agency for teams to chose own tools, enhanced observability
These areas are fully in the realm of developer enablement. There are other important components of enablement as well, including automation, loosely coupled architectures, lean product and management principles. What I have highlighted are cultural elements of enablement because these point to what is the biggest enabler for teams: trust.
Richard Branson* is correct about the value of a strong foundation. It is impossible for leadership to help developers do their best work if leadership is not focused on the right goals. Trusting developers is that foundation and is the starting point for efforts initiated towards raising developer productivity.
You might wonder how to tie “cultural” elements into metrics that show productivity impact. By categorizing measurable aspects of collaboration, learning, and technology, we can define “L1” and “L2” metrics that roll into our “L3” level metrics. Here is how that might look like:
These are simply high level examples. More work would be needed to refine these metrics so that the right correlation is established. The point here is that there is a way to use enablement metrics to build a story that supports developer productivity in an objective way. The way you get there is to bridge lower level functional metrics to higher level strategic metrics.
Does your organization have a mandate to improve developer productivity? How have you gone about measuring productivity and what have been the results?
* It was noted in my challenge network that Richard Branson and his Virgin business empire are on rather shaky ground right now. Despite that, his words still ring true. The right foundation of culture and people and business sense led him to build a massive global business employing over 70K people.
HOW I BUILT THIS with Luis von Ahn of reCAPTCHA and Duolingo
reCAPTCHA and Duolingo: Luis von Ahn
In 2000, Luis von Ahn was starting his PhD in computer science when he attended a talk and happened to learn about one…
One of my favorite podcasts is HOW I BUILT THIS from NPR, with their inspiring stories of founders. This episode is an awesome story of invention and the power of simple solutions (the Captcha) to solve the massive problem of spam and fraud. One of the better tech oriented founder stories on the podcast.
If you want to be a guest on the podcast, have unique thoughts on the state of IT and Engineering, and have a favorite superhero, we would love to have you on the show!
We help IT leaders in enterprises solve the cultural challenges involved in digital transformation and move towards a community based culture that delivers innovation and customer value faster. Learn more about our work here.