You must not measure individual software engineer performance
It is very difficult to explain the nature of software engineering work to stakeholders who don’t work in engineering. We compensate salespeople, for example, partly with commissions that reflect their individual performance. That’s because someone in sales is able to individually improve some small metric for the company. And anyone working in operations or customer support can be judged at least in part by the number of issues they’ve handled and how satisfied the company is with the outcomes of those issues.
But effective engineering is a collaborative and creative discipline, where the team is the performative unit rather than the individual, and the performance cannot be measured by the team’s actions but rather by the outcomes of the system that the team produces.
Like any heist film, a software engineering team comprises multiple individuals with relative strengths and weaknesses that complement each other. One engineer might love to make new features while another might be entirely consumed by improving the quality of the team’s existing software. Either one working alone, given enough time, would eventually become entirely useless to the company. Together, as a team, they can produce value while lowering cost indefinitely.
Attempting to reward or discipline either of these two individuals would throw the team itself off balance. Incentivize new feature addition and the quarter-over-quarter feature development speed will consistently slow. Incentivize improvements and no feature will ever again launch. But what if we reward them, individually, for their specific and different contributions? Yes, we can do that and all we’ll need is to create a metric for the engineer who likes to build features and another metric for the one who fixes existing features. Then measure each engineer by the appropriate metric.
But what are the units of that measurement? How do you compare 6 clumsy features launched in a month against 1 extremely hard to find bug that took all month to discover and required only a one-line change? What if an engineer did neither of these activities but mentored their colleagues?
It’s impossible to say one person performs better than another without a shared measurement. And, while the outputs of these two hypothetical (yet all too real) engineers cannot be compared, their daily activities can be. Number of changes to a code base. Number of lines added or removed. Time spent in the office. Number of messages sent to colleagues. Number of bugs noticed by a QA team.
I know of companies that have, at one point, used all of the above to measure the ‘performance’ of their software developers. And, in each case, the company got a lot of useless (or harmful) activity and enormous employee attrition.
So, if we can’t compare engineers against each other without incentivizing the wrong behavior, we must accept that their success or failure is inherently collaborative. Like a surgical operating team in a hospital whose success is tied to the patient’s. Or a unit of firefighters who enter a hot building together.
One would naturally, then, try to measure the performance of the team instead of the individual.
But software is a unique art: It’s the encoding of human decision-making into machines. The performance that matters isn’t that of the team that builds the software but of the computers themselves.
Imagine if the surgical operating team in a hospital were a robotic AI. Or the firefighters entering a 4-alarm blaze were fire-retardant autonomous robots. We would reward the engineers who created these tools for the outcomes of the surgeries and the lives saved from the fires. No one would ever care if the engineers worked late or took extra vacation — what matters is the outcomes.
All software engineering teams produce these same outcomes, and always indirectly. The way the engineers manage their software is much like how a sales executive manages their team. Throughput, latency, and quality can all be measured and charted over time. A sales leader can identify problem areas and intervene when an individual account executive underperforms. Engineers do the same thing but call it ‘debugging’.
This inherent indirection — that the engineers produce the system that produces value — is what makes software engineering a creative discipline.
Like a gardener thoughtfully removing a blight, any engineer at any time can increase the value of the company in unexpected ways. Or, like a gardener rewarded only for the sheer number of weeds removed, we can task them with goals that produce huge activity while the company gets outpaced by competitors with more creative teams.
The solution is to define what, precisely you want from your engineering teams. It’s never sheer activity. And it’s likely not blindly implementing whatever they’re asked to build because that’s how consultancies operate. And a consultancy only makes money because it doesn’t have to deal with the inevitable failure of most of the things they build for poorly-informed clients.
What you want from your engineering team is to produce the maximum number of market-impactful launches. It’s on your product leadership to effectively measure the market and maximize what that impact is. But it’s up to your engineering leadership to maximize the sheer number of launches that your product function believes could be useful.
Define those top-level metrics, install competent leaders with an unyielding devotion to coaching and developing their people, and watch as your engineers work together to make something impossibly beautiful.
Whatever you do, don’t create metric to measure the performance of your individual engineers.
