Let's talk about Goodhart's law.
🤔 Why you should care about it
"Tell me how you measure me, and I will tell you how I will behave." - Eliyahu Goldratt, Israeli business management guru and originator of the Theory of Constraints (TOC).
Named after British economist Charles Goodhart, Goodhart's law warns managers that in their quest to measure the performance of their team or team members, they can trigger new behaviours and actions that can ultimately harm the desired outcome.
😫 Problem(s)
Increasing production volume usually does not increase user value —> when engineers have targets on coding output (lines of code, story points, number of commits), they tend to produce less efficient code (more lines) or focus on shipping a lot of small tasks, instead of tackling harder, more value-generating, tasks.
Engineers tend to overestimate development time to avoid being late —> this is a classic in organisations that incentive engineers to meet development deadlines, creating horse-trading negotiations with business teams.
😃 Solution
To avoid creating situations where team members game the system while still measuring performance, CTOs can:
1/ Set targets on metrics that serve the engineering team's goal (delivering qualitative features fast that solve users' problems). One such metric can be the Time to Positive User Feedback (see below). In this case, it's the coordinated effort of multiple teams that can achieve the desired outcome, making it more challenging to game at the individual level
2/ Set targets on processes, rather than outcomes, when it comes to individuals. The disciplined repetition of good practices (like doing technical design before coding, doing code reviews, and tackling technical debt) creates good outcomes, even if not all outcomes are good.
💡 Key Concepts
Campbell's law —> the more critical a metric is in social decision-making, the more likely it is to be manipulated. For example, a customer support team disconnects their phone line to decrease the number of customer complaints.
Perverse incentives (aka cobra effect) —> incentives that have unintended and undesirable results that are contrary to the intentions of their designers. For example, people breed cobras, so they would kill them to get a dead cobra bounty.
Gaming the system —> manipulating a system to reach the expected outcome, potentially harming the initial intention in the process.
😡 Detractors
"If you can't measure it, you can't manage it." —> Goodhart's law is not an encouragement to do things without measuring them but is a cautionary tale when using those measures as targets for teams or individuals.
"Targets put so much pressure on team members that we prefer not to use any." —> Research reveals that setting challenging and specific goals can further enhance employee engagement in attaining those goals.
📚 Top books
Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations - Nicole Forsgren PhD, Jez Humble, Gene Kim.
⚙️ Tools
Echoes HQ - Drive engineering success
📝 Content
📝 Can developer productivity be measured? - Isaac Lyman --> a good summary on measuring an engineering team's productivity from the Stack Overflow blog.
📝 CannotMeasureProductivity - Martin Fowler --> why measuring productivity is so seductive, and how false measures only worsen things.
📝 What Should We Measure? - Tim Ottinger --> a few metrics that CTOs can use to gauge the effectiveness of a team at pleasing the project community, improving their processes, and managing their source code.