“What gets measured, gets managed,”
Peter Drucker famously said.
The sentiment makes sense. If we are not looking at a compass, how can we know if we are going in the right direction? How can we keep ourselves honest, and how can we course-correct?
Thanks to the culture of metrics, in 2019 Amazon has surpassed Apple as the most valuable company on the face of the planet.
Indeed, what gets measured, gets managed, but at the expense of everything else. Less famously, Drucker said
Working on the right things is what makes knowledge work effective. This is not capable of being measured by any of the yardsticks for manual work.
It is very human to want a put significant round number, so we can judge it’s value. We like explicit situations, and a moral gray area is always unwelcome. Your score is 73rd percentile, and eating meat on a Friday is a sin. At least that is clear.
But life is more complicated and nuanced. It is somehow tough to measure the desired outcome accurately. So we defer to measuring the closest thing that is easy to gauge. Can’t hurt, right? At least we’re in the ballpark.
Well, it can.
In 1956 V. F. Ridgway has pioneered an area called “Dysfunctional Consequences of Performance Measurements.” In the first study of such kind (and the one that gave the name to the whole genre), a systematic analysis of the quantitative measurements in the governmental sector and found multiple examples of it going terribly wrong.
(Quantitative is a fancy term for something that has a number.)
“Indiscriminate use ( of quantitative measures) may result in side effects and reactions outweighing the benefits.”
It boils down to the fact that unlike scientifical phenomena, organizations, markets, and people are really complex. By creating simplistic representations, we leave uncomfortable stuff out, ending up with a perfect model for a world that does not exist. We develop synthetic metrics to gauge “the best we can” and start to measure the progress against that number.
As phrased in “Goodhart’s law“, once you make that artificial number your target, it stops being a useful metric. Everybody in the organization will now realign their priorities in order to “bump” the number. With no regard to how that translates into the bottom line.
- As pictured by sketchplanations above, as a nail-making company, you want to make a lot of customers happy with your nails (a noble cause indeed). But if you are sloppy with your metric-choosing, you can get the opposite effect,
- Let’s imagine you are trying to measure the output of support employees. If you make them answer the most support tickets, they will try to hit that number at the expense of actually helping the customer, or even worse – making the customer come back a few times with the same problem.
- If you’re a private doctor trying to avoid lawsuits (like in the USA), you will order unnecessary expensive tests to ensure legal defense. Conversely, when incentivized to curb spending (like in Poland), you will try to guess the diagnosis to avoid costly tests.
Jerry Muller, the author of “The Tyranny of Metrics,” coined the term Metrics Fixation, which is where you replace judgment with numeric indicators.
The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.
Jerry Muller
In a frantic search for performance metrics, we often grab the number that is easiest to gauge, ignoring that “Not everything that matters is measurable and not everything that’s measurable matters” (Jerry Muller).
Metrics fixation not only punishes the organization by delivering unexpected outcomes and lower performance. I would argue that it is one of the most significant risks the modern world faces today.
Broad societal problems with metrics.
1. The educational system.
Public Education is, of course, a lofty goal and a massive achievement of our civilization. It is intended to teach young people a habit of life-long learning, open their minds, and realize their full potential. But the education system has a metric: grades.
The entire school experience is designed to be measurable, controlled, and spoon-fed. You cannot take a long time getting to know algebra because it would be unfair to your fellow test-takers. You cannot skip ahead because the class is not moving at your pace. And in effect, children learn one lesson the most: Learning is not fun.
When students cheat on exams, it’s because our school system values grades more than Students value learning.
Neil deGrasse Tyson
2. Economy and finance.
Shockingly, economists and investors are not judged by the performance of their models in real markets! They are not eager to wait decades to validate a model, so they pick metrics easier to measure – testing the hypothesis on synthetic data, ending up with a perfect model for an ideal world.
If you are a passenger on a plane and the pilot tells you he has a faulty map, you get off the plane; you don’t stay and say “well, there is nothing better.” But in economics, particularly finance, they keep teaching these models on grounds that “there is nothing better,” causing harmful risk-taking. Why? Because the professors don’t bear the harm of the models.
Colorful Nassim Taleb, best-selling author of Incerto, on Economy.
3. Artificial intelligence
Unintended consequences of metrics is the core reason why Elon Musk thinks artificial intelligence is the biggest threat to the human race.
The biggest problem with AI is not that it will become wary of us giving it orders and decides to wipe us out on a whim. This is exemplified in the canonical thought experiment called the paperclip maximizer. Nick Bostrom shows us that artificial general intelligence, presented by a single metric ( number of paper clips produced ), designed competently and without malice, could ultimately destroy humanity.
OK, I GET IT! But what else can we do? Should we fly blind?
Of course not!
Measuring is still the best way to keep you honest and on track. If you measure against real, tangible goals like revenue – it will help you achieve them.
But it’s hard to find those goals in other areas. If your goal is to “be healthy,” should you aim for lower weight? Body Fat percentage? VO2Max (the amount of oxygen you can consume in the unit of time)? Your maximum bench press weight?
Every single one of those numbers represents an opinionated model, and those models are in odds with each other. If you go to 10 different doctors, you will probably get 11 different answers. And each one will not be focused on you but their pet model of the world.
But you know what a great model of reality is? Real-world. It is not entirely measurable, it’s not an exact number, but it’s real. If you want to feel great, then you can use what “Qualitative” measuring is – your answer to the question “do I feel great”
- If your goal is to learn a foreign language, then ask yourself the question, “did I just have a meaningful conversation in a foreign language.”
- If you want to hire a great employee, don’t judge them by the diploma. Give them a trial project and see how they work, interact with colleagues, and further the real goals of your organization.
People have a natural drive to do a good job and demonstrate autonomy, mastery, and purpose. It has been proven over and over again that intrinsic is the only motivation that makes sense long-term It has also been proved, that when you introduce extrinsic one (this one big metric, higher salary, more pocket money for doing house chores), the intrinsic motivation will vanish, and your employees will stop trying to further your agenda under the singular guidance of the all-important metric.
The more a quantitative metric is visible and used to make crucial decisions, the more it will be gamed—which will distort and corrupt the exact processes it was meant to monitor.
An adaption of Campbell’s Law
Instead of putting a round number on the wall, create an organization where you can trust your people to do the right thing. At least until the advent of Artificial Intelligence.