“Well, we have to measure something.”, And the perils of metrics.

“What gets measured, gets managed,”

Peter Drucker famously said.

The sentiment makes sense. If we are not looking at a compass, how can we know if we are going in the right direction? How can we keep ourselves honest, and how can we course-correct?

Thanks to the culture of metrics, in 2019 Amazon has surpassed Apple as the most valuable company on the face of the planet.
Indeed, what gets measured, gets managed, but at the expense of everything else. Less famously, Drucker said

Working on the right things is what makes knowledge work effective. This is not capable of being measured by any of the yardsticks for manual work.

It is very human to want a put significant round number, so we can judge it’s value. We like explicit situations, and a moral gray area is always unwelcome. Your score is 73rd percentile, and eating meat on a Friday is a sin. At least that is clear.

But life is more complicated and nuanced. It is somehow tough to measure the desired outcome accurately. So we defer to measuring the closest thing that is easy to gauge. Can’t hurt, right? At least we’re in the ballpark.

Well, it can.

In 1956 V. F. Ridgway has pioneered an area called “Dysfunctional Consequences of Performance Measurements.” In the first study of such kind (and the one that gave the name to the whole genre), a systematic analysis of the quantitative measurements in the governmental sector and found multiple examples of it going terribly wrong.

(Quantitative is a fancy term for something that has a number.)

“Indiscriminate use ( of quantitative measures) may result in side effects and reactions outweighing the benefits.”

It boils down to the fact that unlike scientifical phenomena, organizations, markets, and people are really complex. By creating simplistic representations, we leave uncomfortable stuff out, ending up with a perfect model for a world that does not exist. We develop synthetic metrics to gauge “the best we can” and start to measure the progress against that number.

As phrased in “Goodhart’s law“, once you make that artificial number your target, it stops being a useful metric. Everybody in the organization will now realign their priorities in order to “bump” the number. With no regard to how that translates into the bottom line.

  • As pictured by sketchplanations above, as a nail-making company, you want to make a lot of customers happy with your nails (a noble cause indeed). But if you are sloppy with your metric-choosing, you can get the opposite effect,
  • Let’s imagine you are trying to measure the output of support employees. If you make them answer the most support tickets, they will try to hit that number at the expense of actually helping the customer, or even worse – making the customer come back a few times with the same problem.
  • If you’re a private doctor trying to avoid lawsuits (like in the USA), you will order unnecessary expensive tests to ensure legal defense. Conversely, when incentivized to curb spending (like in Poland), you will try to guess the diagnosis to avoid costly tests.

Jerry Muller, the author of “The Tyranny of Metrics,” coined the term Metrics Fixation, which is where you replace judgment with numeric indicators.

The most characteristic feature of metric fixation is the aspiration to replace judgment based on experience with standardized measurement.

Jerry Muller

In a frantic search for performance metrics, we often grab the number that is easiest to gauge, ignoring that “Not everything that matters is measurable and not everything that’s measurable matters” (Jerry Muller).

Metrics fixation not only punishes the organization by delivering unexpected outcomes and lower performance. I would argue that it is one of the most significant risks the modern world faces today.

Broad societal problems with metrics.

1. The educational system.

Photo by Feliphe Schiarolli on Unsplash

Public Education is, of course, a lofty goal and a massive achievement of our civilization. It is intended to teach young people a habit of life-long learning, open their minds, and realize their full potential. But the education system has a metric: grades.

The entire school experience is designed to be measurable, controlled, and spoon-fed. You cannot take a long time getting to know algebra because it would be unfair to your fellow test-takers. You cannot skip ahead because the class is not moving at your pace. And in effect, children learn one lesson the most: Learning is not fun.

When students cheat on exams, it’s because our school system values grades more than Students value learning.

Neil deGrasse Tyson

2. Economy and finance.

Photo by M. B. M. on Unsplash

Shockingly, economists and investors are not judged by the performance of their models in real markets! They are not eager to wait decades to validate a model, so they pick metrics easier to measure – testing the hypothesis on synthetic data, ending up with a perfect model for an ideal world.

If you are a passenger on a plane and the pilot tells you he has a faulty map, you get off the plane; you don’t stay and say “well, there is nothing better.” But in economics, particularly finance, they keep teaching these models on grounds that “there is nothing better,” causing harmful risk-taking. Why? Because the professors don’t bear the harm of the models.

Colorful Nassim Taleb, best-selling author of Incerto, on Economy.

3. Artificial intelligence

Photo by Arseny Togulev on Unsplash

Unintended consequences of metrics is the core reason why Elon Musk thinks artificial intelligence is the biggest threat to the human race.

The biggest problem with AI is not that it will become wary of us giving it orders and decides to wipe us out on a whim. This is exemplified in the canonical thought experiment called the paperclip maximizer. Nick Bostrom shows us that artificial general intelligence, presented by a single metric ( number of paper clips produced ), designed competently and without malice, could ultimately destroy humanity.

OK, I GET IT! But what else can we do? Should we fly blind?

Photo by Joao Tzanno on Unsplash

Of course not!

Measuring is still the best way to keep you honest and on track. If you measure against real, tangible goals like revenue – it will help you achieve them.

But it’s hard to find those goals in other areas. If your goal is to “be healthy,” should you aim for lower weight? Body Fat percentage? VO2Max (the amount of oxygen you can consume in the unit of time)? Your maximum bench press weight?

Every single one of those numbers represents an opinionated model, and those models are in odds with each other. If you go to 10 different doctors, you will probably get 11 different answers. And each one will not be focused on you but their pet model of the world.

But you know what a great model of reality is? Real-world. It is not entirely measurable, it’s not an exact number, but it’s real. If you want to feel great, then you can use what “Qualitative” measuring is – your answer to the question “do I feel great”

  • If your goal is to learn a foreign language, then ask yourself the question, “did I just have a meaningful conversation in a foreign language.”
  • If you want to hire a great employee, don’t judge them by the diploma. Give them a trial project and see how they work, interact with colleagues, and further the real goals of your organization.

People have a natural drive to do a good job and demonstrate autonomy, mastery, and purpose. It has been proven over and over again that intrinsic is the only motivation that makes sense long-term It has also been proved, that when you introduce extrinsic one (this one big metric, higher salary, more pocket money for doing house chores), the intrinsic motivation will vanish, and your employees will stop trying to further your agenda under the singular guidance of the all-important metric.

The more a quantitative metric is visible and used to make crucial decisions, the more it will be gamed—which will distort and corrupt the exact processes it was meant to monitor.

An adaption of Campbell’s Law

Instead of putting a round number on the wall, create an organization where you can trust your people to do the right thing. At least until the advent of Artificial Intelligence.

Sign up to my “Deliberate thoughts” list for more content like this 👇

Contacting the pigeons….
Success 🚀 ! You're in! Check your inbox to confirm your email address.

Book: Hard Thing about Hard Things

“Do you know the best thing about startups?” Ben: “What?” Marc: “You only ever experience two emotions: euphoria and terror. And I find that lack of sleep enhances them both.”

the-hard-thing

Some people call this book the most important book about startups.

Shane Parish from Farnam Street Blog has been singing praises about this book as well. If you are interested in comprehensive opinion/summary, you should definitely check his out.

Amazon link

 

My key takeaways

  • Struggle is core to being a CEO
  • The biggest problem is the one that blindsides you – when everybody is reporting great success, but problem is brewing
  • Be honest – sharing bad news is good
  • Fire people fast, let them know you are firing them and let the keep respect
  • Company is a completely different place at different size levels
  • Most incompetent person holding a title defines the value of the title
  • Peacetime CEO is different than wartime CEO

 

My highlights

  • Following conventional wisdom and relying on shortcuts can be worse than knowing nothing at
  • Former secretary of state Colin Powell says that leadership is the ability to get someone to follow you even if only out of curiosity.
  • The simple existence of an alternate, plausible scenario is often all that’s needed to keep hope alive among a worried workforce.
  • My father turned to me and said, “Son, do you know what’s cheap?” Since I had absolutely no idea what he was talking about, I replied, “No, what?” “Flowers. Flowers are really cheap. But do you know what’s expensive?” he asked. Again, I replied, “No, what?” He said, “Divorce.”
  • Marc: “Do you know the best thing about startups?” Ben: “What?” Marc: “You only ever experience two emotions: euphoria and terror. And I find that lack of sleep enhances them both.”
  • You need someone who will actually be more excited for you than he would be if it had happened to him. The second kind of friend is somebody you can call when things go horribly wrong—when your life is on the line and you only have one phone call. Who is it going to be?
  • If you are going to eat shit, don’t nibble.”
  • what I’d like you to do. First, reach up to your face and take off your rose-colored glasses. Then get a Q-tip and clean the wax out of your ears.
  • “What Are We Not Doing?”
  • If a warrior keeps death in mind at all times and lives as though each day might be his last, he will conduct himself properly in all his actions.
  • The Struggle is the land of broken promises and crushed dreams. The Struggle is a cold sweat. The Struggle is where your guts boil so much that you feel like you are going to spit blood.
  • My single biggest personal improvement as CEO occurred on the day when I stopped being too positive.
  • A much better idea would have been to give the problem to the people who could not only fix it, but who would also be personally excited and motivated to do so.
  • In any human interaction, the required amount of communication is inversely proportional to the level of trust.
  • individuals should not be “This is great, we are cleaning up performance.” The message must be “The company failed and in order to move forward, we will have to lose some excellent people.”
  • “Ben, you cannot let him keep his job, but you absolutely can let him keep his respect.”
  • 1. People who quit 2. People who got fired 3. People who quit, but it’s okay because the company didn’t want them anyway
  • Fascinatingly, as companies begin to struggle, the third category always seems to grow much faster than the
  • The customers were buying; they just weren’t buying our product. This was not a time to pivot. So I said the same thing to every one of them: “There are no silver bullets for this, only lead bullets.”
  • The customers were buying; they just weren’t buying our product. This was not a time to pivot. So I said the same thing to every one of them: “There are no silver bullets for this, only lead bullets.” They did not want to hear that, but it made things clear: We had to build a better product. There
  • “Bill, nobody cares, just coach your team.”
  • All the mental energy you use to elaborate your misery would be far better used trying to find the one seemingly impossible way out of your current mess. Spend zero time on what you could have done, and devote all of your time on what you might do. Because in the end, nobody cares; just run your company.
  • “We take care of the people, the products, and the profits—in that order.”
  • In good organizations, people can focus on their work and have confidence that if they get their work done, good things will happen for both the company and them personally. It is a true pleasure to work in an organization such as this. Every person can wake up knowing that the work they do will be efficient, effective, and make a difference for the organization and themselves. These things make their jobs both motivating and fulfilling.
  • “Good Product Manager/Bad Product Manager
  • which I used to train the team on my basic expectations. I was shocked by what happened next. The performance of my team instantly improved. Product managers whom I had almost written off as hopeless became effective.
  • What will you do in your first month on the job?
  • How will your new job differ from your current
  • Write down the strengths you want and the weaknesses that you are willing to tolerate.
  • Early on at Loudcloud, many people would do crazy things backed up by “Ben said.” Often I didn’t say any of it, but I definitely didn’t say it in the way they used it. The management principles I share here are connected to many of those experiences.
  • In all three cases, managers got what we asked for, but not what we wanted. How did this happen? Let’s take a look.
  • Sun Tzu, in his classic work The Art of War, warns that giving the team a task that it cannot possibly perform is called crippling the army.
  • Unfortunately, the metrics that I set did not capture those priorities. At a basic level, metrics are incentives. By measuring quality, features, and schedule and discussing them at every staff meeting, my people focused intensely on those metrics to the exclusion of other goals. The metrics did not describe the real goals and I distracted the team as a result.
  • Some things that you want to encourage will be quantifiable, and some will not. If you report on the quantitative goals and ignore the qualitative ones, you won’t get the qualitative goals, which may be the most important ones.
  • Management purely by numbers is sort of like painting by numbers—it’s strictly for amateurs.
  • Was customer satisfaction rising or falling?
  • What did our own engineers think of the products?
  • Every really good, really experienced CEO I know shares one important characteristic: They tend to opt for the hard answer to organizational issues.
  • Sometimes an organization doesn’t need a solution; it just needs clarity.
  • In all my years in business, I have yet to hear someone say, “I love corporate politics.”
  • What do I mean by politics? I mean people advancing their careers or agendas by means other than merit and contribution.
  • At a macro level, a company will be most successful if the senior managers optimize for the company’s success (think of this as a global optimization) as opposed to their own personal success (local optimization).
  • No matter how well the CEO designs the personal incentive programs, they will never be perfect.
  • Dr. Seuss’s management masterpiece Yertle the Turtle.
  • What’s not fun about working here?
  • If we could improve in any way, how would we do it?
  •   What’s the number-one problem with our organization? Why?   What’s not fun about working here?   Who is really kicking ass in the company? Whom do you admire?   If you were me, what changes would you make?   What don’t you like about the product?   What’s the biggest opportunity that we’re missing out on?   What are we not doing that we
  • should be doing?   Are you happy working here?
  • The primary thing that any technology startup must do is build a product that’s at least ten times better at doing something than the current prevailing way of doing that thing.
  • In his bestselling book Built to Last, Jim Collins wrote that one of the things that long-lasting companies he studied have in common is a “cult-like culture.”
  • them. These door desks are not great ergonomically, nor do they fit with Amazon.com’s $150 billion–plus market capitalization, but when a shocked new employee asks why she must work on a makeshift desk constructed out of random Home Depot parts, the answer comes back with withering consistency: “We look for every opportunity to save money so that we can deliver the best products for the lowest cost.”
  • The first rule of organizational design is that all organizational designs are bad. With any design, you will optimize communication among some parts of the organization at the expense of other parts.
  • Figure out what needs to be communicated.
  • Figure out what needs to be decided. Consider
  • Prioritize the most important communication and decision paths.
  • Decide who’s going to run each group.
  • Identify the paths that you did not optimize.
  • Build a plan for mitigating the issues identified in step five.
  • By far the most difficult skill I learned as CEO was the ability to manage my own psychology.
  • Ideally, the CEO will be urgent yet not insane.
  • Tip to aspiring entrepreneurs: If you don’t like choosing between horrible and cataclysmic, don’t become CEO.
  • Be aware that management books tend to be written by management consultants who study successful companies during their times of peace. As a result, the resulting books describe the methods of peacetime CEOs. In fact, other than the books written by Andy Grove, I don’t know of any management books that teach you how to manage in wartime
  • If you really want someone to succeed, then make her feel it. Make her feel you. If she feels you and you are in her corner, then she will listen to you.
  • The very next day I informed the head of Sales Engineering and the head of Customer Support that they would be switching jobs.
  • There are two kinds of cultures in this world: cultures where what you do matters and cultures where all that matters is who you are. You can be the former or you can suck.
  • The answer is that your loyalty must go to your employees—the people who report to your executives. Your engineers, marketing people, salespeople, and finance and HR people who are doing the work. You owe them a world-class management team. That’s the priority.