Monday, November 01, 2021

OKRs - not as Great as You Think!

Recently, I made a short Linkedin post that was quite critical of OKRs, Objectives and Key Results. The post got a few likes, and comments who both agreed and disagreed with me. I found one of the comments that disagreed with me particularly interesting, because it referenced an article that was very positive to OKRs.

That article contained pretty much everything that makes me doubt that OKRs are useful. I’ll walk through it, but first, here is my original Linkedin post:

Here is why I do not trust OKRs, in three quotes:

"When a measure becomes a target, it ceases to be a good measure."

-Goodhart's Law,

"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

-Campbell's Law,

"Key Results benchmark and monitor how we get to the objective. Effective KRs are specific and time-bound, aggressive yet realistic. Most of all, they are measurable and verifiable. You either meet a key result’s requirements or you don’t; there is no gray area, no room for doubt."

-What is an OKR? Definition and examples,

In other words, the OKR system is designed to implement the kind of target setting Goodwin and Campbell warn about.

Judging from what I have seen, OKRs are a great way to train people to fudge results.

Add in that OKRs oversimplify the dependencies between different objectives, and that OKR data is usually presented with to few data points to be useful in decision making, and I have a difficult time understanding the enthusiasm.

I do like metrics, but I doubt that successful companies that use OKRs became successful because they used OKRs.

#strategy #management #leadership

Let’s look at the article used as evidence that OKRs work, and see whether it holds up. The article is named Implementing OKRs. A tale from the trenches.

The case of the misleading case

First of all, the article describes a single case of a company that implemented OKRs, set ambitious targets, and then saw the company meet those targets. That is problematic, because one, or even a few, cases do not prove, or disprove anything.

Look at the picture below.

Which of the three cases in the picture is closest to the truth? The author of the article clearly believes it is Case 1, but there is no evidence of that. The truth may be closer to Case 2, or Case 3.

Selection bias and lack of scientific research

When in doubt, turn to science! I googled “okr scientific research”, and a few other sets of search terms, but found no statistical studies, or metastudies.

The closes I got, was a question on Quora:

“Is there any peer-reviewed research showing that the Objectives and Key Results (OKR) model actually improves efficiency, productivity, or overall company performance?”

Unfortunately, because of the selection bias inherent in the way the question was phrased any answers won’t be much use. Studies that do not show that OKRs work, will never be included in the answers. What you need is studies that prefer statistically significant results. Preferably, we should have metastudies, because they produce better results than any single study can.

As far as I can tell, the research on OKRs is so inadequate that we cannot turn to direct scientific research to either prove or disprove whether it works.

People burning bright, but not for long…

We can, however, use indirect methods to evaluate whether OKRs are likely to work, or not. You create an OKR by setting an agressive, relatively short term, goal, and a target value to reach.

The article author did this by setting nine BHAGs, Big, Hairy Audicious Goals, figured out what reasonable targets would be, and then doubled them. In the words of the article author:

We took those numbers and we doubled them. Why? Because a) why not? That way if we aimed for twice the volume of our plan and we missed by half, we’d still hit our plan. If we actually hit those goals, even better! And b) because you are only supposed to get 70% on your OKRs. These are stretch goals, so come on, stretch a bit. Yeaaaah, that’s it.

The article then continues, describing how quarterly OKRs were set, followed by individual OKRs. The author also asked advice about OKRs on Facebook. That probably yields as reliable advice as asking about Covid-19 advice on Facebook.

Did it work? The article author is convinced it did:

When Brew and I traveled on business, shit got done! Decisions were made! Targets were hit!

He also has a credible sounding explanation for why the targets were hit:

Once you see what other people are signing up for, and you see the plans they are putting in place to achieve them, you get religion. You believe they are going to hit their goals. And you don’t want to be the only one not hitting her goals. So you work. You work hard. Instead of watching what everyone else is doing and reacting, you start focusing much more on achieving your Objectives and just assume they’ll hit theirs. OKRs help everyone focus, and that drives results.

The author does not specify exactly what OKRs they used, but he does provide three examples of OKRs he believes are good ones:

  • Increase leads from 300 to 700
  • Improve conversion funnel efficiency by 30%
  • Increase sales volume by 70%

Sounds pretty clear cut: OKRs are a recipe for success…or is there something wrong with this picture?

Yes, there is something wrong! For one thing, the author claims success comes from everyone working harder. Note that working harder and working smarter are two very different things.

Leads increase from 300 to 700. That is an increase of 133%. If that increase in results came from working harder, it is reasonable to assume there was a corresponding increase in effort. If people worked 40 hour weeks before OKRs, with a 133% increase, they would now work more than 93 hours per week.

Did they work 93 hours per week? We do not know, because there was no OKR for that, but it is quite possible that the OKR system pushed people to work way more than is sustainable. What happened to personnel turnover? The article did not mention that either, so we do not know.

Good metrics are not isolated from each other. They are linked into a system. Such a system would have metrics that prevent people from being overworked. A good metrics system should also be designed to be self-correcting. You have both leading and trailing metrics. You use the leading metrics to guide behavior changes, and trailing metrics to see if you actually got the results you thought you’d get.

OKRs do none of that. OKRs are treated as independent. That is why, with OKRs, it is easy to turn a workplace into a 93 work-hour per week sweatshop.

On the other hand, we do not know that people worked more hours. There might be some other explanation.

I do not have data from the article author’s company, but I do have data from the project teams I lead, so let’s use some of my project data, and see where it gets us. First let’s compare productivity data for two periods, similar to what you would get if you used OKRs. Let’s go:

  • Period 21: 4 value units
  • Period 22: 47 value units

Wow! From 4 units to 47 units! That is an increase of 1075%! How audacious target setters we must have been! What a Big Hairy Audacious Goal we must have had!

Lets look at the productivity the quarter after that:

  • Period 23: 19 value units

That is a drop of 60%. How could the geniuses of the previous quarter turn into such bumbling idiots? Off with their heads!

Well, that last thing might be overreacting a bit. Let’s look at why we have such huge increases and decreases in results.

The picture above is a Process Behavior Chart, also known as a Process Control Chart. It shows that although we have a lot of variation in productivity, the variation always stays within the upper and lower process limits. These limits are calculated using a statistical method. I won’t go into details in this article. The important thing is that the system is stable within the process limits. All the variation is random.

With OKRs, this randomness cannot be distinguished from actual change, so it is entirely possible that the 133% increase in leads the OKR article author has in his example, is entirely due to random variation.

I don’t know, and neither does he! The difference is, Process Behavior Charts make it possible to distinguish real change from random variation. OKRs do not do that. Even worse, OKRs lead you to believe you understand a situation, and that you control it, while you really don’t.

Targets corrupt!

What happens when you set a target? Lets look at the same Process behavior Chart, but this time I have marked some target levels. I have also color graded the area from zero to the upper process limit, in colors from green to red. The more green, the easier it is to make a target. The more red, the more difficult it is to reach a target.

It is important to remember, that we are looking at the system as it actually is. To get the system to behave differently, we need to change it in some way. If we set targets and empower people, but do not teach how to change the system, they have only a few alternatives:

  • Hope they get lucky. Works once in awhile, but the game tends to be stacked against the people doing the actual work, especially since OKRs mandates setting targets without providing any method of figuring out what a reasonable target is.
  • Work more hours. Common, and often works in a very short perspective, but not sustainable over time. Can have disastrous side effects, like increasing personnel turnover, which drains the organization of competence, and destroys organizational memory.
  • Cheat. Usually the safest option. Risk of discovery is small. Even if discovered, risk of punishment is not that high.
  • Teach themselves how to change the system, change the system, and then pray that they really were as empowered as they were told they were. If they were not, punishment for doing actual change is usually swift and unmerciful. This is difficult, time consuming, dangerous, and often provides little benefit to the people who do it, even if the change succeeds.

There are 25 data points in the series. If we want a 70% chance of success, that is we want 17 data points at the target level or higher, we should set the target at around 5. That is well below the average, which is slightly above 20.

Setting the target to a 70% success rate based on future performance would require a time machine. If you don’t have one, trying will just be silly.

Let’s assume for a minute that you know the average productivity is 20. Note that with OKRs, you don’t bother with averages, so you need something else to provide that information. If you set the target at double the average, 40, you have a 20% chance of reaching it, even if you do not change anything about how you work. I’d say doubling the average sounds like an aggressive target.

That means, in our example, 20% of the times you set a target that is double the average, you will trick yourself into believing that OKRs work. People like writing about their successes much more than they like writing about their failures, so the people who write about OKRs are likely to come from a group that sets target like this, or, since they usually do not know what the average is, have set the targets way into the green.

It is a popular belief that if you empower the people who do the work, they will start doing the systemic improvements necessary to meet audacious targets. Unfortunately, that requires that they also know how to improve the system. Sometimes people do, but most of the time, they don’t. Why not? One reason that stands out is that neither schools, nor workplaces, teach effective methods for improving systems. Universities sometimes do, 

Plenty of such methods exist. There is a plethora of different improvement methods based on complexity thinking, systems thinking, and statistics. Some of these methods have tools that are fairly easy to use, like Five Why in Lean/TPS, or Evaporating Clouds in The Logical Thinking Process (which is one of the tool sets in the Theory Of Constraints). Others, like Process Behavior Charts, are more difficult to use. Easy or hard, they all have one thing in common:

They are rarely taught!

Imagine that someone sets a target for you, and you have no idea how to achieve it. Imagine that whoever sets the target also tells you “you are empowered to do whatever it takes to reach the target, just don’t fail”.

If the target looks possible to achieve, you will almost certainly continue the way you are used to, and hope for the best.

If the target is way out there, and looks impossible to achieve, there is only one thing to do:

You cheat!

Some years ago, I worked for a company that set very challenging targets for its sales people, in several different markets. I was in close contact with people from one of the markets, and they told me how they had to cheat to meet their targets. They hated it! It’s just that they had to. They told me all the other markets were in a similar position, and did the same thing.

In their case, they did not manipulate the total number of units sold, but they did mess around with the dates when things were sold, and that was enough to ensure that they were on target often enough to avoid getting punished for missing them.

In software development, which I usually work with, management likes to set targets like “80% automated test coverage” for an application. That is particularly useless, because the need for tests varies between different parts of an application. In general, it is useless to write tests for something called accessor methods, but there are a lot of accessor methods, and they are easy to write automated tests for, so that is what often happens.

Another little trick: If management measures software development team velocity, usually in story points per sprint, or stories per sprints, and sets a target, story points and stories will magically shrink. As a result, velocity will go up, and the target is met, without anything real changing at all.

If we look to research on target setting, it corroborates the stories above.

Here we found that participants who were given a specific performance goal for how much revenue they had to earn were almost twice as willing to use unethical methods to achieve it than those given a vague goal, irrespective of their moral justification.

— When Tough Performance Goals Lead to Cheating, by Colm Healy and Karen Niven

The above quote is from an HBR article about a scientific study with 106 participants.

We found that people with unmet goals were more likely to engage in unethical behavior than people attempting to do their best. This relationship held for goals both with and without economic incentives. We also found that the relationship between goal setting and unethical behavior was particularly strong when people fell just short of reaching their goals.

— Goal Setting as a Motivator of Unethical Behavior, June 2004, The Academy of Management Journal, by Maurice E Schweitzer (University of Pennsylvania), Lisa D. Ordóñez (The University of Arizona), and Bambi Douma (University of Montana)

Here is a quote from a third study:

Our study finds that cheating in goal-based systems occurs due to financial rewards to cheating and to social comparison framing. Assigning a goal on its own without increased pay or social comparison framing did not lead to an increase in cheating relative to a do-your-best counterfactual. However, the use of either financial rewards or social comparison framing led to cheating.

— Why do goal-based incentives cause cheating? Unpacking the confounding effects of goals, social comparisons and pay, by Matthew Chao (Williams College), and Ian Larkin (UCLA Anderson School of Management)

We do need a lot more research about the effects of setting goals and targets. As far as I can tell, setting goals, by itself, does not cause significant problems. However, when we offer financial rewards, or compare people’s performance, then we invite cheating.

If a target is tied to a reward, then people will be prone to cheat to get the reward. The same way, if a target is used for social comparison, as in “Helen reached the target, and you didn’t”, then we also invite cheating.

I haven’t found any scientific research about targets tied to punishment, but since we are geared to react even more strongly to punishment than to rewards, it is highly likely that fear of punishment makes us cheat too.

Goals blinds us to creative solutions!

Goals do make us more focused. When we do something relatively simple, that only requires increased effort, that can indeed be effective. However, our working lives are often way more complex than that.

With goals, people narrow their focus. This intense focus can blind people to important issues that appear unrelated to their goal (as in the case of Ford employees who overlooked safety testing to rush the Pinto to market). The tendency to focus too narrowly on goals is compounded when managers chart the wrong course by setting the wrong goal…

— Goals Gone Wild: The Systematic Side Effects of Over-Prescribing Goal Setting, by Lisa D. Ordóñez Maurice E. Schweitzer Adam D. Galinsky Max H. Bazerman

Case studies and anecdotes are no substitute for scientific research, but if you got the research, they can be used to illustrate a point, so let’s do that:

Several years ago, I worked directly for the CTO of a company that had hired a sub-contractor to improve its IT infrastructure. The project was not going well, and failure would mean the sub-contractor would not get a new contract, and that would mean firing a lot of personnel. This was during a raging global financial crisis, and if there was one thing you did not want to do, it was lose your job.

The sub-contractor management set as a goal to minimize the damage to their company from missing the contract. Everything they did was focused on whom to fire, when to fire, what severance packages should look like, and other damage control measures.

I asked my CTO if I could get a shot at helping the sub-contractor make the project deadline, and he okayed it. There were benefits for both organizations if the sub-contractor made the deadline, and, of course, to the people who would keep their jobs.

The sub-contractor was less than enthusiastic about having me meddle in what everyone knew was a hopeless situation, but the CTO I worked for could be very persuasive when he wanted to be, so I got permission to make an attempt.

The development team was already working as hard as it could. There simply wasn’t enough hours in a day to add more overtime.

I began with an information gathering and brainstorming workshop using a method called Crawford Slip. Then, I showed the team how to set up a kanban board. They were less common back then than they are today. I would have preferred to stay with the team throughout the improvement project, but that proved impossible. Nobody wanted to spring for the cost of having me around full time.

Instead, I told the team to photograph the kanban board twice a week, and email me the photos. I spent several weeks doing nothing but recording the team velocity, and using the data from the brainstorming session to build a Current Reality Tree (CRT). The CRT is a problem solving tool from the Logical Thinking Process toolkit.

Then, I traveled back to the team, showed the project manager what I had found, and made a few suggestions. You can see the results in the graph below.

I met with the project manager week 40. Week 41, productivity went up. Week 42, it went up again. After that, it held steady around a new baseline. In two weeks, the average output increased by 316%.

Remember, they were already working as much overtime as they could, so the productivity did not come from working harder. It came from working smarter.

What does it mean to work smarter?

So, what does it mean to work smarter? You need to understand the process, find problem areas, and then fix them. The picture above shows the situation before we made changes. The developers worked on a bunch of jobs, all with high priority, and all broken down into several tasks. Since everything had high priority, they tried to start each job as soon as possible.

When they had finished one task, they immediately began working on another. The problem was that the new task they started working on, had nothing to do with the preceding task. They switched back and forth between working on different jobs. This lead to tasks belonging to different jobs getting interleaved, and this pushed lead times to become longer.

What we did was reorganizing the tasks, so that all tasks belonging to the highest priority job were finished first. Then all the tasks belonging to the second highest job were done, and so on.

One counter-intuitive thing is that we delayed starting work, in order to finish sooner.

In reality, the benefits are much larger than the diagram shows, because when you eliminate multitasking, you eliminate having to spend time on switching tasks. You do not need to change your tool setup as often, you do not need to backtrack to try to remember what you did the last time, and you do not need to figure out what the heck you were planning to do. That adds up to a lot of time saved.

Of course, eliminating multi-tasking is only one of the things you can do. For practical purposes, there are nearly always dozens, or hundreds, of improvements you can do. Once you have made an improvement, that often opens up the way for other improvements.

With OKRs, there would have been no improvement, because working harder was not the answer. Understanding the process, and eliminating a major problem, that did the trick.

It’s worth noting that a change like this is a sustainable change. No one is pushed to work harder. The developers can actually deliver more value while working less. That leaves more time for thinking, which can be used to find new things to improve.

How did things turn out? Well, the company managed to deliver, it got the contract it was after, and when I checked a year later, they had managed to keep all the personnel.

Had they used OKRs to just push to work harder, people, they would have had close to zero chance of finishing the work in time, and getting the contract.

Summing up

It’s time to sum this article up:

  • There is very little research on the efficacy of OKRs. The existing material consists mostly of testimonies and case studies
    • Testimonies and case studies are notoriously unreliable. We should not base our beliefs about OKRs, or anything else, on them.
  • While we cannot use scientific research to tell whether OKRs work, there is research on the building blocks of OKRs.
    • Agressive target setting: Setting agressive targets increases cheating substantially
    • Use of single data points: Using single data points can be, and very often is, incredibly misleading. Metrics systems should, by default, use long data series. 
  • OKRs ignore the effects of variation, and that is a great way to trick people into making bad decisions. We need some way of separating random variation within the system from special cause variation. We also need to detect trends in the data. Process Behavior Charts does all of this, but there are other methods that can be used.
  • OKRs do not tell us why we reach a target, or why we fail. It may be that we have improved something, or it may be that we got lucky, or unlucky.
  • OKRs do not tell us whether a change, if there is one, is sustainable, or if it will cause detrimental side effects.
  • OKRs present dependent variables as if they are independent. This is bound to cause confusion, and bad decisions. To be useful, a metrics system must link things together, so that we do not inadvertently cause big harm when we make a small improvement.
  • OKRs makes us focus on reaching targets, and this can make us blind to creative solutions. In other words, OKRs can prevent you from reaching the goals set by the OKRs.

There you have it: There is very little that speaks in favor of OKRs, and a lot of reasons to be suspicious.

In addition, we can easily create metrics systems that do not have the problems that OKRs have.

You can use Goal Maps, S&T trees, or strategy maps, to build a model of the organizations goals, and how they interact with each other. (I think Goal Maps and S&T trees work better than Strategy Maps for this, but that is a matter open to debate. There are also other ways to do the mapping.)

Add using Process Behavior Charts, or some other way to distinguish between systems noise and special cause changes, and you have a good start.

Of course, any metrics system has limitations, so you should be aware that in some situations, cause and effects based metrics systems cease to function. That happens when your organization dives into complexity. You don’t have cause and effect anymore, but you still have statistical dispositions, which means that while a Goal Map is less trustworthy, it is probably not totally useless. When you have true chaos, which happens only rarely, you have no causality, so you need to use other methods. (Check out Cynefin, if you are interested.)

Most of the time though, a good metrics system will help, but OKRs probably won’t.

Sunday, January 24, 2021

Tempo 2.0 - Section 3.4 Value Stream Maps

Mapping a Value Stream is an adventure, a bit like exploring an uncharted landscape.

If you are serious about improving the way your organization works… Let me rephrase that: If you are serious about improving any process, for an organization, for yourself, or for a friend, you will sooner or later need information about what the relevant value streams look like. You also need a simple, yet useful, way to map the information. Having that enables you to figure out where to focus your efforts.

A value stream map is easy to learn, yet very useful, tool for mapping value streams.

The purpose of a value stream map is to help you identify waste in a value stream. A value stream map tells you how much of the time that a goal unit[1] spends in the value stream that is value adding time and how much is non-value adding time.

Waste, schock, and incredulity

Schock and incredulity are common reactions the first time someone sees a value stream map. Does it really take that much time to produce a single goal unit? Do we really have that much wasted time?

The harsh reality is that in most value streams, goal units spend a lot of time waiting. Often, the non-value adding time is 99% of the total time, or more.

How can you have 99% waste in a process without anyone complaining about it?

I once worked as an interface designer for a very large enterprise software system. Designing and constructing an interface took about a week, and each connection between two subsystems had its own custom-designed interface. There were many, many subsystems, and two subsystems could have several different connections.

Interfaces were deployed once per year. The project had been ongoing for about seven years.

That meant an interface spent on average six months either in a requirements queue before it was designed, or after design and construction, in another queue waiting to be deployed. Thus, for a week of value-adding time (design and construction), we had about 25 weeks of non-value adding time.

A ratio of 1:25, means we have 4% value adding time, and thus, 96% non-value adding time.

But wait, there is more! I wasn’t the only interface designer. We produced about 60 interfaces per year. In seven years, that is about 420 interface designs.

How many interface designs did we need? One! We could have built the whole thing with one standardized interface, for example HTTPS[2], which is used for the vast majority of computer connections on the Internet. Instead of making a custom data format for each interface, we could have managed with about five standardized data formats.

Sticking to just the interfaces, we need to divide those 4% value-adding time by 420. That means we are down to 0.0095% value adding time, and 99.99% non-value adding time.

Note that we had two different types of waste:

One type was caused by having partially finished work laying around, instead of finishing the work and deploying it, so it could be used. Assuming that there was good economic reasons for building each connection, there was an economic cost attached to not having the connection.

The other type of waste was building things that should not have been built in the first place.

Of course, if you have 420 times as many designs as you ought to have, then you will need quite a lot of people to build the stuff, which means you will need more administrative personnel. The costs do not stop there, because you will need extra people maintaining all the extra stuff the project built, and there will be delays in maintenance too, which will cost even more

A company can easily create cost cascades that plague them for decades, and the funny thing is, identifying such costs, and stopping the bleeding, is usually a low priority.

Identifying and fixing problems like the ones in the story, requires a lot more than knowing how to make a value stream map, but value stream maps are a good place to start. Sooner or later, you will have to present your findings to other people, and value stream maps are very helpful there too.

Value Stream Map Example

Figure: Value Stream Map

The figure above shows a value stream map for a software development company. It shows how a single goal unit, a user story, travels from requirements gathering to the point where customers start paying for it.

The greatest delay, 3,200 hours, is between the initial Profit & Loss estimate, and the monthly meeting where work is prioritized. The queueing time is about two years. This means the greatest delay is in an administrative process that is executed before a development team even knows there is a requirement.

It is very common in software development that the greatest delays occur before, or after, the software development team is working on a user story. And yet, most companies would, in this situation, focus on improving the way developers work.

The mistake happens because no one examines what the value stream looks like before the improvement work begins.

Reducing waste with a Value Stream Map

Some years ago, I worked with a software development team that had not released anything to production for 3-4 months. We set about fixing a number of things that kept the team from delivering, and got to the point where the team released software once every two weeks, and could release more often than that, if they were asked to.

After the improvements, how long was our lead time? It was still 3-4 months!

I mapped out what happened to a user story after the team was done implementing it. It turned out it went on to a team that both trained users, and tested the software.

The problem was that when a new piece of functionality had been implemented, they tested it, wrote training materials, and held a course in a single batch, before approving a user story for release. We very politely asked them if they could test and approve user stories before writing the training material and holding the courses.

They said, “No, problem!”

From then on, we could release software as often as the business side wanted.

The reason it was so easy to make such a major improvement, was that the team’s Product Owner was an employee at the company, and knew the people in the training and testing department really well.

I contributed my theoretical knowledge about how processes and queues ought to work, he contributed both detailed knowledge about how the processes actually worked at his company, and his power to influence processes via his network of contacts, and friends, at the company.

Neither of us could have done it alone, and that illustrates an important point: When you try to fix something, you must work with people at the company, and you must do it all the way through the change. Handing off a change plan and expecting someone who hasn’t been a part of developing it, does not work! It does not work if you are a consultant like me, and it does not work if you are a manager, or leader.

It is often difficult to get an overview of a process. Most companies are still divided into function-oriented parts, where each part is interested in its own little piece of the value stream, and little or nothing else. Most of the time, like in the story above, the greatest delays occur where there are organizational borders.

Worth mentioning that “organizational borders” does not mean just departmental borders. For example, when you have multi-team projects or programs, borders between teams can be, and often are, a major source of delays.

If you are mapping a value stream across organizational borders, you will not always be able to find someone who is responsible for the whole value stream. If so, you will have to follow a goal unit as it travels through the organization, without having approval from a senior manager. Technically, this is easy to do, but you might want to have your CV prepared, in case you find out where someone has buried a skeleton or two.

If you find someone who is responsible for the whole process, it is likely that that person does not know in detail how the process works, or where the delays are.

If you are lucky, as I have been on several occasions, you will find a friend, someone who is as curious about what is going on as you are. You might get a bit of support, but do not expect that person to take personal risks for you, especially if you are a consultant. Even if it looks to you that they could just magically whisk you out of trouble when you have found something sensitive, that is probably not the case in reality.

By the way, do not mistake the organization chart for the reality. You are liable to find that goal units travel weird and mysterious ways before arriving at their destination. I have, on occasion, found that some work items do not actually have a destination. That is, they are still produced, but the intended recipient may have disappeared in a re-organization frenzy years before.

It is part of what makes value stream mapping such an adventure.


  The purpose of a value stream map is to help you identify waste in a value stream.

   In most value streams, goal units spend a lot of time waiting. Often, the non-value adding time is 99% of the total time, or more.

  There is more than one type of waste in processes. For example: Partially finished work waiting in queues, and building things you do not need.

  To find waste, you need to look at the whole value stream, across team and departmental borders.

  You need theoretical knowledge, a good understanding of actual processes, and a strong network of people willing to help you in the organization.

  Involve people from the start! If you hand over a plan to someone who has not been involved in creating it, execution will often fail.

  Get a sponsor within the organization if you can, but do not expect them to work miracles if you get in trouble investigating a value stream. At best, they can advice you where to be careful, and what to steer clear of.

  Value stream mapping is a lot of fun.

  There is a lot more to value stream mapping than this introduction. Be aware that there is a lot more to learn.

[1] A goal unit is the unit of measure you use to measure your goal. In agile software development, user stories, features, and epics are common goal units. For a car factory value stream, the unit might be a car. For an accounting system, the goal units are money.

[2] HTTPS has been around since 1994, so, even though the project was many years ago, no cutting edge technology was required.

Friday, December 04, 2020

Tempo 2.0 - Section 3.3 Value Streams and Process Flows


…that the greatest prosperity can exist only as the result of the greatest possible productivity of the men and machines of the establishments— that is, when each man and each machine are turning out the largest possible output.

The Principles of Scientific Management, by Frederick Taylor, 1911.

The 20th century was a century of economic growth and technological development never seen before in the history of humanity. A significant part of the credit for this has to go to Frederick Taylor, whose book The Principles of Scientific Management, was published in 1911.

Taylor laid down the basic principles of mass production. The ideas worked well for many years, but there were hidden problems.

One thing that happened, because the idea was that each worker and each machine should produce as much as possible as much of the time as possible, was that parts tended to pile up, everywhere. Different people, and different machines, doing different things, produce things at different rates. If you produce a lot of different parts that will eventually be fitted together, you will inevitably end up with too many parts of some things, and too few of other things.

Having big piles of unused parts laying around didn’t bother people much in the early 20th century. One reason was that the economy overall was expanding, except for the occasional depression, and most of the time, the parts in those big piles would eventually be used and sold. Another reason was that the prevalent accounting model, Cost Accounting[Cost Accounting is still the prevalent accounting model. It is still a model with problems, but today there are several alternatives, who have their own pros and cons.], regarded those big piles as assets. At the time, it was a reasonable assumption that whatever you produced would eventually be sold.

“Reasonable” does not always mean correct. Those piles of parts were actually a huge risk. Having capital tied up in parts meant a company had less cash on hand. If something happened, like a depression, or just a change in what the market wanted, the parts might become worthless, and the company stuck with them would suffer a huge loss.

Using an inherently wasteful production system also meant huge economic barriers to entry for new companies…unless someone could figure out how to make a small number of products with a much smaller investment.

Eventually, someone did.

The quality guru W. Edwards Deming is considered one of the greatest management experts of the 20th century. Deming traveled from the US to Japan in the early 1950’s. He taught both managers and engineers statistical methods to improve quality. However, there was something that Deming considered even more important.

Deming taught how to view a company, and the environment it operates in, as a system. He summarized what he taught in a simple sketch.

Figure: Deming’s view of manufacturing organizations as systems.

Deming’s simple sketch was the start of a revolution. Deming described a flow of materials, a value stream. He also described feedback. Information is retrieved from customers, and fed back into the process.

In 1950 Japan was a country in a deep economic crisis. The industry was obliterated during World War II. Nobody knew how to rebuild the country again.

Deming’s sketch held the key to the Japanese economic miracle. In the West, mass production based on Frederick Taylor’s Principles of Scientific Management reigned supreme. There was no way that a poor country like Japan could make the investments necessary to build an industry based on mass production.

The Japanese decided to change the rules, and play a different game. They would outmaneuver their stronger opponents. While Western companies were built around the functions needed for manufacturing, Japanese companies like Toyota and Honda decided to organize around the process flows companies need.

Value Stream became a central concept. The idea was to look at value streams from the perspective of the customer:

What does the customer want to get out of the process?

Looking at the process flow, the value stream, from the perspective of the customer made it possible to divide the flow into value adding time and non-value adding time. An astonishing discovery was that in most processes, only a small part of the time was value adding time. Most of the process time was non-value adding time, and the Japanese companies worked hard to get rid of it.

Getting rid of non-value adding time made it possible to produce things in much smaller volumes than mass production systems, with much smaller investments, and still make a profit. The feedback loop from customers meant that initially shoddy products would be improved over time. Eventually, they became good enough to turn quality into an additional competitive advantage.

Deming’s sketch represents the smartest business strategic move of the 20th century. To be clear, Deming did not, on his own, save Japan’s industry. Nor was he the only one who had bright ideas. The Japanese economic miracle was a lot more complex than that. Still, Deming’s sketch sums up the basic idea in a way that is easy to understand, even if it is a challenge to implement it.

In the West, we never really got it. For a long time, we continued to base our industry on the outdated mass production model. Scientific Management is so integral to how we think that most people who use it don’t even know its name. It’s the only way we know, so there is no need to distinguish it from anything else.

Our terminology has changed. Talking about value streams is popular today, but if you look closely at organizations and processes, they are often built around the old way of thinking. Many companies have redesigned their factories, but even then, the administrative processes, distribution networks, and management paradigms, are often based on Taylor’s ideas.

In the same manner, there is a lot of talk about the “customer”, but what is meant is often the next step in the value stream, not the customer who will use the product or service. This twists the whole idea of thinking in terms of value streams.

The tendency to adopt new terminology, without really changing anything, means a lot of good opportunities are lost. The purpose of this book is not to convince you that any particular idea is better than everything else. I do want you to appreciate the importance of new ideas, and I want to emphasize the importance of understanding ideas, rather than blindly following recipes. The reason is that implementing a good idea requires adapting it to your situation. To do that, you need to understand both the idea, and your situation. Blindly following a recipe for success, is actually a recipe for failure.


  • Optimizing each part of a process often leads to sub-optimization of the whole.
  • If you do not know which management paradigm you are using, you are almost certainly using Scientific Management.
  • Scientific Management will get you, and your company into trouble, unless you have a deep understanding of its merits and limitations. It is safest to assume that you don’t.
  • Understanding value streams is a key to understanding production systems. Do not make the mistake of believing it is the only key!
  • Building organizations around value streams makes it easier to optimize processes than building organizations around functions.
  • Process flows can be divided into value added time and non-value added time. Non-value added time is bad, so try to get rid of it.
  • The customer is the user of the product, not someone in the production process.
  • Adopting value stream terminology does not make your organization focused on value streams. Building, or rebuilding, your organization around value streams does…but watch out for innovative ways of screwing things up.
  • Learn to appreciate the value of new ideas, but don’t be so open to new ideas that your brain falls out.
  • Implementing ideas requires adapting them to your situation, and that requires understanding both the ideas and your situation.
  • Following recipes for success, is a recipe for failure.

Wednesday, December 02, 2020

Tempo 2.0 - Chapter 3.2 Properties of Systems

This is a draft version of a section in Tempo 2.0. Please do comment.

Even simple systems can be difficult to understand. It is not because systems are inherently difficult to wrap your head around. The problem is our brains are trained the wrong way. Fortunately, it is not all that difficult to re-learn. When we do that, many problems that have looked utterly incomprehensible before, are suddenly fairly easy to understand. Doing something about them may still be difficult, but at least we have got a start.

For starters, we will define what a system is:

  • A system is a set of connected parts, real or abstract, that form an integrated whole.
  • Systems have some interesting properties. For example:
  • Systems consists of parts, that are also systems.
  • Systems always belong to larger systems.
  • Systems have properties that do not exist when you examine the parts separately.
  • You cannot understand a system just by studying the parts.
  • When you do something with a part of a system, other parts will nearly always be affected.
  • When you improve part of a system, the system as a whole may get worse.
  • Systems with many connected parts usually have only a few degrees of freedom.
  • Cause and effect in a system can be separated in time. When we talk about organizations, time delays can be years, decades, or even centuries.
  • Cause and effects can be separated in space. It is common that a problem in one part of an organization is caused by a solution to a problem in a different part of an organization.
  • Systems that are not continuously improved, will degrade on their own. Organizations often find that the world around them changes faster than themselves. The organization therefore becomes less, and less, adapted to the world around it. When the gap between the organization and the world around it becomes too large, the organization dies.
  • Things can happen without having identifiable causes. Events can also have effects that are not predictable. Instead, the relationships between parts are statistical trends and dispositions. This is a biggie, because when you are dealing with this kind of a problem, almost everything you have learned in traditional management courses and training, will push you into doing the wrong thing. Though this sounds frightening at first, you are in no way helpless. There are things you can do, and those things can be highly effective.

Leading or managing a system as if the parts are independent of each other does not work well! When we improve part of a system, the system as a whole can get worse!

For example, if your company increases its manufacturing capacity without selling more, you may suddenly find that you have tied up too much capital in stock. Sometimes that gets you into a lot of trouble. Goodbye company!

Figure: Management thought they would increase capacity by increasing the number of people in the project team. Instead, capacity dropped like a rock.

Sometimes, it is less obvious what will happen. A mistake I see over and over again, is when management adds people to late software development projects. What often happens, is that the production capacity goes down instead of up.

The picture above is from production velocity measurements I made in a project several years ago. Both before and after, I have seen the same thing happen in many different projects. It is a problem that can be easily predicted, and prevented, or, if it has already happened, it can be fixed. Fixing it afterwards, is more difficult though.

Right now, you are probably thinking “That can’t be true! If it was, no project should have more than one person”. The answer is that it is not an either or situation. Adding more people has very different effects depending on the situation, and on how you add them. Sometimes velocity goes up. Sometimes velocity goes down. Sometimes, you will notice no difference.

If you know the basic principles involved, and if you know a little bit about the situation the team is in, and how the team is organized, figuring out what will happen is quite easy. It is also easy to measure the results of actions you take, to see if you found the right solution. Easy, that is, if you know how to do it. That, among many other things, you can learn from this book. We’ll take it one step at a time though, so back to basic properties of systems.

Because every part in a system is connected to one or more other parts, it follows that we cannot optimize one part without affecting other parts. As we saw in the graph above, the effects of optimizing a part can be a bit counter-intuitive.

Most of the time, cause and effect are easier to understand, but we keep shooting ourselves in our feet anyway. In software development, we often track how long it takes to finish each piece of work. We then push software developers to finish each piece of work faster. When we do that, quality goes down. The code gets more complicated, and there are a lot of bugs. Working with complicated code takes a lot of time, and fixing errors in complicated code also takes a long time. As a result the entire project slows down, because someone pushed the developers to speed up.

How much can you speed projects up by optimizing the whole? Back in 2010, when I wrote the original version of Tempo! I wrote that optimizing the whole instead of the parts could lead to delivering 500% faster than with waterfall methodologies. Since then, a lot has happened.

The original assessment by Jeff Sutherland was based on a single team producing software. Nowadays, I mostly work with programs where many teams collaborate. That makes the effects of sub-optimization much greater. I have seen multi-team programs where an estimated two weeks of work takes two years, or more.

You might think that is an extreme case, but no, it isn’t. What is even worse, when it happens, entire organizations go into denial. You hear people, otherwise very smart people, claim that a two year lead time is perfectly normal. They claim that even if it is a long lead time for a single piece of work, it does not matter, because they are going to deliver everything at once.

As a result, companies lose the revenues they would have had from early partial deliveries. Risk increases, because nobody knows if that big batch of software that will be delivered all at once, will work, or if it was what the customers wanted.

The more complicated our companies, their environments, and their products become, the worse traditional methods for managing them become. Traditional methods of leading companies, and planning what they do, are quickly becoming obsolete.

There are fields of science that helps us understand complicated and complex systems: Systems Thinking, and Complexity Thinking. You will encounter both of them in this book. I will do my best to present them in ways that are practically useful, and as easy to understand as possible, without being dumbed down.

One thing we have learned, often very painfully, is that there are no “best practices” that always work. Most of the problems you are facing today, are caused by the solutions of yesterday. You can be pretty sure that many of the problems you will be facing tomorrow, will be caused by the solutions you create and implement today.

Complexity Thinking and Systems Thinking help you to understand the causes of the problems you are facing, and make it easier to find solutions. They also give you a little bit of insight into what problems your solutions might cause in the future, and can help you reduce risks, and create opportunities.

Strategic Navigation, the business strategy framework this book is about, is based on a military strategic framework, Maneuver Conflict, created by Colonel John Boyd. It is an open system, based on systems and complexity thinking, that works well with several other frameworks and methods. You will encounter the ones I use in this book.

There is one thing I would like to encourage you to do:

I will introduce several frameworks and methods in this book. That means, I have picked and chosen parts I believe are useful to you, but by necessity, there is a lot I have left out. Do go look at the sources!

This book is a starting point. It is not the management book to end all management books. There is much more in-depth knowledge that you will almost certainly find useful, so embark on a journey of your own to find it. You will have to learn for the rest of your life, so make it interesting and fun.

It will be worth the effort.


  • A system is a set of connected parts, real or abstract, that form an integrated whole.
  • Systems always belong to larger systems.
  • Systems have properties that do not exist when you examine the parts separately.
  • You cannot understand a system just by studying the parts.
  • When you do something with a part of a system, other parts will nearly always be affected.
  • When you improve part of a system, the system as a whole may get worse.
  • Systems with many connected parts usually have only a few degrees of freedom.
  • Cause and effect in a system can be separated in time.
  • Cause and effects can be separated in space.
  • Systems that are not continuously improved, will degrade on their own.
  • Things can happen without having identifiable causes.
  • Events can have effects that are not predictable.
  • You, your family, your team, your department, your company, the location where you live, your country, and the world, are all systems, and they are connected, and can and do affect each other. You cannot control or accurately predict how they interact, but you can influence, and make educated guesses.
  • This book is a starting point for learning.
  • You will have to learn for the rest of your life, so make it interesting and fun.

Saturday, August 15, 2020

Tempo 2.0 - Chapter 3: Your company is a system and section 3.1: The Monty Hall Problem

To manage a system effectively, you might focus on the interactions of the parts rather than their behavior taken separately.

— Dr. Russel Ackoff

Companies and other organizations are systems. All systems have some properties in common. As you will soon see, these properties can be counter-intuitive. If we rely only on intuition, we will make the wrong decision, even when we have all the information we need to make better ones.

The Monty Hall problem

We are used to solve problems through analysis. To analyze means to split a problem into parts, and trying to understand the whole by understanding each part separately. The idea is that understanding the parts enables us to understand the whole. This way of viewing the world is called reductionism, and it has been the dominating way of viewing the world in Western culture for hundreds of years.

Analysis is indeed a powerful tool, but it has limits. When we try to understand a system by examining its parts, we loose the full picture. This can lead us to making bad decisions. Here is a classic example, the Monty Hall problem:1

You are a contestant in a game show. The game show host, Monty, puts three cards on a table. All cards are face down. Monty tells you that one of the cards is a winning card, the other two are blanks.

Monty asks you to pick a card, but keep it face down on the table. Then, Monty flips one of the two remaining cards over. It’s a blank.

Now you get to choose: You can either stick with the card you have, or you throw it away and pick the remaining card on the table.

Think it over before you turn the page and read the answer. What would you do, and why? Does it matter what you choose?

Figure: A Future Reality Tree with the solution to the Monty Hall problem.

The diagram above has the solution. Read the diagram in the direction of the arrows, from the bottom, and up, to the top.

The diagram is called a Future Reality Tree (FRT) and it is one of the tools you will learn from this book. It is fairly easy to read the diagram even if you have never seen a Future Reality Tree before.

Boxes with rounded corners contain text describing reality as it is right now. Boxes with straight corners describe things we do to change reality. The ellipses mean and. No ellipsis means the content of any one box can cause the effect on its own. Note that in a complete tree, all the boxes must describe things that actually do exist. Speculation and guesswork should not be present in the diagram itself.

The four boxes at the bottom of the diagram are read like this:

If there are three cards on the table face down and only one of the cards is a winning card, and I pick a card, but I do not look at the face side, then there is a 1/3 chance I have picked a winning card.

Analysis can easily lead you astray when you are dealing with anything even slightly complicated. If you break the problem down in separate parts, you end up in a situation where you have two cards to pick from, the one in your hand and the only remaining card on the table. It is easy to conclude that there is a 50-50 chance of winning, regardless of whether you switch cards or not. Because of this, most people choose to keep the card they picked at the start.

If you follow the diagram all the way to the top, you will see that after Monty has flipped a blank card, there is a 2/3 chance that the remaining card on the table is the winning card!

Really! Please do not take my word for it, after all, I could be wrong, or even trying to trick you. Walk through the diagram yourself. Even better, go get a deck of cards, and try it out a couple of times.

The Future Reality Tree helps us combine two different ways of thinking, analysis, and synthesis:

  Analysis is breaking down the problem into parts. That is what I did when I split the problem up in little boxes.

  Synthesis means putting different ideas together to form a whole. I did that when I organized the boxes, and drew arrows and ellipses.

You can think of it as laying a puzzle. Analysis is when you look at the shape and color of each piece. Synthesis is when you put the pieces together, so we can see the entire picture.

In this book, I use one of the best methods I know to understand complicated problems, The Logical Thinking Process (TLTP). I explain how you can use The Logical Thinking Process to solve problems and develop both strategies and tactics in Part C: Navigation.

It is fairly easy to read TLTP diagrams even if you do not yet know how to create them, so I’ll use them whenever I want to be extra careful to show cause and effect, and to build logical arguments.

Like all other tools, TLTP diagrams are based on certain assumptions of how things work, and that means they have certain limitations. We will have a look at those limitations later on, when we get to complexity theory and Cynefin. The reason I mention the limitations now, is I do not want you to believe that just because I show you a really great hammer, all problems are nails. We will deal with both nail-shaped, and not-nail-shaped problems and solutions in this book.


  Analysis is when you break a problem into smaller pieces to try to understand each piece.

  Synthesis is when you organize and connect pieces of a problem, so you get an understanding of the whole.

  We need analysis and synthesis together to understand and solve complicated problems.

  Visual methods, like The Logical Thinking Process can be a great help when analyzing and synthesizing problems and solutions.

  Just because you have a hammer, not every problem will be a nail. The tools in this book has limitations, and to use them well, you must understand not just how and when they work, but also when they do not work.

1 The Monty Hall problem got its name from Maurice “Monty Hall” Halperin, who led the game show Let’s Make a Deal in the sixties and seventies.