Friday, August 18, 2006
Sunday, August 13, 2006
Systems Archetype: Shifting the Burden to the Intervenor
There is a special case of Shifting the Burden which I am especially familiar with because I have worked as a consultant for many years: Shifting the Burden to the Intervenor.
The problem may occur when an organization decides to rely on outside competence to solve an internal problem. The problem may be solved. The catch is that the competence to deal with the internal problem now resides outside the organization. This makes the organization vulnerable if the problem occurs again.
"The organization" may be a company that outsources development work to a consultant, or outsources a part of their infrastructure to a subcontractor. Departments within an organization are also vulnerable. They may (not always voluntarily) let another part of the organization take over an important function, only to find that the other part of the organization is not able to process requests quickly (usually because of queues), may not fully understand the requests (development department vs. IT department), or may see some political advantage in not fulfilling a request.
A company needs to get a new product to market fast. There may be an economic, or strategic need, or political pressure. The company outsources development to consultants, quite often from several different companies. The consultants are done on time (well, this part usually does not happen,) but the knowledge about how the system works resides in the heads of people from other companies.
When the company needs to maintain the application, or develop a new version, it does not have the capability to do so in an efficient manner. The people who worked on the original project has dispersed, and now work on other projects, for other customers. Assembling the original team, or even a fraction of it, is not possible.
The total cost, over a few years, of outsourcing the work may be far greater than keeping the necessary expertise on staff.
Any situation where outsourcing is considered should raise an alarm. That does not mean that outsourcing is always bad, just that the consequences should be explored thoroughly. One way to do that is with a Future Reality Tree (FRT).
As with Shifting the Burden, I would like to stress the importance of making appropriate (I.e. measuring the right thing is more important than measuring with great precision.) measurements once an outsourcing plan is under way. In addition to the measurements mentioned in Shifting the Burden, it is a good thing to keep a close eye on code quality. If the quality of the code drops, development costs, in current and future projects, will rise dramatically. I have seen many companies where management are not aware of the implications of poor code quality. Software projects at these companies are always extremely expensive, deadlines are often missed, and the return on investment of the projects is much less than it could be.
Shifting the Burden to the Intervenor is often built right into the organizational structure. The most common structure today is the functional organization. In functional organizations, work is divided according to functions. For example, there may be specialized departments for IT services, methodology, and so on. This causes a number of problems, but the one we are concerned with here, is the loss of competence in software develpment departments. I have worked in development projects where nobody knows how to set up a web server, or a database server, or other important pieces of equipment. What happens is that the project organization is forced to make a request to the appropriate department, and then they will have to wait until the request is processed. In this way, even a relatively simple job can stretch out several weeks, months, or may never get done at all.
There is no "grunt work" in software development. Writing software is all about designing highly complex systems, and writing very detailed specifications describing those systems. The grunt work, building software according to design specifications, is something compilers and interpreters do, never humans. Every developer in a software development organization is a knowledge worker, must be regarded as such, must be expected to behave as such, and must be treated as such. (For more on this, I recommend you read the classic Peopleware, and the more recent (2001) Slack.)
Organize to optimize the value stream (flow organization), not to keep people busy (functional organization)! Both Lean and Theory Of Constraints have proved, both empirically and theoretically, that this is the best way to go.
Description
The problem may occur when an organization decides to rely on outside competence to solve an internal problem. The problem may be solved. The catch is that the competence to deal with the internal problem now resides outside the organization. This makes the organization vulnerable if the problem occurs again.
"The organization" may be a company that outsources development work to a consultant, or outsources a part of their infrastructure to a subcontractor. Departments within an organization are also vulnerable. They may (not always voluntarily) let another part of the organization take over an important function, only to find that the other part of the organization is not able to process requests quickly (usually because of queues), may not fully understand the requests (development department vs. IT department), or may see some political advantage in not fulfilling a request.
Example
A company needs to get a new product to market fast. There may be an economic, or strategic need, or political pressure. The company outsources development to consultants, quite often from several different companies. The consultants are done on time (well, this part usually does not happen,) but the knowledge about how the system works resides in the heads of people from other companies.When the company needs to maintain the application, or develop a new version, it does not have the capability to do so in an efficient manner. The people who worked on the original project has dispersed, and now work on other projects, for other customers. Assembling the original team, or even a fraction of it, is not possible.
The total cost, over a few years, of outsourcing the work may be far greater than keeping the necessary expertise on staff.
Early Warning Symptoms
Quite often, management is aware that outsourcing may cause long term problems. However, the short term gains are over-estimated, and the long term costs are under-estimated.Any situation where outsourcing is considered should raise an alarm. That does not mean that outsourcing is always bad, just that the consequences should be explored thoroughly. One way to do that is with a Future Reality Tree (FRT).
As with Shifting the Burden, I would like to stress the importance of making appropriate (I.e. measuring the right thing is more important than measuring with great precision.) measurements once an outsourcing plan is under way. In addition to the measurements mentioned in Shifting the Burden, it is a good thing to keep a close eye on code quality. If the quality of the code drops, development costs, in current and future projects, will rise dramatically. I have seen many companies where management are not aware of the implications of poor code quality. Software projects at these companies are always extremely expensive, deadlines are often missed, and the return on investment of the projects is much less than it could be.
Shifting the Burden to the Intervenor is often built right into the organizational structure. The most common structure today is the functional organization. In functional organizations, work is divided according to functions. For example, there may be specialized departments for IT services, methodology, and so on. This causes a number of problems, but the one we are concerned with here, is the loss of competence in software develpment departments. I have worked in development projects where nobody knows how to set up a web server, or a database server, or other important pieces of equipment. What happens is that the project organization is forced to make a request to the appropriate department, and then they will have to wait until the request is processed. In this way, even a relatively simple job can stretch out several weeks, months, or may never get done at all.
Management Principles
Don't outsource your brains! While many managers in knowledge organizations know this, as a general principle, they are sometimes a bit confused about where the brainpower in their organization resides. They believe it is with them. Not so! Nor should it be. Knowledge organizations have most of their brain power at the base of the organizational structure, by the power of sheer numbers, if nothing else. (Besides, good managers want smart people to work for them. If I ever start a company again, I'm going to make damn sure I only hire people smarter than me.)There is no "grunt work" in software development. Writing software is all about designing highly complex systems, and writing very detailed specifications describing those systems. The grunt work, building software according to design specifications, is something compilers and interpreters do, never humans. Every developer in a software development organization is a knowledge worker, must be regarded as such, must be expected to behave as such, and must be treated as such. (For more on this, I recommend you read the classic Peopleware, and the more recent (2001) Slack.)
Organize to optimize the value stream (flow organization), not to keep people busy (functional organization)! Both Lean and Theory Of Constraints have proved, both empirically and theoretically, that this is the best way to go.
Systems Archetype: Shifting the Burden
Description
Shifting the Burden is one of the more common systems archetypes. The problem occurs when a short term fix is applied without regards for possible side effects. The short term fix seems to work, but over time the side effects escalate. In many cases, the capability to apply a long term solution atrophy over time, making it very hard to correct the problem.
Example
A classic Shifting the Burden example in software development is when management tries to apply a brute force solution, like increasing the manpower of a project, or forcing developers to work an excessive amount of overtime, to compensate for low productivity, or poor planning.
The diagram above describes a situation where a project has productivity problems. In other words, it won't make the deadline. The obvious, but often flawed, solution is to add more people to the project.
Most projects measure progress in terms of hours worked. Unless you live on Mars, you are almost certainly familiar with the basic idea: if a job has been estimated to take 100 hours, and someone has worked on it for 80 hours, then there are 20 hours of work left to do. However, this simplistic notion of progress fails to take several important factors into account: the original estimate may be wrong; statistical fluctuations (see The Variance Trap) may reduce the productivity rate; time lost to task switching will be mistakenly counted as "progress"; developers may have to spend time on non-productive tasks, which will also be counted as progress, etc.
The upshot is, management often has no idea what happens to productivity when they increase manpower. Even worse, because the number of hours spent per week does go up when more people is added, and management assumes there is a direct relationship between number of workers and productivity rate, it is very hard for them to notice any side effects. All seems to be well.
Under the surface there may be a witches brew of problems. The first thing that happens when more people are added to a project is that productivity goes down. Think of it like this: the first day on the job, a developer will not be very productive. She will need help though, setting up the development environment, being shown around, taught administrative routines, etc. Someone else, usually several someones, will have to spend time helping her to settle in. Therefore, the net productivity of the project is reduced on the first day. Over time the new developer will learn the ropes, and will gradually become more productive. This does take time though. If the system under development is complex, or just messy, there may be several months, maybe a year, before the new developer is up to speed. Until then, the project will not get the productivity increase the management hoped for.
If too many developers have been added at once, there may be major coordination problems. What should all the new people do? The system under development may not have an infrastructure that allows very many people to work on it efficiently. Management usually finds ways to keep everyone busy anyway, in the interest of "cost efficiency". The result is predictable: the software becomes incredibly messy. Consequently, changes become very hard to implement, and the defect count skyrockets. Even if there was an initial improvement in productivity, it is not sustainable.
Early Warning Symptoms
In software development, an early warning symptom is when developers begin to complain of poor quality, high complexity, or that the software "isn't object oriented" (assuming it is supposed to be). Most developers won't complain. They don't care two figs worth whether the software is a mess, as long as they get paid every month. I'm sad to say, they often do not even know the difference between high and low quality code. The people who will complain are the nerds, the people for whom software development is an all-consuming interest. (You know, the oddballs management often wish they hadn't hired in the first place.)In general, a situation where workers begin to worry, and management can't understand why, is a sign of Shifting the Burden.
Management Principle
Focus on fundamental solutions! The only reason ever to use a short term solution is to gain time necessary for implementing a fundamental solution.Of course, the trick is to be able to distinguish fundamental and short term solutions from each other. This can be difficult, but effective tools for doing that, such as the TOC Thinking Tools, do exist. All management has to do is learn to use them, and apply them. (There are no miracle cures, of course. However, I do believe that effects have causes. Therefore, using methods for reasoning about cause and effect is very useful.)
Another thing management can do is to just listen. Keep ears and minds open. Very often there are people in the organization who will recognize a false fix for what it is, and who are able to predict the long term effects very accurately.
The problem is accerbated by measuring the wrong thing, as we in the software development business are especially prone to do. Measuring the wrong thing blinds us to what is really going on, and makes it hard to take effective action. Most managers, project managers, middle management, and upper management, focus on measuring effort (hours worked, lines of code, task completion time, etc.). Measuring effort does not work, because there is no direct cause-effect relationship between effort and productivity, only a correlation. (The correlation isn't quite what most managers believe. The Critical Chain theory of business process management shows that a system yielding a high return on investment by necessity has parts that work at less than full capacity. Conversely, a system where the parts all work at full capacity, is never yielding optimal return on investment.)
I have seen several cases where management did not understand that something was wrong until years after the symptoms first occurred. Even worse, because the measurements where wrong, it was very hard for them to see what to fix.
On the other hand, productivity can be measured directly, so it is a good thing to measure. There is a direct relationship between Work-In-Process (WIP, or Inventory) and return on investment, (the ROI equation), so that is also a good thing to measure. Know those things, and you will know what really happens in a project.
Subscribe to:
Posts (Atom)