Truth, Damned Truth, and Statistics, Part 2
This is part 2 in a series of articles. You can read the first part if you click here.
In the previous article in this series, I discussed how to measure and visualize the Throughput part of the Return On Investment (ROI) equation. This time I'll focus on Inventory (I).
As you may recall, Inventory is defined as "money tied up in the system". Inventory includes all the equipment you use in a development project, computers, software, chairs, tables, etc. In a manufacturing process, Inventory would also include the partially finished material being worked on in the process itself, the Work-In-Progress, or WIP.
In a software development process there is no WIP, but there is an analog: partially finished software. This includes requirements, design specifications, and any code that is not fully finished, tested, and ready for release. Partially finished software is sometimes called Design-In-Progress, or DIP. The DIP has a monetary value. The value of the DIP is the amount of money lost producing DIP that is never used because of requirements changes, plus the cost of removing partially finished software from the system. This cost can be quite high.
The less DIP there is, the less risk we take that requirements changes will cause the project to waste effort.
We cannot have zero DIP, because then the project team would have nothing to work on, but it is obvious we want to keep the DIP as low as possible.
There are two basic approaches to entering material into a production process: push systems, and pull systems. As it turns out, these two models have vastly different effects on the DIP.
A push model, as used in RUP and other traditional software development methodologies, means that each step in the production chain pushes material to the next step: a project leader assigns work to team members, analysts work as fast as possible to push work to designers, designers push to programmers, programmers push to testers.
Push systems designed to be cost cost efficient, that is, they are an attempt to maximize the number of work items per hour, per person. Unfortunately, the production capacity of different parts of a production process is never balanced. Some parts will have higher capacity than others. In addition, the capacity varies. As a result, queues with DIP will build up in the process. Perhaps the testers can't keep up, or the DBA can't keep up with the programmers, or the analyst finished a truck load of requirements before the designers even got started.
In a pull system, each step in the production chain signals the step before when it is ready to take on more material. Work is pulled into the system no faster than it can be handled. This keeps the DIP to a minimum. The most well know pull system technique is called Kanban. It is the technique used by Extreme Programming. (Though the name "Kanban" is used very rarely.) There is a slightly different pull system model called Drum-Buffer-Rope, which is used by Feature Driven Development. All agile methodologies use pull systems. It is part of what makes them agile. (It is also one of the most misunderstood parts of agile.)
Figure 1: Design-In-Progress in Agile and Traditional Projects.
Figure 1 shows how DIP builds up in two projects. The agile project, using a pull model, never processes more than five stories concurrently. From this it is possible to surmise that the team either consists of five solo developers, or ten developers working in pairs. The process runs smoothly, with the DIP winding down to zero at the end of each iteration.
The non-agile project is different. DIP is allowed to build unchecked, and gets much higher than the DIP in the agile project. Thus, the non-agile project risks to loose more money if requirements change. we can also see that in iteration two and three, DIP builds until late in the project, and then suddenly drops. There is a big batch of material building up, and getting released at the end of the project. This indicates that there is a process step at the end that can't keep up with the steps before it. This is likely to be the testers, working frantically to test what the developers produce.
If the testers are overloaded with work, and the project still makes the iteration goals every time, it is most likely that the testers have been pressured into skipping tests. This in turn may indicate that the testers are using a brute force "test everything" approach. This is bad, because it may indicate that the quality of the code the team produces is low. It is much better if the developers use defect prevention techniques (unit testing, pair programming, refactoring, etc.) to keep the code quality acceptable. Of course, it may also indicate that the testers just do not know how to test statistically significant samples. Either way, it is up to the management to step in and fix the process.
Note that the DIP does not quite reach zero at the end of an iteration. There is a backlog of unfinished work building up. This is a project destined for large delays. Traditional methodologies, like RUP, may cause enormous amounts of DIP to build, but they have no mechanism for monitoring it! This is why project delays often come as a surprise to management very late in a project. (That, and the fact that Management By Fear causes many project managers to actively hide information about problems in a project.)
How To Measure DIP
As you can see, monitoring the DIP can tell you a lot about the state of a project. Measuring the DIP is easy. In the project I am currently working in, we use an andon board, i.e. a whiteboard where we have a table with a column for each step in the development process. At the start of an iteration the team writes a sticky note for each story. At the beginning of an iteration, all stories are in the leftmost column, the backlog column. When someone begins to work on a story (s)he moves the corresponding sticky note to the next column. Eventually, each story has travelled to the Done! column.
To measure the DIP, all I have to do is to keep track of how when stories are moved from one column to the next. I use a spreadsheet to do this. Then I use a small Ruby program to a DIP graph (and several other graphs).
In the previous article in this series, I discussed how to measure and visualize the Throughput part of the Return On Investment (ROI) equation. This time I'll focus on Inventory (I).
As you may recall, Inventory is defined as "money tied up in the system". Inventory includes all the equipment you use in a development project, computers, software, chairs, tables, etc. In a manufacturing process, Inventory would also include the partially finished material being worked on in the process itself, the Work-In-Progress, or WIP.
In a software development process there is no WIP, but there is an analog: partially finished software. This includes requirements, design specifications, and any code that is not fully finished, tested, and ready for release. Partially finished software is sometimes called Design-In-Progress, or DIP. The DIP has a monetary value. The value of the DIP is the amount of money lost producing DIP that is never used because of requirements changes, plus the cost of removing partially finished software from the system. This cost can be quite high.
The less DIP there is, the less risk we take that requirements changes will cause the project to waste effort.
We cannot have zero DIP, because then the project team would have nothing to work on, but it is obvious we want to keep the DIP as low as possible.
There are two basic approaches to entering material into a production process: push systems, and pull systems. As it turns out, these two models have vastly different effects on the DIP.
A push model, as used in RUP and other traditional software development methodologies, means that each step in the production chain pushes material to the next step: a project leader assigns work to team members, analysts work as fast as possible to push work to designers, designers push to programmers, programmers push to testers.
Push systems designed to be cost cost efficient, that is, they are an attempt to maximize the number of work items per hour, per person. Unfortunately, the production capacity of different parts of a production process is never balanced. Some parts will have higher capacity than others. In addition, the capacity varies. As a result, queues with DIP will build up in the process. Perhaps the testers can't keep up, or the DBA can't keep up with the programmers, or the analyst finished a truck load of requirements before the designers even got started.
In a pull system, each step in the production chain signals the step before when it is ready to take on more material. Work is pulled into the system no faster than it can be handled. This keeps the DIP to a minimum. The most well know pull system technique is called Kanban. It is the technique used by Extreme Programming. (Though the name "Kanban" is used very rarely.) There is a slightly different pull system model called Drum-Buffer-Rope, which is used by Feature Driven Development. All agile methodologies use pull systems. It is part of what makes them agile. (It is also one of the most misunderstood parts of agile.)
Figure 1: Design-In-Progress in Agile and Traditional Projects.Figure 1 shows how DIP builds up in two projects. The agile project, using a pull model, never processes more than five stories concurrently. From this it is possible to surmise that the team either consists of five solo developers, or ten developers working in pairs. The process runs smoothly, with the DIP winding down to zero at the end of each iteration.
The non-agile project is different. DIP is allowed to build unchecked, and gets much higher than the DIP in the agile project. Thus, the non-agile project risks to loose more money if requirements change. we can also see that in iteration two and three, DIP builds until late in the project, and then suddenly drops. There is a big batch of material building up, and getting released at the end of the project. This indicates that there is a process step at the end that can't keep up with the steps before it. This is likely to be the testers, working frantically to test what the developers produce.
If the testers are overloaded with work, and the project still makes the iteration goals every time, it is most likely that the testers have been pressured into skipping tests. This in turn may indicate that the testers are using a brute force "test everything" approach. This is bad, because it may indicate that the quality of the code the team produces is low. It is much better if the developers use defect prevention techniques (unit testing, pair programming, refactoring, etc.) to keep the code quality acceptable. Of course, it may also indicate that the testers just do not know how to test statistically significant samples. Either way, it is up to the management to step in and fix the process.
Note that the DIP does not quite reach zero at the end of an iteration. There is a backlog of unfinished work building up. This is a project destined for large delays. Traditional methodologies, like RUP, may cause enormous amounts of DIP to build, but they have no mechanism for monitoring it! This is why project delays often come as a surprise to management very late in a project. (That, and the fact that Management By Fear causes many project managers to actively hide information about problems in a project.)
How To Measure DIP
As you can see, monitoring the DIP can tell you a lot about the state of a project. Measuring the DIP is easy. In the project I am currently working in, we use an andon board, i.e. a whiteboard where we have a table with a column for each step in the development process. At the start of an iteration the team writes a sticky note for each story. At the beginning of an iteration, all stories are in the leftmost column, the backlog column. When someone begins to work on a story (s)he moves the corresponding sticky note to the next column. Eventually, each story has travelled to the Done! column.
To measure the DIP, all I have to do is to keep track of how when stories are moved from one column to the next. I use a spreadsheet to do this. Then I use a small Ruby program to a DIP graph (and several other graphs).
Comments