Is Timeboxing Bad for Software Development?

I recently made a video, An Introduction to Reference Class Forecasting, where I show how to use a simple, yet reliable, alternative to estimates in software projects. In the video, I touch on the topic of timeboxing, and why it can have the opposite of the desired effects in software development.

Timeboxing is generally accepted as a good, even indispensable thing. It is an integral part of Scrum, and other Agile methodologies. (Even the ones I like a lot.) It is worth going into a little bit of detail about how, and why, timeboxing can backfire.

What is timeboxing?

Timeboxing is a project planning technique where a fixed time period is allocated to an activity. For example, in Scrum, Sprints, Sprint Planning, Sprint Review, and Daily Scrum, are all timeboxed.

If we look at other Agile methodologies, like Extreme Programming, Lean Software, Development, and Dynamic systems Development Method (DSDM), they use timeboxing in some shape or form. Kanban is an exception.

Note that in Lean Software Development, iterations are not timeboxed!Timeboxing is used for large releases. This means, you can have several variable length iterations inside a timeboxed release schedule, for example.

When we timebox iterations and increments in Agile, we also use variable scope. This is necessary, because we cannot predict exactly how long our work packages will take.

In the case of Scrum, this means it must always be possible to remove sprint backlog items without compromising the Sprint Goal.

If you have worked in a Scrum team, this is the point where you may start to become slightly suspicious. Is the scope of you Sprint Goal always negotiable?

The advantages of timeboxing

There are a number of advantages to timeboxing. Here are some that are commonly listed:

• Fight procrastrination
• Focus on things that matter
• Create a feeling of achievement
• Productivity boost, provided that you can work uninterrupted for the duration of the timebox

If we have a short timebox, like a fifteen minute daily Scrum, a 25 minute Pomodoro, or a two hour Sprint Retrospective, this usually works.

Note that all of the examples I have given have fairly short time periods. A couple of hours, at most.

The disadvantages of timeboxing

Timeboxing also has disadvantages. Too tight timeboxes can:

• create pressure to do shoddy work
• create feelings of never being able to catch up
• disrupt a pull process

If a team struggles to finish work within a timeboxed Sprint, this can create pressure to skimp on writing tests, refactoring code, and generally thinking problems through. This creates technical debt, and technical debt, is just another name for economic debt. Sooner or later, this debt will have to be paid, often with a very high interest rate.

Note that technical debt increases the risk of unforeseeable delays. This means failing to plan a timebox accurately, increases the risk of failing to plan the next timebox accurately, and so on…

If timeboxes are missed repeatedly, this can create a feeling of losing control, of never being able to catch up. I have seen many Scrum teams that have been demoralized because they can never finish the work they have taken on in a Sprint.

The natural response is to spend more time and effort making time estimates. This leads to a downwards spiral, because the problem is not poor estimation. The problem is random variation in the process. This is a solvable problem, but it cannot be solved with time estimates.

In this situation, time estimates are the wrong thing to do!

Finally, Agile frameworks and methodologies use pull processes, and these put developers, and everyone else, in control of how much work they do, and when they do the work.

In pull processes, production is triggered based on current capacity and demand. That is, a developer has to be free to start new work, and there must be something available for the developer to do. An example is when a developer pulls a User Story from the Sprint backlog and begins to work. Work is available, and the developer is in control.

Pull processes are a good idea if we have a process with a lot of inherent variation. They are very good at controlling the amount of Work-In-Process (WIP). The WIP is directly proportional to lead time. (Little’s Law, for those of you interested in queuing theory.)

Timeboxing, on the other hand, requires that we anticipate future capacity and demand. This makes timeboxing a push process. Push processes work when we have almost no variation in the process. Even a little variation can make a push process go wildly out of control.

Putting a high variation pull process inside a timebox designed to create a push process for low variation work, creates a conflict between the process models.

In Scrum, and several other Agile frameworks and methodologies, we put a pull process (working with a Sprint backlog and WIP limits) inside a push process (estimating how much work we can do in a timeboxed Sprint).

If a team has taken on more work than it can finish in a Sprint, this creates pressure on developers to take on more work than they can handle, and thus disrupts the pull process!

At this point, we can sum up:

• If whatever we want to do has a predictable duration, or we have a high degree of control over the scope of what we want to do, then timeboxing has a number of advantages.
  • The requirement to fulfill a Sprint Goal limits our ability to reduce scope.
• If the durations of the things we want to do in the timebox are highly unpredictable, then timeboxing has a number of disadvantages.

It is generally easier to predict how long it takes to do a little bit of work, than it is to predict how long it takes to predict large chunks of work. We can conclude that:

• Small timeboxes, like a 25 minute Pomodoro, or a three hour Sprint Retrospective, are fairly easy to timebox.
• Large timeboxes, like a Sprint, a SAFe Program Increment, or a project, are inherently more difficult to plan than small timeboxes.
• Whether timeboxes are good or bad depends on:
  • The size of the timebox
  • The variation inherent in the work you are planning

By now, I think you can see where this is going.

Can we estimate Sprints?



The correlation between estimates and cycle times is nearly always so low the estimates have no predictive value.

When I work with teams, one of the things I do, is checking how well their Sprint Planning has worked in the past. Here are two easy methods:

• Create a scatter plot with estimates of User Stories on the X-axis, and actual durations on the Y-axis. (Don’t tell me “we estimate effort, not time”! You are doing capacity planning, so the purpose of estimating is to figure out how much work you can do in a timebox, the Sprint.) If the dots in the plot form a line, or close to it, your estimates work. If you see randomly scattered dots, your estimates do not work. I have made a whole video about it. (Not the Reference Forecasting video this time. It’s another one.)
• You can also calculate the statistical correlation coefficient in Excel. There is material about it in my Reference Class Forecasting video.

In most cases, you will find that there is little, or no, correlation between estimates and actual durations. In general, the answer is no, we cannot estimate Sprints in a meaningful way! There is too much random variation in the development process.

The implementation time of a User Story is a probability range, not a number!



The implementation time of a User Story, is not a number. It is a probability range. In the graph, there is a range of different probabilities over a range of 2-62 days.

So, we cannot say how long it will take to implement a User Story, because of the inherent randomness. On the other hand, we do know that we can implement most User Stories.

This means there must exist a range of possible durations. At each point on the range, there exists a probability that the User Story will be finished, so what we have, is a probability range.

The good news is that, if we have data on how long it takes to implement User Stories, or whatever type of requirements we use, we can decide what probability we want of having a correct answer, and then calculate how long it takes to implement a User Story. We can do this with Reference Class Forecasting.

Better yet, it is easier to get a good answer with Reference Class Forecasting, than it is to get a bad answer with various estimation techniques. I know I’m getting a bit repetitive, but I’ll plug my Reference Class Forecasting video again. It has details, and a demo.

We should be a bit cautious here, though, because, it turns out we can eliminate the timeboxed Sprint altogether.

Eliminating the timebox (and the Sprint Backlog)

Let’s have a look at what the timebox is supposed to do for us again:

• Fight procrastrination
• Focus on things that matter
• Create a feeling of achievement
• Productivity boost

We already have another box that does that, a unit of work-box. What’s the unit of work? Any vertically sliced unit of work will do for our purposes, provided that it is fairly small.

A User Story fits the bill very well. It helps the developer :

• Fight procrastrination — The developer pulls the User Story from the backlog, and gets to work. This is even more effective if developers pair-program, or work n a troika.
• Focus on things that matter — The developer does what is needed to implement the User Story, nothing else.
• Create a feeling of achievement — When a User Story is Done, the developer has made real progress, and finished a thing that actually works from the perspective of a user. This is a real achievement, as opposed to “I clocked in a two week Sprint again”.
• Productivity boost — The tight focus on implementing the User Story reduces the time spent on other things, with a resulting boost in productivity.

In addition, using a pull process allows the developer to be in control! She/he can pull new work from the backlog, when there is free capacity, and finish it without undue time pressure. This makes it easier to maintain high quality, which means less technical debt than you get with a timebox.

The difference from timeboxing, is that the box is a unit of work instead of a unit of time.

Oh, since we have scrapped timeboxing, we do not need the Sprint Backlog. We can pull directly from the Product Backlog.

Conclusion

Whether timeboxing is bad, or good, for you, depends on context!

Timeboxes are primarily useful for short duration, predictable, activities, like pomodoro sessions, and meetings. In such cases, they can increase focus, contribute to a feeling of achievement, and give real productivity boosts.

For activities of longer duration, and more inherent variation, timeboxing is often counterproductive. Timeboxing iterations, and Sprints, in software development is, in most cases, a bad idea. It creates pressure to do shoddy work. Planning failures are inevitable, and these failures are often demoralizing. Timeboxing can also torpedo pull processes, which leads to increased lead times, increased cost, and unnecessarily increased pressure on developers.

Looking at the Agile Manifesto credo:

“We are uncovering better ways of developing
software by doing it and helping others do it.”

…we still have a lot of “uncovering better ways” to do. Getting rid of timeboxed Sprints is one thing we can do fairly easily.