Are You Still Using the Wrong Control Levers in your Agile projects? Part 2: Business Value, it's Use and Abuse
In the first part of this article series, I wrote about how using Cost and Capacity to control an agile software project can trap an organization in a hire and fire cycle that increases project duration and cost.
This time, we will take a closer look at an agile control lever that works very well, except when it doesn’t: Business Value.
When used right, the Business Value lever can be the most powerful tool you have to steer an agile project or program towards success. When used wrong, and it often is, the Business Value lever can be completely disabled, leaving management to pull a lever that no longer works, and no longer has the ability to steer the project.
How to Deliver Business Value the Agile Way
Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.
— Principles behind the Agile Manifesto, https://agilemanifesto.org/principles.html
Let’s start by looking at how the Business Value lever is supposed to work:
Most of the original agile methods, often called lightweight methods, were designed for small systems development. There was usually a single team, and that team built something that could be sliced up in many small deliveries that had value to a customer. For example, a website, or a payroll system, can be built and delivered with minimal functionality. After that, new functionality can be delivered in small increments at relatively short intervals.
In order to deliver in small increments, you need a way of slicing up the application you are building. Thus, the User Story was born! A User Story is a short, non-technical description of something a user wants to do, written from the user’s perspective. The User Story was not the only way, nor the first, to describe vertical slices, little pieces of functionality that worked end-to-end, but it became the most popular.
With Critical Path and Waterfall methods, if you had a twelve month project, you would not see any business value from the project until after 12 months. With an agile method, you would get something of value after one or two months, and then, with each delivery, the business value would increase.
Getting business value sooner meant making money sooner. If you have ever played around with a Profit & Loss statement, you know that shipping working software just a little bit faster can have enormous impact on the total business value over the product life cycle.
Critical Path became obsolete, replaced by User Stories, and the idea of shipping a minimal product as quickly as possible, followed by incremental increases in functionality.
The Decline and Fall of the User Story
…user stories are based on real customers who will use the product once it’s released. As such, the Product Owner may choose to talk to potential customers in order to get a feel for what they want. There could be focus group discussions, interviews and any other kind of research needed in order to gain the intelligence needed to create a viable user story.
— Cohn, Jeff. Scrum Fundamentals: A Beginner’s Guide to Mastery of The Scrum Project Management Methodology
In the beginning User Stories worked very well, but then something happened: Scrum became the dominant agile method! Scrum did away with the idea that an end user should write the User Stories, and inserted an intermediary, the Product Owner.
Note the phrase “the Product Owner may choose to talk to potential customers” in the quote above. The Product Owner should talk to end users, but isn’t required to. In many large organizations, it’s difficult for a Product Owner to even find an end user.
With most original agile methods, like eXtreme Programming (Kent Beck), Crystal (Alistair Cockburn), and Lean Software Development (Mary and Tom Poppendieck), it was explicit that the organization would have to change in order to support agile teams. One such change would be tearing down organizational barriers, so the development team could meet and talk to users directly.
It is worth noting that some agile methods, notably Crystal, took a more scalable approach to requirements management. Crystal used lightweight use cases instead of user stories. Use cases also represent vertical slices of functionality. However, they capture a lot more information than user stories, while still being reasonably quick and easy to create. The use cases were created by a troika, an expert user, a business expert, and a designer/developer. Crystal was explicit about the high degree of expertise needed to create good requirements.
Scrum took a different path. Instead of changing the organization, Scrum inserted the Product Owner as an intermediary, isolating the development team from the organization around it. This was attractive to management in many organizations, because they did not have to change how they worked. Instead, the team could be treated more or less like a black box.
Isolating development teams from users came at a cost:
Loss of information!
In some organizations, the Product Owner works in close contact with the users. In those organizations, inserting an extra degree of separation might not matter much. In other organizations, where there is less contact between Product Owner and end users, the information loss may be substantial, and significantly reduce business value.
Note that User Stories are not complete requirements. Originally, they were intended as placeholders for conversations between developers and end users. With Scrum, developers will have those conversations with the Product Owner instead. For this to work well, the Product Owner must have both deep and broad knowledge of the end user’s area of expertise! However, the Official Scrum Guide does not require Product Owners to have any such expertise. Instead, Scrum focused on accountability:
The Product Owner is accountable for maximizing the value of the product resulting from the work of the Scrum Team. How this is done may vary widely across organizations, Scrum Teams, and individuals.
— The Scrum Guide, p. 5, https://scrumguides.org/download.html
The Scrum Guide goes into more detail about the areas of accountability, but is silent on the corresponding skills needed to do the job well. This focus on areas of accountability in lieu of skills is not limited to the Product Owner role. All other roles in Scrum are defined the same way.
Eliminating skill requirements contributed to making Scrum an easy sell. Unfortunately, the lack of skill requirements lead to an overall reduction in skill levels for all Scrum roles, compared to other agile methods.
Why would user stories being vertically sliced matter?
— Agile Way of Working responsible in a major SAFe program
Many, perhaps most, Product Owners did not get much training in capturing requirements. Nor did they learn why vertical slicing was important. At the same time, they had to do a difficult job. Instead of an end user writing a story, a Product Owner had to imagine what an end user would write, and then write that. To make it worse, the Product Owner often had to do it without expertise in the subject matter, and without expertise in the relevant business processes. The Product Owner job became very difficult indeed, and that could not help but reduce the quality of User Stories.
The problem with this is that we do not have anything that structures the tasks into vertical slices of functionality anymore. Product Owners were supposed to organize the work order from the item with the highest business value to the item with the lowest, but tasks don’t have business value!
Business value is an emergent property of a vertical slice of functionality, or a set of vertical slices of functionality! That rarely stopped anyone from trying to do the impossible though. I have occasionally met Product Owners, and other people, who suspect that something fishy is going on with how we write requirements, but they are few and far between.
So, why does it matter whether a team uses User Stories or tasks?
Whenever I see this phenomenon, teams still do deliveries, and still do demos. It’s just that they do internal deliveries, and the demos are Powerpoint presentations, or demos of some internal function of interest only to developers. End users cannot use the software in the internal delivery. End users are never invited to these demos.
Companies that do this can be even worse off than they were before agile, because they first got rid of the Critical Path based project management they had before, and then disabled the vertical slicing which was supposed to replace Critical Path management.
Yeah. And don't try to tell us there is no way to go but up because the truth is, there is always more down.
— Gunn, in the Angel TV-series episode Happy Anniversary
Confusing user stories with tasks causes big enough problems when doing small systems development. What happens when you scale up to large projects with many teams?
If front facing teams worked with real User Stories, then we could break those stories down into tasks that supporting teams worked on, and continue producing vertical slices of software.
We could also design the system architecture so that each team can handle a vertical slice of functionality.
In most cases, we would have to balance the two approaches.
What I have seen happening though, is that everyone keeps working on tasks. Architectural dependencies are not resolved. Team topology is ignored. Work consequently slows down, a lot!
I worked in one large project some years ago where deliveries were slowed down by a factor of 50! In other words, what should have been a two week user story took two years to implement instead.
Most other projects I have seen have had similar problems.
Mirror, Mirror on the Wall, Tell Me Which is the Most Fake User Story of Them All
As far as I know, no method has done more to confuse the issue of user stories versus tasks than the Scaled Agile Framework (SAFe).
SAFe has a requirements model with two kinds of stories: User Stories, and enabler stories.
User stories are, according to SAFe, vertical slices of functionality, but SAFe does not mention why that is important, or what happens to SAFe’s own requirements prioritization system if they are not. Then SAFe tops it off by introducing enabler stories, which are a kind of tasks.
Hierarchically, both user stories and enabler stories are the children of features. There are two kinds of features, business features, which are vertical slices of functionality, and enabler features which are a kind of tasks created by architects. Both kinds of features may contain either kind of story.
Moving up the hierarchy to business capabilities and enabler capabilities, both of them may contain both business and enabler features.
The top level is the epic. There are three main types, portfolio epic, solution epic, and program epic. Any type of epic may be of one of two subtypes, business epic and enabler epic, which gives you a total of six different kinds of epic.
Any kind of epic may contain any kind of capability, and any kind of feature. Any of these work items may be standalone. You may, for example, have enabler stories and enabler features who are standalone and not connected to anything that has any business value.
It’s a mess! I don’t think anyone has a grip on it. I may be wrong, but I haven’t met anyone who has figured it out yet.
What’s Bad for the Geese, May Be Good for the Gander
All of the confusion about agile requirements is bad for the industry as a whole, but it is actually an opportunity for those who manage to fix their user stories. When user stories describe vertical slices of functionality, the Business Value lever starts working, and that means the basic mechanism for making small, fast deliveries is working.
That, in turn, makes it possible to use other control levers more effectively.
Comments