Going agile the wrong way - How enterprise adaptation hurts agile software development

The topic of how to make agile and enterprise organizations fit together better has garnered some interest. No matter what your position on the issue, if you blog about it, and tell me, I will list a link to your post here. I do believe this is an important issue:

• Pascal Pinck wrote some thoughtful comments on his blog Hanzatsu.

Now on to my post:

There is a rather active thread in the Agile Alliance forum on LinkedIn regarding the Agile Manifesto and whether it is in need of an overhaul. The premise is that enterprise organizations have been slow to adopt agile methodologies, and therefore basic agile principles should be revised in order to make adoption easier.

I disagree strongly. While I can see ways in which the principles behind the Agile Manifesto can be improved, I believe increased enterprise compatibility is the wrong goal to set.

Here is one of my posts in the thread:

@Alan - I liked the blog post. (Alan Shalloway's blog post is here.)

@Everyone - I looked through the reference sections of two of my oldest books about agile this morning: Beck's eXtreme Programming and Highsmith's Agile Software Development Ecologies.

Both Beck and Highsmith were inspired by Systems Thinking. (The 2nd ed. of eXtreme Programming explicitly mentions TOC as a source of inspiration.)

That was a good start, but somehow it never caught on. Instead, the focus turned inwards, to the team. Interaction between the team and its super-system has received very little attention.

Highsmith made the observation that while agile can fit a broad range of organizational cultures, there is one type that does not fit very well at all: Command & Control culture. Highsmith was correct. Agile and C&C culture fit very poorly. This is partly because of different mind-sets, but also for structural reasons.

C&C structures are hierarchical. Hierarchical structures cannot accommodate self-organization and mission orientation very well. Agile teams are supposed to self-organize. They are also supposed to be mission oriented. (C&C hierarchies and mission oriented organizations have incompatible command structures - commands flow in different directions during missions/projects.)

Thus, Agile teams make a good fit with Three Ring organizations (Semco), Lattice organizations (Gore & Associates) and Boydian networks (BNI is the closest I can think of in the business world at the moment), but a lousy fit with traditional C&C hierarchical organizations.

There are also, as Alan pointed out in his blog post, issues with flow. Agile teams are (supposed to be) geared to global optimization of flow. C&C hierarchies are intended to achieve local optimization of resource utilization. (Interestingly, this means Agile projects must execute within the context of portfolio management, something many enterprises do very badly. There are some brilliant exceptions, of course.)

C&C hierarchies have worked well for many years, but ultimately they are a dead end. When the complexity of the environment and the rate of change in the environment increases, C&C hierarchies find it more and more difficult to cope. No big surprise, because C&C hierarchies are designed for relatively static environments. A bit simplified, they are built to sacrifice economy of speed for economy of scale.

Today economy of speed is gaining in importance. Economy of scale is losing importance. (Toyota vs. most other car companies, print-on-demand vs. traditional publishing, fashion industry...)

Trying to adapt agile to fit C&C hierarchies will inevitably rob agile of the properties that make agile competitive in the super-system outside the enterprise. Most (but not all) enterprises are a poor fit with agile and with their own super-system.

Agile is supposed to be adaptive to customer demand. That does not mean agile should be adaptive to enterprises that are themselves not very adaptive to customer demand.

Trying to do both will lead to a failure to achieve either.

BTW, there are two things I would have liked to see in the Agile Manifesto: Explicit commitments to continuous learning and Systems Thinking.

Here is a follow-up to the first post:

It is worth noting that the Agile Manifesto is not a declaration of agile principles per sé. The principles are here: http://agilemanifesto.org/principles.html (Until this point, no-one in the thread had made the distinction between the text in the manifesto and the principles the text is based upon.)

If you look at the principles you can easily see there is a problem with fitting agile to C&C, for example:

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

C&C ideas about motivation are Tayloristic. This is reflected in companies in various ways, for example by various kind of monetary reward programs. Today many of these ideas have been disproved by scientific research. For example we know today that when you offer monetary rewards to knowledge workers (CEOs as well as software developers) you stimulate a brain center, Nucleus Accumbens, the pleasure center. And we know that when N. Accumbens gets stimulated, the ability to solve complicated problems drops. A lot. (Some books that touches on the subject: Sway, Predictably Irrational, Brain Rules, Punished by Rewards. I haven't read Daniel Pink's latest yet, but it looks promising.)

About "trust" in the second sentence: In addition to achieving economy of scale an important reason for using C&C hierarchies is to impose narrow spans of control for individuals, and enable extensive control mechanisms. You do this because you do not trust subordinates.

The problem is that this environment is not only engendered to replace trust with control, it also actively reduces trust in various ways. The division of labor model, as well as authority flow and information flow structures make it difficult to build trust among people in C&C hierarchies.

For example, compare a department in an enterprise with a BNI team. (BNI is a large business referral network, about 125,000 people. I am a member.):

In a BNI team leadership rotates. All members take turns leading the team. We have a shared common goal, designed so that each team member benefits from helping other members. We are evaluated on how much we support other members. Team members do active trust building exercises, and actively seek opportunities to get to know each other outside of team activities.

We do these things because we need to trust each other to be effective. Compare this to, say, the forced ranking systems used in many enterprises. See http://kallokain.blogspot.com/2009/06/performance-evaluations-business.html

Which type of environment do you believe to be more conducive to effective knowledge work and close collaboration? Which environment do you believe is most effective in motivating members of the organization?

The agile principles can be improved upon. The problem is that if you improve them to be more in line with science (physics, queueing theory, statistics, neuroscience, psychology, systems thinking), you make agile _less_ compatible with most enterprises.

If you improve the principles for the purpose of increasing compatibility with enterprises, you will have to remove a significant portion of the scientific basis. You will also make agile less responsive to customer's needs and desires.

The solution, I believe, is to redesign the enterprises. Most enterprises are built on old paradigms. Those paradigms have become obsolete. They are no longer compatible with the environments in which the enterprises exist.