Can Starting with Waterfall Lead to Better Agile?

Guest Post by Howard Wiener (first posted on CERM ® RISK INSIGHTS – reposted here with permission)

We should all be aware by now that business agility is the primary enabler for companies seeking sustainability. In the past, companies would evolve in chunks, a project at a time. The costs and risks of implementing change, whether system-related or otherwise, were large, so designing and planning upfront (the Waterfall approach) was applied to create predictability and moderate risks.

Unfortunately, a side-effect of Waterfall was often to constrain companies’ ability to pivot to respond to changing circumstances at speed. Around the turn of this century, Agile arose and was adopted by many companies seeking a better way to become responsive to a changing environment. Now much evolution takes place as a series of small improvements in rapid succession.

Dissonance immediately formed around the desire for predictability and control and the need for speed and flexibility. Waterfall (and traditional project management methods) and Agile were and are seen as two opposite ends of a spectrum.

Waterfall:

Assumes that the business case on which the investment in a project is predicated is and will remain valid,
Is predicated on knowing nearly everything required to build a solution before planning and implementing it, similar to a plan for a building,
Is usually associated with a legacy funding model in which a project’s scope, budget and schedule (the ‘triple constraint’) is approved on an all-or-nothing basis,
Employs project management techniques that focus on completing within the triple constraint parameters established at the time the project plan was made and
Presumes a fixed lifecycle process that prescribes activities and deliverables with little allowance for deviation, regardless of what is learned or how the validity of the business case may change along the way.

Agile (at least in its initial incarnation):

Is often funded and executed similarly to Waterfall projects, i.e., initiatives are defined within a scope, schedule, budget constraint and progress is monitored with traditional techniques,
Is often predicated on a business case, the ongoing validity of which is not revisited regularly as initiatives progress,
Begins with a focus on articulating requirements for the solution that will be built (user stories entered into the backlog) but accommodates review and evolution along the way,
Glosses over many elements of architecture, such as data architecture, and non-functional requirements,
Is usually managed in short sprints of two to four weeks, with a focus on completing a pre-determined amount of work in each sprint and then building and making the solution available to users for evaluation and
Has a number of fixed activities and ceremonies, many of which divert developers’ time and are of questionable value.

We have learned that this form of Agile isn’t agile; it’s WaterScrumFall and it does not lead to the functional agility that will provide competitive advantage or sustainability to companies that employ it.

To be blunt, if you wrap an Agile process in something not Agile, it isn’t Agile anymore.

Managing in a Different Context

Managing a business now is very different than it was when Agile was codified in the Manifesto. Things happen much more quickly, and technology advancements have empowered new and smaller competitors to become viable risks to your business. Innumerable authorities have advocated an approach in which a portfolio of prospective business ideas is maintained, matured and experimented with in order to identify potential winners. Two of these are Eric Ries and Strategyzer. I should note, here, that potential investments in the evolution of existing products should be treated similarly; they should occupy a place in the portfolio with all of the from-scratch ideas and their relative risks and value should be assessed in competition with them.

This is a major shift from traditional project funding. If you adopt this approach you will be experimenting. You will be operating as Venture Capital investors do—they make a number of bets to spread risk and increase the odds of getting a win. You will invariably invest in many initiatives that generate experience and knowledge but produce no financial payoff and, hopefully, a few that generate business success. Unlike traditional projects, which assume that completion within parameters equals success, this sort of experimentation has no such predication. You will need to monitor each of your initiatives, frequently checking to see whether it still has the potential to produce a business success or not. If at any point in the execution you determine that it does not, you should either redesign it to increase its chance for success or terminate the effort and reallocate your resources as quickly as possible.

Early termination due to invalidation of the business case in the context of a traditional project would be viewed as a terrible failure. From the standpoint of a WaterScrumFall project, it would be viewed similarly. Many political and other factors conspire to inhibit initiative managers’ willingness admit failure, so they soldier on to create Zombie projects, wasting money, time, effort, resources and generating opportunity costs.

Of course, it’s also possible that you might ascertain that an initiative is demonstrating signs of success, in which case, you may want to take steps to exploit it, such as accelerating or expanding it.

To be successful, you must equip yourself to recognize signs of initiative success or failure at the earliest possible moment, and the quickest way to do this is to establish detailed criteria at its outset. This primes the pump of the OODA loop—Observe, Orient, Decide and Act. If you have articulated what signs you will be looking for, what conditions will move you to action and what action you will take in response, you will act and react faster. In addition to accelerating your decision processes, you will have also established a framework that can guide teams to apply what they discover in the course of implementation to change course when it makes sense to.

The linearity and regimentation of the Waterfall approach are anathema to Agile practitioners, yet there are many elements in traditional Agile frameworks and approaches that impair agility. Executives that approve investments in developing or evolving products are loathe to treat them on a fire-and-forget basis. They want to, need to and deserve to have control over their investments. The best way to do that is to establish a framework of expectations and understanding around what is being done so that intelligent decisions can be made over the course of their execution. Everyone involved in an initiative should understand what was assumed, what was known, what was unknown, what must be confirmed or refuted and what is expected from the effort from the time it is initiated until it is completed.

This is where elements of the Waterfall approach may be usefully applied at the beginning of an Agile initiative’s lifecycle, and we will look into this in some detail in the following article.