Why Large Complex Projects Often Fail
Leadership
June 10, 2011
John Ikeda
Topics
complexity, failure, Project Management, risk, take a riskYou don't have to go very far to find a failed multi-million dollar project. To be clear, let's define failure as a multi-million dollar project that ran consistently over budget and behind schedule; some came in years late. If you've been around for any length of time, I'm sure you've involved with a few. I know I've seen my fair share. To give you a few examples, here are some interesting articles on failed projects:
10 Famous ERP Disasters, Dustups and Disappointments: From CIO Magazine
Famous Failures of Complex Engineering Systems: CalTech
Catalog of Catastrophe Why Technology Project Fail: Calleam Consulting
So... why are these projects failing? I'm sure you've seen the typical answers, Poor Project Selection, Scope Creep, Bad Estimates, Poor Project Management. However, there may be another reason unrelated to project management or organization dysfunction that may surprise you...
I am in the middle of reading The Black Swan by Nassim Nicholas Taleb and while it deals with life in general, I couldn't help but relate it to project management. His basic premise is that life is unpredictable. This is not good news for project management. Don't get me wrong. As a project manager I obviously think project management can improve the project success rate. But... can it always? If the project is small, and the length of time is short, odds are that a managed project will come in as planned. But even that is not always guaranteed. Life is unpredictable. Key resources could leave, the organization could change its vision, the company could be bought out.
What about large projects? If life is unpredictable, large projects will most certainly come in late and over budget. Think about it. Name one, just one, successful project over $10 million and over a year long, that didn't run into some sort of problem that pushed the date and budget out. This is especially true in technology projects, or construction projects that deal with new technology.
So my question is, are we just kidding ourselves into thinking we can manage large complex projects, and if so, why? Well as a species, humans don't like uncertainty. We try so hard to create the illusion that life is in our control. We go about our lives in such a way as to give us the comfort that life is predictable. Bad things happen... but not to us. I don't understand why people in flood or earthquake zones are so surprised when it floods, or when there is an earthquake where they live.
If you consider our leaders in government and business, most of them are successful because they are good at creating the illusion of certainty. CEOs are hired because they WILL get things done. Politicians are elected because they WILL bring home the bacon. Executives, Managers and Project Managers are all in their position because the WILL get their teams to perform. But on large complex projects... can they... really?
My answer is no. Because we believe in success (things getting done), because we worship power and control, because bad things happen to others and not to us, we have created the illusion that large complex projects can be managed successfully.
I know that some of you will point to a successful project that will be the exception to the rule. But... could this be due to chance, and not to project management itself? Odds are that some projects will be successful, but are they the outliers? And if they are the outliers, can we really take the best practices at those projects and apply them to other projects and expect the same results? Again, on a small project, perhaps. On a larger project? I don't think so.
Some of you may be wondering if I am saying that we shouldn't manage large complex projects? Not at all. What I am proposing, is a new paradigm for looking at large complex projects. If you are considering a multi-million dollar project, there is a HUGE risk that the project will not adhere to the original plan. The question you must ask yourself is how to reduce the risk as much as possible, and to admit, from the very beginning, that the project could double in cost and time. Is the organization willing to absorb that possibility?
Executives and Project Managers will bring in charts, graphs, numbers, assesssments, all showing that the project can be done. But, on large projects, this is just an attempt at creating a sense of control. Control that we don't have. Too many things could go wrong, and they will. Accept this up front before you begin.
Why do large complex projects fail? Because life is unpredictable and the larger the project, the greater the odds of something happening that will negatively impact the project. In other words, large projects will undoubtedly be late and overbudget, because they are large.
Thoughts?
All the best!
All the time!
JT
This post was originally published over on John's Coaching Leaders blog.
Great food for thought John!
Although I would define failure differently. I think failure is not delivering on the promise of the project. Schedules and budgets are best guess estimates and need to be constantly monitored. They have to be constantly looked at and adjusted given the actual circumstances because of exactly what you suggest – the more complex the project the more impossible it is to predict how it will go. If things go too far off course the best choice may not be to adjust but rather to cancel the project and cut your losses.
I do think a project manager is failing when they consistently go over budget and miss deadlines. More often that not you can see the failure coming and a great project manager will be ahead of the breakdown and adjusting accordingly rather than trying to clean up the mess (or justifying the breakdown) after the fact.
The single biggest source of complex project failure I have seen though is a lack of leadership, specifically a lack of successfully managing the relationships among the groups who have to work together, and keeping everyone’s attention on and commitment to the outcome of the overall project. In the absence of leadership people tend to succumb to the natural tendency to get tunnel vision from focusing on their individual success as a function of doing their part rather than doing whatever it takes to deliver on the promise of the project.
Absolutely agree that we can’t control all events & that thinking that Project Management, for example, can always deal with that uncertainty just doesn’t match the evidence. After all, the only thing that is certain about any Project Plan is that it is wrong.
Human Systems Dynamics teaches us that indeed life is unpredictable and chaotic, and control is just a dream.
As soon as we drop that expectation from our business, political, everyday affairs, and learn tools that help us readily adapt to the uncertainty – including honesty about it – we’ll behave more rationally and be better off.
Thanks John for speaking out on this reality.
Cheers, Paul Reeves
But isn’t there a problem with the definition of success? Is a project that meets all its “requirements”, but is immediately discarded after delivery, a success or failure? Is a project that is over budget, late, and buggy, but immediately has customers clamouring for improvements, a success or failure?
I think we lack a language to describe how long complex projects take, or how the definition of success may change during execution. I have seen work lost to someone who claims they can do the work for the budget, knowing full well they can’t; I have seen cases where development had good evidence for the costs that would be incurred, only to be given budget of less than a 3rd of that; and I have seen the case where no one had any idea what the cost would be, but the program was deemed a failure when it exceeded the magic number pulled out of someone’s butt.
I don’t think success is a fluke, and I think I have seen many very successful projects. But my trick is to stay focused on providing value to the end user, and let the cost/time/goals fight take care of itself. If nothing else it keeps the operational community on my side during the ditch/defer/do decision meetings.
If the issue was simply that bad things happen to good projects, then it would be a simple case of management reserve. I have never known a housing project to fit in the “2 weeks” the builder always promises.