Lawyering Major Projects: Nine Techniques to Keep Black Swans at Bay

The Sydney Opera House, which recently celebrated its 40th anniversary, is by any measure one of the greatest buildings of the twentieth or any other century. It is a monument to human imagination and ingenuity. But it is also a monument to human fallibility: at commencement, in 1957, its planners estimated that the construction project would cost $7 million and be completed in 1963. It was instead completed a decade late and more than 14 times (1400%) over budget.

The data suggest that contemporary governments and their suppliers have not got much better at forecasting costs or schedules or at estimating benefits than their 1950s Australian counterparts. But this has not diminished their enthusiasm for embarking on major projects: the extent of major project investment globally is already huge and set to get even bigger.¹ Sydney Opera House-style errors in forecasting and project execution affect us all. As citizens and taxpayers, we find ourselves funding unviable projects, with attendant opportunity costs; as professionals, we have duties to clients and an eye to our own reputations to ensure that the projects we support are executed expeditiously and economically.

This paper examines some recent important material about the nature of major projects and why they go wrong. In particular I hope to alert you to the dangers posed by ‘Black Swans,’ a term coined by the writer Nassim Nicholas Taleb in connection with the limitations of human knowledge derived from observation and experience. Black Swans are emblematic of low probability, high impact events that can – and too often do – derail major projects.

Although I hope that there is much in here of interest to all professionals who are engaged in major project activity, my intended audience is primarily other lawyers. I suggest nine techniques, broken down by project phase, that a lawyer might find helpful to keep Black Swans at bay. Another way of putting that, to use Taleb’s terminology, is how one might move a project away from the ‘Fourth Quadrant.’ Although I wasn’t present during the construction of the Sydney Opera House, due largely to not being born for most of it (that said, as a boy I did see it shortly after it had opened), I have learned much whilst acting for over a decade as a lawyer in connection with some transformational projects in the United Kingdom, both from personal experience and from long discussions with industry veterans over cold pizza at late hours in meeting rooms in otherwise deserted offices.

What You Need to Know about Major Projects

What is a major project and why should you care? A major project is typically a large-scale, complex venture that costs at least $US 1 billion, takes years to develop and build, involves numerous stakeholders and affects very many people (often millions).² Major projects are typically thought of as government projects, not least because not many actors besides states have the resources to pay for them, although they certainly exist in the private sector too. The combination of scale, expense, complexity, duration and publicity means that they are qualitatively different from smaller projects. The failure of a merely ‘big’ project is an embarrassment; the failure of a major project could, in a democratic society, conceivably bring down a government.

But why should you care? Well, as citizens, an awful lot of the money our governments raise from us in taxation (or borrow on our behalf ) is spent on major projects, especially the ones involving ‘transformational’ schemes. This is true all around the world and is only likely to increase: global infrastructure spending was recently estimated at a bit under $5 trillion per year; but by 2025 it is forecast to rise to $9 trillion per year, with nearly $78 trillion to be spent globally in the intervening 11 years, a colossal sum.³ A huge proportion of our tax pounds, dollars, euros, yuan and yen will be spent on them. Essential services will depend on them being delivered.

But why should you care? Well, if you are reading this, then I expect that you are a lawyer or commercial manager tasked with helping to steer one to completion. Your job is to balance the demands of many actors, including but by no means limited to executive sponsors, suppliers, politicians, lenders, the media, regulators, engineers, end users and project management teams. It’s a difficult job.

But why should you care? Well, because major projects have a terrible track record. Not only do they often run over budget, but an alarmingly high proportion of them blow those budgets substantially. Research by Oxford University in 2011 into 1,471 large IT projects (the average cost was $167 million) showed that although the average overrun was 27% – itself a disturbing result – that figure masked a far more alarming one: fully one in six of the projects studied overran their budgets by an average of 200%, with a timetable overrun of almost 70%.⁴

This research suggests that the ‘fat tailed’ nature of cost and timetable overrun is a fact of life for major IT projects. Large blowouts, which tend to be blamed on a project-by-project basis as being caused by bad luck, or events beyond anyone’s control, or some other unforeseeable cause, happen too often to be dismissed as merely outliers against a track record of generally good project management practice.⁵ Even though IT projects may be especially demanding – Fred Brooks famously warned that there is ‘no silver bullet’ we can fire to overcome software’s ‘essential difficulties’⁶ – the complexity of a modern major project means that we should be just as concerned about infrastructure, transport or construction projects too.

So: major projects are often late, blow their budgets and fail to deliver the things that led to their establishment in the first place – and worse, a substantial proportion of them do those things in a catastrophic way.

Break-fix is Business as Usual

In early 2014 Professor Bent Flyvbjerg of Oxford University’s Saïd Business School published an article in the Project Management Journal⁷ that summarises many of the key features of major projects. The most striking (if depressing) feature is gloriously negative: they usually fail. Professor Flyvbjerg identifies their tendency to run ‘over budget, over time, again and again’ as the ‘iron law’ of major projects. This is not a comforting thought, especially in light of the forecast for future global infrastructure spend described above. Why does this happen? Professor Flyvbjerg suggests that many major projects follow a ‘break-fix’ trajectory, in which:⁸

planners and managers charged with designing and implementing major programmes are neither competent to deliver them well, nor are they incentivised to do so;
those projects ‘break’ sooner or later, when reality ‘catches up’ with optimistic or manipulated estimates of costs, timescales or benefits;
there is a pause to ‘fix’ the problems: lock-in and escalation make it impossible to drop the project altogether; and
the ‘fix’ occurs at great and unexpected cost to stakeholders who were unaware of what was going on and who were unable or lacked the foresight to abandon the project before the break.

Professor Flyvbjerg comments that:

The cure to the break-fix model is to get projects right from the outset so that they don’t break, through proper front-end management.⁹

‘Proper front-end management’ must of course be a good thing and UK initiatives such as the Major Projects Leadership Academy, a collaboration of the Major Projects Authority (an agency of the UK government) and Oxford University in which senior British civil servants are trained in best practice in major project management, are very audable. There are very many ways major projects can fail. Raising the competence of those involved with their p!lanning, design and execution must improve matters.

The Black Swan

But there is an important limitation to the role that front-end management can play. A substantial proportion of major projects have not merely proved too late or too costly: they have been disastrously late and costly. The project managers involved in many of those projects were no doubt suitably qualified and executed their jobs professionally. No amount of planning, however, will eliminate a species of problem that besets complex human enterprises: the occurrence of ‘Black Swan’ events.¹⁰

(A note to the reader: this section and the next (‘Escaping the Fourth Quadrant’) explore the ideas of Nassim Taleb in some technical detail. You may prefer to cut to the chase. If so, please scroll down to ‘Practical Steps for Lawyers Working on Major Projects.’)

It is difficult to convey the erudition of Nassim Nicholas Taleb’s The Black Swan,¹¹ let alone describe what it is ‘about,’ but one important theme throughout is the fundamental limitations that apply to human knowledge. The book’s title alludes to an old story about the fragility of inductive reasoning: Europeans used to be convinced that all swans were white because all empirical evidence supported that conclusion; but the sighting of a single black swan (at Sydney Harbour, perhaps) destroyed that theory utterly.

In Taleb’s usage a Black Swan is an event that not only surprises but also carries a large payload:

First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility. Second, it carries an extreme impact (unlike the bird). Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable.¹²

Taleb argues that the occurrence of Black Swans is an intractable problem. To use Taleb’s classification there are, broadly speaking, two different types of environment or process in which events occur (and, if follows, about which predictions and decisions are made). Let’s use a couple of examples to see what this means.

Mediocristan is an environment or process that is dominated by the ‘mediocre,’ the average event. An example would be the measurement of human heights (for example, the average adult male height in the UK is 5ft 9in) or measured IQ. In both cases the vast majority of observations are clustered around the mean value. So if I were to stand outside my office in London and measure the height of the first 1,000 adult males who walk by, then in addition to some attention from the City of London Police, I can safely predict that the mean value of those heights will be very close to 5ft 9in. No single measurement will greatly affect the average result.

More technically, Mediocristan is an environment where predictions that obey Gaussian (‘thin-tailed’) probability distributions, where probabilities face a very strong ‘headwind’ the further one moves away from the mean. The probability of my encountering an adult UK male who is 10ft tall, or even 8ft tall, is effectively zero.

Extremistan, on the other hand, is an environment or process that can be dominated by unique events. Making accurate predictions in Extremistan is very difficult, if not impossible. Examples include ‘social’ matters such as incomes, book sales per author, the population of cities and deaths in modern wars. Let’s take as an example the mean income of members of the acting profession in the UK. I might predict that the mean value would be very close to the typical wage of a Starbucks barista. But if I were to ignore the police cautions and stand outside my office in London and demand that the first 1,000 actors I met tell me their incomes then a single interview – say, with a fabulously well-paid Tom Cruise or Helen Mirren¹³ – could completely undermine my estimate’s accuracy.

Technically, Extremistan is the domain of the scalable. The impact of any event that occurs in Extremistan is not constrained in size or scale by another, overriding, consideration: it can have a huge, devastating effect. Many Extremistan events do not obey a probability distribution that can usefully be described by a mathematical model. Those that do (‘Grey Swans’) are subject to power law (‘fat tailed’) probability distributions, which for practical purposes renders them scarcely predictable at all.¹⁴

Major projects, and much else in our complex modern society, belong squarely to Extremistan. If you accept this classification – and I personally find it compelling – then no matter how good your planning process becomes, major projects will always be subject to Black Swan events and are therefore always in jeopardy of going off the rails. Better up-front planning, in the sense of trying to predict and cater for all the things you think might affect your project’s trajectory, will never be make you impervious to break-fix. You can do nothing to prevent the occurrence of Black Swans.

Escaping the Fourth Quadrant

All is not lost. Even though you can do nothing to prevent the occurrence of Black Swans, you may be able to do something to neutralise their impact. The second edition of The Black Swan included Taleb’s ‘map’ of the Black Swan domain and some guidance to help you get out of there.

In best programme management tradition, this calls for a diagram:

	I Simple payoffs	II Complex payoffs
A Mediocristan	First Quadrant Extremely safe	Second Quadrant (Sort of) safe
B Extremistan	Third Quadrant Safe	Fourth Quadrant Black Swan Domain

This table, which appears in the revised edition of The Black Swan,¹⁵ illustrates how one might go about classifying a decision against the background of different types of ‘event generators.’ For these purposes an event generator may be one that belongs to Mediocristan, which is entirely predicable, or to Extremistan, which is wild.

The distinction between simple and complex payoffs is a bit more straightforward. A simple payoff is one in which you only care whether something is true or false, whether an event happens or not. A complex payoff is one in which you care not only about whether an event occurs or not but also about its impact.

By way of example, an event with a simple payoff could be guessing the result of a single toss of a coin – say, with your child, just for fun. The coin will either land heads or tails, which matters in a sense to you and especially your child, but nothing really turns on it. With a complex payoff you also care about the effect of the event. An example might be an outbreak of influenza in Shanghai next winter: what matters is not only the probability that people will get sick, but also its impact. How many people will require hospitalisation? How many days’ work will be lost to the city’s economic output?

Major projects live in the Fourth Quadrant. They are located in Extremistan, the complex human world. What’s more, you care not only about the probability that destabilising events will affect them but also about the magnitude of those effects: for example, you care not only whether a key milestone has been achieved on time or not, but also about the knock-on effects of missing it. A missed milestone may mean that a supplier isn’t paid when it expects. The milestone may relate to a deliverable that is in turn critical for the successful achievement of a whole series of milestones due in six months’ time. Putting those milestones in jeopardy may in turn trigger a cascade of yet more undesired effects, and so on.

Taleb’s proposed cure to Black Swan problems is, in a nutshell, for us to arrange our affairs so that we move from the Fourth Quadrant to the Third Quadrant. In other words, instead of our affairs being subject to complex payoffs in Extremistan, we should arrange them so that they are subject to simple payoffs in Extremistan.

By way of example, consider the Fukushima disaster in 2011, in which a nuclear reactor was badly damaged by the tidal wave resulting from unprecedentedly large earthquake. How could this have happened? Aren’t nuclear reactors designed to be safe?

In 2003 the Japanese Nuclear Safety Commission set the following quantitative goal for the country’s light water reactors:

The mean value of acute fatality risk by radiation exposure resultant from an accident of a nuclear installation to individuals of the public, who live in the vicinity of the site boundary of the nuclear installation, should not exceed the probability of about 1x10^-6 per year.¹⁶

The reactor at Fukushima presumably complied with the risk model implied by this standard, so a Fukushima-style disaster should occur only once in a million years. Instead it occurred merely eight years later, which suggests that both the regulators and the reactor’s designers got their model very badly wrong. Moreover, the consequences of getting it wrong were evidently very significant. We are very clearly in the Fourth Quadrant.

What is an appropriate way to shift a decision about nuclear plants into the Third Quadrant? Intelligent nuclear energy firms now give up trying to predict how probable a source of failure might be and instead focus on minimising their exposure to failure, making the prediction (or not) quite irrelevant. This approach leads to building a greater number of smaller nuclear reactors and burying them deep enough in the ground with enough layers of protection around them so that a failure will not affect people much if it should occur.¹⁷

In other words, instead of trying to quantify (guess) the rate of occurrence of low-probability events, instead get out of the Fourth Quadrant by making your solution impervious to harm no matter which of those low-probability events occurs. This advice applies just as well to a European IT project as it does to a Japanese nuclear reactor.

Practical Steps for Lawyers Working on Major Projects

So what sort of front-end management will tend to move a major project from the Fourth Quadrant to the Third Quadrant?

A project’s planning and implementation occurs in at least three stages. First, the project is selected: someone decides what the project is meant to achieve and draws up requirements. This tends to be a customer responsibility. For major projects, these decisions are generally made at Ministerial level (for government projects) or C-level (for private sector projects).

Secondly, projects go through a design phase. Someone (or, more likely, a team of someones) conceives of a design that will satisfy the requirements and writes a specification that describes how the project will be built. In major projects this is largely an activity lead by the supplier, although customers will typically be heavily involved. Lastly, projects are implemented. The teams (led by the supplier) get on with the business of doing the things described in the specification, ideally in the order and to the timetable set out in that specification. Along the way the parties may have to deal with a crisis or two. (In this context a Black Swan is often a very big crisis.)

For a lawyer the key document associated with a major project is the project agreement. For present purposes, a key feature of the project agreement is that it is usually executed after the requirements and design phases but before the implementation phase.

The lawyer’s role is typically quite different on either side of contract execution. Before contract execution many clients (and some lawyers) see the lawyer’s role as largely ‘documenting the deal’ that the technical and business people devise. But this underestimates the role that an experienced project lawyer can play in creating a ‘Third Quadrant’ project structure.

Once the project has commenced the lawyers are often involved only when a crisis occurs, or if the implementation is off-schedule, and executive teams want to understand their contractual powers or liabilities. In a break-fix scenario, lawyers will also be heavily involved in negotiating and documenting the ‘fix.’

As a lawyer supporting a major project there are important contributions you can make to ensure that a project is as Black Swan-proof as possible. Perhaps controversially, I don’t think that they really require what is commonly thought of as core legal expertise, such as drafting or negotiating contracts. I think they depend more on other attributes often found amongst those practising the profession.

First, lawyers are trained critical thinkers: lawyers develop over many years an advanced ability to focus on facts and to see through the logic of an argument to its conclusion, regardless of the appeals to emotion or the political pressure that are often brought to bear. Secondly, lawyers are by training often more sensitive to ethical issues than colleagues in other disciplines: during the course of our careers we are often required to explain to clients the difference between ‘is it legal?’ and ‘is it right?’ Finally, lawyers are not ‘prisoners of the org chart’ in the same manner as many other business disciplines. In order to do their job well – whether private practice or in-house – lawyers often need to ‘surf’ all levels of seniority and disciplines of an organisational hierarchy. Facts that are known only to a junior delivery officer may be critical to the strategy devised by the CEO: a lawyer is often the only person who deals with both people directly.

The next three sections set out some thoughts about how you lawyers might be most effectively deployed in a major project, broken down by project phase.

I: Project Initiation and Requirements phase

At the outset a project is concerned with planning, scheduling and gathering requirements (also known as ‘figuring out what we should do’). This is the prime period to get the project into the Third Quadrant. In this phase a projects lawyer can make the following contributions:

Encourage your team to ‘bake’ redundancy into the requirements. Instead of mandating a single large but ‘efficient’ end project requirement, encourage your team to aim for several smaller parts to the solution that in combination provide a solution, even though that requires some duplication of outcome. If the Japanese had built several small nuclear reactors deep underground rather than the single large facility at Fukushima, the 2011 tidal wave would not have been able to cause the disaster it did, with severe knock-on effects to the environment, human health, local agriculture and the Japanese economy.
Encourage as much re-use of existing technology as possible. There are two reasons for doing this. Time is an excellent stressor. New technology has a tendency to break or to become obsolete quickly. By contrast, things that have been around for a long time have had many opportunities to break and have not (if you are sensibly considering them). The second reason is that existing solutions are likely much cheaper than ‘cutting-edge’ or experimental technology.
Be vigilant against agency issues. In previous important research¹⁸ Prof Flyvbjerg proposed that bad luck and the effects of optimism bias did not alone account for major project underperformance. He suggested that something else is going on: cases where political and organisational pressures where ‘politicians, planners or project champions deliberately and strategically overestimate benefits and underestimate costs in order to increase the likelihood that their projects, and not their competitors’, gain approval and funding.’ Prof Flyvbjerg labels this behaviour ‘strategic misrepresentation.’ A lawyer can play a valuable role in unearthing these behaviours, dispassionately identifying the associated risks and in ensuring that proper governance arrangements are brought to bear.

II: The Design Phase

Once the project’s requirements are in place the parties can get to work designing solutions for them. As there are always many potential solutions for any given problem, attention turns to which solution is best, how long it will take to deliver it and what it will all cost. The project lawyer’s contributions outlined above are generally also applicable to this phase, but some useful extra activities are:

Test the solution’s budget and timetable estimates. Even in circumstances where strategic misrepresentation is not present one must always guard against ‘optimism bias’: in short, humans systematically underestimate how long projects will take, how much they’ll cost and how much things may have to change along the way, with the result that many initial major project forecasts prove wildly optimistic. Black Swans, by definition, are particularly good at escaping our attention. One important technique to combat optimism bias is reference class forecasting, which I have previously written about: if your team has the data to support the use of reference class forecasting, encourage them to do so.
Promote a solution design that ensures that poor solutions will fail quickly. If a major project is going to break (due to a Black Swan event or otherwise) then it is best if it breaks early, while the rectification costs are relatively low. Ask the design team ‘How soon will we know if our proposed solution is a dud? What will flush that out as soon as possible?’ You want failures to be small-scale and private. They are opportunities to learn, not shameful events. (Although it can be difficult to convince people to see them like that.) The worst sort of failure is the one that occurs only when all the budget and available time has been spent and the project has become ‘too big to fail.’
Use prototypes. A corollary of the last point is that you should encourage your team to use prototypes wherever possible. On technology projects, for example, a test system should first be built at a local level, be shown to work under robust testing, and then carefully scaled to meet all users’ needs – bearing in mind that scaling up itself brings with it different challenges. Prototypes (whether created via a formal ‘proof of solution’ stage or, better still, as small-scale production systems) allow designers to use everything that’s currently known about the best design for the requirements and to deploy that knowledge without taking on the often catastrophic risks that can arise with full implementation. They allow you to fail fast, while it’s still cheap to do so.

III: The Implementation Phase (including crisis management)

By their very nature, of course, Black Swan events are high-impact, low-probability events. They have a habit of turning up when they’re least expected. (Moreover, as they are in retrospect ‘predictable’ there is much scope for finger-pointing after the fact.) Lawyers are very often wheeled onstage in the immediate aftermath of a Black Swan event. Schedules are slipping, management is getting nervous and shouty, costs are blowing out and everyone is concerned about the threat of litigation: this is the ‘break’ of the ‘break-fix’ model we encountered earlier. When all about you are losing their heads (if not blaming it on you), what can you do? Here are some ideas:

Get the best information you can about how your project status maps to your legal position: embed your commercial and legal personnel in the front-line project teams. Whether or not the Black Swan will lead to war (e.g., litigation) or peace (e.g., ad-hoc variation of the project agreement or even major renegotiation), a programme’s leadership teams need the best available picture of what is happening ‘on the ground’ and how that maps to the legal position: will you enter the inevitable and inevitably painful negotiations with the other side from a position of strength or weakness? Good lawyers come equipped with formidable forensic and analytic skills. Deploy them. Require them to present the information relevant to your position on A3 paper in graphical form (the ‘programme on a page’ technique) rather than in pages of prose littered with professional qualifications. Take advantage of the fact that communications to and from lawyers can attract legal professional privilege.
Be aware that tough contractual remedies are less useful than you might think. A well-drafted contract will (rightly) identify project milestones and describe the consequences of those milestones being missed. For example, a supplier’s provision of a deliverable by its due date will often trigger a payment. Conversely, failure to provide the deliverable by the due date will often trigger customer remedies such as compensation (e.g., liquidated damages) or, more severely, rights of step-in or even contract termination. At first sight it may appear that the more diverse and potent the customer’s remedies, the stronger its position – but this is not always the case. If, for example, the customer is able to punish even a relatively minor breach of contract by a remedy as nuclear as termination there is very little incentive for the supplier to engage in frank conversations about what has gone wrong. On the contrary: it encourages the supplier to do everything in its power to blame the customer for the delay. There is a great risk that management time will then be preoccupied in pre-litigation activity rather than doing something really useful, such as fix the problem.
Make sure the contract includes a ‘safe’ mechanism to allow the parties to discuss project change. Even when you have a clear understanding of the facts on the ground and the underlying legal position, you still have to fix or walk away from the broken project. The underlying contract can certainly facilitate candour (although it can’t guarantee it). On one project, for example, I drafted into the project agreement a regular ‘without prejudice’ meeting in which the parties could speak candidly about things that might jeopardise upcoming contractual milestones: if they were able to reach agreement about a consequential variation to the prevailing deployment implementation plan, then great; but if they couldn’t, the ‘without prejudice’ discussions would remain permanently off the record.

The Black Swans of Sydney

According to an old story, a lord of ancient China asked his physician, a member of a family of healers, which of them was most skilled. The physician, whose reputation was such that his name had become synonymous with medicine in China, replied: ‘My eldest brother sees the spirit of sickness and removes it before it takes shape, so his name does not get out of the house. My elder brother cures sickness when it is still extremely minute, so his name does not get out of the neighbourhood. As for me, I puncture veins, prescribe potions and massage skin, so from time to time my name gets out and is heard amongst the lords.’¹⁹

My objective with this paper is to encourage the reader to be a little bit more like the eldest brother. But this story also illustrates one of the ironies of managing a major project on time and on budget: it’s no more than what outsiders expected to happen in the first place. (Notoriety is far easier to achieve. As JK Galbraith observed, immortality can always be assured by spectacular error.)

Indeed, if you were to follow the principles set out in this paper you may find yourself subject to a different form of criticism: that the redundancy that acted as ‘insurance’ against Black Swans was never needed in the first place, that the absence of a crisis is somehow evidence that the project built in too much allowance for ‘risk,’ that it was inefficiently designed or that it lacked ambition.

In many senses this would be a nice problem to have but, to be blunt, I’m afraid there’s really no way to eradicate that criticism. We don’t get to run an alternative version of history to see what would happen if we made decisions differently. You can, however, be very confident that without a change in how we think about project design, Black Swans will continue to imperil major projects all around the world, from London to New York to Qatar to Hong Kong and even, yes, to Sydney.

The initial version of this post was published on twobirds.com on 27 August 2014.

PriceWaterhouseCoopers LLP, ‘Capital project and infrastructure spending: Outlook to 2025 (2014), accessed 21 September 2015, http://www.pwc.com/gx/en/industries/capital-projects-infrastructure/publications/cpi-outlook.html. ↩
Definition adapted from Bent Flyvbjerg, ’What You Should Know about Megaprojects and Why: An Overview,’ Project Management Journal 45 (2) (2014) 6-19 at 6 (accessible at http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2424835). I use the expression ‘major project’ throughout this paper for the sake of simplicity. It can be read more or less interchangeably with other similar terms, such as ‘major programmes,’ ‘transformational programmes’ and ‘megaprojects.’ There are taxonomies that purport to distinguish between them, usually based on cost, but the key point is always that the sums of money involved are huge. ↩
PriceWaterhouseCoopers LLP, ‘Capital project and infrastructure spending,’ overview, page 2. ↩
Bent Flyvbjerg and Andrew Budzier, ’Why Your IT Project May Be Riskier Than You Think,’ Harvard Business Review 89 (9) (2011): 24-27. ↩
Andrew Budzier and Bent Flyvbjerg, ‘Making Sense of the Impact and Importance of Outliers in Project Management through the Use of Power Laws.’ Preprint of paper submitted to the 11th IRNOP Conference, 1 June 2013, accessed 21 September 2015, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2289549 ↩
Frederick R Brooks Jr, ‘No Silver Bullet – Essence and Accident in Software Engineering,’ in The Mythical Man-Month: Essays on Software Engineering (anniversary edition) (Boston: Addison-Wesley 1995), Chapter 16. ↩
Flyvbjerg, ‘What You Need to Know about Megaprojects.’ Disclosure: the author has presented to students on the M.Sc. course at Saïd Business School led by Professor Flyvbjerg. ↩
Flyvbjerg, ‘What You Need to Know about Megaprojects’, 12. ↩
Flyvbjerg, ‘What You Need to Know about Megaprojects’, 12. ↩
Despite taking issue with the suggestion in Prof Flyvbjerg’s paper that the ‘cure’ for break-fix is ‘better front-end management’, I am sure he does not mean to say that improved front-end management is a panacea: he has written extensively on the insights on project management afforded by behavioural economics and warmly endorses the works of Nassim Taleb. But the notion that one can improve major project outcomes by better advance planning – which I think is often taken to mean identifying risks in advance and developing contingencies for them (or worse still, convincing oneself that a given risk is so improbable as to be worth ignoring – is still, I think, very prevalent in senior management. ↩
Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable (revised edition) (London: Penguin 2010). ↩
Taleb, The Black Swan, page xxii. ↩
Frankly, I don’t recommend standing around the streets of London attempting some star-spotting. ↩
Taleb, The Black Swan, Chapter 15. Note that a feature of a project schedule being subject to a power law is that the later you are, the worse your time to complete becomes. The example Taleb gives (at page 159) is as follows: let’s say a project that is the subject of a scalar variable is due to complete in 79 days. If it hasn’t completed in 79 days then you can expect it to complete in another 25 days. But if it hasn’t completed on the 90th day then the estimated extra time to complete is another 58 days. By day 100, there should be 89 days to go, and so on to day 1000, when you will be expected to need another 1590 days! The longer you wait, the longer you will be expected to wait. See also Budzier and Flyvbjerg, ‘Making Sense of Outliers.’ ↩
Taleb, The Black Swan, 365. ↩
‘Nuclear Safety Commission (NSC) of Japan: Safety Goals’ accessed 8 July 2014. http://www.nsr.go.jp/archive/nsc/NSCenglish/topics/safety_goals.htm (Note that an attempt to access this page at the time of this article's re-publication failed: a translation of the page as seen in September 2015 appears to say that the relevant source material has been deleted from the NSC archive. See also the discussion ‘Time to understand a few facts about small probabilities’ by Taleb at paragraph 142 of http://www.fooledbyrandomness.com/notebook.htm, accessed 21 September 2015. ↩
See also Nassim Nicholas Taleb, Antifragile: How to Live in a World We Don’t Understand (London: Allen Lane 2012) especially Chapter 8, ‘Prediction as a Child of Modernity’. ↩
Bent Flyvbjerg, ‘Over budget, over time, over and over again: managing major projects’ in The Oxford Handbook of Project Management, ed. Peter Morris et al. (Oxford: Oxford University Press 2011), 321-344. ↩
The story is set out in this form in the translator’s introduction of Sun Tzu, The Art of War, trans. Thomas Cleary (Boston: Shambhala, 1988), 1. ↩