Fiona Macmillan kindly provided PDM with a copy of her PhD thesis “Risk, Uncertainty and Investment Decision-Making in the Upstream Oil and Gas Industry.” The research, conducted at the University of Aberdeen, was funded by CSIRO Australia. Macmillan believes that despite four decades of work on decision analysis tools, no research has been able to show conclusively what works and what does not. Indeed such analysis has been slow to appear in the literature, doubtless because of the threat it represents to the decision analysts. As one previous worker put it: “Asking whether decision analysis works is risky. What if the answer is negative? The contribution will clearly be scientifically valuable, but many individuals – consultants, academics, instructors – with a vested interest in decision analysis could lose standing, clients, or even jobs.” MacMillan’s study remedies this situation by researching the use of decision analysis in investment appraisal decision-making by upstream majors. The thesis seeks to answer three questions: what techniques are appropriate, which are actually used, and is there a relationship between their use and corporate success?
Interviews
In the 250 page tome, Macmillan traces the history of risk analysis from Laplace and Bayes in the early nineteenth century, through its first use in the oil industry in the 1960s, to its limited application today. The research was conducted via in-depth interviews of some 20 companies working in the UK. All of the companies use some software to assist with decision making, usually some form of Monte Carlo simulation. The most popular packages are Crystal Ball, @risk and Merak PEEP. There was a general recognition that whilst the mechanics of the simulation is straightforward, the tools could be misused. One interviewee remarked “the clever bit is in the model that you set up where you’ve got the risk … and you’ve got the relationship. So you can have a fantastic tool that does Monte Carlo inside and out but [it’s] garbage in-garbage out.” Definitions also were a source of problems. One interviewee commented “I would say that there’s about as many different definitions of risk and uncertainty in our company, as you found in your literature search.” Another - “Every time we start to discuss risk we have arguments and rows!” Problems were encountered when reporting the results of probabilistic analysis to management: “We do all this beautiful simulation of the distributions but people still want one figure. You could say, ‘The range is this, or your expectation is this at various probability levels.’ You know, a nice little cumulative distribution. But people don’t look at it. They want one number, ‘What’s the mean? What’s the expected value?’ So sometimes I question why we do it because people just land on one number.”
Competitive advantage
Notwithstanding such doubters, Macmillan contends that the use of decision analysis techniques and concepts in investment appraisal decision-making is a source of competitive advantage, claiming that there are “strong positive correlations between the use and sophistication of decision analysis techniques and concepts used, and various measures of business success in the upstream.” Macmillan proposes a complete, if theoretical, decision support workflow for the upstream as follows. First, assess the chance of success based on historical statistics and analogues of other basins and plays with similar geological characteristics. Next apply sensitivity analysis to determine critical reservoir parameters. A probabilistic analysis of reserves using Monte Carlo techniques follows to provide high, mid and low cases. Influence diagrams are used to draft a decision tree and for each reserve case, the chance of success estimated in the first step is combined with economics and decision tree analysis to generate expected monetary values. These can be further analyzed with option theory. So much for a ‘perfect’ workflow. But what do companies actually do today? Not a lot seemingly! Most companies use Monte Carlo simulation to generate estimates of prospect reserves and run the economics on only one reserve case. In fact, Macmillan found that even Monte Carlo simulation is not in general use for economic analysis in E&P decision making. Option, portfolio and preference theories “are hardly used at all by any firm.” Macmillan concludes that there is a gap between current theory and practice in the quantitative techniques used for investment appraisal in the upstream.
Does it work?
The majority of the results produced suggested that there is a positive association between the use of decision analysis in investment appraisal and good organizational performance in the upstream oil and gas industry. This thesis has highlighted that decision analysis should not be perceived to be providing ‘a dictatorial straitjacket of rationality’ (French, 1989). Rather it should be seen to be ‘a delicate, interactive, exploratory tool which seeks to introduce intuitive judgments and feelings directly into the formal analysis of a decision problem’ (Raiffa, 1968). The decision analysis approach is distinctive because for each decision, it requires inputs such as executive judgment, experience and attitudes, along with the “hard data”. It helps decision-makers tread the fine line between ill-conceived and arbitrary investment decisions made without systematic study and reflection (“extinction by instinct”) and a retreat into abstraction and conservatism that relies obsessively on numbers (“paralysis by analysis”) (Langley, 1995). The thesis has demonstrated that such an approach contributes positively to organizational performance in the upstream oil and gas industry.
Macmillan’s thesis “Risk, Uncertainty and Investment Decision-Making in the Upstream Oil and Gas Industry,” conducted at the University of Aberdeen, is available on www.oilit.com/papers/macmillan.doc.
© Oil IT Journal - all rights reserved.