This month’s lead is a critical analysis of a current ‘best practice’ in risk analysis from Reidar Bratvold (University of Stavanger) and Eric Bickel of the University of Texas at Austin. For those of you who don’t subscribe to the full text edition (cheapskates!), the analysis was presented1 at the Society of petroleum engineers’ annual conference and technical exposition held this month in New Orleans. The author’s critique strikes at the heart of a popular mechanism for ranking industrial risk, the risk matrix, and shows it to be pseudo science.
Bratvold was quizzed for an alternative to the risk matrix approach and he pointed us to the body of work that has come out of Stanford University on how decisions are made. I googled around on the Stanford website and just about the first thing I found was a ‘breakfast briefing’ on, you guessed, risk matrices! A quick email exchange with the authors put me on the right track.
Bratvold pointed us at risk management specialists Elisabeth Paté-Cornell at Stanford and Terje Aven at Stavanger. Co-author Eric Bickel pointed us to a video where George Kirkland explains how Chevron uses decision analysis and also pointed us at the Decision analysis society. I confess that I have not had the time to follow up on all these leads myself but thought that they might be useful to Oil IT Journal readers.
I have a lot of sympathy for Bratvold’s iconoclasm. We have reported previously from conferences on risk and safety where ‘bow tie’ and ‘Swiss cheese’ models have been put forward as ‘best practices.’ Whether these rather poetic approaches are ‘best’ or not, they are hard to qualify as scientific.
Lets now look at the alternative, mathematical modeling of the decision making process. Subject matter experts are tasked with analyzing small bits of the enterprise and making judgment calls on the likelihood of this and that happening and the associated costs. These are then combined, usually with Monte Carlo techniques, to provide a big picture of all options and risks.
One can imagine a situation where the whole enterprise, risk, outcomes and decisions are encompassed in some massive computer model. Value judgments (how much is a life worth, what is the risk of a 40 year old pipeline exploding, how much will it cost to fix) are taken by the modelers. Numbers are duly crunched and the in a process that I call ‘dumbing-up’ because the hard (not to say intractable) decisions are made by the experts leaving the boss to rubber stamp the model.
I admit that this is a dystopian picture and I probably would not have painted it had it not been for another presentation2 made during the SPE session on safety management by ENI’s Annamaria Petrone on ‘Evaluating the HSE risks and costs of major accidents in the upstream.’ Petrone outlined the results of ENI’s Ergo project that seeks to put a monetary value on the cost of major accidents.
While recognizing that the exercise is not easy, Petrone puts forward a methodology that puts a monetary value on the ‘cost of risk’ associated with the production of a barrel of oil. The cost of risk integrates factors such as personnel safety, environmental risk and risk to the asset. The computation assumed inter alia that a lost life ‘costs’ $100 million, a figure drawn from the UK’s health and safety executive.
Petrone’s talk outlined the use of the ‘bow tie’ risk analysis approach and (you guessed it) risk matrices, along with accident statistics from the oil and gas producer’s association. All of which is rolled up using a ‘baseline risk assessment tool’ into a ‘cost of risk’ metric. The study found that the main contribution to the metric was from blowout risk. Not much of a surprise there. What was surprising was the monetary value of the overall risk—which came out to be ‘of the order of one eurocent per barrel.’
It so happened that while Petrone was making her presentation, I had a copy of the New Orleans daily, the Times-Picayune hidden under my computer, which offered some additional data on the cost of a major accident. The Times informed me that BP was potentially liable for $18 billion (€14 billion) damages in respect of the Macondo blowout.
So if you are putting aside one eurocent per barrel, how many barrels do you need to produce to offset this liability? By my reckoning that comes to 1.4 trillion barrels! That is more than the whole world has produced to date. So something is wrong here, with ENI’s sums or with how the legislator has figured the damages or perhaps both.
How does all this relate to dumbing-up in decision making? Well I think that it is saying that the monetary/mathematical approach to decision making and risk management is tricky. While the modelers and statisticians undoubtedly have a role to play, I can sympathize with a management that shoos them all out of the door before taking a major decision. How will this be made in reality? I don’t know. It does appear that current analyses are flawed. Bratvold, in a short discussion following his presentation, observed that management discourse can also be rather misleading, ‘Managers always claim that safety is the number one priority, but if that were true we would not drill at all!’
© Oil IT Journal - all rights reserved.