In his presentation1 at the Society of petroleum engineers’ (SPE) annual conference and technical exposition held this month in New Orleans, Reidar Bratvold of the University of Stavanger began with an inventory of the published use of risk matrices to analyze risk and implement mitigation strategies. Risk matrices (RM) are widely used in oil and gas, frequently cited as best practices and embedded in national and international standards from ISO, Norsok and the American Petroleum Institute. But, asks Bratvold, ‘do they work?’
RMs rank risks so that mitigation efforts can be focused on higher likelihood and cost events. But, for Bratvold, risk matrices contain three irremediable flaws as follows. First, risk matrices produce different results depending on how scoring is done. In one drilling example, using an ascending scoring, the risk category of ‘severe losses’ was prioritized over ‘blowout prevention.’ But simply reversing the scoring scale was enough to change the risk priority order. Bratvold asked ‘Would such a technique withstand scrutiny in a court of law?’
The next flaw is the sensitivity of the risk matrix approach to small changes in cut-offs. A tiny difference here can change the ranking, making for instability in the analysis. None of the standards bodies cited above has anything to say about this flaw.
Thirdly there is the ‘lie factor,’ a concept borrowed from visual display guru Edward Tufte. Here a graphical representation is used to ‘game’ the analysis by presenting information in a misleading way.
Bratvold’s team has analyzed some 1,300 SPE papers—all of which had lie factors greater than one on at least one axis. Such distortion means that risk matrices give the illusion of communicating information simply. But in reality the technique leads to arbitrary risk management decisions. The flaws are inherent to the technique and it is ‘scary that they are considered best practices.’
Bratvold was quizzed on what techniques he recommended other than risk matrices. He replied that risk management was just a subset of decision making and that there is a century or so of analysis and advice on the topic—as practiced at Stanford University’s management sciences department. Other industry practitioners queried Bratvold’s conclusions observing that despite the flaws, RMs were good communication tools. Bratvold was unrepentant, ‘Risk management is important, why should the goal be to keep it simple? There is no need for over sophisticated Monte Carlo analytics but using risk matrices mean we are having the wrong conversation.’
1. SPE paper 166269— www.oilit.com/links/1310_0102.
© Oil IT Journal - all rights reserved.