‘Lots of brilliant ideas, too bad they don't work’

There are plenty of AI/ML ‘success stories’ in the literature. Milano Poly’s ‘Predicting Lithology using ML’ FORCE presentation is a refreshingly frank account of AI that didn’t work and why.

Vincenzo Lipari presented Milano Poly’s* entry to the Norwegian Force Predicted Lithology using Machine Learning competition as ‘lots of brilliant ideas, too bad they don't work’. Lipari observed that ‘almost all the top scores in the Force predicted lithology competition derive from the use of ‘classic’ machine learning methodologies and a standard approach to the problem. It is therefore puzzling that more innovative approaches and more sophisticated reasoning do not improve the result in any way’.

Lipari referred to the 2016 SEG Machine Learning contest which resulted in a ‘very influential’ standard approach to lithological ML. However, applying the usual techniques of ‘trees, xgboost and random forest’ produced poor results. Lipari suggests that the Force dataset may contain a lot of data, but that ‘maybe from a deep learning perspective this is still too small a data set’.

Handling geographical information also proved problematical. Lipari (who confessed to being better versed in signal processing than in geology) found no evidence that new geological meaning was being derived from geographical information although this possibly due to the model already embedding XY coordinates. In the end, the most successful model used a XG boost method. More sophisticated ‘modern’ deep learning based sequence models all failed, again probably because ‘big’ well data is ‘small’ for deep learning. Even the standard approach appears to have an upper limit to its precision score of around 80%. Lipari concluded that to improve performance, it is probably useful to be guided by a geologist and to embed geological knowledge into the model. Watch Lipari’s and other Force presentations on Youtube.

*Politecnico di Milano Image and Sound Processing Lab.

Comment: The ‘big data is too small’ may affect other applications of ML in oil and gas such as Total’s work on ESP monitoring.

Click here to comment on this article

Click here to view this article in context on a desktop

© Oil IT Journal - all rights reserved.