Speaking at the LFE Summit in Paris, Bryce Bartmann who is chief digital technology advisor in Shell’s computational science and digital innovation organization, expounded on the major transition in energy systems and the need for oil and gas companies to respond. The energy mix is changing, decarbonizing and decentralizing. Oils have new competitors with new business models and digitalization is also happening ‘at a rapid pace’. One key enabler of the transforming energy system is open source software, ‘open source is fundamental to us’. This represents a changed culture in Shell from proprietary systems, especially with regard to data. Moreover, ‘people want to work with open source and communities.’ To boost both its new energy and open source credentials, Shell joined LF Energy in 2022 and Bartmann is now a strategic member representative and member of the LFE governing board.
Bartmann presented Shell’s real time data ingestion platform RTDIP. Real time data is ‘being generated everywhere around’ us. When combined with data science and AI, real time provides insights that were simply not possible before. The RTDIP has been four years in the making and currently ingests data from over three millions sensors in real time and houses some 4 trillion records in its Databricks Delta Lake. The latter, ‘built on open source technologies’, powers over 50 Data Science, AI and analytics projects at Shell and is now ‘available as a project on LF Energy’.
Bartmann wants to make open source a part of how Shell’s users work and think. Open source is leveraged in Shell’s strategic values, defining standards and support interoperability. Shell does not want to suppliers’ clouds. Standards are good for Shell and ensure a level playing field, speeding the journey through the energy transition. Particularly in the context of electrifying systems and industries.
Building the data lake was a great decision. Now some 50 apps are up and running providing data science, AI and predictive maintenance. RTDIP runs in LNG, upstream and smart buildings, collecting data and providing visualization. A core RTDIP capability is the pipeline SDK that ingests data from windfarms, smart meters and oil rigs. Other LFE projects have been rolled-into the mix such as LFE Fledge Power, a multi-protocol translation gateway for power systems. Connectivity is provided by Fledge LFE’s IoT solution and an Apache Spark pipeline into the Delta Lake.
RTDIP is now a fully-fledged LFE project that promises ‘easy access to high volume, historical and real time process data for analytics applications, engineers, and data scientists’. The system can runs locally or in the cloud. Components also include Databricks a/a, the Azure key vault, AWS secrets manager and Dagster for scheduling. Development is currently hosted on Azure – the team is also working with AWS and ‘talking to’ Google. RTDIP also uses Apache Spark and Python and supports BI tools including Databricks and Power BI. Users include digital twin developers.
Returning to potential application in the energy transition, Bartmann said, ‘think bigger’ about applying AI to renewables and energy solutions. Data scientists can help decarbonize hard to abate sectors like cement and steel with ‘system level modeling’. Look to solutions like Avelia, a ‘blockchain-powered sustainable aviation fuel’ co-developed by Shell Aviation, Accenture, and the Energy Web Foundation. In the Q&A, Oil IT Journal asked Bartmann about how Shell’s massive and incumbent OSIsoft/PI System platform fitted in with the RTDIP. It turns out that RTDIP was originally built on top of PI. This showed that the idea worked and became the justification for building an open source stack that would better suit Shell’s strategy and support data science. Shell has built lots of connectors including OPC-UA to bypass PI but Shell is ‘not actually planning to replace (all of) its OSISoft platform.
Comment: One thing is for sure. Shell is not short of data projects. In the last couple of years we have reported on the Shell-backed open subsurface data universe OSDU, the Shell/Baker Hughes/C3.ai/Microsoft ‘Open AI Energy’ and now LF Energy RTDIP. What can one say? The more the merrier!
Read our short report from the LF Energy Summit elsewhere in this issue and visit the LFEnergy Summit home page.
© Oil IT Journal - all rights reserved.