Our standards special this month homes-in on use of the HDF5 protocol for the storage of large oil and gas data sets. HDF, the hierarchical data format, was originally developed at the National center for supercomputing applications at the University of Illinois and today is managed by the HDF Group. HDF Group was spun out of the University of Illinois in 2004 as a not-for-profit tasked with the continued development of HDF5.
HDF Group CEO Mike Folk told Oil IT Journal ‘HDF5 is well suited to meet the increasing demands in oil and gas to handle ever bigger and more complex data. Current industry implementations that we are aware of include Energistics’ Resqml standard for reservoir data and EMGS’ H5EM-TS exchange standard for field EM data which is also used by Statoil. The Passcal Instrument Center at New Mexico Tech has developed PH5, an alternative to SEG-Y for seismic data archival based on HDF5 and New Zealand seismic Globe Claritas uses the format in its processing software.’
Folk elaborated on the HDF Group’s status as a not-for-profit. ‘As a non-profit, our goal is to promote and support HDF. HDF5 is open source in the sense that we publish the source code for the basic HDF5 library and tools, and it has a BSD-like license. But it’s not developed and maintained by a large community of people the way most open source software is. Most development and maintenance is done in-house. Today, all our revenues come from two organizations to whom HDF is critical.’
‘We are now investigating an expansion of our business model, developing tools that we might sell and maybe a ‘pro’ version of the library. We also have a project to investigate HDF software-as-a-service, particularly within a cloud environment.’
For those of you located in Houston can catch up with HDF5 at the Rice Oil & Gas HPC Workshop in March.
© Oil IT Journal - all rights reserved.