Speaking at a ‘Lunch and Learn’ session of Society of HPC professionals, Dan Bedard executive director of the iRods* consortium showed how it is addressing the challenge of multi-terabyte data sets.
iRods is a cross industry organization with backing from IBM, EMC, Seagate, Nasa and others. The consortium develops open source software for managing ‘big, important and complex’ unstructured data. iRods also provides data virtualization and discovery along with workflow automation.
Bedard considers seismic data as a prime candidate for the iRods approach. Land seismic data growth has been spectacular, up from 400,000 sensors per sq. km. in 2005 to as much as 36 million in 2009. Bedard also stressed the importance of open source software in data management to avoid vendor lock-in.
iRods is managed by the Renci, the Renaissance Computing Institute, a research unit of UNC Chapel Hill and the Data intensive cyber environments (Dice) group, a spin out of the San Diego supercomputer center, now jointly housed, in ‘bi-coastal’ fashion, at Chapel hill and UCSD.
* Integrated rule-oriented data system.
© Oil IT Journal - all rights reserved.