BP’s bet on a software/hardware combo from Composite Software and Netezza (Oil IT Journal September 2008) has paid off big time, according to Stuart Bonnington, BP America’s head of information architecture, upstream IT and services. Bonnington’s team of data modelers and architects work on MDM, mapping, search and business intelligence.
BP has some 3,000 applications in its upstream portfolio and wanted a ‘single logical source for all data access, conformant to a common business model.’ The data virtualization solution comprises a stack of connectors to 40 upstream data sources, a data virtualization layer and applications on top. To date, 32 applications are connected covering a range of production, well data and financial needs. The data virtualization layer provides a single point of access for some 600 ‘common canonical’ entities, modeled with the Embarcadero Studio tool.
A critical component of the system is the ‘semantic’ metadata model. Bonnington explained, ‘We ensure that all data conforms to a semantic structure. Applications query the record of reference in the virtualization layer rather than the data source.’ Total query throughput on the system is nearly 50,000 queries per day.
The IBM Netezza data warehouse appliance provides data storage inside the virtualization layer. Data in source systems is copied to the Netezza staging database. Refresh intervals are determined by data administrators according to the business case. Some real-time data sources may bypass the staging process. Bonnington added, ‘The Netezza appliance is what makes data virtualization work for us, from a scalability and performance perspective. Netezza is essentially a massive engine underneath Composite that makes it run faster.’
It addition to the raw power provided by Netezza’s 200 processors, it offers a more sophisticated alternative to Composite’s native caching capability by handling refresh schedules and dependencies.
The system has been running for two years with only three hours downtime. Its 15 terabytes of data require minimal support—around 10 minutes of DBA time per day for the Netezza and one and a half full time employees for the whole environment.
Results have been spectacular. Bonnington said, ‘The whole process is so fast that there is no need to create any data marts. The partnership between Composite and Netezza is super powerful. Our development costs have been cut by 40%.’
Bonnington received Composite’s ‘Virtualization Champion Award’ this month for ‘achieving and promoting data virtualization across his organization and in the broader data integration space.’ More from www.oilit.com/links/1111_1 (Composite) and www.oilit.com/links/1111_2 (Netezza).
© Oil IT Journal - all rights reserved.