What is Complex Event Processing (CEP)?
CEP takes multiple streams of real time data and applies standing queries to identify underlying root causes of events in the data stream.
How will the final PI/CEP combo look? CEP is an SQL Server-based time series solution, and PI is a stand-alone RT database. Does the CEP announcement imply a migration or merger of the PI RTDB with SQL Server? Or will the two run in parallel?
CEP is not a database, although databases are an important aspect of CEP. A database could be the source of data for CEP queries and events, once identified, could be stored in a database. There are three main components of CEP: input adaptor, the CEP engine and output adaptor. The adaptors can be written by 3rd parties such as OSIsoft; Microsoft will supply adaptors to and from SQL Server. The CEP engine supports the concept of a standing query, a query that runs continuously on the flow of data into the engine. Microsoft provides a rich query language and toolset that handles just about any conceivable CEP scenario.
How is it going to impact PI System users?
We’ll be leveraging this technology to support high speed, sophisticated analysis of data as it flows into the PI System. Identified events will be stored in PI and can be leveraged by our notification or visualization products. We will do this by creating an input adaptor to CEP, a way to route data heading to the PI Server into the CEP engine. We will also build a query configuration tool tuned to specific customer scenarios at first, and gaining in sophistication over time. And there will be an output adaptor to write query results to the PI System.
It seems that CEP is a pretty good description of what is already done by PI/AF. Pat Kennedy used the CEP term back in 2006*. I am not quite sure if this story is a technical one or if it just reflects the ongoing ‘symbiosis’ between OSIsoft and Microsoft.
I’d say it is both. If you look at our existing analytics—PEs, ACE, Totalizer, there are some CEP aspects. But there are some limitations. Our syntax and queries are rather simple (with the exception of ACE which is VB.Net) and are limited to 1000’s of events/second and to 1000’s of simultaneous queries. Also, to support disparate data sources we must first get the data in PI. Microsoft’s CEP technology will address all of the above. We will likely first leverage the areas where better scaling is required.
Where did CEP start?
Stream prossessing and CEP started making noise in data processing about 7 years ago. Michael Stonebraker (StreamBase) was one of the first. When we read his papers we realized it was similar to some of the things we have been doing for years. But, we were different in that we could easily store all the data coming into our system and store the events we detected or calculated. Of course, CEP is much more generalized then we are—we stick to value/timestamp pairs. The most mainstream CEP scenario is around financial transactions such as stock trading.
This brings up another aspect—CEP will be mainstream technology produced by the big players such as Microsoft and Oracle. They are investing significant resources to this technology and in the long run we are better off leveraging this work. We will focus on the scenarios and issues around manufacturing, utilities and process industries rather than the details of query languages, parsing, etc. Of course, being a strong Microsoft shop, we will leverage their technology.
More from www.osisoft.com.
© Oil IT Journal - all rights reserved.