Big Data Analytics

140 D. Sitaram and K.V. Subramaniam
requests that are queued. Further, given a large factory setting, with a multitude of
such machines, the velocity of data ingestion requires non-traditional means.
While this appears to be an example of an IoT (Internet of Things) system, a
more careful examination of the system reveals that the intelligence is derived from
not only sensors that read data but also a mixture of cameras that take images for
tracking quality, process configuration files, and log files produced by the controlling
machines. In other words, given the volume and variety of the data that is to
be ingested, it is better to treat the system as a big data system so that the analysis
can derive value for the manufacturing unit.
The above considerations necessitate the introduction of systems that can process
and draw inferences from the events in real time. As argued earlier, for many
applications, an event model that captures the relationship between the different
events is needed. In the next section, we give a brief overview of various features of
such a complex event processing system.
3 Basic Features of Complex Event Systems
A real-time complex event processing system that is capable of processing millions
of events from various types of sources that can be viewed as illustrated in Fig. 4
[12]. The event observers or the sources on the left generate events. In our
multi-tank cleaner example, the sources refer to timestamped values of temperature,
pressure, and batch codes of the wafers being processed. The brokers or event
processing agents encode the business logic to act upon the events. For example, to
decide if the temperature reached its critical value over an interval, the brokers have
to process a sequence of values to determine if the operating temperature is valid. If
so, it signals an event indicating the operating temperature which is valid. If not, an
operating temperature invalid event is raised; this generated event can act as a
source event for subsequent stages in the event processing network. Observe that
the graph of the event processing network is logical and does not imply the physical
distribution of the brokers. The processing agents can be distributed across a set of
machines and need not be on a single machine. Also, this network must not have
any cycles. There may be consumers that are interested in the events being generated
by the system. These are represented by the sinks in Fig. 3.
A complex event processing system such as the one described above will have to
consider the following aspects in designing the system. These are based on the eight
requirements enumerated by [34].
1. Data Model/Semantics—this provides a mechanism to handle data from a
variety of sources and also to handle relationships across various events. Consider
our example of the manufacturing intelligence application. Herein, we
need not only a model for being able to extract events from various types of
sensors and log files, but also a model that can help express the relationship
among the various events.