Cost-Based Optimization of Integration Flows - Datenbanken ...

2 Preliminaries and Existing Techniques
A. Data Transfer Optimization
Essentially, the problem of expensive access to external systems (by actively invoking the
existing outbound adapters) is tackled with data transfer optimization. In this context,
we distinguish between (1) streaming transfered data (where the amount of exchanged
data is almost unchanged) and (2) reducing the amount of transfered data.
Streaming Transfered Data. The standard invocation model of external systems or Web
services is the transfer of materialized messages, i.e. the complete message is serialized
(converted to a byte stream) or, more generally, marshalled and transfered to the external
system, where the message is unmarshalled and processed. In contrast, the streaming
invocation of Web services for each tuple achieves higher throughput by leveraging pipeline
parallelism, but it also causes overhead, incurred by parsing SOAP/XML headers and
network latencies. Thus, there is a trade-off between pipeline parallelism achieved by
streaming and the existing overhead. There are approaches that exploit this trade-off
by passing batches of tuples (chunks of messages) in the form of individual messages to
the external system. Srivastava et al. introduced the adaptive data chunking [SMWM06],
where the chunk size k was determined—based on the measured response time of a batch
c i (k i )—by minimizing the response time per tuple c i (k i )/k. This concept was refined by
Gounaris et al. to the use of an online extremum-control approach [GYSD08b, GYSD08a]
that was adapted from control theory. Both approaches use a combination of control-flow
semantics with iterator, instance-local execution. In contrast, Preißler et al. introduced
the concept of stream-based Web services [PVHL09a, PVHL09b] over multiple process
instances, which is thus classified as control-flow semantics with iterator, instance-global
execution. Here, splitting rules are defined and message buckets are determined by the
message content according to those splitting rules. These buckets are then streamed to
and from the external Web service without the overhead (network latency) of passing
individual messages. The concept of stream-based Web services has the advantage that
both the data transfer (similar to the previously mentioned approaches, but with less
latency) and the processing within the Web service (that is able to statefully work on
the defined subtrees of XML messages) are optimized. In addition, streaming subtrees of
messages is also advantageous for local processing steps [PHL10]. Although some of these
approaches are first steps towards the adaptive behavior of integration flows, they neglect
local processing costs in the sense of optimizing only calls to external systems or applying
static (rule-based) optimizations only.
Reducing Transfered Data. The aforementioned approaches use a stream-based data
transfer between the integration platform and the external systems. This does not affect
the amount of exchanged data (payload). In contrast, there are also approaches that can
reduce the amount of transfered data. First, BPEL-DT and BPEL/SQL achieve this by
using references to data sets instead of physically exchanging data. BPEL-DT [HRP + 07,
Hab09] reduces the transfered data by passing references to a data layer and using ETL
tools when possible. This reduction is achieved by using proprietary exchange formats
instead of XML. Furthermore, Vrhovnik et al. introduced the rule-based optimization
of BPEL/SQL processes [VSES08, VSS + 07], where several rewrite rules were defined in
order to condense sequences of SQL statements and to push down certain operations to the
DBMS. In particular, the tuple-to-set rewrite rules reduce the amount of transfered data.
Second, Subramanian and Sindre introduced the rule-based optimization of ActiveXML
workflows [SS09a, SS09b]. An ActiveXML document is an XML document that contains
several Web service calls in order to load dynamic external content. In this approach,
16