Cost-Based Optimization of Integration Flows - Datenbanken ...

4.2 Plan Vectorization
the control-flow semantics of an integration flow as well as the Copy operator for data flow
splits. The semantics of these operators are described in Table 4.1.
In order to realize the pipes-and-filter execution model, operators are executed in socalled
multi-threaded executed buckets. Figure 4.4 illustrates the conceptual model of
such an execution bucket.
input queue
q in1
sync input queue
sq in
…
dequeue()
Thread
input queue
q in k1
sync input queue
sq in
input queue
q in
dequeue()
Thread
input queue
q in1
dequeue()
Thread
input queue
q in2
operator o i
operator o i
operator o i
output queue
q out1
enqueue()
…
output queue
q out k2
enqueue()
output queue
enqueue()
output queue
sync output queue
sq out
q out
sync output queue
sq out
q out
sync output queue
sq out
(a) Generic Operator
(b) Unary Operator
(c) Binary Operator
Figure 4.4: Conceptual Model of Execution Buckets
In general, Figure 4.4(a) shows the generic model of an execution bucket that contains
a set of input message queues, a set of output message queues, a single so-called input
synchronization queue, a single output synchronization queue, and an operator. The
bucket is a thread with an endless loop, where in each iteration it dequeues from all
input message queues, dequeues from the synchronization queue, executes the operator,
and enqueues the results into all output queues. However, with regard to the defined
flow meta model, unary (see Figure 4.4(b)) and binary (see Figure 4.4(c)) operators with
one or two inputs and a single output are most common. Note that only the unary
operators Xor and And require synchronization input queues, while each operator can have
a synchronization output queue, which depends on its position within the plan and the
need for serialization. Apart from the synchronization queues, similar operator models
are commonly used in the context of data stream management systems [GAW + 08] and
related system categories [CEB + 09, CWGN11].
4.2.2 Rewriting Algorithm
In this subsection, we first describe the core rewriting algorithm and second, we specify
the control-flow-specific rewriting techniques, which preserve the external behavior. Both
aspects are required in order to enable plan vectorization as an optimization technique.
Core Algorithm
The basic rewriting algorithm that is described in the following can be applied for all types
of operators of our integration flow meta model.
In detail, Algorithm 4.1 consists of two parts. In a first part, the dependency analysis
is performed by determining all data dependencies between operators. There, we create
a queue instance for each data dependency between two operators (the output message
of operator o i is the input message of operator o j ). Internally, our optimizer reuses the
existing dependency graph that was described in Subsection 3.2.1. In a second part, we
93