Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
Algorithm 4.1 Plan Vectorization (A-PV)
Require: operator sequence o
1: B ← ∅, D ← ∅, Q ← ∅
2: for i ← 1 to |o| do // for each operator o i
3: // Part 1: Dependency Analysis
4: for j ← i to |o| do // for each following operator o j
5:
δ
if ∃o j → oi then
6: Q ← Q ∪ q with q ← createQueue()
7: D ← D ∪ d〈o j , q, o i 〉 with d〈o j , q, o i 〉 ← createDependency()
8: // Part 2: Graph Creation
9: if o i ∈ Switch, Iteration, Validate, Signal, Savepoint, Invoke then
10: // see rules on rewriting context-specific operators
11: else
12: b(o i ) ← createBucket(o i )
13: for k ← 1 to |D| do // for each dependency d
14: d〈o y , q, o x 〉 ← d k
15: if o i ≡ o x then
16: b(o i ).addOutputQueue(q)
17: else if o i ≡ o y then
18: b(o i ).addInputQueue(q)
19: if b(o i ).countOutputQueues() ≥ 2 then
20: createCopyOperatorAfter(b(o i ), B, D, Q)
21: else if b(o i ).countOutputQueues() = 0 then
22: createLogicalQueue(b(o i−1 ), b(o i ), B, D, Q)
23: B ← B ∪ b(o i )
24: return B
create the graph of execution buckets. This part contains the following three steps. First,
we create an execution bucket for each operator. Second, we connect each operator with
the referenced input queues. Clearly, each queue is referenced by exactly one operator,
but each operator can reference multiple queues. Third, we connect each operator with
the referenced output queues. If one operator must be connected to n output queues with
n ≥ 2 (its results are used by multiple following operators), we insert a Copy operator
after this operator in order to send the message to all dependency sources. Within the
createCopyOperatorAfter function the new operator, execution bucket and queue are
created as well as the dependencies and queue references are changed accordingly. Furthermore,
if an operator is connected to zero input queues, we insert an artificial dummy queue
(where empty message objects are passed) between this operator and its former temporal
predecessor. Based on this basic rewriting concept, additional rewriting rules for contextspecific
operators (e.g., Switch, Iteration) and for serialization and recoverability are
required (line 10). We use an example to illustrate the A-PV in more detail.
Example 4.2 (Automatic Plan Vectorization). Recall our example plan P 2 as well as the
vectorized plan P 2 ′ of Example 4.1. Figure 4.5(a) and 4.5(b) illustrate the core aspects
when rewriting P 2 to P 2 ′ . First, we determine all direct data dependencies δ and add those
to the set of dependencies D. If an operator is referenced by multiple dependencies (e.g.,
operator o 1 in this example), we need to insert a Copy operator just after it. Furthermore,
94