Cost-Based Optimization of Integration Flows - Datenbanken ...

3.3 Periodical Re-Optimization
Algorithm 3.2 Pattern Matching Optimization (A-PMO)
Require: operator op, dependency graph DG
1: if type(op) is Plan then
2: apply Plan techniques on op (Before) // WD10 (DPSize), WD13, MFO
3: o ← op.getSequenceOfOperators()
4: apply WC2 on o // apply technique for all sequences
5: for i ← 1 to |o| do // for each operator of the sequence
6: if type(o i ) ∈ (Plan, Switch, Fork, Iteration, Undefined) then // complex
7: o i ← A-PMO(o i , DG)
8: apply operator techniques on o i // WC1, WC4, WD1, WD2, WM1, WC3
9: else // atomic
10: apply operator techniques on o i // WM1, WM2, WM3, WD3, WD4, WD5,
// WD6, WD8, WD9, WD11, WD12
11: if type(op) is Plan then
12: apply Plan techniques on op (After) // Vect, HLB
enumeration is only executed once for the complete plan. Within this full optimization
algorithm, we use the DPSize [SAC + 79, Moe09] join enumeration algorithm. Another example
is the optimization technique multi-flow optimization (see Chapter 5). Second, for
complex operators (line 6), we recursively invoke this algorithm and subsequently, apply
available optimization techniques. Third, we apply operator-type-specific techniques for
the individual atomic operators (line 9). Note that from each operator, parent nodes and
all other operators are reachable. From the perspective of a single optimization technique,
however, only successors (following operators) are considered (forward-only) in order to
avoid to consider the same operator multiple times. Fourth, we apply all techniques, which
need to be executed on top level of a plan and after the operator-type-specific techniques.
Among others, we invoke the optimization technique vectorization (see Chapter 4). In
contrast to the full plan enumeration with dynamic programming approaches, this iterative,
transformation-based algorithm preserves the control-flow semantics of the given
plan and it iteratively improves the current solution. Thus, it can be aborted between
applying optimization techniques without loss of intermediate optimization results.
The worst-case time complexity of the A-PMO is given by the optimization technique
with the highest individual complexity. In our case this is the optimization technique
join enumeration, where we use the DPSize join enumeration algorithm. According to the
complexity analysis of DPSize [MN06] by Moerkotte and Neumann, it is given by O(n 4 )
for chain and cycle queries as well as by O(c n ) for star and clique queries, where n denotes
the number of joined input data sets.
Example 3.4 (Optimization Algorithm). Recall the plan P 1 that is shown in Figure 3.7(a).
The A-PMO recursively iterates over all operators and applies available optimization techniques.
We start at the top-level sequence of operators, where we apply the rewriting of
sequences to parallel flows 5 because no data dependencies exists between o 7 and o 8 . The
resulting plan P 1 ′ is shown in Figure 3.7(b). Then, we iterate over the individual operators.
5 Rule-based optimization techniques are applied during the initial deployment of a plan. As an example
consider the operators o 4 and o 6, which would be detected as redundant work and thus merged to a
single operator after o 2. This operator would have been included in the parallel flow of operator o 7.
For simplicity of presentation, we did not apply rule-based optimization techniques in the example.
49