Cost-Based Optimization of Integration Flows - Datenbanken ...

5.2 Horizontal Queue Partitioning
Algorithm 5.2 MFO Plan Rewriting (A-MPR)
Require: operator sequence o, partitioning scheme ba
1: D ← ∅
2: for i ← 1 to |o| do // for each operator o i
3: // Part 1: Dependency Analysis
4: for j ← i to |o| do // for each following operator o j
5:
δ
if ∃o j → oi then // existing data dependency over data object var
6: D ← D ∪ d〈o j , var, o i 〉 with d〈o j , var, o i 〉 ← create dependency
7: // Part 2: Plan Rewriting
8: for k ← 1 to |D| do // for each dependency d
9: d〈o y , q, o x 〉 ← d k
10: if o i ≡ o y then // o i is target of data dependency
11: if ba(o y , var) < ba(o x , var) then // o i benefits from lower-level partition
12: o x .insertAfter(o s ) with o s ←createPSlit
13: o y .insertBefore(o iter ) with o iter ←createIteration
14: ba cur ← ba(o y , var)
15: for j ← i to |o| do // for each following operator o j
16: if ba(o j ) = ba cur or ba(o j ) = NULL then
17: o.remove(o j )
18: o iter .insertLast(o j )
19: D.modify(o iter , o j ) // change the old dependencies
20: else if ba(o j ) < ba cur then
21: recursive plan rewriting (lines 12-24)
22: else // ba(o j ) > ba cur
23: o j .insertBefore(o m ) with o m ←createPMerge
24: break
25: return o
o, the derived partitioning scheme ba and the created set of dependencies D. For each
operator o i in o, we iterate over all dependencies in D followed by the main rewriting
rules. If the partitioning attribute of operator o x (source of the data and thus, target
of data dependency) is higher than the partitioning attribute of the source of the data
dependency o y , we insert a PSplit operator as well as an Iteration operator. We then
iterate over all following operators (including o y ) and evaluate if they can be included
into the iteration body by comparing the partitioning attributes with ba(o j ) = ba cur . If
the required level is a lower-level attribute, we recursively invoke the insertion of PSplit
and Iteration operators. In contrast, if we determined an operator with a higher-level
attribute, we found the end of the current Iteration operator and insert a PMerge operator.
For binary operators (e.g., Join operator), there might be a difference between
partitioning attributes (for one side an Iteration was inserted). In this case, we do not
use a combined iteration but a direct positional lookup at the higher-level partition. A
single PSplit or PMerge operator can bridge multiple levels of the partitioning scheme
such that for plan rewriting, the lower, equal, and higher comparison is sufficient.
This algorithm is used whenever the partitioning scheme (derived from the plan) changes
during periodical re-optimization. Thus, although the algorithm is not executed during
each re-optimization step, it might be used during runtime and hence, it is worth to
141