Cost-Based Optimization of Integration Flows - Datenbanken ...

6 On-Demand Re-Optimization
The resulting PlanOptTree for a sequence of m = 5 operators that has been distributed
to k = 3 execution buckets is illustrated in Figure 6.12. With regard to cost-based vectorization,
we need to include operator nodes for all operators, but only the execution time as
atomic statistic nodes. Furthermore, we use the complex statistic node W (o max ) + λ and
the aggregated bucket execution costs W (b i ) for each bucket with more than two operators.
Then, there are three optimality conditions (oc 1 -oc 3 ), which check that the bucket
execution costs (or operator execution costs) are below this maximum. In addition, two
optimality conditions (oc 4 and oc 5 ) are used in order to check if all operators belong to
the right bucket with regard to producing the same result as the A-CPV does.
Whenever one of the optimality conditions is violated, we trigger directed re-optimization.
In this case we know the operator or execution bucket, respectively, which reasoned
this violation. In contrast to the the full A-CPV, we directly start the cost-based plan
vectorization at this bucket and hence, might not need to evaluate all operators. However,
the worst-case time complexity of O(m) is not changed by the directed cost-based plan
vectorization because we might start directed re-optimization at the first operator o 1 .
6.4.3 Multi-Flow Optimization
Similarly to the cost-based vectorization, on-demand re-optimization can also be applied
to the data-flow-oriented optimization technique multi-flow optimization that has been
discussed in Chapter 5. It is based on the algorithms of deriving partitioning schemes
(A-DPA) and on the waiting time computation (A-WTC). Furthermore, we follow the
optimization objective of minimizing the total latency time with
φ = max |M ′ |
∆t
= min T L (M ′ ), (6.11)
where T L (M ′ ) is computed by
⌈ |M
ˆT L (M ′ , k ′ ′ ⌉
|
) = · ∆tw + Ŵ (P ′ , k ′ ). (6.12)
k ′
and Ŵ (P ′ , k ′ ) is computed by Ŵ (P ′ , k ′ ) = W − (P ′ )+W + (P ′ )·k ′ for arbitrary k ′ = R·∆tw.
Due to our specific cost model extension, this minimum is given at ∆tw = W (P ′ , ∆tw · R)
such that we can compute ∆tw by ∆tw = W − (P ′ )/(1−W + (P ′ )·R). In order to ensure the
latency time constraint at the same time, we additionally evaluate the validity condition
of (0 ≤ W (P ′ , k ′ ) ≤ ∆tw) ∧ (0 ≤ ˆT L ≤ lc). In general, there are different cases, which
reason different optimality conditions. Here, we concentrate on the default case, where
the computed waiting time fulfills the validity condition.
For a plan P with m = 5 operators and h = 2 partitioning attributes, there exist
h + 3 = 5 optimality conditions. First, h − 1 = 1 optimality conditions are required
with regard to the derivation of partitioning schemes, where we order the partitioning
attributes according to the monitored selectivities with oc 1 : sel(ba 1 ) ≥ sel(ba 2 ). Second,
for this case of a valid waiting time, we require four optimality conditions—independent
of the number of partitioning attributes—in order to represent the validity condition with
oc 2 : 0 ≤ W (P ′ , k ′ ), oc 3 : W (P ′ , k ′ ) ≤ ∆tw, oc 4 : 0 ≤ ˆT L , and oc 5 : ˆT L ≤ lc).
The PlanOptTree that represents these optimality conditions is shown in Figure 6.13.
Essentially, it contains operator nodes for all five operators. Only for operators with
partitioning attributes, we monitor the selectivity according to this attribute, while for all
186