Cost-Based Optimization of Integration Flows - Datenbanken ...

6 On-Demand Re-Optimization
evaluating the MEMO structure. However, due to the requirement of robustness, we use the
POT3 configurations for all end-to-end comparison scenarios. Furthermore, we show only
the total statistic maintenance time because these times scale linear with an increasing
number of statistic tuples due to the use of the incremental EMA aggregation method. Finally,
compared to the performance benefit of on-demand re-optimization, the overhead
for statistic maintenance is negligible.
Additionally, we evaluated the algorithm overhead when creating, using and updating
PlanOptTrees. There, we used the already described POT3 configuration. Figure 6.18(b)
illustrates the results, varying the number of operators m of a plan and hence, indirectly
varying the number of optimality conditions. The PlanOptTree is created over all
m operators (where we simulated the actual optimization techniques and directly provided
the partial PlanOptTrees), while triggering re-optimization and the subsequent
update of the PlanOptTree addressed violated optimality conditions only, which depend
on the randomly changed statistics. We observe a moderate execution time, where the
creation and the update of PlanOptTrees were dominated by the merging of partial
PlanOptTrees. Due to the different numbers of addressed optimality conditions the update
of PlanOptTrees is more efficient than the creation of an initial PlanOptTree. It
is important to note the almost linear scaling of creating PlanOptTree, triggering reoptimization
and updating PlanOptTrees. In conclusion, the overhead of PlanOptTree
algorithms is fairly low and can be neglected as well because it is only required for initial
deployment or during on-demand re-optimization.
Robustness
Compared to periodical re-optimization, on-demand re-optimization is more sensitive with
regard to workload changes because it directly reacts on detected violated optimality conditions.
According to the robustness of optimization benefits, there are two major effects
that are worth mentioning. First, the on-demand re-optimization does not require the
specification of an optimization interval. Therefore, it is more robust to arbitrary workload
changes compared to periodical re-optimization as shown in Figure 6.16. Second,
for on-demand re-optimization there is a higher risk of frequently changing plans due to
correlated data or almost equal selectivities that alternate around equality (see Example
6.5). Therefore, we introduced the strategies of the correlation table, minimum existence
time and lazy condition evaluation. The minimum existence time simply ensures
a lower bound of time between two subsequent re-optimization steps and thus, linearly
reduces the number of re-optimization steps. Furthermore, the strategy of lazy condition
evaluation overcomes the problem of almost equal selectivities. While the effects of these
two strategies are fairly obvious, the use of the correlation table requires a more detailed
discussion using a series of experiments.
Therefore, we reused the correlation experiment of Chapter 3 in order to evaluate the
on-demand re-optimization on correlated data with and without the use of our lightweight
correlation table. We executed 100,000 instances of our example plan P 5 and compared
the resulting execution time. We used a minimum existence time of ∆t = 5 s, a lazy
condition evaluation count of ten, and a re-optimization threshold of τ = 0.001. Figure
6.19(a) recaps the conditional selectivities P (o 2 ), P (o 3 |o 2 ), and P (o 4 |o 2 ∧ o 3 ) of the
three Selection operators (with P (o 3 |¬o 2 ) = 1 and P (o 4 |¬o 2 ∨ ¬o 3 ) = 1), which lead to
a strong dependence (correlation) of o 3 on o 2 as well as of o 4 on o 2 and o 3 .
Figures 6.19(c) and 6.19(b) illustrate the resulting optimization times and execution
196