Cost-Based Optimization of Integration Flows - Datenbanken ...

6.5 Experimental Evaluation
abstractions of statistics maintenance, cost estimation and re-optimization for enabling the
use of both alternative optimization models. Most importantly, we extended the optimizer
interface in the form of exchanging partial PlanOptTrees, which includes the modification
of optimization techniques in order to enable directed re-optimization. Furthermore, we
ran the experiments on the same platform as used within the rest of this thesis. With the
aim of simulating arbitrary changing workload characteristics, we synthetically generated
XML data sets with varying selectivities and cardinalities as input for our integration
flows.
Simple-Plan End-to-End Comparison
In a first series of experiments, we compared periodical re-optimization with on-demand
re-optimization (both asynchronous with inter-instance plan change). In order to conduct
a fair evaluation, we used our fairly simple example plan P 5 because the benefit of ondemand
re-optimization increases with increasing plan complexity. Essentially, we reused
the end-to-end comparison experiment from Chapter 3, where we executed 100,000 instances
for the non-optimized plan as well as for both optimization approaches, and we
then measured re-optimization and plan execution times. The execution time already includes
the synchronous statistic maintenance and the evaluation of optimality conditions
in case of on-demand re-optimization. During execution, we varied the selectivities of the
three selection operators (see Figure 6.14(a)) and the input cardinality (see Figure 6.14(b)).
The input data was generated without correlations. With regard to re-optimization, there
are four points (∗1, ∗2, ∗3, and ∗4) where a workload change (intersection points between
selectivities) reasons the change of the optimal plan. For periodical re-optimization, we
used a period of ∆t = 300 s, while for on-demand re-optimization, we used a minimum
existence time of ∆t = 1 s and a lazy condition evaluation count of ten.
Figure 6.14(c) shows the re-optimization times, while Figure 6.14(e) illustrates the cumulative
optimization time. There, we used the elapsed scenario time as the x-axis in
order to illustrate the characteristics of periodical re-optimization. We see that periodical
re-optimization requires many unnecessary optimization steps (36 steps), while on-demand
re-optimization is only triggered if a new plan will be found (6 steps). For workload shifts
∗2 and ∗3, two on-demand re-optimizations were triggered due to the two intersection
points of selectivities and the used exponential moving average, which caused a small
statistics adaptation delay that led to exceeding the lazy count before converging to the
final statistic measure. With a different parameterization, only one re-optimization was
triggered for each workload shift. Despite directed re-optimization, a single optimization
step is, on average, slightly slower than a full re-optimization step due to the small
optimization search space of the applied optimization techniques (selection reordering
and switch path reordering). The reason is that directed re-optimization requires some
constant additional efforts and benefits only if several operators can be ignored during
optimization. Further, the re-optimization time of this plan is dominated by the physical
plan compilation and the waiting time for the next possible exchange of plans (due to
the asynchronous optimization). As shown in Chapter 3, if we would use a larger ∆t for
periodical re-optimization, we would use suboptimal plans for a longer time and hence, we
would miss optimization opportunities. As a result, over time, on-demand re-optimization
yields optimization time improvements because it requires fewer re-optimization steps.
Figures 6.14(d) and 6.14(f) show the measured execution times. The different execution
times are caused by the changing workload characteristics in the sense of different input
189