Cost-Based Optimization of Integration Flows - Datenbanken ...

6.5 Experimental Evaluation
demand re-optimization directly reacts to the workload change and switches plans. Hence,
over time, execution time reductions are yielded due to the fast adaptation. In order to
achieve this with periodical re-optimization, a really small ∆t is required. However, this
would increase the total re-optimization time. Finally, note that the small differences
of absolute execution times compared to Chapter 3 are caused by randomly generated
messages for this experiment according to the given selectivities.
Complex-Plan End-to-End Comparison
In addition to the simple-plan scenario, we executed a second series of experiments using
the more complex example plan P 7 ′. The difference of plan P 7 ′ compared to the already
introduced plan P 7 is that we explicitly changed the join query type chain to a clique
type in order to have the possibility of arbitrary join reordering. This plan receives a data
set, loads data from four different systems, and executes schema transformations with
translation operators (XSLT scripts). Finally, the five data sets are joined using four join
operators and the result is sent to a fifth system. We executed 20,000 plan instances using
the input cardinalities shown in Figure 6.15(b) and the same parameter configurations as
for the simple plan scenario. For the four loaded data sets, we used input cardinalities of
d ∈ {2, 4, 8, 16} (in 100 kB). After every 2,000 instances, we changed the input cardinalities
of the four external systems round-robin as shown in Figure 6.15(a).
Regarding the results that are shown in Figures 6.15(c)-6.15(f), we observe similar characteristics
as within the simple-plan scenario. It is important to note that (1) the higher the
optimization opportunities of a plan, and (2) the higher the number of workload changes,
the higher the execution time improvements achieved by on-demand re-optimization due
to the higher importance of immediate adaptation (see Figures 6.15(d) and 6.15(f)). However,
we restricted ourself to moderate differences of input data sizes in order to show only
this main characteristic rather than showing arbitrarily high improvements. Further, we
also observe, on average, higher re-optimization time improvements as within the simpleplan
comparison scenario due to the higher influence of immediate re-optimization and
directed re-optimization (see Figures 6.15(c) and 6.15(e)). The outliers for periodical reoptimization
have been cause by the Java garbage collection because our implementation
of the full join enumeration (DPSize) requires many temporary data objects, which are
lazily deleted if space is required. This effect is only visible for periodical re-optimization
because there, we used full join enumeration, where more objects have been created and the
join enumeration is invoked more often than our on-demand re-optimization. In conclusion,
the benefit of on-demand re-optimization increases with increasing plan complexity
and increasing frequency of workload changes.
Scalability
In order to investigate the influencing aspects such as the number of workload shifts wc,
the input data size d and the optimization interval ∆t in more detail, we conducted an
additional series of scalability experiments, where we varied these parameters. We compare
the unoptimized case with periodical and on-demand re-optimization using the cumulated
plan execution time, the cumulated optimization time, and the number of re-optimizations.
As the experimental setup, we executed 5,000 plan instances of P 5 for each configuration,
where each workload shift switches between the two selectivity configurations of (sel(o 2 ) =
0.8, sel(o 3 ) = 0.6, sel(o 4 ) = 0.1) and (sel(o 2 ) = 0.1, sel(o 3 ) = 0.6, sel(o 4 ) = 0.8). As
191