Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
period is longer than the sliding time window size (∆t ≥ ∆w). In detail, we aggregated
700,000 statistics (execution times W (o i ) only) and we observed that all statistics were
aggregated in less than 20 ms. The single aggregation methods differ only slightly in their
execution time, where MA is the fastest method but only minor differences are observable.
If ∆t < ∆w or no ∆w is used, incremental statistics maintenance is required. Thus, we
repeated the experiment with our incremental aggregation methods. When comparing full
and incremental maintenance, we see that the incremental methods are a factor of 1.5 to
3 slower than the full methods because they require additional computation efforts for
producing valid intermediate results and for many method invocations. EMA is the fastest
incremental method based on its incremental nature. Our Estimator comprises all of these
aggregation methods and some additional infrastructural functionalities, where we use the
incremental EMA as default aggregation method. The maintenance of all three statistics
(|ds in |,|ds out |, and W (o i )) for all plan instances of the test set (2,100,000 statistic tuples)
using our Estimator is illustrated as Estimator (EMA). In conclusion, the overhead for
statistics maintenance during the full comparison scenario was 106 ms. This is negligible
compared to the cumulative execution time of 140 min in the optimized case.
Workload Adaptation
Due to changing workload characteristics, the sensibility of workload adaptation has high
importance. According to Subsection 3.3.3, there are three possibilities to influence the
sensibility of workload adaptation: (1) the workload sliding time window size ∆w, (2) the
optimization period ∆t, and (3) the workload aggregation method Agg. We evaluated
their influence in the following series of experiments.
Figure 3.27: Workload Adaptation Delays
Figure 3.27 shows the results of an experiment, where we executed n = 20,000 instances
of plan P 3 and a modified plan P 3 ′ (with eager group-by) with disabled periodical reoptimization.
After n = 5,000 and n = 15,000 instances, we changed the cardinality of
one of two input data sets (workload changes WC1 and WC2). While in the first part,
the eager group-by was most efficient, the simple join and group-by performed better after
WC1. We fixed a sliding window size of ∆w = 5,000 s and MA as the workload aggregation
method. It took 2,100 plan instances to adapt to the workload shift and the plan changed
(PC1 at break even point between estimated plan costs). This adaptation delay depends
on the used sliding time window size ∆w and the aggregation method.
82