Cost-Based Optimization of Integration Flows - Datenbanken ...

3.5 Experimental Evaluation
First, in order to investigate the benefits of rewriting patterns to parallel subflows in
more detail, we executed a speedup experiment. Figure 3.23 shows the results of this
experiment on the rewriting of sequences to parallel flows (WC2). We used a plan that
contains a sequence of m = 100 independent Delay operators, and we explicitly varied
the number of threads k (concurrent subflows) used for parallelism (Fork operator) in
order to evaluate the speedup. Due to the distribution of m operators to k forklanes, a
theoretical speedup of m/ ⌈m/k⌉ is possible. Then, we varied the waiting time of each
single Delay operator from 10 ms to 40 ms in order to simulate different network delays
and waiting times for external systems, respectively. This experiment was repeated ten
times. As a result, an increasing maximum speedup (at an increasing number of threads)
was measured with increasing waiting time (note that the fall-offs were caused by the Java
garbage collector). This strong dependence of multi-tasking benefit on the waiting time
of involved operators justifies the decision to use the waiting time as main cost indicator
when rewriting sequences and iterations to parallel flows.
Second, we used our running example plans in order to investigate the scalability of
optimization benefits with increasing input data size. Based on the experimental results
shown in Figure 3.22, we re-used the workload configuration and all plans except plan P 2
because for this plan no optimization technique could be applied. In detail, we executed
20,000 plan instances for each running example plan and compared the periodical reoptimization
with no-optimization varying the input data size d ∈ {1, 2, 3, 4, 5, 6, 7} (in
100 kB messages), which resulted in a total processed input data size of up to 13.35 GB
(for d = 7). Note that we varied this input data size only for the initially received message
of a plan, while for plans P 3 , P 6 and P 8 , we changed the cardinality of externally loaded
data sets because these plans are time-based initiated. Further, we fixed an optimization
interval of ∆t = 5 min, a sliding window size of ∆w = 5 min and EMA as the workload
aggregation method. For all investigated plans, we observe that the relative benefit of
optimization increases with increasing data size as shown in Figure 3.24. Essentially, the
same optimization techniques were applied and thus, the optimization time is unaffected
by the used input data size. The highest benefits were reached by the data-flow oriented
optimization techniques. For example, the unoptimized versions of plans P 4 , P 6 , P 7 show
a super-linearly increasing execution time with increasing data size, while the optimized
versions shows almost a linear increasing execution time. Thus, the optimized versions
exhibit a better asymptotic behavior, which is for example, caused by rewriting nested
loop Joins to combinations of Orderby and merge Join subplans. With this in mind, it
is clear that arbitrarily high optimization benefits can be reached with increasing input
data size (input cardinalities).
Optimization Overhead
In addition to the evaluation of performance benefits and the scalability with increasing
input cardinalities and increasing number of operators, we evaluated the optimization
overhead in more detail. Therefore, we conducted a series of experiments, where (1) we
compared the optimization overhead of our exhaustive optimization approach versus the
heuristic optimization approach and (2) we analyzed the overhead of statistics monitoring
and aggregation in detail.
First, we evaluated the influence of using the exhaustive optimization algorithm A-PMO
or the heuristic A-HPMO. We already discussed when it is applicable to use the second
heuristic A-CPO and therefore did not include it into the evaluation. Essentially, the ex-
79