Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
We ran our experiments using the same platform as described in Section 3.6. Further,
we executed all experiments on synthetically generated XML data (using our DIPBench
toolsuite [BHLW08c]) due to only minor influence of the data distribution of real data sets
on the benefit achieved by vectorization because it is a control-flow-oriented optimization
technique. However, there are several aspects with influences on vectorization. In general,
we used five scale factors for all three execution approaches: the data size d of input
messages, the number of operators m, the time interval t between two arriving messages,
the number of plan instances n, and the maximum constraint of messages in a queue q.
End-to-End Comparison and Scalability
Similar to the general comparison experiment of optimized and unoptimized plan execution,
which results are shown in Figure 3.22, we first evaluated the impact of vectorization
and cost-based vectorization compared to the unoptimized execution for our example use
case plans. In detail, we executed 20,000 plan instances for all asynchronous, data-driven
example plans (P 1 , P 2 , P 5 , and P 7 ) and for each execution model. We fixed the cardinality
of input data sets to d = 1 (100 kB messages) and used the same workload configuration
(without workload changes and without correlations) as in the mentioned experiment of
Chapter 3. Note that the normal cost-based plan rewriting is orthogonal to vectorization,
where vectorization achieves additional improvements except for the effects of rewriting
patterns to parallel flows. In order to be focused on vectorization, we disable all other
optimization techniques. Furthermore, we fixed an optimization interval of ∆t = 5 min,
a sliding window size of ∆w = 5 min and EMA as the workload aggregation method. To
summarize, we consistently observe significant total execution time reductions (see Figure
4.19(a)) of 71% (P 1 ), 72% (P 2 ), 69% (P 5 ), and 55% (P 7 ). In contrast to Chapter 3, we
measured the scenario elapsed time (the latency time of the message sequence) because
for vectorized execution, the execution times of single plan instances cannot be aggregated
due to overlapping message execution (pipeline semantics).
(a) Scenario Elapsed Time
(b) First Optimization Time
Figure 4.19: Use Case Comparison of Vectorization
First, the full vectorization approach leads to a significant reduction of the total elapsed
time for execution of the sequence of 20,000 plan instances. We achieved a speedup of factor
three for the plans P 1 , P 2 , and P 5 , while for the plan P 7 we achieved a speedup of factor
two. Furthermore, the cost-based vectorization further improved the full vectorization by
about 10%. However, there are cases, where the cost-based vectorization caused only a
minor improvement because plans such as P 1 are too restrictive with regard to merging
execution buckets (e.g., the combination of a Switch operator with specific paths is not
120