Cost-Based Optimization of Integration Flows - Datenbanken ...

5.5 Experimental Evaluation
data-driven integration flow use cases (plans P 1 , P 2 , P 5 , and P 7 ), which have been described
in Section 2.4. Furthermore, we used the following scale factors: the number of
messages |M|, the message rate R, the selectivity according to the partitioning attribute
sel, the batch size k ′ , the message rate distribution function D, the maximum latency
constraint lc, and the data size d of input messages (in 100 kB).
End-to-End Comparison and Optimization Benefits
First of all, we investigate the end-to-end optimization benefit achieved by multi-flow optimization
and the related optimization overhead. We compared the multi-flow optimization
with no-optimization, while all other optimization techniques have been disabled. Similar
to the use case comparison in Section 3.5 and 4.6, we executed 20,000 plan instances for
each asynchronous, data-driven example plan (P 1 , P 2 , P 5 , and P 7 ) and for both execution
models. We reused the same workload configuration as already presented (without correlations
and without workload changes). Furthermore, we fixed the cardinality of input
data sets to d = 1 (100 kB messages), an optimization interval of ∆t = 5 min, a sliding
window size of ∆w = 5 min and EMA as the workload aggregation method. With regard to
multi-flow optimization, we did not use the computed waiting time but directly restricted
the batch size to k ′ = 10 in order to achieve comparable results across the different plans.
(a) Cumulative Execution Time
(b) Cumulative Optimization Time
Figure 5.15: Use Case Comparison of Multi-Flow Optimization
Figure 5.15(a) shows the resulting total execution times. To summarize, we consistently
observe significant execution time reductions that have been achieved as follows:
• P 1 : The plan P 1 benefits from MFO in several ways. First, the Switch operator
o 2 is executed once for a message batch because the switch expression attribute
/material/type is used as the only partitioning attribute. Furthermore, the Assign
operators o 4 , o 6 , and o 8 are also executed only once because the result is exclusively
used by the partition-aware Invoke operators o 7 , and o 9 . These writing Invoke operators
show additional benefit because a single operator instance is used to process
all messages of a batch. Overall, this achieves a throughput improvement of 62%.
• P 2 : The plan P 2 mainly benefits from executing the Invoke operator o 3 and the
predecessor Assign operator o 2 only once for a whole partition. There, the predicate
part /resultsets/resultset/row/A1 Custkey is used as the partitioning attribute.
Additional benefit is achieved by the final Assign and Invoke operators o 5 and o 6 .
In total, an improvement of 53% has been achieved.
157