Cost-Based Optimization of Integration Flows - Datenbanken ...

4.6 Experimental Evaluation
approach was designed for multi- and many-core systems, in general, it can be extended
to the distributed case as well. There, extensions with regard to the communication costs
between several nodes as well as heterogeneous hardware (different execution times on different
server nodes) would be required. However, in this distributed setting, the cost-based
vectorization would be even more important because the number of involved server nodes
could be reduced without sacrificing the degree of parallelism.
4.6 Experimental Evaluation
In this section, we provide experimental evaluation results for both the full vectorization
and the cost-based vectorization of integration flows. The major perspectives of this
evaluation are (1) the performance in terms of message throughput, (2) the influence on
latency times of single messages, as well as (3) the optimization overhead and influences
of parameterization. In general, the evaluation shows that:
• Significant performance improvements in the sense of increased message throughput
are achievable. The benefit of vectorization increases with increasing number of
operators, with increasing data size, and with increasing number of plan instances.
• The latency of single messages is moderately increased by vectorization. However,
due to Little’s Law [Lit61], in case of high load of messages, the total latency time
(including waiting time) is reduced by vectorization.
• The deployment and optimization overhead imposed by vectorization is moderate
as well. In addition to the influence of the parameters of periodical optimization,
also the cost-constraint-parameter λ has high influence on the resulting performance.
Typically, the default setting of λ = 0 leads to highest performance.
• The cost-based optimization typically finds the global optimal plan according to the
number of execution buckets. Thus, the number of buckets should not be restricted.
Finally, we can state that the cost-based vectorization achieves significant throughput
improvements, while accepting moderate additional latency for single messages. In conclusion,
this concept can be applied by default if a high load of plan instances exists and
moderate latency time is acceptable. The theoretical optimality and latency guarantees
also hold under experimental performance evaluation.
The evaluation is structured as follows. First, we present the end-to-end comparison of
unoptimized and vectorized execution using our running example plans. Second, we use a
set of additional template plans in order to evaluate the aspects throughput improvement,
latency time, and optimization overhead in more detail on plans with variable number of
operators. Third, we present evaluation results with regard to multiple deployed plans.
Experimental Setting
We implemented the presented approach within our WFPE (workflow process engine). In
general, the WFPE uses compiled plans and an instance-based execution model. Then, we
integrated components for the full vectorization (VWFPE) and for the cost-based vectorization
(CBVWFPE). For this purpose, new deployment functionalities have been introduced
and several changes in the runtime environment were required because those plans are
executed in an interpreted fashion.
119