Cost-Based Optimization of Integration Flows - Datenbanken ...

3.5 Experimental Evaluation
techniques. Traditional techniques can be adapted also for the optimization of integration
flows. In addition to this, there are hybrid techniques, where the control flow must
be taken into account for data-flow-oriented techniques as well. Finally, there are also
techniques that are tailor-made for integration flows. In conclusion, we observe the presence
of many optimization opportunities, where ensuring semantic correctness is a major
challenge with regard to concrete optimization techniques. Therefore, our transformationbased
optimization algorithm iteratively applies optimization techniques. Many of these
techniques are independent but for dependable techniques, we need to prevent local suboptima.
Note that applying techniques independently (1) reduces the optimization overhead
due to a reduced search space, and (2) less development effort for new techniques. The
presented general optimization framework can then be extended with arbitrary additional
optimization techniques.
3.5 Experimental Evaluation
In this section, we present results of our exhaustive experimental evaluation with regard
to the three evaluation aspects: (1) optimization benefits and scalability, (2) optimization
overheads, as well as (3) workload adaptation. In general, the evaluation shows that:
• Significant performance improvements can be achieved by periodical re-optimization
in the sense of minimizing the average execution time of a plan. According to Little’s
Law [Lit61], this has also direct influence on the message throughput improvement.
Scalability experiments showed that the benefit increases with increasing amount of
input data as well as with increasing plan complexity.
• The overhead for statistic maintenance and periodical re-optimization is moderate.
Thus, this overhead is typically subsumed by the achieved execution time reduction.
Even in the worst-case, where the initial plan constantly exhibits the optimality
property, this additional runtime overhead is moderate.
• The right choice of parameterization (workload aggregation, optimization interval,
sliding time window size) in combination with correlation awareness can ensure an
accurate but still robust adaptation to changing workload characteristics. In detail,
even after specific workload changes, the self-adjusting cost model consistently
converges to the real costs.
In conclusion of these major experimental findings, the periodical re-optimization can
be applied by default. The available parameters of the optimization algorithm can additionally
be used to fine-tune the adaptation sensibility (and thus, influence the execution
time reduction) and the optimization overhead.
The detailed description of our experimental results is structured as follows. First, we
explain the end-to-end comparison of no-optimization versus periodical re-optimization
for all plans of our use cases. This already includes the optimization overhead. Second,
we analyze this optimization overhead in more detail with regard to statistics monitoring
and re-optimization. Third, we evaluate the cost model according to the adaptation to
changing workload characteristics as well as how to set the existing parameters.
73