Cost-Based Optimization of Integration Flows - Datenbanken ...

7 Conclusions
Existing approaches before this thesis either used the rule-based optimize-once model,
where the integration flow is only optimized once during the initial deployment, or the
optimize-always model, where optimization is triggered for each plan instance. In contrast,
we presented the first entire cost-based optimizer for imperative integration flows using the
cost-based optimization models of periodical and on-demand re-optimization. The major
advantage of these new optimization models is robustness in terms of (1) high optimization
opportunities that allows (2) the adaptation to changing workload characteristics with (3)
low risk of optimization overheads.
Beside the investigation of additional optimization techniques, we see four major research
challenges regarding future work of the cost-based optimization of integration flows:
• Mid-Instance Optimization: Our asynchronous re-optimization model is based on the
assumption of many plan instances with rather small amounts of data per instance.
In order to make it also suitable for (1) long running plan instances and (2) adhoc
integration flows (as required for situational BI and mashup integration flows),
an extension to synchronous mid-instance re-optimization would be necessary. The
challenge is to define a hybrid model for both use cases of integration flows.
• Multi-Objective Optimization: We focused on the optimization objectives of execution
time and throughput. However, facing new requirements, this might be extended
by multiple objectives including for example, monetary measures, execution and latency
time, throughput, energy consumption, resiliency or transactional guarantees.
• Optimization of Multiple Deployed Plans: The cost-based optimization of integration
flows—with few exceptions—aims to optimize a single deployed plan. However,
typically, multiple different integration flows are deployed and concurrently executed
by the integration platform. The major question is if we could exploit the knowledge
about workload characteristics of all plans and their inter-influences for more efficient
scheduling of plan instances or influence-aware plan rewriting.
• Optimization of Distributed Integration Flows: With regard to load balancing and
the emerging trend towards virtualization, distributed integration flows might be
used as well. The resulting research challenge is to optimize the entire distributed
integration flow, rather than just the subplans of local server nodes. One aspect of
this might be the extension of cost-based vectorization to the distributed case.
Thus, we can conclude that there are many directions for future work in this new field
of the cost-based optimization of integration flows. Similarly to cost-based query optimization,
this might be a starting point for the continuous development of new execution and
optimization techniques:
”In my view, the query optimizer was the first attempt at what we call autonomic
computing or self-managing, self-timing technology. Query optimizers
have been 25 years in development, with enhancements of the cost-based query
model and the optimization that goes with it, and a richer and richer variety
of execution techniques that the optimizer chooses from. We just have to keep
working on this. It’s a never-ending quest for an increasingly better model and
repertoire of optimization and execution techniques. So the more the model can
predict what’s really happening in the data and how the data is really organized,
the closer and closer we will come [to the ideal system].”
— Patricia G. Selinger, IBM Research – Almaden, 2003 [Win03]
200