Cost-Based Optimization of Integration Flows - Datenbanken ...

3.3 Periodical Re-Optimization
3.3 Periodical Re-Optimization
Based on the presented prerequisites, we now explain the core algorithm for the cost-based
optimization of imperative integration flows. First, we formally define the periodic plan
optimization problem including the existing parameters and show that this problem is
NP-hard. Due to the complexity of this optimization problem, we additionally introduce
two search space reduction approaches. Then, we describe how the parameters can be
leveraged in order to influence the sensibility of workload adaptation. Finally, we sketch
how to handle conditional probabilities and correlation without the knowledge about data
characteristics (e.g., value distributions) of external systems.
The core optimization algorithm is independent of any concrete optimization technique.
For that reason, it can be extended with arbitrary new techniques. However, we use
selected optimization techniques in order to illustrate the properties and the behavior of
our optimization algorithm. In Section 3.4, we will explain various concrete optimization
techniques in more detail.
3.3.1 Overall Optimization Algorithm
Existing approaches of integration flow optimization, which also take execution statistics
into account [SMWM06, SVS05], use the optimize-always model, where the given plan
is optimized for each initiated plan instance. While this is advantageous for changing
workload characteristics in combination with long running plan instances, it fails under
the assumption of many plan instances with rather small amounts of data because in this
case the optimization time might be even higher than the execution time of the plan. In
consequence, we introduce an optimization algorithm that exploits the integration-flowspecific
characteristics of (1) being deployed once and executed many times as well as (2)
the presence of an initially given imperative integration flow.
Optimization Problem
The goal of plan optimization is to rewrite (transform) a given plan into a semantically
equivalent plan that is optimal in the average case with regard to the estimated costs. The
major differences to DBMS are (1) the average case optimization of a deployed plan and
(2) the transformation-based plan rewriting that takes into account the specified controlflow
semantics of imperative integration flows. As a first step, we define the optimal plan
as follows:
Definition 3.3 (Optimal Plan). A plan P = (o, c, s) is optimal at timestamp T k with
respect to a given workload W (P, T k ) if no plan P ′ = (o ′ , c ′ , s) with lower estimated execution
time Ŵ (P ′ ) < W (P ) exists. Thus, the optimization objective φ of any optimization
algorithm is to minimize the estimated average execution time of the plan with:
φ = min Ŵ (P ). (3.5)
The plan P is optimal according to the monitored statistics at the timestamp of optimization
T k . In case of changing workload characteristics, over time, the plan P might
loose this property of optimality. For this reason, we require periodical re-optimization if
we do not want to employ an optimize-always model. Hence, as a second step, we formally
define this time-based optimization problem as follows:
45