Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
δ msg1
O
δ msg1
-1
Assign (o1)
[out: msg1]
Invoke (o2)
[service s4, in: msg1, out: msg2]
δ msg1
D δ msg1
D
Assign (o1)
[out: msg1]
Fork (o-1)
δ msg2
D
δ msg3
D
Invoke (o3)
[service s5, in: msg1, out: msg3]
Join (o4)
[in: msg2,msg3, out: msg4]
Groupby (o5)
[in: msg4 out: msg5]
Assign (o6)
[in: msg5 out: msg1]
Invoke (o7)
[service s5, in: msg1]
δ msg4
D
δ msg5
D
δ msg1
D
δ msg1
-1
Invoke (o2)
[service s4, in: msg1, out: msg2]
Join (o4)
[in: msg2,msg3, out: msg4]
Groupby (o5)
[in: msg4, out: msg5]
Assign (o6)
[in: msg5, out: msg1]
Invoke (o7)
[service s5, in: msg1]
Invoke (o3)
[service s5, in: msg1, out: msg3]
(a) Dependency Graph DG(P 3)
(b) Plan P ′ 3
Figure 3.2: Example Dependency Graph and its Application
While our approach focuses on the optimization of complete plans, Vrhovnik et al. use a
so-called Sphere Hierarchy—in addition to a dependency analysis—to determine optimization
boundaries that must not be crossed [VSS + 07]. Thus, they independently optimize
partitions (spheres) of a plan. For example, the operator subsequence of a complex operator
(e.g., each subflow of the Fork operator) is optimized only locally. However, since
this is BPEL-specific [OAS06] (scope activity) and reduces the optimization potential, we
do not restrict ourselves to these boundaries. Another approach [WPSB07] generates a
minimal dependency set. For this purpose, they merge and optimize explicitly modeled
dependencies. In contrast, we do not use explicitly modeled dependencies but analyze
implicit dependencies (given by the data flow) in order to ensure semantic correctness. Finally,
our automatic dependency-awareness reduces the development effort of integration
flows and it is an essential prerequisite for both rule-based and cost-based optimization.
3.2.2 Cost Model and Cost Estimation
Referring back to the problem of changing workload characteristics (Problem 3.2), a tailormade
cost model reflecting these workload characteristics is required as a foundation of
cost-based plan optimization. Due to the problem of missing statistics (Problem 3.3),
execution statistics must be incrementally maintained as input for the defined cost model.
In this context, the problem is to determine costs (cardinalities as well as execution times)
for rewritten parts of a plan, where no statistics exist so far. Furthermore, the challenge of
representing (1) data-flow-, (2) control-flow-, and (3) interaction-oriented operators arises
(Problem 3.1), where the different operator categories are described by different execution
statistics. While for the data-flow- and interaction-oriented operators, cardinality is a
widely-used metric, this is not applicable for the control-flow-oriented operators because
the costs of those operators are mainly described by means of execution times. In addition,
concrete costs of the interaction-oriented operators strongly depend on the involved
external systems and their individual performance. Hence, also for interaction-oriented
operators the execution time rather than the cardinality should be used as the metric.
In consequence of the aforementioned characteristics, in this subsection, we propose the
double-metric cost model [BHLW09c] for enabling precise cost estimation of a plan.
Cost models in other domains typically follow a different approach. A widely used
38