Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
size of messages as good indicator for the cardinalities. Based on these monitored atomic
statistics, derived more complex statistics such as the relative frequencies of alternative
paths P (path i ) or operator selectivities sel(o i ) = |ds out1 (o i )|/|ds in1 (o i )| are computable.
In order to allow cost estimation for integration flows with control-flow semantics, in the
following, we define a double-metric cost model.
Definition 3.2 (Double-Metric Cost Estimation). The costs of a plan P are defined as
an aggregate of operator costs, where these operator costs are defined by two metrics:
1. Abstract costs C(o i ) are defined for all data-flow- and interaction-oriented operators
in the form of their time complexity using cardinalities as the metric.
2. Execution times W (o i ) are then used as the second metric in order to weight the
abstract costs. For control-flow-oriented operators, we only use execution times.
Both types of input statistics (cardinalities and execution times) are used in the form of
aggregates over the monitored atomic statistics of executed plan instances.
For multiple deployed integration flows, it can be necessary to normalize the monitored
execution statistics (in particular execution times) when aggregating them. We presented
detailed cost normalization algorithms [BHLW09c] that we omit here for the sake of being
focused on the core cost model.
Based on Definition 3.2, Tables 3.1-3.3 show the double-metric costs of all operators of
our flow meta model (see Subsection 2.3.1). In the following, we substantiate the different
cost formulas according to the two steps of our cost estimation approach.
1: Abstract Costs: In a first step, we consider the mentioned abstract costs C(P ) that
are based on the cardinality metric. Costs for interaction-oriented operators include the
costs for the operators Receive, Reply and Invoke. The costs for the Receive operator
are determined as |ds out |, i.e., by the cardinality of the received data set ds out and they
comprise costs for transport, protocol handling, format conversion as well as decompression.
The costs for the related Reply operator are similarly computed with |ds in |. Finally,
the costs of the Invoke operator are computed by |ds in | + |ds out | because it actively sends
and receives data. A data set |ds out | might be NULL (e.g., in case of pure write interactions);
in that case, we assume a cardinality of 0. Control-flow-oriented operators have no
abstract costs. An exception to this is the Switch operator because it evaluates expressions
over input data sets, which requires costs of |ds in | for a single path expression and
due to the if-elseif-else semantic total costs of ∑ n
i=1 (P (path i) · i · |ds in |) for all n paths,
where P (path i ) denotes switch path probabilities (relative frequencies). For computing
the abstract costs of the data-flow-oriented operators, we partly adapted a cost model from
a commercial RDBMS [Mak07] to the specific characteristics of integration flows. With
regard to physical operator alternatives, we excluded hash-based algorithms because they
Table 3.1: Double-Metric Costs of Interaction-Oriented Operators
Operator
Name
Abstract Costs C(o i ) Execution Time W (o i )
Invoke |ds in | + |ds out | W (o i )
Receive |ds out | W (o i )
Reply |ds in | W (o i )
40