Cost-Based Optimization of Integration Flows - Datenbanken ...

5 Multi-Flow Optimization
• Case 2: ∆tw > W (P ′ ) (Figure 5.10): The execution time is shorter than the waiting
time. This can improve the throughput and is advantageous for multiple plans.
• Case 3: ∆tw < W (P ′ ): The waiting time is shorter than the execution time. Thus,
the result would be (1) temporally overlapping plan instances for different partitions
or (2) growing message queue sizes (increasing k ′ ).
Problem 5.5 (Waiting Time Inconsistency). When computing the waiting time ∆tw, we
must prevent case 3 (∆tw < W (P ′ )) because it would lead to the parallel execution of
plan instances, where we could not ensure serialization. Furthermore, there might be an
a-priori violated latency constraint with T L (M ′ ) ≥ lc or the latency constraint lc might be
set to impossible values with regard to the execution time W (P ′ ).
In order to overcome Problem 5.5, we define the validity condition: For a given latency
constraint lc, there must exist a waiting time ∆tw such that (0 ≤ W (P ′ ) ≤ ∆tw) ∧ (0 ≤
ˆT L (M ′ ) ≤ lc); otherwise, the constraint is invalid. In other words, (1) we avoid case 3, and
(2) we check if the worst-case latency of a single message—in the sense of the estimated
total latency of 1/sel distinct partitions—fulfills the latency constraint lc.
5.3.2 Extended Cost Model and Cost Estimation
In order to estimate the costs of plan instances for specific batch sizes k ′ with k ′ = |b i |,
which is required for computing the optimal waiting time, we need to adapt the used cost
model for integration flows that has been described in Subsection 3.2.2. Basically, only
operators that benefit from partitioning require extensions.
The cost model extensions address the abstract costs as well as the execution time
because interaction-, control-flow-, and data-flow-oriented operators are affected by partitioning.
Table 5.2 shows these extended costs C(o ′ i , k′ ) and execution times W (o ′ i , k′ ) in
detail. The extended costs of operators that directly benefit from partitioning, namely
Invoke, Receive, Switch, and Assign, are independent of the number of messages k ′ .
For example, the Invoke operator executes an external query once for the whole batch.
Further, also the Switch operator evaluates its path expressions on the partitioning attribute
and thus, for the whole batch rather than on individual messages. However, the
Switch operator recursively contains the extended costs of arbitrary child operators that
might benefit from partitioning or not. In contrast, the costs of all other operators (that
do not benefit from partitioning) linearly depend on the number of messages in the batch
k ′ . This is also true for Invoke, Receive, Switch, and Assign operators if they do not
Table 5.2: Extended Double-Metric Operator Costs for Message Partitions
Operator Abstract Execution Time W (o ′ i , k′ )
Name Costs C(o ′ i , k′ )
Invoke |ds in | + |ds out | W (o i )
Receive |ds out |
⎛ ⎛
W (o i )
⎞⎞
n∑
i∑
Switch |ds in |
⎝P (path i ) · ⎝ W ( ) ∑m i
expr pathj + W ( o ′ i,l, k ′) ⎠⎠
Assign |ds in | + |ds out | W (o i )
other C(o i ) · k ′ W (o i ) · k ′
i=1
j=1
l=1
144