Cost-Based Optimization of Integration Flows - Datenbanken ...

5.3 Periodical Re-Optimization
Based on this waiting time, we compute the hypothetical total latency time of
ˆT L (M ′ , R, ∆tw) =
⌈ ⌉ 1
sel
= 100 · 1.8056 s +
· ∆tw + W (P ′ , R · ∆tw)
(
0.325 + 0.205 · 4 1 )
· 1.8056 s ≈ 182.3656 s.
s
Finally, we check the validity conditions of (0 ≤ W (P 2 ′, k′ ) ≤ ∆tw) ∧ (0 ≤ ˆT L ≤ lc), where
we observe that all constraints are given 15 . Comparing instance-based and partitioned
execution with regard to executing for example |M| = 1,000 messages, we reduced the total
⌈
latency time from ˆT L (M) = 1,000 · 0.53 s = 530 s (simplified due to overload situation) to
1,000/(4
1
s ) · 1.8056 s⌉ · 1.8056 s + 1.8056 s = 252.784 s and the total execution time from
530 s to 250.978 s (W (M, k ′ ) = ˆT L (M, k ′ ) + ∆tw because we computed ∆tw according to
the lower border of the defined interval).
The whole computation approach is designed for the general case of arbitrary cost
functions. Based on the observation that our concrete extended cost model only uses
two categories of operator costs (operators that are independent of k ′ and operators that
depend linearly on k ′ ), we use an efficient tailor-made waiting time computation algorithm
as a simplification of this computation approach. In contrast to the general solution of
computing the optimal waiting time, this algorithm does not require the derivation of the
latency time function.
Algorithm 5.3 Waiting Time Computation (A-WTC)
Require: rewritten plan P ′ , message rate R, latency constraint lc, selectivity sel
1: ∆tw ← 0, k ′ ← 0, W − (P ′ ) ← 0, W + (P ′ ) ← 0
2: for i ← 1 to m do // for each operator o i
3: if W (o ′ i ) is independent of k′ then
4: W − (P ′ ) ← W − (P ′ ) + W (o ′ i )
5: else
6: W + (P ′ ) ← W + (P ′ ) + W (o ′ i )
7: ∆tw ← W − (P ′ )
// ∆tw = W (P ′ , ∆tw · R) = W − (P ′ ) + W + (P ′ ) · ∆tw · R
1−W + (P ′ )·R
8: T L ← ⌈ 1
sel⌉
· ∆tw + W (P ′ , R · ∆tw)
9: k ′ ← R · ∆tw
10: if ¬(0 ≤ W (P ′ , k ′ ) ≤ ∆tw) ∨ ¬(0 ≤ ˆT L ≤ lc) then
11: k ′ ← k ′ | W (M ′ , k ′ ) + ∆tw = T L (M ′ , k ′ ) = lc // overload situation
12: return ∆tw, R, k ′
Algorithm 5.3 illustrates this automatic waiting time computation approach for a given
message rate R, the latency constraint lc, and number of distinct items sel. First, there
are some initialization steps (line 1). Then, we iterate over the operators of the rewritten
plan P ′ (lines 2-6) in order to compute the costs W − (P ) that are independent of k ′ and
the costs W + (P ) that depend linearly on k ′ . Using a simplified Equation 5.8, we compute
15 In this example, the result depends on the message arrival rate R. For example, R = 10 msg /s would result
in an invalid waiting time ∆tw ≈ −0.3095 s. In this case we would try to compute ˆT L(M ′ , R · ∆tw) =
W (M ′ , ∆tw · R) but would observe an overload situation and therefore, would proceed accordingly and
compute the maximum batch size k ′′ .
149