Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
P’a
o1
W(o1)=4
o2
W(o2)=3
o3 k=2
W(o3)=1
P’b
o1 o2 o3 o4 o5
W(o1)=3 W(o2)=2 W(o3)=5 W(o4)=4 W(o5)=3
k=4
P’c
o1
W(o1)=1
o2 o3 o4
W(o2)=9 W(o3)=4 W(o4)=2
k=3
Figure 4.16: Problem of Solving P-CPV for all Plans
for each plan and compute the restricted vectorized plan accordingly. The major challenge
of solving the P-MPV is to determine the best distribution of all operators of the h different
plans across the maximum number of K execution buckets such that we get the highest
overall performance and do not exceed the maximum constraint. In the next subsection,
we present a computation approach that addresses this challenge.
4.4.2 Computation Approach
The core idea of computing the optimal operator distribution across buckets for multiple
plans is to use the costs of each plan in order to assign more execution buckets to more
cost-intensive plans. In detail, we use those costs to weight the maximum constraint K
of execution buckets and determine a local maximum constraint K i for each plan P i . We
use the plan execution time W (P i ) as well as the message arrival rate R i (that can be
monitored as the number of plan instances per time period).
In a first step, we determine the local maximum constraint K i for each plan. If all h plans
would exhibit the same message arrival rate and execution times, we could compute it by
K i = ⌊K/h⌋. However, based on the monitored statistics, we determine the constraint by
⌊
⌋
R i · W (P i )
K i = 1 + ∑ h
j=1 R j · W (P j ) · (K − h) , (4.19)
where K i is at least one execution bucket and at most K execution buckets. Due to the
lower bound, a constraint of K < h (smaller than the number of plans) is invalid.
In a second step, after we have determined the local maximum constraints, we use a
heuristic computation approach to determine the cost-based operator distribution for each
plan. If the maximum constraint is not exceeded, we use the computed scheme. If the
used number of execution buckets k i is smaller than the local constraint, in a third step,
we further redistribute the K i − k i open execution buckets across the remaining plans,
where the local constraint is exceeded.
In detail, Algorithm 4.4 illustrates our heuristic approach. First, we determine the local
maximum constraints and number of free buckets according to the statistics (lines 2-4).
Second, for each plan, we execute the heuristic cost-based plan vectorization (lines 5-10)
using the A-CPV. If the determined number of buckets is smaller than or equal to the
local maximum constraint, we accept this plan and add the remaining buckets to the free
buckets. Third, after all plans have been processed, we redistribute the remaining free
buckets according to the monitored statistics of the individual plans (lines 11-12) in an
upper-bounded fashion. Fourth, we use the heuristic cost-based plan vectorization but
restrict it to the extended maximum constraints (line 13-19) using the A-RCPV. Here, we
114