Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
P’a
o1
W(o1)=4
o2
W(o2)=3
o3 k=2 | Ka=1
W(o3)=1
X
Ka=2
k=2
P’a
o1
W(o1)=4
o2
W(o2)=3
o3
W(o3)=1
P’b
o1 o2 o3 o4 o5
W(o1)=3 W(o2)=2 W(o3)=5 W(o4)=4 W(o5)=3
k=4 | Kb=1
X
Kb=2
k=2
P’b
o1 o2 o3 o4 o5
W(o1)=3 W(o2)=2 W(o3)=5 W(o4)=4 W(o5)=3
P’c
o1
W(o1)=1
o2 o3 o4
W(o2)=9 W(o3)=4 W(o4)=2
k=3 | Kc=3
(a) Cost-Based Plan Vectorization
(b) Restricted Cost-Based Plan Vectorization
Figure 4.17: Heuristic Multiple Plan Vectorization
Finally, we execute the restricted cost-based plan vectorization with K i as a parameter.
Figure 4.17(b) shows the result of this step, where K a = 2 was used, but K b = 2 because
for P b , the local constraint exceeded the number of free buckets. As a result, we ensured
that the global maximum constraint was not exceeded.
This is a heuristic computation because (1) we directly assign free buckets (independent
of relative cost improvements) and (2) the order of considered plans might influence
the resulting distribution. However, it often solves this problem adequately. An exact
computation approach would use a while loop and redistribute free buckets as long as no
more changes are made. Just after this, we would restrict k explicitly. However, an exact
computation approach would require using the exhaustive cost-based computation.
Our heuristic algorithm has a time complexity of O(h · m) because we call h times the
A-CPV that has a time complexity of O(m) (see Theorem 4.3). Furthermore, we might
call at most h times the heuristic restricted plan vectorization algorithm that has also a
linear time complexity of O(m).
In conclusion, the multiple plan vectorization takes into account the workload characteristics
in order to restrict the maximum number of used execution buckets. There,
plans with high load are preferred and get more execution buckets assigned to them. At
the same time, this algorithm applies the cost-based vectorization and hence, can ensure
near-optimal performance even in the case of large numbers of plans.
4.5 Periodical Re-Optimization
We have shown how to compute cost-based vectorized plans in case of both single and
multiple deployed plans. Those exhaustive and heuristic algorithms rely on continuous
gathering of execution statistic and periodical re-optimization in order to be aware of
changing workload characteristics. Finally, the whole vectorization approach is embedded
as a specific optimization technique into our general cost-based optimization framework.
Note that our transformation-based optimization algorithm (A-PMO, Subsection 3.3.1)
applies the cost-based vectorization after all other rewriting techniques, where techniques
for rewriting patterns to parallel flows are not used if vectorization is enabled.
With regard to the whole feedback loop of our general cost-based optimization framework,
there are two additional challenges that need to be addressed for vectorized plans.
First, if vectorization produces a plan that differs from the currently deployed plan, we
have to evaluate the re-optimization potential in the sense of the benefit of exchanging
plans. Second, if an exchange is beneficial or one of the other optimization techniques
116