Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
(a) P with m = 100 (b) P with m = 200
Figure 4.8: Speedup Test with Varying Degree of Parallelism
of the P-PV and present our cost-based vectorization approach [BHP + 09a, BHP + 11] that
overcomes the two drawbacks of vectorization. The instance-based plan and the fully
vectorized plan are then specific cases of this more general solution. This cost-based
vectorization directly relies on the foundations of our general cost-based optimization
framework in the form of a cost-based, control-flow-oriented optimization technique.
4.3.1 Problem Generalization
The core idea of this problem generalization is to rewrite an instance-based plan to a costbased
vectorized plan with a minimal number of execution buckets, where each bucket
can contain multiple operators. All operators of a single execution bucket are executed
instance-based, while the set of execution buckets use the pipes-and-filter execution model.
Example 4.5 (Cost-Based Vectorization). Recall the vectorized plan P 2 ′ (shown in Figure
4.9(a)) from Example 4.1. If we now use cost-based vectorization, we search for the
cost-optimal plan P 2 ′′ with k execution buckets. Figure 4.9(b) illustrates an example of
such a cost-based vectorized plan. There, k = 4 execution buckets are used, while buckets
2 and 4 include two operators. The individual operators of each bucket are executed with
the instance-based execution model. As a result, we require only four instead of six threads
for this plan.
The input (instance-based plan) and the output (vectorized plan) of the P-PV are
extreme cases of this generalization. In order to compute the cost-optimal vectorized
plan, we generalize the P-PV to the Cost-Based P-PV:
Definition 4.2 (Cost-Based Plan Vectorization Problem (P-CPV)). Let P denote a plan,
and p i ∈ {p 1 , p 2 , . . . , p n } denotes the implied plan instances with P ⇒ p i . Further, let
each plan P comprise a sequence of atomic and complex operators o i ∈ {o 1 , o 2 , . . . , o m }.
For serialization purposes, the plan instances are executed in sequence with t 1 (p i ) ≤
t 0 (p i+1 ). The P-CPV describes the search for the derived cost-optimal plan P ′′ according
to the optimization objective φ with k ∈ [1, m] execution buckets b i ∈ {b 1 , b 2 , . . . , b k },
where each bucket contains l operators o i ∈ {o 1 , o 2 , . . . , o l }. Here, the constraint conditions
(t 1 (p ′′
i , b i) ≤ t 0 (p ′′
i , b i+1)) ∧ (t 1 (p ′′
i , b i) ≤ t 0 (p ′′
i+1 , b i)) and (t 1 (b i , o i ) ≤ t 0 (b i , o i+1 )) ∧
100