Cost-Based Optimization of Integration Flows - Datenbanken ...

4.4 Cost-Based Vectorization for Multiple Plans
4.4 Cost-Based Vectorization for Multiple Plans
So far, we have described how to compute the cost-optimal vectorized plan for a single
deployed plan. While this approach significantly improves the performance of this plan, the
cost-based vectorization can also hurt the overall performance in case of multiple deployed
plans (independent) due to a possibly high number of execution buckets regarding all plans.
In conclusion, it is not appropriate to simply use the cost-based vectorization approach or
the fixed number of execution buckets for the set of all plans as well. In this section, we
present an approach that takes into account executions statistics of all deployed plans. As
a result, this approach achieves robustness in terms of the overall performance and hence,
allows for more predictable performance of the integration platform.
4.4.1 Problem Description
In many real-world scenarios, multiple independent plans P i ∈ {P 1 , . . . , P h } are deployed
within an integration platform that executes instances of these plans concurrently. Costbased
vectorization overcomes the problems of full vectorization, i.e., the number of required
threads and the work-cycle domination by single operators with regard to a single
deployed plan. When executing multiple cost-based vectorized plans concurrently, a similar
problem arises. Here, the number of threads required by all h plans depends on the
number of plans. In detail, it is upper-bounded by ∑ h
i=1 m i, where m i denotes the number
of operators of plan P i .
In order to overcome this problem in case of a high number of deployed plans, we define
an extended vectorization problem. The core idea is to restrict the maximum number
of threads by K in the sense of a user-defined parameter. Then, we compute the fairest
distribution of all operators of the h plans across the K execution buckets according to the
current workload characteristics and execution statistics. First of all, we formally define
the extended cost-based vectorization problem for multiple plans.
Definition 4.4 (Cost-Based Multiple Plan Vectorization Problem (P-MPV)). Let P with
P i ∈ {P 1 , . . . , P h } denote a set of h plans. The P-MPV then describes the problem of finding
a restricted cost-optimal plan P i
′′ with k i execution buckets for each P i ∈ P according
to the P-CPV. There, the constraint of the maximum overall number of execution buckets
of ∑ h
i=1 k i ≤ K must hold.
Obviously, when simply solving the standard cost-based optimization problem for each
single plan P i , we might exceed the maximum number of execution buckets with ∑ h
i=1 k i >
K. The following example illustrates this problem.
Example 4.10 (Problem when Solving the P-MPV). Assume three plans P a , P b and P c
with different numbers of operators and monitored costs as shown in Figure 4.16. We
set the maximum total number of execution buckets to K = 7. Further, the P-CPV is
solved for each single plan using the heuristic computation approach. For this example,
we observe that we get ∑ h
i=1 k i = 9 execution buckets and hence, we exceed the maximum
constraint of K = 7.
As a result, the P-MPV cannot be solved by applying the P-CPV for each single plan.
In contrast to restricting k, (see Subsection 4.3.3), the given maximum constraint K is
only an upper bound and therefore we have to consider more solution candidates. In
addition, we might not fully use the optimization potential if we simply use K/h buckets
113