Cost-Based Optimization of Integration Flows - Datenbanken ...

5 Multi-Flow Optimization
decreasing relative improvement because it mainly benefits from the reduced costs for the
Switch operator. However, this operator is executed after several Selection operators
that significantly reduced the amount of input data, which led to this almost constant
absolute improvement. Similarly, also plan P 7 shows a decreasing relative improvement
because the size of external data was not changed. In conclusion, the scalability with
increasing data size strongly depends on how a plan benefits from partitioning.
Second, we investigated the scalability with increasing batch sizes k ′ . There, we reused
our example plans P 1 , P 2 , P 5 , and P 7 and the workload configuration from the previous
experiment. For each example plan and compared the multi-flow optimization with nooptimization
varying the batch size k ′ ∈ {1, 10, 20, 30, 40, 50, 60, 70}. Figure 5.17 shows
the results of this experiment. Essentially, we make three major observations. First, the
overhead for executing single-message-partitions is marginal. Despite the fact that MFO
theoretically cannot decrease the performance, there is some overhead due to horizontal
partitioning at the inbound side and additional message abstraction layers. However, the
experiments show that this overhead is negligible. As a result the multi-flow optimization
is robust in the sense that it ensures predictable performance even in special cases, where
we do not benefit from partitioning. Second, the theoretically analyzed monotonically
non-increasing total execution time function with increasing batch size and the existence
of a lower bound of the total execution time do also hold under experimental evaluation.
Third, we observe that the lower this lower bound (the higher the optimization potential),
the higher the batch size that is required until we asymptotically tend to this lower bound
(e.g., compare Figure 5.17(a) and Figure 5.17(c)). This effect is reasoned by the higher
relative amount of time that is logically shared among messages.
Execution Time
Until now we have evaluated the optimization benefit and scalability of multi-flow optimization
using restricted batch sizes k ′ . In this subsection, we investigate in detail the
inter-influences between arbitrary message arrival rates R, waiting times ∆tw, partitioning
attribute selectivities sel and the resulting batch size k ′ . In addition, we evaluate the
effects of the resulting batch size k ′ on the plan execution time W (P ′ , k ′ ).
(a) Execution Time W (P ′ 2, k ′ ) (b) Relative Execution Time W (P ′ 2, k ′ )/k ′
Figure 5.18: Execution Time W (P ′ 2 , k′ ) with Varying Batch Size k ′
First, we evaluated the execution time of the resulting plan instance for message partitions
compared to the unoptimized execution. We executed instances of plan P 2 with
varying batch size k ′ ∈ [1, 20], where we measured the total execution time W (P ′ 2 , k′ )
(Figure 5.18(a)) and computed the relative execution time W (P ′ 2 , k′ )/k ′ (Figure 5.18(b)).
160