Cost-Based Optimization of Integration Flows - Datenbanken ...

5 Multi-Flow Optimization
Thus, T L (m i ) ≤ ˆT L (M ′ ) ≤ lc is true due to the subsumtion of W ∗ (P ′ , k ′ ) by ∆tw because
the waiting time is longer than the execution time, for the valid case of ∆tw ≥ W (P ′ , k ′ ).
This is also true for arbitrary serialization concepts. Hence, Theorem 5.5 holds.
Based on this formal analysis, we can state that the introduced waiting time computation
approach (1) optimizes the message throughput by minimizing the total latency
time of a message subsequence, and ensures the additional restrictions of (2) a maximum
latency time constraint for single messages, and (3) the serialized external behavior.
5.5 Experimental Evaluation
In this section, we present experimental evaluation results with regard to the three evaluation
aspects of (1) optimization benefits/scalability, (2) optimization overheads, and (3)
latency guarantees under certain constraints. In general, the evaluation shows that:
• Message throughput improvements are yielded by minimizing the total latency time
of message sequences. Compared to the unoptimized case, the achieved relative
improvements decrease with increasing data sizes for some plans, while they stay
constant for other plans. In contrast, the relative improvement increases with increasing
batch sizes.
• The runtime optimization overhead for deriving partitioning schemes, rewriting
plans, and computing the optimal waiting time is moderate. In addition, the overhead
for horizontal partitioning of inbound message queues is, for moderate numbers
of distinct items, fairly low compared to commonly used transient message queues.
• Finally, the theoretical maximum latency guarantees for arbitrary distribution functions
also hold under experimental investigation. This stays true under the constraint
of serialized external behavior (additional serialization at the outbound side) as well.
The detailed description of our experimental findings is structured as follows. First, we
evaluate the end-to-end throughput improvement as well as the overheads of periodical reoptimization.
Second, we present scalability results with regard to increasing data sizes as
well as increasing batch sizes. Third, we analyze in detail the execution time with regard
to influencing factors such as message rate, selectivities, waiting time, and batch sizes.
Fourth, we present the influences on the latency time of single messages with and without
serialized external behavior and with arbitrary message rate distribution functions. Fifth
and finally, we discuss the runtime overhead of horizontal message queue partitioning.
Experimental Setting
We implemented the approach of MFO via horizontal partitioning within our Java-based
WFPE (workflow process engine) and integrated it into our general cost-based optimization
framework. This includes the partitioned message queue (partition tree and hash partition
tree), slightly changed operators (partition-awareness) as well as the algorithms for deriving
partitioning attributes (A-DPA), rewriting of plans (A-MPR) and automatic waiting
time computation (A-WTC).
We ran our experiments on the same platform as described in Section 3.5 and we used
synthetically generated data sets. As integration flows under test, we use all asynchronous,
156