Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
(a) Number of Operators m
(b) Input Data Size d
Figure 4.24: Influence of λ with Different Numbers of Operators and Data Sizes
detail, we see the near-optimal solution with λ = 0. Typically, when increasing the number
of execution buckets from this point, the elapsed time increases. However, there are cases
such as the plan with m = 5, where we observe that the instance-similar execution (with
one bucket for all operators) performs better. The difference to traditional instance-based
execution is caused by (1) reused operator instances (e.g., pre-parsed XSLT stylesheets of
Translation operators), and (2) pipelined inbound processing. Further, for m = 20, we
see the optimal total execution time at λ = 10. Note that even in this case of k = 1, we
reach better performance than in the instance-based case because there are no synchronous
(blocking) calls through the whole engine but only within the single plan. In addition,
for small numbers of processed messages n, the instance-based execution model performs
worse due to the overhead of just-in-time compilation of generated plans. Furthermore,
we observe that the higher the number of operators m, the higher the influence of the
parameter λ. In conclusion, we typically find a very good solution with λ = 0, but when
required, this parameter can be used to easily adjust the degree of parallelism.
Furthermore, in the second sub-experiment, we used a single plan with m = 20, we fixed
t = 0, q = 50 and we executed n = 250 messages with different λ and for different data sizes
d ∈ {1, 4, 7} (in 100 kB). Figure 4.24(b) shows the results with regard to the execution
time as well as the number of execution buckets (annotated at the top of each point) when
varying λ. In general, we see similar behavior as in Figure 4.24(a) (for m = 20). The
different numbers of execution buckets for d = 1 and λ ∈ (0, 10) are caused by dynamically
monitored operator costs, which varied slightly. The major difference when comparing the
influence of varying the data size with the previous sub-experiment is that the data size
significantly increases the execution time of single operators. As a result, we observe that
we require higher values of λ to reduce the number of execution buckets. In conclusion,
λ should be configured with context knowledge about current workload characteristics.
We could overcome this workload dependency with a relative value of λ according to
the maximum operator costs. However, with λ as an absolute value, we can explicitly
determine the maximum work-cycle increase of the data flow graph.
In conclusion, there are several parameters with significant influence on the total execution
time and on the behavior of cost-based vectorization. As a general heuristic, one
should use a maximum costs increase of λ = 0. This simplest configuration typically results
in near-optimal throughput. However, if more context knowledge about the workload
is available, the described parameters can be used as tuning knobs.
126