Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
plan with Orderby and Setoperation (UNION DISTINCT MERGE) operators. The
sorted output order between the two Setoperation operators was exploited by the
optimizer in order to reduce the number of required Orderby operators. Finally, the
Setoperation and Orderby operators were parallelized by WC2.
• P 7 : Similar to P 6 , this plan was also affected by WC1 in the sense of rescheduled
subflows of the existing Fork operator. Furthermore, the techniques WD10 and WD8
changed the Join operators o 14 , o 15 and o 16 from nested loop joins to subplans of
Orderby and merge join operators. Finally, note that join enumeration did not
resulted in a new join order.
• P 8 : This plan was mainly affected by control-flow oriented optimization techniques.
In detail, the operator sequence (o 3 -o 9 ) was rewritten into two parallel subflows of a
Fork operator. The last operator o 10 was not included because both o 9 and o 10 are
two writing interactions to the same external system. Finally, the technique WC1
was applied once again for rescheduling the created subflows.
In addition to these consistent optimization benefits, Figure 3.22(b) shows the required
cumulative optimization time. The significant differences between the optimization times
of different plans are caused by two facts. First, the different total execution time influences
the number of periodical re-optimization steps required in this scenario because these
optimization steps are triggered periodically. Second, different techniques (with different
time complexity) are applied according to the specific operator types used in the concrete
plan. For example, plans P 4 and P 7 are dominated by the costs for join enumeration,
where we did not found different join orders due to ensuring semantic correctness (P 4 )
and the chain query type (P 7 ).
Putting it all together, we can conclude that execution time reductions are possible,
while only a fairly low overhead is required by periodical re-optimization.
Scalability
In addition to the presented comparison of optimized and unoptimized execution, scalability
is one of the most important aspects. Hence, we conducted a series of experiments
that examines the scalability with regard to increasing number of operators as well as with
regard to increasing input data size.
Figure 3.23: Speedup of Rewriting Sequences to Parallel Flows
78