Cost-Based Optimization of Integration Flows - Datenbanken ...

3 Fundamentals of Optimizing Integration Flows
Fork (o1)
Fork (o1)
Assign (o2)
[out: msg1]
Assign (o4)
[out: msg3]
Assign (o6)
[out: msg5]
Assign (o2)
[out: msg1]
Assign (o4)
[out: msg3]
Assign (o6)
[out: msg5]
Invoke (o3)
[service s3, in: msg1, out: msg2]
Invoke (o5)
[service s4, in: msg3, out: msg4]
Invoke (o7)
[service s5, in: msg5, out: msg6]
Invoke (o3)
[service s3, in: msg1, out: msg2]
Invoke (o5)
[service s4, in: msg3, out: msg4]
Invoke (o7)
[service s5, in: msg5, out: msg6]
Orderby (o12)
[in: msg2, out: msg2]
Orderby (o13)
[in: msg4, out: msg4]
Orderby (o16)
[in: msg6, out: msg6]
Setoperation (o8)
[in: msg2,msg4, out: msg7]
UNION DISTINCT
Setoperation (o8)
[in: msg2,msg4, out: msg7]
UNION DISTINCT
(Merge)
Setoperation (o9)
[in: msg7,msg6, out: msg8]
UNION DISTINCT
Setoperation (o9)
[in: msg7,msg6, out: msg8]
UNION DISTINCT
(Merge)
Assign (o10)
[in: msg8, out: msg9]
Assign (o10)
[in: msg8, out: msg9]
Invoke (o11)
[service s6, in: msg9]
Invoke (o11)
[service s6, in: msg9]
(a) Plan P 6
(b) Plan P ′ 6
Figure 3.18: Example Setoperation Type Selection
the techniques orderby insertion and setoperation type selection, we created the rewritten
plan P 6 ′ shown in Figure 3.18(b). Here, we use the efficient merge algorithm for both
Setoperation operators and hence, require to sort all three input data sets. Sorting the
result of the first Setoperation operator is not required because the output of the merge
algorithm is already ordered. Consider two cases with different statistics for input and
output cardinalities of the Setoperation o 8 . Figure 3.19 (left) shows the abstract costs of
the two possible subplans—P 6 : (o 8 ) versus P 6 ′ : (o′ 12 , o′ 13 , o′ 8 )—in both cases.
Statistics C(o 8 ) C(o ′ 12 , o′ 13 , o′ 8 )
|ds in1 (o 8 )| = 1,000
case 1 |ds in2 (o 8 )| = 1,000, 501,000 21,932
|ds out (o 8 )| = 1,000
|ds in1 (o 8 )| = 1,000,
case 2 |ds in2 (o 8 )| = 10, 6,000 11,009
|ds out (o 8 )| = 1,000
Figure 3.19: Example Setoperation Cost Comparison
We observe that the optimality of these subplans depends on current workload characteristics,
while the subplan (o ′ 12 , o′ 13 , o′ 8 ) is more robust6 over arbitrary statistic ranges than
the subplan (o 8 ) as shown in Figure 3.19 (right).
Finally, this optimization technique is also applicable for other operators such as Projection
with duplicate elimination or for forcing a merge-based join algorithm (WD9). Thus,
in general, the technique WD8 (orderby insertion) should be applied before selecting different
physical types of an operator.
To summarize, we presented selected control-flow- and data-flow-oriented optimization
6 Robustness is an alternative optimization objective, which is beyond the scope of this thesis. However,
in contrast to existing work [ABD + 10], we would identify these robust (insensitive to input statistics)
plans by simply choosing one of the plans with lowest asymptotic time complexity with regard to their
overall abstract cost functions of all plan operators.
72