Cost-Based Optimization of Integration Flows - Datenbanken ...

3.4 Optimization Techniques
2,205.05, where for example, the cost of the Switch operator o 3 are determined with
C(P ) = 1,000 · 0.5 · 0.7 = 350 by the input data size and the selectivity of previous operators.
If we would reorder the Selection operators traditionally, we would get the operator
sequence (o 7 , o 8 , o 1 , o 2 ) shown in Figure 3.16(b). The costs of this sequences (using Equation
3.23) are given by C(P ′ ) = 1,000+0.05·1,000+1,000·0.965+1,000·0.965·0.4+1,000·
0.965 · 0.4 · 0.5 = 2,594 and thus, the costs are higher than the initial costs. In contrast,
if we reorder Selection operators in a control-flow-aware manner, we get the operator
sequences (o 8 , o 1 , o 2 , o 7 ) shown in Figure 3.16(c). The costs of this sequence are computed
by C(P ′′ ) = 1,000 + 1,000 · 0.4 + 1,000 · 0.4 · 0.5 + 200 + 1,000 · 0.4 · 0.5 · 0.7 · 0.05 = 1,807.
As a result, control-flow-aware selection reordering reduced the costs from 2,594 to 1,807.
To summarize, the control-flow aware selection reordering exhibits a slightly worse time
complexity than the traditional selection ordering. However, the opportunity of a significant
plan cost reduction justifies the application of this technique. Finally, note that
the same concept of effective operator selectivities (P (o i ) · f oi + (1 − P (o i ))) is in general,
also applicable for all selective operators such as Groupby, Join, Setoperation, and
Projection. For the sake of a clear presentation, we will not mention this during the
discussion of the following data-flow-oriented optimization techniques.
Eager Group-By and Pre-Aggregation
Similar to reordering selective operators, it can be more efficient to apply specific operators
as early as possible in order to reduce the cardinalities of intermediate results, where the
earliest possible position can be determined using the dependency graph. The core concept
is to reduce the cardinality of intermediate results and thus, to improve the execution time
of the following operators.
We concentrate only on WD6: Early Groupby Application, which was considered for
DBMS [CS94] (with complete [YL95] or partial [Lar02] aggregation) and for EII (Enterprise
Information Integration) frameworks (adjustable partial window pre-aggregation
[IHW04]). For early group-by application, a sequence of Join and Groupby is rewritten
to a construct of Groupby and Join (invariant group-by also known as eager group-by) or
to a construct of Groupby, Join, and Groupby (pre-aggregation). The assumption is that
it can be more efficient—with respect to the monitored cardinalities and selectivities—to
compute the Join on pre-aggregated partitions rather than on the single tuples. The
precondition is that the list of grouping attributes G contains the Join predicate jp.
First of all, we need some additional notation, where we use the relational algebra for
simplicity of presentation. Assume a join of n data sets (with arbitrary multiplicities) and
a subsequent group-by, where the join predicate and group-by attributes are equal with
γ F (X);A1 (R ⋊⋉ R.A1 =S.A 1
S). For left-deep join trees, without cross products, and only one
join implementation, there are then 4n! possible plans because for each join operator, four
possibilities exist to apply the Groupby operator (for invariant group-by, the final γ in
P c -P f can be omitted):
P a (opt) : γ(R ⋊⋉ S) P c : γ((γR) ⋊⋉ S) P e : γ(R ⋊⋉ (γS)) P g : (γR) ⋊⋉ (γS)
P b : γ(S ⋊⋉ R) P d : γ(S ⋊⋉ (γR)) P f : γ((γS) ⋊⋉ R) P h : (γS) ⋊⋉ (γR).
Without loss of generality, we assume n = 2 and concentrate on the four possibilities
to arrange group-by and join for a given join order. In addition to the join costs
69