Cost-Based Optimization of Integration Flows - Datenbanken ...

6.2 Plan Optimality Trees
of nodes within a PlanOptTree for a given plan, which indirectly implies the worst case
complexity for any operation that evaluates at most all nodes of such a PlanOptTree.
Theorem 6.1 (Worst-Case Complexity). The worst-case time and space complexity of a
PlanOptTree for a plan of m operators is O(m 2 ).
Proof. Assume a plan P with m operators. A minimal PlanOptTree has at most m
ONodes, m · s SNodes, 2 · |oc| CSNodes (two complex statistics nodes for each binary
optimality condition), and |oc| OCNodes. Each operator can be included in one optimality
condition per dependency (in case of a data dependency this subsumes any temporal
dependency) and in one additional optimality condition for binary operators. Now, let
us assume a sequence of operators o. Then, an arbitrary operator o i with 1 ≤ i ≤ m
can—in the worst case—be the target of i − 1 dependencies δi − , and it can be the source
of m − i dependencies δ
i + . Based on the equivalence of δ− = δ + and thus, |δ − | = |δ + |, the
maximum number of optimality conditions is given by
|oc| =
Hence, Theorem 6.1 holds.
m∑
(i − 1) + m =
i=1
m−1
∑
i=1
i + m =
m · (m + 1)
. (6.1)
2
After we have created the initial PlanOptTree, we can apply the following two optimizations
with an additional single pass over all nodes of the PlanOptTree. However, for
simplicity of presentation and due to a small impact on our use cases, we did not apply
them in the examples.
• Collapsing Statistic Hierarchies (CSNodes): The hierarchies of statistic nodes of
partial PlanOptTrees are defined for each optimization technique with regard to
re-usability. Thus, there might be unnecessarily fine-grained CSNodes. We collapse
these hierarchies by merging CSNodes with their children if only a single child exists.
Similarly to the merging of prefix nodes within a patricia trie [Mor68] this can reduce
the number of levels of the PlanOptTree.
• Reusing Atomic Statistic Measures (SNodes): For operators of the PlanOptTree
with a data dependency between them, we reuse cardinalities across those operators
in order to eliminate redundancy. Due to the data dependency, the output
cardinality of operator o i is equal to the input cardinality of operator o i+1
(|ds out (o i )| = |ds in (o i+1 )|). Hence, we remove the latter SNode (|ds in (o i+1 )|) and
modify the references. This requires awareness when updating the PlanOptTree after
successful re-optimization, because the re-optimization might have changed the
ordering of operators and thus, also changed the data dependencies and statistics.
As a result of creating the initial PlanOptTree, we obtain a structure that represents
optimality of a plan with the properties sketched in Subsection 6.2.1. It includes all cost
conditions that must be satisfied for plan optimality with regard to the current statistics.
Thus, only the update of statistics can trigger re-optimization.
6.2.3 Updating and Evaluating Statistics
We use the PlanOptTree for statistics maintenance and for immediate evaluation of optimality
conditions. This triggers re-optimization if optimality conditions are violated.
175