Cost-Based Optimization of Integration Flows - Datenbanken ...

3.3 Periodical Re-Optimization
Table 3.4: Example Execution Statistics
PID NID OType Start End ... W [ms]
1 (p 1 ) 1 Receive 4.1 5.7 ... 1.6
1 (p 1 ) 2 Invoke 5.9 7.7 ... 1.8
1 (p 1 ) Plan 4.0 8.0 ... 4.0
2 (p 2 ) 1 Receive 19.1 20.3 ... 1.2
2 (p 2 ) 2 Invoke 20.4 21.7 ... 1.3
2 (p 2 ) Plan 19.0 22.0 ... 3.0
3 (p 3 ) 1 Receive 24.2 26.1 ... 1.9
3 (p 3 ) 2 Invoke 26.1 27.9 ... 1.8
3 (p 3 ) Plan 24.0 28.0 ... 4.0
Basically, the optimization algorithm is triggered with period ∆t = 16 ms. At those timestamps,
only statistics of plan instances p i ∈ [T k − ∆w, T k ] are used for cost estimation.
Hence, at T 1 = 13, only statistics of p 1 are included, while at T 2 = 29 (T 1 + ∆t), statistics
of p 2 and p 3 are used.
As a result, we are able to periodically estimate the costs of a plan with the aim to
optimize this plan according to the current workload characteristics.
In the following, we discuss the complexity of this approach. Essentially, the periodic
plan optimization problem that includes the creation of the optimal plan is NP-hard. This
claim is justified by the known complexity of two subproblems (concrete optimization
techniques). First, the merging of parallel flows (see Fork operator) to a minimal number
of parallel flows with a maximum constraint on the total costs of such a flow is reducible
to the NP-hard bin packing problem. Second, also the subproblem of join enumeration is,
in general, NP-hard but requires a more detailed argumentation:
• A plan is a hierarchy of sequences (Definition 2.1) with control-flow semantics (that
subsume the data-flow semantics). Hence, all types of join queries are possible.
• The join enumeration is NP-hard in general (if all types of join queries are supported)
[Neu09]. For a comprehensive analysis of known results, see [Moe09].
• If a cost model has the ASI property, join enumeration can be computed with polynomial
time. There, ranks are assigned to relations and the sequence of ordered
ranks is optimal [IK84, KBZ86, CM95].
• Our cost model does not exhibit the ASI property (see Subsection 3.2.2).
As a result, the periodic plan optimization problem belongs to the complexity class of
NP-hard problems. Obviously, the analogous problem of cost-based query optimization
in DBMS is also NP-hard. However, in [SMWM06], it was shown that the optimal Web
service query plan can be computed in O(n 5 ), where n is the number of Web services.
The difference is caused by their assumption of negligible local processing costs (including
joins and other data-flow-oriented operators) such that no join enumeration has been used.
In contrast, our optimization objective is to minimize the average total execution costs
including local processing steps. In order to ensure efficient periodical re-optimization, we
will introduce tailor-made search space reduction heuristics in Subsection 3.3.2.
47