Cost-Based Optimization of Integration Flows - Datenbanken ...

2 Preliminaries and Existing Techniques
plan. Further, progressive optimization [HNM + 07, MRS + 04, EKMR06] uses checkpoint
operators to determine if validity ranges of subplans are violated and thus, can avoid
unnecessary re-optimization. The major drawback is that validity ranges are defined as
black boxes for subplans. Hence, directed re-optimization is impossible and it cannot be
guaranteed that the current plan is not optimal. In addition, there might be trashing
of intermediates, and the use of materialization points might be too coarse-grained. The
latter problems were addressed by corrective query processing [IHW04] (reactive, intraoperator)
that uses data partitioning across plans. Here, new plans are used only for new
data and stitch-up phases combine the results of different plans. This is disadvantageous
if there are large intermediate results in combination with large operator states because
these results must be post-processed and then merged with a union.
In contrast to these reactive re-optimization approaches, proactive, inter-operator reoptimization
in Rio [BBD05a, BBD05b] computes bounding boxes around all used estimates
to express their uncertainty before optimization. The bounding boxes are then
used to create robust or switchable plans. During execution, a switch operator can choose
between three (low, estimate, high) different remaining plans based on a random sample
of its input. However, those bounding boxes are used as black boxes with regard to single
estimates. Again, this makes directed re-optimization impossible and the suboptimality
of the current plan cannot be guaranteed.
To summarize, all plan-based adaptation approaches rely on the assumption of longrunning
queries, where mid-query re-optimization can significantly improve the query execution
time. Optimization is triggered synchronously at materialization points or asynchronously
in the case of intra-operator re-optimization.
2.2.3 Continuous-Query-Based Adaptation
In contrast to plan-based adaptation in DBMS, the adaptation of continuous queries in
DSMS is typically realized with another approach: The optimizer specifies which statistics
to gather, requests them from the monitoring component, and re-optimization is triggered
periodically or whenever significant changes have occurred [BB05]. CQ-specific aspects
are the extensive profiling of stream characteristics [BMM + 04] and the state migration
(e.g., tuples in hash tables) during re-optimization [ZRH04] in order to prevent missing
tuples or duplicates and to ensure the tuple order. Examples for this optimization model
are CAPE [ZRH04, RDS + 04, LZJ + 05], NiagaraCQ [CDTW00], StreaMon [BW04], and
PIPES [CKSV08, KS09, KS04]. There exist high statistics monitoring overhead and the
mentioned problem of when to trigger re-optimization.
In order to tackle the problem of state migration and to allow for fine-grained adaptation
as well as load balancing, also tuple routing strategies can be applied. The routing-based
adaptation does not use any optimizer but combines optimization, execution, and statistics
gathering. The most prominent example of such a system is Eddies [AH00, MSHR02].
An eddy operator is used to route single tuples along different operators rather than
using predefined plans. Due to the dynamic evaluation of applicable operators as well
as the decisions on routing paths by routing policies [AH00, TD03, BBDW05], there can
be significant overhead compared to plan-based adaptation [Des04]. This problem was
weakened by the self-tuning query mesh [NRB09, NWRB09] that uses a concept drift
approach to route groups of tuples instead of single tuples.
Both approaches of continuous-query-based adaptation and tuple routing strategies rely
on the assumption that continuous queries process infinite tuple streams. Specific charac-
20