Cost-Based Optimization of Integration Flows - Datenbanken ...

2 Preliminaries and Existing Techniques
none of the existing approaches addresses the basic characteristic of integration flows that
they are deployed once and executed many times. On the one side, there are approaches
that follow the optimize-once model, where rule-based optimization is applied during the
initial deployment of a plan only. These approaches cannot adapt to changing workload
characteristics, which may lead to poor plans over time, and many optimization techniques
that require cost-based decisions cannot be applied at all. On the other side, there
are approaches that follow the optimize-always model, where optimization is triggered
whenever an instance of an integration flow is executed. This might lead to tremendous
optimization overhead if many instances with rather small amounts of data are executed
over time because in such scenarios, the optimization time can be even higher than the
execution time of a single instance. In conclusion, we observe the lack of a tailor-made
cost-based optimization approach for integration flows that allows the continuous adaptation
to changing workload characteristics and that exploits the specific characteristics of
integration flows in the form of being deployed once and executed many times.
2.2 Adaptive Query Processing
In contrast to the mainly rule-based optimization of integration flows, there is a large
body of work on the cost-based optimization in various system categories. With regard
to the aim of adaptation to changing workload characteristics as well as to unknown and
misestimated statistics, the literature refers to this field as Adaptive Query Processing
(AQP). In this section, we classify and discuss the existing techniques. For this purpose,
we present an extended AQP classification that combines the known, system-oriented classification
of Babu and Bizarro [BB05] with the time-oriented classification of Deshpande
et al. [DHR06] and extend it by the category of integration flows. We focus only on
the main characteristics and drawbacks but refer to the surveys [BB05, DIR07] and the
tutorials [DHR06, IDR07] for a more detailed analysis of the individual categories.
2.2.1 Classification of Adaptive Query Processing
From a system perspective [BB05], we distinguish between (1) the plan-based adaptation of
ad-hoc queries in DBMS, (2) the adaptation of deployed integration flows (see Section 2.1)
in integration platforms, (3) the adaptation of continuous queries (CQs) in DSMS, and
(4) tuple routing as a specific execution model for CQs. We use this system-oriented
classification when surveying existing techniques in the following subsections. All of those
different system categories exhibit specific characteristics that are reflected by the specific
optimization approaches.
In addition to this system-oriented classification, we use a time-based classification in
the sense of when re-optimization is triggered. Figure 2.7 illustrates the resulting overall
classification. According to the spectrum of adaptivity [DHR06], there are essentially four
types when re-optimization can be triggered. First, the coarse-grained inter-query optimization
refers to the standard optimization model of DBMS as established in System R
[SAC + 79], where each query is optimized during compile-time and thus, before execution
(optimize-always). For OLTP systems with rather short query execution times, plan or
QEP (query execution plan) caching [ZDS + 08, BBD09, Low09] is a widely used approach
in order to reduce the optimization time by compiling a new QEP only if it does not
exist or if statistics have changed significantly (optimize once). Second, late binding uses
18