Cost-Based Optimization of Integration Flows - Datenbanken ...

2 Preliminaries and Existing Techniques
tion model [OAS06], where compensation flows are modeled by the user (e.g., the compensation
of an INSERT would be a DELETE with the appropriate identifier). These compensations
are executed for successfully executed parts of an integration flow. As a result, the
compensated parts are rolled back (compensated) and completely re-executed after that.
With regard to arbitrary external systems and applications, there might exist operations
where no compensation exists at all. Second, there is the recovery-based transaction model
[BHLW08a, SWDC10] that tries to address the problem of missing compensations. Here,
REDO-images—in the sense of output messages of successfully executed operators—are
stored in order to resume integration flows after the last successful operator. In conclusion,
the problems of message lost and message double processing are typically addressed with
persistent message storage and a tailor-made recovery model. Thus, the contract of an integration
platform can be extended from store-and-forward to a form that guarantees that
each received message will be successfully delivered exactly once to the external systems.
Beside these data-related guarantees also temporal guarantees must be ensured. From
the perspective of integration flow optimization, we would consider executing subsequent
plan instances in parallel. Unfortunately, the problem of message outrun would arise.
Problem 2.3 (Message Outrun). Assume two messages m 1 and m 2 , where m 1 arrives
earlier at the integration platform than m 2 , with t 1 < t 2 . If we execute the two resulting
plan instances p 1 and p 2 in parallel, an outrun of messages in terms of changed sequential
order of messages at the outbound side might take place and the result of p 2 is sent to
the external system s 1 before the result of p 1 . For example, if customer master data is
propagated to the external system s 1 with the customer’s first order, a message outrun can
result in a referential integrity conflict within the target system s 1 . Additional examples
from the area of financial messaging that also require serialization are financial statements
and stock exchange orders.
To tackle this problem, typically, inbound message queues are used in combination with
single-threaded plan execution. This serialized execution of plan instances guarantees that
no message outrun can take place. This is comparable to snapshot isolation in DBMS
[LKPMJP05, CRF08]. Hence, internal out-of-order processing would be possible, because
we only need to ensure the serialized external behavior in the sense that the inbound order
is equivalent to the outbound order of messages. More formally, eventual consistency
[Vog08] with the property of monotonic writes (serialize the writes of the same plan), and
thus, with convergence property, must be guaranteed. In addition, also monotonic reads
with regard to individual data objects must be ensured.
The mentioned transactional properties have several implications for the cost-based
optimization of integration flows. First, when rewriting plans during optimization, we must
be aware of the problems of message lost, message double processing, and message outrun.
Second, the contract of an integration platform with any client application or system
is that each received message must be successfully delivered, in arrival-order (monotonic
writes), with monotonic reads from external systems, exactly once to the external systems.
2.4 Use Cases
From a business perspective, we distinguish between horizontal and vertical integration of
information systems [Sch01]. In this section, we illustrate an example scenario for both
use cases, including concrete integration flows that we will use as running examples and
26