Cost-Based Optimization of Integration Flows - Datenbanken ...

4 Vectorizing Integration Flows
Over time and hence, with an increasing number of plan instances n, the performance
improvement regarding the total execution time grows linearly. We use the term performance
in the sense of high throughput and low execution time of a finite message sequence
M ′ . For this case of a finite sequence, where the incoming order of this sequence must be
preserved, (1) the execution time W (P, M ′ ), (2) the message throughput |M ′ |/∆t, and (3)
the latency time T L (M ′ ) are correlated. According to Little’s Law [Lit61], the rationale
for this is the waiting time within the system because instances of one plan must not
be executed in parallel. In more detail, decreasing total execution time of the message
subsequence decreases the waiting time, increases the message throughput and thus finally
decreases the total latency time of this message sequence:
W (P, M ′ ) ∝
1
|M ′ |/∆t ∝ T L(M ′ ). (4.3)
However, the latency of single messages can be higher for vectorized plans compared to
instance-based plans.
Message and Flow Meta Model
In order to allow for transparent vectorization as an internal optimization technique, the
control-flow semantics must be preserved when vectorizing a plan. Therefore, we extended
the Message Transformation Model (MTM) (Subsection 2.3.1) in order to make it applicable
for vectorized plans, where we refer to it as VMTM.
In the VMTM, we extend the message meta model from a triple to a quadruple with
m i = (t i , d i , a i , c i ), where the context information c denotes an additional map of atomic
name-value attribute pairs with c ij = (n j , v j ). This extension is necessary due to processing
of multiple messages within one single standing (vectorized) plan instead of independent
plan instances. Thus, instance-related context information such as local variables
(e.g., counters or extracted attribute values) must be stored within the messages.
In contrast to the MTM flow meta model, in the VMTM, the flow relations between
operators o i do not specify the control flow (temporal dependencies) but the explicit data
flow in the form of message streams. Additionally, the Fork operator is removed because
in the vectorized case, operators are inherently executed in parallel. Finally, we introduce
the additional operators And and Xor for synchronization of operators in order to preserve
Table 4.1: Additional Operators of the VMTM
Name Description Input Output Complex
And Reads a synchronization ID and a single
message and forwards the read message.
Xor Reads a synchronization ID and/or multiple
messages and outputs all messages,
which synchronization IDs have already
been seen. Thus, this operator has an intraoperator
state of IDs and messages.
Copy Gets a single message, then copies it n − 1
times and puts those messages into the n
output queues.
(2,2) (1,1) No
(1,*) (0,*) No
(1,1) (2,*) No
92