Cost-Based Optimization of Integration Flows - Datenbanken ...

More documents

Recommendations

Info

List of Figures 3.27 Workload Adaptation Delays . . . . . . . . . . . . . . . . . . . . . . . . . . 82 3.28 Influence of Parameters on the Sensibility of Workload Adaptation . . . . . 83 3.29 Comparison Scenario of Periodical Re-Optimization with Correlation . . . . 85 4.1 Example Instance-Based Execution of Plan P 2 . . . . . . . . . . . . . . . . 90 4.2 Example Fully Vectorized Execution of Plan P 2 ′ . . . . . . . . . . . . . . . . 90 4.3 Temporal Aspects of Instance-Based and Vectorized Plans . . . . . . . . . . 91 4.4 Conceptual Model of Execution Buckets . . . . . . . . . . . . . . . . . . . . 93 4.5 Example Plan Vectorization . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 4.6 Rewriting Switch Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 4.7 Rewriting Invoke Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 4.8 Speedup Test with Varying Degree of Parallelism . . . . . . . . . . . . . . . 100 4.9 Example Cost-Based Plan Vectorization . . . . . . . . . . . . . . . . . . . . 101 4.10 Spectrum of Cost-Based Vectorization . . . . . . . . . . . . . . . . . . . . . 101 4.11 Work Cycle Domination by Operator o 3 . . . . . . . . . . . . . . . . . . . . 102 4.12 Plan-Dependent Search Space . . . . . . . . . . . . . . . . . . . . . . . . . . 103 4.13 Bucket Merging with Different λ . . . . . . . . . . . . . . . . . . . . . . . . 108 4.14 Heuristic Operator Distribution with Fixed k . . . . . . . . . . . . . . . . . 111 4.15 Example Operator Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . 112 4.16 Problem of Solving P-CPV for all Plans . . . . . . . . . . . . . . . . . . . . 114 4.17 Heuristic Multiple Plan Vectorization . . . . . . . . . . . . . . . . . . . . . . 116 4.18 Example Periodical Re-Optimization . . . . . . . . . . . . . . . . . . . . . . 118 4.19 Use Case Comparison of Vectorization . . . . . . . . . . . . . . . . . . . . . 120 4.20 Evaluated Example Plan P m . . . . . . . . . . . . . . . . . . . . . . . . . . 121 4.21 Scalability Comparison with Different Influencing Factors . . . . . . . . . . 122 4.22 Latency Time and Execution Time of Single Messages (for n = 250) . . . . 124 4.23 Vectorization Deployment Overhead . . . . . . . . . . . . . . . . . . . . . . 125 4.24 Influence of λ with Different Numbers of Operators and Data Sizes . . . . . 126 4.25 Restricting k with Different Numbers of Operators and Data Sizes . . . . . 127 5.1 Example Instance-Based Plan Execution . . . . . . . . . . . . . . . . . . . . 130 5.2 Example Message Batch Plan Execution . . . . . . . . . . . . . . . . . . . . 131 5.3 Common Cases of Rewritten Queries to External Systems . . . . . . . . . . 132 5.4 Query Execution Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 5.5 Partitioned Message Batch Execution P 2 ′ . . . . . . . . . . . . . . . . . . . . 133 5.6 Example Partition Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 5.7 Example Hash Partition Tree . . . . . . . . . . . . . . . . . . . . . . . . . . 137 5.8 Inverse Operators PSplit and PMerge . . . . . . . . . . . . . . . . . . . . . 140 5.9 Example Plan Rewriting Using the Split and Merge Approach . . . . . . . . 140 5.10 P-MFO Temporal Aspects (with ∆tw > W (P ′ )) . . . . . . . . . . . . . . . 143 5.11 Relative Execution Time W (P 2 ′, k′ )/k ′ . . . . . . . . . . . . . . . . . . . . . 145 5.12 Search Space for Waiting Time Computation . . . . . . . . . . . . . . . . . 147 5.13 Monotonically Non-Increasing Execution Time with Lower Bound . . . . . 152 5.14 Waiting Time Computation With Skewed Message Arrival Rate . . . . . . . 154 5.15 Use Case Comparison of Multi-Flow Optimization . . . . . . . . . . . . . . 157 5.16 Use Case Scalability Comparison with Varying Data Size d . . . . . . . . . 159 5.17 Use Case Scalability Comparison with Varying Batch Size k ′ . . . . . . . . 159 5.18 Execution Time W (P 2 ′, k′ ) with Varying Batch Size k ′ . . . . . . . . . . . . 160 220
List of Figures 5.19 Varying R and sel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 5.20 Latency Time of Single Messages T L (m i ) . . . . . . . . . . . . . . . . . . . . 162 5.21 Latency in Overload Situations . . . . . . . . . . . . . . . . . . . . . . . . . 162 5.22 Runtime Overhead for Enqueue Operations with Different Message Queues 163 5.23 Use Case Scalability Comparison with Varying Data Size d . . . . . . . . . 164 6.1 Drawbacks of Periodical Re-Optimization . . . . . . . . . . . . . . . . . . . 168 6.2 Example Plan P 5 and Monitored Statistics . . . . . . . . . . . . . . . . . . . 169 6.3 Partitioning of a Plan Search Space . . . . . . . . . . . . . . . . . . . . . . . 170 6.4 PlanOptTree of Plan P 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 6.5 General Structure of a PlanOptTree . . . . . . . . . . . . . . . . . . . . . . 172 6.6 Merging Partial PlanOptTrees . . . . . . . . . . . . . . . . . . . . . . . . . 174 6.7 Determining Re-Optimization Potential . . . . . . . . . . . . . . . . . . . . 178 6.8 Partial PlanOptTree Replacement . . . . . . . . . . . . . . . . . . . . . . . 180 6.9 Example Join Enumeration . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 6.10 Example Eager Group-By . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 6.11 Example Union Distinct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 6.12 Example PlanOptTree of Cost-Based Vectorization . . . . . . . . . . . . . . 185 6.13 Example PlanOptTree of Multi-Flow Optimization . . . . . . . . . . . . . . 187 6.14 Simple-Flow Comparison Scenario . . . . . . . . . . . . . . . . . . . . . . . 190 6.15 Complex-Flow Comparison Scenario . . . . . . . . . . . . . . . . . . . . . . 192 6.16 Scalability of Plan P 5 Varying Influencing Parameters . . . . . . . . . . . . 193 6.17 Directed Join Enumeration Benefit . . . . . . . . . . . . . . . . . . . . . . . 194 6.18 Overhead of PlanOptTrees . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 6.19 Simple-Flow Correlation Comparison Scenario . . . . . . . . . . . . . . . . . 197 221
Page 1:
Cost-Based Optimization of Integrat
Page 4 and 5:
of traditional data management syst
Page 7 and 8:
Contents 1 Introduction 1 2 Prelimi
Page 9:
Contents 6.5 Experimental Evaluatio
Page 12 and 13:
1 Introduction for integration flow
Page 14 and 15:
1 Introduction violated. We present
Page 16 and 17:
2 Preliminaries and Existing Techni
Page 18 and 19:
Page 20 and 21:
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
Page 44 and 45:
3 Fundamentals of Optimizing Integr
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
Page 58 and 59:
Page 60 and 61:
Page 62 and 63:
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
Page 74 and 75:
Page 76 and 77:
Page 78 and 79:
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
Page 90 and 91:
Page 92 and 93:
Page 94 and 95:
Page 96 and 97:
Page 98 and 99:
4 Vectorizing Integration Flows •
Page 100 and 101:
4 Vectorizing Integration Flows exe
Page 102 and 103:
4 Vectorizing Integration Flows Ove
Page 104 and 105:
4 Vectorizing Integration Flows Alg
Page 106 and 107:
4 Vectorizing Integration Flows two
Page 108 and 109:
4 Vectorizing Integration Flows Inv
Page 110 and 111:
4 Vectorizing Integration Flows (a)
Page 112 and 113:
4 Vectorizing Integration Flows o 2
Page 114 and 115:
4 Vectorizing Integration Flows In
Page 116 and 117:
4 Vectorizing Integration Flows 2.
Page 118 and 119:
4 Vectorizing Integration Flows The
Page 120 and 121:
4 Vectorizing Integration Flows ord
Page 122 and 123:
4 Vectorizing Integration Flows 4.3
Page 124 and 125:
4 Vectorizing Integration Flows P
Page 126 and 127:
4 Vectorizing Integration Flows P
Page 128 and 129:
4 Vectorizing Integration Flows t1:
Page 130 and 131:
4 Vectorizing Integration Flows We
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
4 Vectorizing Integration Flows 4.7
Page 140 and 141:
5 Multi-Flow Optimization cannot be
Page 142 and 143:
5 Multi-Flow Optimization the query
Page 144 and 145:
5 Multi-Flow Optimization The incom
Page 146 and 147:
5 Multi-Flow Optimization example p
Page 148 and 149:
5 Multi-Flow Optimization not allow
Page 150 and 151:
5 Multi-Flow Optimization partition
Page 152 and 153:
5 Multi-Flow Optimization mention i
Page 154 and 155:
5 Multi-Flow Optimization • Case
Page 156 and 157:
5 Multi-Flow Optimization k ′ . F
Page 158 and 159:
5 Multi-Flow Optimization partition
Page 160 and 161:
5 Multi-Flow Optimization the waiti
Page 162 and 163:
5 Multi-Flow Optimization Execution
Page 164 and 165:
5 Multi-Flow Optimization Thus, for
Page 166 and 167:
5 Multi-Flow Optimization Thus, T L
Page 168 and 169:
5 Multi-Flow Optimization • P 5 :
Page 170 and 171:
5 Multi-Flow Optimization decreasin
Page 172 and 173:
5 Multi-Flow Optimization (a) Fixed
Page 174 and 175:
5 Multi-Flow Optimization reached,
Page 176 and 177:
5 Multi-Flow Optimization (2) plan
Page 178 and 179:
6 On-Demand Re-Optimization categor
Page 180 and 181: 6 On-Demand Re-Optimization present
Page 182 and 183: 6 On-Demand Re-Optimization stratum
Page 184 and 185: 6 On-Demand Re-Optimization o 3 o 4
Page 186 and 187: 6 On-Demand Re-Optimization For on-
Page 188 and 189: 6 On-Demand Re-Optimization 6.3.1 O
Page 190 and 191: 6 On-Demand Re-Optimization such th
Page 192 and 193: 6 On-Demand Re-Optimization Join En
Page 194 and 195: 6 On-Demand Re-Optimization f γ((
Page 196 and 197: 6 On-Demand Re-Optimization The res
Page 198 and 199: 6 On-Demand Re-Optimization project
Page 200 and 201: 6 On-Demand Re-Optimization (a) Sel
Page 202 and 203: 6 On-Demand Re-Optimization (a) Loa
Page 204 and 205: 6 On-Demand Re-Optimization ical re
Page 206 and 207: 6 On-Demand Re-Optimization evaluat
Page 208 and 209: 6 On-Demand Re-Optimization 6.6 Sum
Page 210 and 211: 7 Conclusions Existing approaches b
Page 212 and 213: Bibliography [BBD05a] Shivnath Babu
Page 214 and 215: Bibliography [BHP + 09b] [BHP + 11]
Page 216 and 217: Bibliography [CM95] Sophie Cluet an
Page 218 and 219: Bibliography [GZ08] [HA03] [Haa07]
Page 220 and 221: Bibliography [IKNG09] [INSS92] [Ioa
Page 222 and 223: Bibliography [LX09] [LZ05] Rubao Le
Page 224 and 225: Bibliography [OMG07] OMG. XML Metad
Page 226 and 227: Bibliography [Sto02] Michael Stoneb
Page 228 and 229: Bibliography [ZRH04] Yali Zhu, Elke
Page 233: List of Tables 2.1 Interaction-Orie
Page 237: Selbstständigkeitserklärung Hierm
show all

Cost-Based Optimization of Integration Flows - Datenbanken ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?