Performance Modeling and Benchmarking of Event-Based ... - DVS

More documents

Recommendations

Info

2 CHAPTER 1. INTRODUCTION it difficult to monitor and analyze system performance during operation. Closely related to EBS is the publish-subscribe paradigm that is nowadays used as a building block in major new software architectures and technology domains such as enterprise service bus (ESB), enterprise application integration (EAI), service-oriented architecture (SOA) and event-driven architecture (EDA) [50, 51, 101]. Modern EBS are implemented using several technology platforms such as centralized systems based on message-oriented middleware (MOM), e.g., IBM Web- Sphere and TIBCO Rendezvous, or large-scale distributed event-based systems (DEBS), e.g., SIENA [40], Hermes[179] or REBECA [154]. Several standards such as Java Message Service (JMS) [221], Advanced Message Queuing Protocol (AMQP), and Data Distribution Service (DDS) have been established and are supported by middleware vendors. A major task of system architects, developers and deployment managers is to choose adequate technologies and deploy the EBS in such a way that the QoS requirements are fulfilled. While developing, deploying, and maintaining event-based applications (and their underlying middleware), the following questions are often raised: • What performance would the system exhibit for a given deployment topology, configuration, and workload scenario • What is a typical workload scenario • What would be the expected notification and subscription delays as well as the utilization of the various system components (e.g., brokers, network links) • What maximum load (number of publishers and subscribers, event publication rates) would the system be able to handle without breaking the service level agreements • What would be the optimal number of brokers and the optimal system topology • Which components would be most utilized as the load increases and when are they potential bottlenecks • Will the event-based middleware scale for future loads • Which product offers the best performance for a certain workload scenario Benchmarks and performance modeling techniques help answering these questions. However, there is a lack of test harnesses and benchmarks using representative workloads for EBS. The same applies to performance models. Only a few approaches for modeling EBS have been published and they have limitations such as unrealistic assumptions or scalability issues. Furthermore, most traditional performance modeling techniques have not yet been evaluated using EBS. As far as the author of this thesis knows, neither an EBS benchmark implementing a real-world workload nor a performance modeling methodology focusing on EBS was available when this thesis effort was started. 1.3 Approach and Contributions of This Thesis In this section, we summarize the results of this thesis. First, we provide a brief overview of these results in the area of performance engineering. Second, we discuss those that lie within the area of benchmarking of EBS. We summarize our contributions and conclude this section with an overview of related activities and publications.
PC Server 310 PC Server 310 PC Server 310 PC Server 310 PC Server 310 PC Server 310 PC Server 310 PC Server 310 1.3. APPROACH AND CONTRIBUTIONS OF THIS THESIS 3 WAN WAN WAN WAN Figure 1.1: A (Distributed) Event-based System 1.3.1 Contributions in Performance Engineering Modeling EBS is challenging because of the decoupling of the communicating parties, on the one hand, and the dynamic changes in the system structure and behavior, on the other hand. If a request is sent in a traditional request/reply-based distributed system, it is sent directly to a given destination. This makes it easy to identify the system components and resources involved in its processing. In contrast, when an event is published in an EBS, as illustrated in Figure 1.1, it is not addressed to a particular destination, but rather routed along all those paths that lead to event consumers. The event might have to be processed by multiple system nodes on its way to consumers and it is difficult to predict in advance which nodes will be involved in delivering the event. Moreover, individual events published by a given producer might be routed along completely different paths, visiting various sets of system nodes. Another difficulty stems from the fact that every time a new subscription is created or an existing one is modified, this might lead to significant changes in the system behavior. Thus, the dynamics of EBS require that workload characterization be done on a regular basis in order to reflect changes in the system configuration and workload. We proposed a novel approach to workload characterization and performance modeling of EBS, aiming to address the above challenges. We developed a workload model based on monitoring data that captures the system routing behavior and resource consumption at a level that allows us to use this information as an input to performance models. The workload model we proposed does not make any assumptions about the algorithms used at the event routing and overlay network layers of the system. Using the workload model and applying operational analysis techniques, we were able to characterize the message traffic and determine the utilization of system components. This in turn enabled us to derive an approximation of the mean event delivery latency. For more accurate performance prediction, we proposed detailed performance models based on queueing Petri nets (QPNs). Furthermore, we discussed how different features of EBS can be reflected in performance models and introduced eleven performance modeling patterns for the most common settings and features of EBS. We used QPNs as a modeling technique to illustrate the patterns. However, several of our patterns are not (or only with high effort) convertible with standard QPNs.
Page 1: Performance Modeling and Benchmarki
Page 4 and 5: priorities. Finally, we validated o
Page 6 and 7: 4.3.4 Tool Extension . . . . . . .
Page 8 and 9: 5.7 Proportions of the Interactions
Page 11 and 12: List of Acronyms Acronym λ t,k j A
Page 13 and 14: Acronym SPEC-OSG SPE SPN SQL ST SUT
Page 15: Chapter 1 Introduction 1.1 Motivati
Page 19 and 20: 1.3. APPROACH AND CONTRIBUTIONS OF
Page 21 and 22: 1.4. THESIS ORGANIZATION 7 • Stan
Page 23 and 24: Chapter 2 Background In this chapte
Page 25 and 26: 2.1. EVENT-BASED SYSTEMS 11 is gene
Page 27 and 28: 2.2. TECHNOLOGY PLATFORMS OF EVENT-
Page 35 and 36: 2.3. INTRODUCTION TO QUEUEING PETRI
Page 37 and 38: 2.3. INTRODUCTION TO QUEUEING PETRI
Page 39 and 40: Chapter 3 Related Work In this chap
Page 41 and 42: 3.2. BENCHMARKING OF EVENT-BASED SY
Page 43 and 44: 3.2. BENCHMARKING OF EVENT-BASED SY
Page 45 and 46: 3.4. CONCLUDING REMARKS 31 hierarch
Page 47 and 48: Chapter 4 Performance Engineering o
Page 49 and 50: 4.1. MODELING METHODOLOGY FOR EBS 3
Page 57 and 58: 4.2. PERFORMANCE MODELING PATTERN 4
Page 67 and 68:
4.2. PERFORMANCE MODELING PATTERN 5
Page 69 and 70:
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
4.3. EXTENSIONS OF QPNS 73 ,-).$%&'
Page 89 and 90:
4.4. CONCLUDING REMARKS 75 Without
Page 91 and 92:
Chapter 5 Benchmarking of Event-Bas
Page 93 and 94:
5.1. SPECJMS2007 - A STANDARD BENCH
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
5.2. CASE STUDY I: SPECJMS2007 101
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 127 and 128:
5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 129 and 130:
5.4. CASE STUDY II: JMS2009-PS 115
Page 131 and 132:
5.4. CASE STUDY II: JMS2009-PS 117
Page 133 and 134:
Chapter 6 Performance Modeling of E
Page 135 and 136:
6.2. MODELING SPECJMS2007 121 Scena
Page 137 and 138:
6.2. MODELING SPECJMS2007 123 BASE*
Page 139 and 140:
6.2. MODELING SPECJMS2007 125 1‘O
Page 141 and 142:
6.2. MODELING SPECJMS2007 127 Sc. 1
Page 143 and 144:
6.2. MODELING SPECJMS2007 129 Perce
Page 145 and 146:
6.2. MODELING SPECJMS2007 131 Serve
Page 147 and 148:
6.3. CONCLUDING REMARKS 133 the mod
Page 149 and 150:
Chapter 7 Conclusions and Outlook W
Page 151 and 152:
7.1. ONGOING AND FUTURE WORK 137 JA
Page 153 and 154:
7.1. ONGOING AND FUTURE WORK 139 th
Page 155 and 156:
Bibliography [1] J. Abbott, K. B. M
Page 157 and 158:
BIBLIOGRAPHY 143 [29] BiCEP Researc
Page 159 and 160:
BIBLIOGRAPHY 145 [61] G. Cugola and
Page 161 and 162:
BIBLIOGRAPHY 147 [92] R. Henjes, M.
Page 163 and 164:
BIBLIOGRAPHY 149 [128] S. Kounev an
Page 165 and 166:
BIBLIOGRAPHY 151 [162] Object Compu
Page 167 and 168:
BIBLIOGRAPHY 153 [197] K. Sachs, S.
Page 169 and 170:
BIBLIOGRAPHY 155 [228] P. Tran, P.
Page 171:
Erklärung Hiermit erkläre ich, di
show all

Performance Modeling and Benchmarking of Event-Based ... - DVS

Create successful ePaper yourself

Delete template?

Save as template?