Performance Modeling and Benchmarking of Event-Based ... - DVS

More documents

Recommendations

Info

96 CHAPTER 5. BENCHMARKING OF EVENT-BASED SYSTEMS Figure 5.16: Driver Framework Agents Each logical agent represents one of the locations in the application scenario. This means that, for example, a distribution center agent will contain a set of DC event handlers pertaining to that location. Agents connect back to the Controller who coordinates the stages of the test. Once all agents are connected, the event handlers (implemented as a Java thread each) start connecting to the JMS server and the warm-up phase of messaging begins. The controller manages the life cycle of the test by monitoring progress, coordinating phase changes and collecting statistics from the other components. When complete, it validates and combines the statistics into summary output files and presents the final metric for the test. Steps of a SPECjms2007 Run A SPECjms2007 run is a sequence of eight periodes[214], which can be grouped according three phases: Phase I: Benchmark Preparation 1. Starting Driver Framework 2. Starting Agents 3. Starting Event Handlers Phase II: Benchmark Run 4. Warmup Period 5. Measurement Period 6. Drain Period Phase III: Post Run 7. Stopping Event Handlers 8. Post-processing Results Period 1: Starting Driver Framework The controller component reads in all of the configuration and topological layout preferences given by the user. This will include items such as the number of different types of location and lists of the nodes across which they are distributed.
5.1. SPECJMS2007 - A STANDARD BENCHMARK 97 For vertical and horizontal scaling, many of these values are either fixed or are automatically calculated based upon the scaling factor. With this knowledge, the controller instantiates the software components of the SUT. It begins this by starting an RMI server and connecting to a satellite process on each node machine identified as part of this test to give it specific instructions. In all places (throughout the benchmark) where lists are given, work is distributed equally using a round-robin algorithm. Period 2: Starting Agents The satellite is a simple part of the framework that knows how to build the correct environment to start the required JVM processes. It takes the controller’s configuration and starts the agents relevant to that node. There is an architectural lower bound of one Agent-JVM per class of location (SP, SM, DC, HQ), meaning a minumum of four AgentJVMs in the SUT. Each agent connects back to the controller to signal their readiness. Period 3. Starting Event Handlers The controller signals all agents to initialise their event handler threads and connect to the JMS resources they will be using (this includes both incoming and outgoing Destinations). Each event handler is implemented as a Java thread. Period 4: Warmup Period Load-generating threads (drivers) ramp up their throughput from zero to their configured rate over the Warmup Period. This helps ensure the SUT is not swamped by an initial rush when many of its constituent elements may not yet be fully prepared. The agents are the only parts of the SUT which perform JMS operations (i.e. talk directly to the JMS middleware). Period 5: Measurement Period The measurement period is also known as the steadystate period. All agents are running at their configured workload and no changes are made. The controller will periodically (thirty seconds by default) check that there are no errors and may also collect periodic realtime performance statistics. Period 6: Drain Period In order to make sure all produced messages have an opportunity to be consumed, the controller signals agents to pause their load-generating threads (drivers). This period is not expected to be long in duration as a noticeable backlog in messages would invalidate audit requirements on throughput anyway. Period 7: Stopping Event Handlers Agents will terminate all event handlers but remain present themselves so that the controller can collect final statistics. Period 8: Post-processing Results Having collected statistics from all parties, the controller begins post-process including auditing and preparation of the final results. Reporter Framework The benchmark includes a reporter framework that prepares detailed reports about the run. For this purpose, the reporter framework enriches collected measurement data with other information, e.g., throughput predictions and configuration settings. The reporter framework has two major components: 1. Final Reporter The controller takes formal measurements at three points during the run (see Figure 5.17). The first two, the beginning and end of the measurement period, are used to audit the messaging throughput. The final measurement, at the end of the drain period, is used to audit the final message counts.
Page 1:
Performance Modeling and Benchmarki
Page 4 and 5:
priorities. Finally, we validated o
Page 6 and 7:
4.3.4 Tool Extension . . . . . . .
Page 8 and 9:
5.7 Proportions of the Interactions
Page 11 and 12:
List of Acronyms Acronym λ t,k j A
Page 13 and 14:
Acronym SPEC-OSG SPE SPN SQL ST SUT
Page 15 and 16:
Chapter 1 Introduction 1.1 Motivati
Page 17 and 18:
PC Server 310 PC Server 310 PC Serv
Page 19 and 20:
1.3. APPROACH AND CONTRIBUTIONS OF
Page 21 and 22:
1.4. THESIS ORGANIZATION 7 • Stan
Page 23 and 24:
Chapter 2 Background In this chapte
Page 25 and 26:
2.1. EVENT-BASED SYSTEMS 11 is gene
Page 27 and 28:
2.2. TECHNOLOGY PLATFORMS OF EVENT-
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
2.3. INTRODUCTION TO QUEUEING PETRI
Page 37 and 38:
2.3. INTRODUCTION TO QUEUEING PETRI
Page 39 and 40:
Chapter 3 Related Work In this chap
Page 41 and 42:
3.2. BENCHMARKING OF EVENT-BASED SY
Page 43 and 44:
3.2. BENCHMARKING OF EVENT-BASED SY
Page 45 and 46:
3.4. CONCLUDING REMARKS 31 hierarch
Page 47 and 48:
Chapter 4 Performance Engineering o
Page 49 and 50:
4.1. MODELING METHODOLOGY FOR EBS 3
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
4.2. PERFORMANCE MODELING PATTERN 4
Page 59 and 60: 4.2. PERFORMANCE MODELING PATTERN 4
Page 87 and 88: 4.3. EXTENSIONS OF QPNS 73 ,-).$%&'
Page 89 and 90: 4.4. CONCLUDING REMARKS 75 Without
Page 91 and 92: Chapter 5 Benchmarking of Event-Bas
Page 93 and 94: 5.1. SPECJMS2007 - A STANDARD BENCH
Page 109: 5.1. SPECJMS2007 - A STANDARD BENCH
Page 115 and 116: 5.2. CASE STUDY I: SPECJMS2007 101
Page 125 and 126: 5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 127 and 128: 5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 129 and 130: 5.4. CASE STUDY II: JMS2009-PS 115
Page 131 and 132: 5.4. CASE STUDY II: JMS2009-PS 117
Page 133 and 134: Chapter 6 Performance Modeling of E
Page 135 and 136: 6.2. MODELING SPECJMS2007 121 Scena
Page 137 and 138: 6.2. MODELING SPECJMS2007 123 BASE*
Page 139 and 140: 6.2. MODELING SPECJMS2007 125 1‘O
Page 141 and 142: 6.2. MODELING SPECJMS2007 127 Sc. 1
Page 143 and 144: 6.2. MODELING SPECJMS2007 129 Perce
Page 145 and 146: 6.2. MODELING SPECJMS2007 131 Serve
Page 147 and 148: 6.3. CONCLUDING REMARKS 133 the mod
Page 149 and 150: Chapter 7 Conclusions and Outlook W
Page 151 and 152: 7.1. ONGOING AND FUTURE WORK 137 JA
Page 153 and 154: 7.1. ONGOING AND FUTURE WORK 139 th
Page 155 and 156: Bibliography [1] J. Abbott, K. B. M
Page 157 and 158: BIBLIOGRAPHY 143 [29] BiCEP Researc
Page 159 and 160: BIBLIOGRAPHY 145 [61] G. Cugola and
Page 161 and 162:
BIBLIOGRAPHY 147 [92] R. Henjes, M.
Page 163 and 164:
BIBLIOGRAPHY 149 [128] S. Kounev an
Page 165 and 166:
BIBLIOGRAPHY 151 [162] Object Compu
Page 167 and 168:
BIBLIOGRAPHY 153 [197] K. Sachs, S.
Page 169 and 170:
BIBLIOGRAPHY 155 [228] P. Tran, P.
Page 171:
Erklärung Hiermit erkläre ich, di
show all

Performance Modeling and Benchmarking of Event-Based ... - DVS

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?