Performance Modeling and Benchmarking of Event-Based ... - DVS

More documents

Recommendations

Info

98 CHAPTER 5. BENCHMARKING OF EVENT-BASED SYSTEMS Figure 5.17: Formal Measurement Points during SPECjms2007 Run [214] Collecting Runtime stats 1 for the last 300 sec. statistics complete in 0 seconds =======Interaction 1 ======= - Predicted Sent/Received: 61672.71 / 61672.71 - Measured Sent/Received: 62509(+1.36 %) / 62510(+1.36 %) =======Interaction 2 ======= - Predicted Sent/Received: 27341.57 / 31658.66 - Measured Sent/Received: 28143(+2.93 %) / 32604(+2.99 %) =======Interaction 3 ======= - Predicted Sent/Received: 674.55 / 6745.45 - Measured Sent/Received: 703(+4.22 %) / 7030(+4.22 %) =======Interaction 4 ======= - Predicted Sent/Received: 21877.14 / 21877.14 - Measured Sent/Received: 21899(+0.1 %) / 21896(+0.09 %) =======Interaction 5 ======= - Predicted Sent/Received: 77869.59 / 77869.59 - Measured Sent/Received: 79321(+1.86 %) / 79323(+1.87 %) =======Interaction 6 ======= - Predicted Sent/Received: 1919.96 / 19199.58 - Measured Sent/Received: 1860(-3.12 %) / 18599(-3.13 %) =======Interaction 7 ======= - Predicted Sent/Received: 1383.68 / 13836.83 - Measured Sent/Received: 1414(+2.19 %) / 14139(+2.18 %) Time to collect and process periodic runtime stats (msec): 428 Figure 5.18: Output of Run Time Reporter The final reporter generates a set of detailed result files. These provide the data in different granularity and allow in-depth analysis of the run. 2. Run Time Reporter The run time reporter collects periodically measurements and statistics on the satellites and forwards them to the controller node, where they are processed to generate run time reports. These reports provides information on the running system and how current message counts differ from model’s prediction (see Figure 5.18). By this, an early identification of message backlogs at run time is possible.
5.1. SPECJMS2007 - A STANDARD BENCHMARK 99 Audit Test Scope Description Input rate is within +-5% of configured value Interaction For each Interaction, the observed input rate is calculated as count/time for the measurement period. This must be within 5% of the value prescribed by Total message count is within +-5% of configured value Input rate distribution deviations do not exceed 20% 90th percentile of Delivery Times under 5000ms All messages sent were received Interaction Interaction Message Destination Message Destination the topology. Using a model of the scenario, the benchmark knows how many messages should be processed as part of each Interaction. The observed number of messages sent and received by all parties must be within 5% of this value. The percentage of pacing misses (in driver threads) the benchmark will allow. A miss is when the timing code found itself behind where it thought it should be. Messages are timestamped when sent and received. The consequent delivery time is recorded as a histogram (for the measurement period only) in each event handler and the 90th percentile of this histogram must be on or under 5000ms. Fails if the final results (taken after the Drain Period) show that not all sent messages were received. For publish-subscribe topics this means received by all subscribers. Table 5.8: Audit Tests Auditor To validate the correctness of a benchmark run, SPECjms2007 comes with two auditor components: • Pre-Auditor (executed in Phase I): The pre-auditor checks the availability of JMS message destinations (queues / topcis) and whether messages exist before starting a benchmark run which would invalidate a run (but might otherwise only be detected when the test is completed). If the pre-audit fails for any reason the benchmark is not started. • Auditor (executed in Phase III): The auditor component is responsible for making sure that the run has been executed properly and that the results are valid. The Auditor is automatically launched at the end of the run to validate the results. It performs an explicit audit of the results against the specified requirements marking each audit test as P ASS or F AIL in the summary reports. A list of all audit tests in provided in Table 5.8. Minimizing the Impact of Non-MOM-Related Components As discussed in Section 5.1.1, the SPECjms2007 workload should be focused on measuring the performance and scalability of a MOM server’s software and hardware components and minimize the impact of other components that are not directly related to the MOM services. While developing SPECjms2007 two concerns had to be addressed in order to achieve this goal. The first one is how to avoid using a database and the second one is how to minimize the message processing overhead on the client. Given the fact that MOM servers, in their role as mediators in interactions, are typically less loaded than database servers or application servers,
Page 1:
Performance Modeling and Benchmarki
Page 4 and 5:
priorities. Finally, we validated o
Page 6 and 7:
4.3.4 Tool Extension . . . . . . .
Page 8 and 9:
5.7 Proportions of the Interactions
Page 11 and 12:
List of Acronyms Acronym λ t,k j A
Page 13 and 14:
Acronym SPEC-OSG SPE SPN SQL ST SUT
Page 15 and 16:
Chapter 1 Introduction 1.1 Motivati
Page 17 and 18:
PC Server 310 PC Server 310 PC Serv
Page 19 and 20:
1.3. APPROACH AND CONTRIBUTIONS OF
Page 21 and 22:
1.4. THESIS ORGANIZATION 7 • Stan
Page 23 and 24:
Chapter 2 Background In this chapte
Page 25 and 26:
2.1. EVENT-BASED SYSTEMS 11 is gene
Page 27 and 28:
2.2. TECHNOLOGY PLATFORMS OF EVENT-
Page 29 and 30:
Page 31 and 32:
Page 33 and 34:
Page 35 and 36:
2.3. INTRODUCTION TO QUEUEING PETRI
Page 37 and 38:
2.3. INTRODUCTION TO QUEUEING PETRI
Page 39 and 40:
Chapter 3 Related Work In this chap
Page 41 and 42:
3.2. BENCHMARKING OF EVENT-BASED SY
Page 43 and 44:
3.2. BENCHMARKING OF EVENT-BASED SY
Page 45 and 46:
3.4. CONCLUDING REMARKS 31 hierarch
Page 47 and 48:
Chapter 4 Performance Engineering o
Page 49 and 50:
4.1. MODELING METHODOLOGY FOR EBS 3
Page 51 and 52:
Page 53 and 54:
Page 55 and 56:
Page 57 and 58:
4.2. PERFORMANCE MODELING PATTERN 4
Page 59 and 60:
4.2. PERFORMANCE MODELING PATTERN 4
Page 61 and 62: 4.2. PERFORMANCE MODELING PATTERN 4
Page 87 and 88: 4.3. EXTENSIONS OF QPNS 73 ,-).$%&'
Page 89 and 90: 4.4. CONCLUDING REMARKS 75 Without
Page 91 and 92: Chapter 5 Benchmarking of Event-Bas
Page 93 and 94: 5.1. SPECJMS2007 - A STANDARD BENCH
Page 111: 5.1. SPECJMS2007 - A STANDARD BENCH
Page 115 and 116: 5.2. CASE STUDY I: SPECJMS2007 101
Page 125 and 126: 5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 127 and 128: 5.3. JMS2009-PS - A PUBLISH /SUBSCR
Page 129 and 130: 5.4. CASE STUDY II: JMS2009-PS 115
Page 131 and 132: 5.4. CASE STUDY II: JMS2009-PS 117
Page 133 and 134: Chapter 6 Performance Modeling of E
Page 135 and 136: 6.2. MODELING SPECJMS2007 121 Scena
Page 137 and 138: 6.2. MODELING SPECJMS2007 123 BASE*
Page 139 and 140: 6.2. MODELING SPECJMS2007 125 1‘O
Page 141 and 142: 6.2. MODELING SPECJMS2007 127 Sc. 1
Page 143 and 144: 6.2. MODELING SPECJMS2007 129 Perce
Page 145 and 146: 6.2. MODELING SPECJMS2007 131 Serve
Page 147 and 148: 6.3. CONCLUDING REMARKS 133 the mod
Page 149 and 150: Chapter 7 Conclusions and Outlook W
Page 151 and 152: 7.1. ONGOING AND FUTURE WORK 137 JA
Page 153 and 154: 7.1. ONGOING AND FUTURE WORK 139 th
Page 155 and 156: Bibliography [1] J. Abbott, K. B. M
Page 157 and 158: BIBLIOGRAPHY 143 [29] BiCEP Researc
Page 159 and 160: BIBLIOGRAPHY 145 [61] G. Cugola and
Page 161 and 162: BIBLIOGRAPHY 147 [92] R. Henjes, M.
Page 163 and 164:
BIBLIOGRAPHY 149 [128] S. Kounev an
Page 165 and 166:
BIBLIOGRAPHY 151 [162] Object Compu
Page 167 and 168:
BIBLIOGRAPHY 153 [197] K. Sachs, S.
Page 169 and 170:
BIBLIOGRAPHY 155 [228] P. Tran, P.
Page 171:
Erklärung Hiermit erkläre ich, di
show all

Performance Modeling and Benchmarking of Event-Based ... - DVS

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?