Performance Modeling and Benchmarking of Event-Based ... - DVS

110 CHAPTER 5. BENCHMARKING OF EVENT-BASED SYSTEMS
CPU Utilization
100
80
60
40
20
0
0 2 4 6 8 10 12 14 16 18 20
# Queues
Exp. 1
Exp. 2
Message Throughput (Msg/sec)
25000
20000
15000
10000
5000
0
0 2 4 6 8 10 12 14 16 18 20
# Queues
Exp. 1
Exp. 2
CPU Time Per Msg (ms)
1
0.8
0.6
0.4
0.2
0
0 2 4 6 8 10 12 14 16 18 20
# Queues
Exp. 1
Exp. 2
CPU Time Per KByte Payload (ms)
1
0.8
0.6
0.4
0.2
0
0 2 4 6 8 10 12 14 16 18 20
# Queues
Exp. 1
Exp. 2
Figure 5.29: Scenarios 1 and 2: NPNT vs.
Queues
PT P2P Messaging with Increasing Number of
5.2.6 Conclusions of the SPECjms2007 Case Study
In this section, we presented a case study of a leading JMS platform, the WebLogic server, conducting
an in-depth performance analysis of the platform under a number of different workload
and configuration scenarios. We evaluated the server performance for both the point-to-point
and publish/subscribe messaging domains studying the effect of individual workload characteristics
on the server CPU utilization, the message throughput, the CPU processing time per
message/KByte payload, the message delivery latency, etc. Two groups of scenarios were tested.
The first group uses complex workloads based on the standard horizontal and vertical topologies
provided by the benchmark. The second group includes scenarios that focus on specific aspects
and features of MOM, e.g the overhead of persisting messages, the influence of the message size,
the effect of increasing the number of message producers/consumers and the maximum throughput
that can be processed through a given number of queues. In most cases, the system scaled
in a linear fashion and did not exhibit any unexpected behavior. Interesting observations are the
relation between the number of the producer threads and message throughput, the number of
topic consumers and CPU load, and the influence of pub/sub and P2P messaging in the vertical
scenario.
5.3 jms2009-PS - A Publish /Subscribe Benchmark
While SPECjms2007 includes some limited publish/subscribe communication as part of the
workload, the focus of the benchmark is on point-to-point communication via queues which
dominate the overall system workload. Moreover, the SPECjms2007 workload implementation