Performance Modeling and Benchmarking of Event-Based ... - DVS

6.3. CONCLUDING REMARKS 133
the model is optimistic in that the predicted completion times are lower than the measured ones.
This behavior is typical for performance models in general since no matter how representative
they are, they normally cannot capture all factors causing delays in the system.
6.2.4 Conclusion
In summary, the model proved to be very accurate in predicting the system performance, especially
considering the size and complexity of the system that was modeled. The proposed model
can be used as a powerful tool in the software engineering lifecycle of event-driven systems. For
example at system design time, predictive performance models can be exploited for comparing
alternative system designs with different communication and messaging patterns. At system deployment
time, models help to detect system bottlenecks and to ensure that sufficient resources
are allocated to meet performance and QoS requirements.
6.3 Concluding Remarks
In this chapter, we presented two novel case studies of representative state-of-the-art event-based
systems showing how our proposed methodology can be exploited for workload characterization,
performance modeling and prediction.
In the first case study, we applied our performance methodology to the SIENA publish/subscribe
system and validated our approach by providing a workload characterization and performance
model for a basic workload. In the second case study, we studied the SPECjms2007 standard
benchmark deployed on a leading commercial middleware platform. A detailed model was developed
in a step-by-step fashion and ways to customize the model for a particular deployment
scenario were demonstrated. The model contains a total of 59 queueing places, 76 token colors
and 68 transitions with a total of 285 firing modes. To validate our modeling technique we considered
a real-life deployment in a representative environment comparing the model predictions
against measurements on the real system. A number of different scenarios with varying workload
intensity and interaction mixes were considered and the accuracy of the developed models was
evaluated.
The results demonstrated the effectiveness and practicality of the proposed modeling and prediction
approach. The technique can be exploited as a powerful tool for performance prediction
and capacity planning during the software engineering lifecycle of message-oriented event-driven
systems.