Actas JP2011 - Universidad de La Laguna

Actas XXII Jornadas de Paralelismo (JP2011) , La Laguna, Tenerife, 7-9 septiembre 2011C.4 SummaryFollowing this analysis, the quality of olsrd v.0.5.6and v.0.6.0 do not really differ in presence of the consideredfaults, and they could be used indistinctlybut when the network is perturbed by flooding attacksor consumption is the main concerns of theevaluator, where v.0.6.0 reacts better. Otherwise,olsrd v.0.4.10 is the best candidate to meet the requirementsestablished for this case study.V. ConclusionsTraditional analysis of results performed duringthe resilience benchmarking of components and systemscan be feasible for those cases where a reducedset of measures is considered. However, as far asthe amount of measures increases, the analysis complexityrises as well. This is an important issue thatcan compromise the analysis thoroughness, despitethe rest of the benchmark process has been correctlycarried out.This paper describes a practical approach to systematisethe analysis of resilience benchmarkingwith a well-known technique in the domain of softwareengineering: the Logic Score of Preferences(LSP). Conversely to other measures-aggregationtechniques, LSP plays an active role during thebenchmark definition. It addresses how to adequatelyselect and gather the types of measures torepresent the system, thus assisting the benchmarkuser to minimise errors during the results interpretation.In this line of reasoning, the case study selectedto illustrate the applicability of this technique hasbeen been focused on the promising domain of adhoc networks, allowing the reader to understand thatthe steps described in the paper can be designed tofit different applications. The LSP technique resultsa very useful approach to overcome the problem ofmeasures scalability and eases a more concise visionof the system. Nevertheless, regarding previous results,the application of this technique requires theadequate definition of the quality thresholds (X minand X max ) for each criterion functions, the weight(w i ) assigned to each score within the same aggregationblock, and the operator type (o i ) in charge of themeasures aggregation. All these aspects highly dependon the applicative context the ad hoc networksis conceived to be deployed.Considering the points previously detailed is a firststep towards the characterisation of the wide amountof applicative domains ad hoc networks are presentin, such as Wireless/Underground and SubaquaticSensor Networks, Wireless Mesh Networks and Mobileand Vehicular Ad hoc Networks, among others.We argue that this type of approaches can be usefulnot only to quantify the impact of faults with respectto the actual application context (where componentsand systems are planned to be deployed),but for the comparison and selection of those targetswhich best fit the system requirements. In the futurework, we ambition to provide evaluators differenttemplates with precomputed parameters that theycould customise for their particular deployments tosemi-automate the application of LSP for the quantitativebenchmarking of ad hoc routing protocols.AcknowledgementsThis work has been funded by the Spanish Ministryof Science and Innovation through the SEMSE-CAP Project (TIN-2009-13825).References[1] Karama Kanoun and Lisa Spainhower, DependabilityBenchmarking for Computer Systems, Wiley and IEEEComputer Society Press, 2008.[2] Hung-Chin Jang, Yao-Nan Lien, and Tzu-Chieh Tsai,“Rescue information system for earthquake disastersbased on manet emergency communication platform,” inInternational Conference on Wireless Communicationsand Mobile Computing (IWCMC), 2009, pp. 623–627.[3] P. Serrano et al., “A CARMEN mesh experience: deploymentand results,” in 10th IEEE WoWMoM, 2009,pp. 1–8.[4] Jesus Friginal, David de Andres, Juan-Carlos Ruiz, andPedro Gil, “On selecting representative faultloads toguide the evaluation of ad hoc networks,” in FifthLatin American Symposium on Dependable Computing(LADC), 2011.[5] J.J. Dujmovic and R. Elnicki, A DMS Cost/Benefit DecisionModel: Mathematical Models for Data ManagementSystem Evaluation, Comparison, and Selection, NationalBureau of Standards, Washington D.C., No. GCR 82-374.NTIS No. PB 82-170150, 1982.[6] M. F. Morris, “Kiviat graphs: conventions and figures ofmerit.,” ACM/Sigmetrics Performance Evaluation Review,vol. 3, no. 3, pp. 2–8, 1974.[7] David de Andres, “Using dependability, performance,area and energy consumption experimental measures tobenchmark ip cores,” in Forth Latin American Symposiumon Dependable Computing (LADC), 2009.[8] Giulio Concas, Michele Marchesi, Sandro Pinna, andNicola Serra, “Power-laws in a large object-oriented softwaresystem,” IEEE Trans. Softw. Eng., vol. 33, pp.687–708, October 2007.[9] Yazeed A. Al-Sbou, Reza Saatchi, Samir Al-Khayatt,Rebecca Strachan, Moussa Ayyash, and MohammadSaraireh, “A novel quality of service assessment of multimediatraffic over wireless ad hoc networks,” in Proceedingsof the 2008 The Second International Conferenceon Next Generation Mobile Applications, Services, andTechnologies, 2008, pp. 479–484.[10] I. F. Akyildiz and others, “Wireless mesh networks: asurvey,” IEEE Radio Communications, vol. 43, pp. S23–S30, 2005.[11] “Hillsdale WMN,” Online: http://dashboard.openmesh.com/overview2.php?id=Hillsdale,2010.[12] David Andrés, Jesús Friginal, Juan-Carlos Ruiz, andPedro Gil, “An attack injection approach to evaluatethe robustness of ad hoc networks,” in IEEE PacificRim International Symposium on Dependable Computing(PRDC), 2009, pp. 228–233.[13] Jasmeet Chhabra et al., “Real-world experiences withan interactive ad hoc sensor network,” in Proceedings ofthe 2002 International Conference on Parallel ProcessingWorkshops, 2002, pp. 143–151.[14] Weiquan Lu, Winston K. G. Seah, Edwin W. C. Peh,and Yu Ge, “Communications support for disaster recoveryoperations using hybrid mobile ad-hoc networks,”in Proceedings of the 32nd IEEE Conference on LocalComputer Networks, 2007, pp. 763–770.JP2011-378