Download PDF - Krishna Institute of Engineering and Technology

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013Software Usability Assessment Models and Metrics: A SurveyVINITA SHARMA 1 , ARUN SHARMA 2Department of Computer Science and EngineeringKrishna Institute of Engineering and Technology, MTU, Ghaziabad, INDIA1 vini.itsharma123@gmail.com, 2 arunsharma2303@gmail.comAbstract: In this technical world the use ofsoftware is increased day by day. Software plays avery important role in the industry and society. Soas the role of software in the industry and thesociety becomes vital, it becomes very crucial todevelop high-quality and easy to use software. Thispaper deals with the Usability of software. TheUsability of the software refers to the ease of usethe software. It is also measured in terms of itscapability to provide the user satisfaction andability to fulfill the needs of the users by providingthe user friendly interfaces. For the long survival ofany software it must be user friendly. So theassessment of Usability of the software becomesvital for both the developer as well as for the user.The purpose of this paper is to do a survey ofUsability assessment standards and models andalso other Usability assessment techniques bypointing out their findings and limitations.Keywords: Usability, Usability Models, Usabilityassessment, User Satisfaction, Factors and Metrics1.IntroductionUsability is very important for the success of anyhuman made application. These applications maybe a software application, website, mobileapplications or any other machine. WithoutUsability, a software product cannot survive. Forbeing in competition in this competitive world thesoftware companies have to be care about the userinterface. The application should be as easy aspossible to operate. It should be user friendly,because user may be an inexperienced user or maybe a naive user. An inexperienced user has no priorknowledge that how to operate that particularsystem. The novice user also new to a system andrequired very simple user interface, but they mayhave some experience with similar application.7 | P a g eFor example: On the web, Usability is a veryimportant parameter for its survival. If a website isnot easy to use, people will not use that website andwill move to another website. In the similar way ifany user is using any software for achieving hisgoal, but the user interface of that software is not soeasy for that user then user will leave it and switchto another similar easy application. Current bestpatterns are said to spend about 10% on Usabilityof total budget for designing a project.2.UsabilityBasically Usability is a quality attribute ofsoftware that evaluates how easy user interfaces areto use. Usability also refers to the ease of use for aparticular software application. Means how easilyusers can use the software to carry out theirrequired task.The various definitions of Usability are as follows:―The capability of the software product to beunderstood learned, used and attractive to the user,when used under specified conditions.‖ (ISO/IEC9126-1, 2000) [1].―The extent to which a product can be used byspecified users to achieve specified goals witheffectiveness, efficiency and satisfaction in aspecified context of use.‖ (ISO9241-11, 1998) [2].―The ease with which a user can learn to operate,prepares inputs for, and interprets outputs of asystem or component.‖ (IEEE Std.610.12-1990)[3].

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 20133.Usability assessment standards and models:Literature reviewUsability measurement standards and modelsdefine some metrics or parameters to assess theUsability of a particular software application. Thereare various models proposed to assess theUsability. Here a review is provided of theseexisting models. This section will contain theirwork done and some of the limitations of thesemodels which gave the motivation to propose anew model. Some of these models are ISO9241,ISO/IEC9126, IEEE Std.610.12-1990, metrics forUsability standard in computing (MUSiC), qualityin use integrated measurement (QUIM).3.1 Traditional quality models addressing theUsabilityThere are various traditional quality models thataddress the Usability. These models are as follows:3.1.1 McCall [4]This is a hierarchical model which has 11 qualityfactors, 25 quality criteria and 41 quality metrics.This model has 4 criteria for Usability named asConstantine &Lockwood [8]Schneiderman[7]Nielsen [6]operability, training, communicativeness, I/O rate,I/O Volume.3.1.2 Boehm model [5]This model defines the Usability as the context towhich product is convenient and practical to use.Similar to McCall model this is also a hierarchicalmodel. It has total 19 quality factors includingproduct utility, portability and maintainability.Utility is further divided into reliability, Usabilityand efficiency. In this model unlike Mc Call modelit is not necessary that metrics are related to samefactors. These can vary according to circumstances.This causes uncertainty.3.1.3 Nielsen and Schneiderman model [6, 7]These models define the Usability through fivequality attributes. These five attributes of Usabilityare learnability, effectiveness, satisfaction, errortolerance and memorization. This model iscriticized by various graphics designer because itlacks in readability and typography.Different models have defined the Usabilitythrough different attributes. These are listed infollowing table1.Preece et al.[9]Shackel [10]Efficiency in useSpeed ofperformanceEfficiencyuseofThroughputEffectiveness(Speed)learnability Time to learn Learnability(Ease oflearning)Learnability(Ease oflearning)Learnability(Time tolearn)RememberabilityRetentionovertimeMemorabilityLearnability(Retention)Reliability in useUser satisfactionRate of errorsby usersSubjectivesatisfactionErrors/safety Throughput Effectiveness(Errors)Satisfaction Attitude AttitudeTable 1 — Usability attributes of various models3.2 ISO standards for Usability8 | P a g e

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013There are some ISO/IEC standards which are usedto measure the Usability of software.The description of these standards and consideredapproaches is given in following table2.Different standards have considered differentevaluation approaches for estimating the Usability.Standard Description Evaluation approach [16]ISO/IEC14598-19 | P a g e(ISO/IEC25000)[11]ISO 9241-11 [2]ISO13407[12]ISO 18529[13] Defines Usability as the internalsoftware quality attribute. Measures the Quality in Use Defines Usability as a high levelquality objective. Major attributes are efficiency,effectiveness and satisfaction Abstract in nature Evaluation approach is not given indetail Estimate Usability of software byevaluating the design phase as wellas by supervising the system incontext with user centered designprocess. Measure the Usability against users’requirements. This standard provides theguidelines for the execution ofactivities in different stages of theevaluation process, But this standard didn’t specify thathow these activities have to becarried out. This model has given the attentionon designing the life cycle processrather on the hardware, software orFour phases evaluation process, applicablethroughout the software product life-cycle Quality requirements definition Specify the evaluation Plan the evaluation Execute the evaluationDistinction between specification andevaluation of Usability during design andduring use of system: During design: Usability inputto a quality plan. During use: evaluate how changes incomponents of context of use affectefficiency, effectiveness, and/orsatisfaction of the userIt include following phases prior to evaluation(human centered design approach) Identification of the context of use andits components represents a basis forevaluation. Specification of user andorganizational requirements should bedone in terms that permit subsequenttesting and should be confirmed orupdated during the life of the project. Produce design solutions Evaluate designs against requirementsIt has following evaluation activities:Specify context of evaluationEvaluate for requirementsEvaluate to improve designEvaluate against system requirementsEvaluate against required practiceEvaluate in use.

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013ISO/IEC9126-1[14]ISO/IEC9126-4[15]interactive system. Defines Usability as externalattribute of software quality. Defines the Usability as ―quality inuse‖. Provides difference between internaland external quality in use.In evaluating the Usability five factors are usedhere: Learnability Understandability Attractiveness Operability Usability compliance with user guidesUsability depends on context in which softwareis used. In evaluating the Usability mainly threefactors are used:EffectivenessProductivitySafetyTable 2 — Evaluation approaches of variousmodels3.3 Neural network based modelsHere the description of models is given which usesthe neural network approach for evaluating theUsability models use neural networks approach forevaluating the Usability. These models are asfollows:3.3.1 Koffka et al [17]Here, authors propose the neural network basedmodel for measuring the Usability of ecommercesystems. Mainly the model is proposed forevaluating the Usability of B2C eCommerce webservices. Glowworm swarm optimization (GSO)neural networks-based model is used here.3.3.2 Márcioe [18]Here, author proposed a model for estimating theUsability of software industrial automation whichuses the artificial neural networks. This modelconsiders the Usability as crucial need of anysoftware industry. Due to the higher complexity oftoday software industry this model uses theartificial neural network technology.3.4 Fuzzy based modelsSome models use Fuzzy approach for evaluatingthe Usability empirically. These models are asfollows:3.4.1 Sharma et al [19]Here, authors provide some important factorsinfluencing the Usability of object oriented system.This paper also defines a model for Usabilityevaluation with the help of soft computingtechnique. This uses the fuzzy logic approach todetermine the Usability of object oriented softwaresystem. The sub-characteristics considered here areeffectiveness, efficiency, satisfaction, andlearnability.3.4.2 Sanjay et al [20]Here authors propose the new model for measuringthe Usability of software using fuzzy AHPtechnique. This proposed method uses the existingmodel 9241 as the base model and augmented thenew factor learnability to this existing model. Thismodel considers the Usability as an importantparameter of interactive software system. But herethis model is limited to use only object orientedmethodology for designing the software. This canbe validated by other methodologies. This proposedmodel considers the following factors and subfactors of Usability in hierarchical form shown infigure1:10 | P a g e

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013Figure 1 — Usability model [1]3.5 Other Models and Metrics for UsabilityEvaluationThere are some another models for evaluation ofthe Usability such as MUSiC, SANe, AIDE, andQUIM etc. A description about these models isgiven as follows:3.5.1 Metrics for Usability standard in computing(MUSiC) [21]This method is developed by the European MUSiCproject .It provides the genuine and certain meansof specifying and measuring the Usability. It alsoconcerns with the feedback through whichUsability can be enhanced by changing the design.MUSiC includes the tools and techniques tomeasure the satisfaction and performance. Formeasuring the user satisfaction University CollegeCork has developed the SUMI (Software UsabilityMeasurement Inventory) which is a part of MUSiCproject. SUMI has a standard questionnaire of 50items and it is available in 7 languages. SUMIprovides the overall assessment and Usabilityprofile which is further divided into five submeasures named as affect, efficiency, control,learnability, and helpfulness.3.5.2 Skill Acquisition Network (SANe;Macleod) [22]This model was designed specifically formeasuring the Usability of interactive systems. Ittakes in to consideration the user tasks. It definesthe five factors to define the Usability named aslearnability, adaption, efficiency, workload, anderror correction.3.5.3 AIDE; Sears model [23]The semi-Automated Interface Designer andEvaluator (AIDE) model defines a software tools toassess the Usability of static HTML pages on thebasis of predefined instructions or rules fordesigning of web pages. These rules contain theinstruction such as positioning and adjustment ofelements such as text, links and buttons. Thismodel assesses the Usability with the help of fivefactors which include horizontal balance, verticalbalance, efficiency, alignment and constraintsdefined by the designer.11 | P a g e

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 20133.5.4 GOMS; John and Kieras model [24]The Goals, Operators, Methods, and Selection rules(GOMS) model was designed for a specific taskwhich contains the methods to achieve the desiredgoals with the help of a software system. Thesemethods are considered as a series of operationsthat must be executed by the user while working onthat system. Furthermore goals can also be calledby a method to perform a task. Hence this modelhas a hierarchical structure. This model is used topredict the user performance in task analysis phase.Here user performance can be measured in tern oftime taken by user to perform the task. But thismodel lacks in verifying these predictions.3.5.5 NIST; Scholtz and Laskowski model [25]The National Institute of Standards andTechnology (NIST) Web Metrics provide s a set ofsix computer tools and different metrics whichprovide the support for remote, rapid andautomated testing and evaluation of web Usability.3.5.6 Quality in use integrated measurement(QUIM) [22]This model agglomerates various factors andmeasures all together. This model also defines therelationship among these factors, criteria andmetrics which are used to evaluate the Usability.This model defines the Usability with the help often factors. These factors are efficiency,effectiveness, safety, satisfaction, learnability,truthfulness, accessibility, usefulness, universalityand productivity. These all factors in this model areconsidered to be dependent and give the consistentdefinitions of these models. This model alsoprovides the QUIM editor which provides the toolsupport up to some extant for measuring theUsability to both expert users as well as to the userwhich are not familiar with Usability engineering.This model tried to provide a broader theory ofUsability measurement but it lacks in evaluatingthe Usability more empirically.3.5.7 Mohd. Haleem et al [26]This paper recognizes the Usability as the ―qualityin use‖. Here authors provide a new quality model12 | P a g ewhich is used to quantify the Usability matrices.This new proposed model is considered asframework to quantify and specify the quality inuse components. This model includes various typesof factors, criteria, matrices and also datacharacterized in different Human ComputerInterface and Software Engineering. It is ahierarchical model which can be viewed as thecombinations of other existing models. It differsfrom others in the way of decomposition of level ofUsability such as factors, criteria and metrics. Thismodel has Usability criteria in this model areeffectiveness, efficiency, satisfaction,internationability, Accessibility, Productivity,Safety, Learnability, Trustfulness, Universality,Adjustability, Portability, and Modifiability. Thismodel can be supportive for creations of Usabilitymeasurements plans only in the conditions wherespecific metrics are identified and defined certainhigher level goals of Usability such as satisfaction,learnability, and efficiency. It can be extended tosupport other broad field of engineering.3.5.8 Piyush Jain et al [27]Here authors have tried to evaluate the Usability ofsoftware through the use of fuzzy logic technique.Here Usability is defined as the factor of softwarequality model which deals with the userrequirements. Authors also tried to include theUsability engineering phase in the SDLC (SoftwareDevelopment Life Cycle) model to enhance thesoftware performance. This paper has given thedescription of some Usability estimation methodsand proposed a technique called fuzzy logic systemfor evaluating the software Usability. Thistechnique is more useful than other techniquesdescribed because this deals with the uncertainty.This technique can be used where conditions arenot constant and varies according to the time. Asuser requirements are also not constant and changesaccordingly so this technique is used. But for moreaccurate estimation of Usability we can use herethe combination of two techniques named as neuralnetwork and fuzzy system. This technique is calledneuro-fuzzy approach. This is more effectivebecause it combines the benefits of both thetechniques.

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 20133.5.9 Nektarios et al [28]This paper provides the description of software toolcalled Usability logger. This tool is used forUsability evaluation of human computer interfaces.The main features and capabilities of Usabilitylogger software tool are described here. Usability isevaluated by recording the various actionsperformed by the users with the softwareinterfaces. Mainly this software includes therecording of the use of keyboard and mouse. Theactions which are taken into consideration here aredistance covered by mouse, keyboard use, timeinvolved in user activity, speed of cursor, and useractivities’ storage in log files. This tool can beinstalled in any Usability laboratory or in any otherplace where application is being used. This tool hastwo parts: the recording part which is responsiblefor the recording of the movement of mouse andother parameters and the database which is used tostore the data received by the recorder part. But thistool is used here only for the Usability evaluationof two software applications named as ―GATES‖and ―Electronic Lite‖. Both are educationalsoftware which provides the help in understandingthe use of logic gates and electrical & electronicsengineering respectively.3.5.10 Munesh Chandra et al [29]Rather specifying any method to evaluate theUsability of software applications, this paper dealswith the role of physical and social context inUsability evaluation. It is important because theresult of evaluation also depends upon theenvironment in which Usability is evaluated. Thispaper tells about the importance of the context inUsability evaluation. It also emphasis on the factthat context should be considered as a vitalcomponent in the Usability evaluation model. Butthis paper provides the motivation for developingthe Usability evaluation framework which alsoincludes the context within it.4 SummaryWe have seen that there are various models,standards, and other methods that are proposed formeasuring the Usability of different applications.13 | P a g eThese all models Provides the different factors andmetrics for assessing the Usability. Most of thesemodels are static in nature. Some of these are onlyproposed, not implemented or validated. Very fewof them are concerned with the experimental formof calculating the Usability of a particularapplication. This provides the motivation toprovide a consistent view of Usability factors andto provide an empirical method to assess theUsability rather than the broader theory so that onecan quantify the Usability in a well identifiedmanner.So based on several papers and studies weconclude that Usability may be defined using fivemajor parameters. These basic factors arelearnability, efficiency, memorability, errortolerance, and user satisfaction. Other parametersmeasuring the Usability may include effectiveness,safety, truthfulness, accessibility, usefulness,universality, and productivity.5 ConclusionThis survey contains the study of various modelsand metrics for measuring the Usability ofsoftware. Factors taken into consideration ofvarious models and standard has been studied,compared and analyzed. In future author canpropose new software Usability assessment modelby taking into consideration different measure andmetrics for estimating the Usability empirically andmay also implement or validate the model by anysoft computing technique for better results.References[1] ISO/IEC 9126 Software ProductEvaluation - Quality Characteristicsand Guidelines for the User.[2] ISO: ―International StandardOrganization: ISO 9241-11(Guidance on usability)‖,International Standard: ISO 9241-11(Guidance on usability), 1998.[3] IEEE Std. 610.12-1990. IEEEStandard Glossary of SoftwareEngineering Terminology

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013[4] McCall, J. A., Richards, P. K.Walters, G. F., ―Factors in SoftwareQuality‖, Springfield, VA: NationalTechnical Information Service.[5] Boehm, B.W. Abts, C. Brown, W.Chulani, S. Clark, B.F. Steece, B.Brown, ―Software Cost Estimationwith COCOMO II, Prentice-Hall,New York 2000.[6] Nielsen, ―Usability Engineering‖,London, UK: Academic Press.[7] Schneiderman, ―Designing the UserInterface: Strategies for EffectiveHuman-Computer Interaction (2nded.), Reading‖, MA: Addison-Wesley.[8] Constantine, L.L. and Lockwood,―Software for Use: A Practical Guideto the Models and Methods ofUsage-Centered Design‖, New York:Addison-Wesley.[9] Preece, J. Rogers, Y. Sharp, H.Benyon, D. Holland, S. Carey,―Human Computer Interaction,Wokingham‖, UK: Addison Wesley.[10] Shackel, ―Usability—Context,framework, definition, design andevaluation‖, in B. Shackel and S.Richardson (Eds.), Human Factorsfor Informatics Usability,Cambridge, MA: University Press,pp. 21–38.[11] ISO/IEC 14598-1 (1999) InformationTechnology – Software productevaluation – Part 1: Generaloverview.[12] ISO 13407 (1999) Human-centereddesign processes for interactivesystems.[13] ISO 18529 (2000) Ergonomics –Ergonomics of human-systeminteraction – Human-centeredlifecycle process descriptions.[14] ISO / IEC, IS 9126-1, ―SoftwareEngineering – Product Quality – Part1: Quality Model‖, GenevaSwitzerland: InternationalOrganization for Standardization,2001.[15] ISO/IEC 9126 Software ProductEvaluation - Quality Characteristicsand Guidelines for the User.[16] Dornima Marghescu, ―Usabilityevaluation of information Systems: AReview of Five InternationalStandards‖, Turku Centre forComputer Science, Abo AcademyUniversity.[17] Koffka Khan, Ashok Sahai "AGlowworm Optimization Method forthe Design of Web Services", I.J.Intelligent Systems and Applications,10, 89-102, 2012.[18] Márcio José Moutinho da Ponte,Antonio Morais da Silveira, "AMethodology for Evaluation theUsability of Software for IndustrialAutomation Using Artificial NeuralNetworks: Case Study--Eletrobrás,"cimca, pp.430-435, 2008.[19] Sanjay Kumar Dubey, Ajay Rana,Arun Sharma, ―Usability evaluationof object oriented software systemusing fuzzy approach‖, InternationalJournal of Computer Applications(0975-8887) Volume 43-No.19,April 2012.[20] Sanjay Kumar Dubey, Arpan Mittal,Prof. Dr. Ajay Rana, "Measurementof Object Oriented SoftwareUsability using Fuzzy AHP",International Journal of ComputerScience and Telecommunications,Volume 3, Issue 5, May 2012.[21] M. Macleod, R. Bowden, N. Bevan,I. Curson, ―The MUSiC performancemethod‖, Behavior and InformationTechnology, Vol.16, pp.279-293,1997.[22] A. Seffah, M. Donyaee, R. B. Kline,H. K. Padda, ―Usability measurementand metrics: A consolidated model‖,Software Quality Control, Vol. 14,No. 2, pp. 159–178, 2006[23] Sears, ―AIDE: A step toward metricbasedinterface development tools,Proceedings of the ACM Symposiumon User Interface Software andTechnology‖, New York: ACMPress, pp. 101–110, 1995[24] B.E. Kieras, ―Using GOMS for userinterface design and evaluation:Which technique?‖, ACMTransactions on Computer-HumanInteraction 3: 287–319.14 | P a g e

International KIET Journal of Software and Communication Technologies (IKJSCT)Volume 1, Issue 1, pp: 7-15, April 2013[25] Scholtz, J. and Laskowski,―Developing usability tools andtechniques for designing and testingweb sites, Proceedings of the FourthConference on Human Factors & theWeb‖, Basking Ridge, NJ. RetrievedJuly 3, 2005[26] Mohd.Haleem, Prof (Dr)Md.RizwanBeg, Syed Umar Amin, "A Model forQuantifying Usability Metrics: AnEffective Approach", UACEEInternational Journal of ComputerScience and its Applications -Volume 1 : Issue 1 [ISSN 2250 -3765][27] Piyush Jain , Sanjay Kumar Dubey ,Prof.(Dr.)Ajay Rana, "SoftwareUsability Evaluation Method",International Journal of AdvancedResearch in Computer Engineering& Technology Volume 1, Issue 2,April 2012.[28] Nektarios KOSTARAS , MichalisXENOS , ChristosKOUTSOGIANNIS, "Usabilityevaluation of software applicationswith the use of Usability Logger",8th Joint Conference on Knowledge-Based Software Engineering,JCKBSE 08, pp. 395-404, Piraeus,Greece, August 25-28, 2008.[29] Munesh Chandra Trivedi,Mohammadi Akheela Khanum,"Role of context in usabilityevaluations: a review", AdvancedComputing: An International Journal( ACIJ ), Vol.3, No.2, March 2012.[30] J. Nielsen, & J. Levy, ―Measuringusability: Preference vs.performance‖, Communications ofthe ACM, Vol. 37(4), pp.66-75,2003.[31] Elizabeth Chang, Tharam S. Dillon,―A Usability-Evaluation MetricBased on a Soft-ComputingApproach‖, IEEE Transactions onsystems, man, and cybernetics—parta: systems and humans, Vol. 36, no.2, march 2006.[32] R.V. Siva Balam, Dr. M.Punithavalli, ―Software Architecturefor Object Oriented Systems-Usability Patterns‖, IJCSNSInternational Journal of Computer15 | P a g eScience and Network Security,Vol.10 No.10, October 2010.[33] Constantinos K.Coursaris, DanJ.Kim, ―A meta Analytical Review ofempirical mobile usability studies‖,Journal of usability Studies, Vol. 6,pp. 117-171, 3 May 2011.[34] Rajinesh Ravendran, MacColl,Michael Docherty, ―Usabilityevaluation of a tag based interface‖,Journal of usability Studies, Vol. 7,Issue 4, pp. 143-160, August 2012.