WWW/Internet - Portal do Software Público Brasileiro

IADIS International Conference WWW/Internet 20102.1 Heuristic Evaluation MethodHeuristic evaluation involves having a small set of evaluators examine an interface and judge its compliancewith recognized usability principles (the 'heuristics'). This method was developed by Nielsen and hiscolleagues in 1990, guided by a set of usability principles, to evaluate whether a user interface conforms tothese principles (Jacobsen Nielsen, 1992) (Hix et al., 2004). These principles or criteria are referred to asguidelines or general rules that can guide a design decision or be used to critique a decision that has alreadybeen made. In general, heuristic evaluation is difficult for a single individual to do because one person willnever be able to find all the usability problems in an interface. Moreover, many different projects have shownthat different people find different usability problems (Morten Hertzum and Nielsen Jacobsen, 2001). As aresult, to improve the effectiveness of the method significantly, it should involve multiple evaluators.According to a number of researchers, such as Jakob Nielsen, a single evaluator is able to identify only 35%of all the usability problems in an interface, which is the result of averaging more than six of his projects.However, since different evaluators tend to find different problems, it is possible to achieve a betterperformance by increasing the number of evaluators. Figure 1, clearly shows that there is profit frominvolving many evaluators. Nevertheless, there is no agreement about the number of evaluators that areenough to identify a sufficient number of usability problems. According to Jakob Nielsen (1994), three tofive evaluators typically identify about 75-80% of all usability problems. Koyani (2003) asserted that two tofive is enough.Figure 1. The relationship between the number of evaluators and the number of the found problems (Jakob Nielsen,2005).2.2 Advantages and Disadvantages of Heuristic EvaluationHeuristic evaluation is recommended as the fastest, cheapest, most flexible and effective way for identifyingusability problems (Saul Greenberg et al., 2000). Also, heuristic evaluation is especially valuable when timeand resources are short (Laurie Kantner and Stephanie Rosenbaum, 1997). Jeffries and his colleaguescompared four different techniques used to evaluate a user interface. These were heuristic evaluation,software guidelines, cognitive walkthroughs and user testing. Overall, the heuristic evaluation techniqueidentified the most usability problems (Jeffries, 1991). In addition, heuristic evaluation has a number of othermore specific advantages, such as low cost relative to other evaluation methods, advanced planning notrequired, can be used early in the development process, may be used throughout the development process,effective identification of major and minor problems, during a short session, a small number of experts canidentify a range of usability problems, and because of their experience with many system interfaces, it is easyfor evaluators to suggest solutions to the usability problems (Nielsen, 1990) (Holzinger, 2005) (Jenny Preeceet al., 1993).In contrast, there are a number of major drawbacks to heuristic evaluation, such as: it is highly dependenton the skills and experience of the evaluators, the evaluators only emulate users (they are not actually usersthemselves), user feedback can only be obtained from laboratory testing, and it does not essentially evaluatethe whole design because there is no mechanism to ensure the whole design is explored (evaluators can focustoo much on one section). Also, it does not include end users, and thus is unable to specifically identify users’needs. Furthermore, it produces a large number of false positives that are not usability problems, which169