WWW/Internet - Portal do Software Público Brasileiro

IADIS International Conference WWW/Internet 2010checklist; they also liked the idea of the usability problem profile as it seems to work as a double-check forthe identified usability problems.Table 3. Number and percentage of usability problems uncovered on Website B by group G1.CustomisedGuidelinesEvaluator1Evaluator2Evaluator3Evaluator4Total Problemsby group G1Total problems discoveredby both groups (G1, G2)Total 28 22 15 20 34 38% of TotalProblems74.47% 57.89% 39.47% 52.63% 89.47% 100%Group G2, who used the customised guidelines in the first study, then used the HE method to evaluateWebsite B. Each evaluator in this group was able to reveal a number of problems in Website B. The resultsare summarised in Table4. They were able to reveal more than 21% of the total of the identified usabilityproblems. The results ranged from 10% to 21% of the total number of the problems. Comparing these resultsto group G1 using the same tool in the previous results, there is not a huge difference as both of them werenot able to identify more than 23% of total number of the identified usability problems.Table 4. Number and percentage of usability problems uncovered on Website B by group G2.HE Evaluator5Evaluator6Evaluator7Evaluator8Total Problemsby group G2Total problems discoveredby both groups (G1, G2)Total 5 8 4 5 12 38% of TotalProblems13.15% 21.05% 10.52% 13.15% 31.57% 100%In addition, 7 out of the 8 evaluators stated that they would prefer to use the customised guidelinesmethod in future evaluations rather than the HE method. The given reasons for this were that the customisedguidelines method is easier to understand and use than HE. They also stated that the customised methodseems to encourage them to be more through. On the other hand, evaluators spent less time when they usedHE for both websites, they spent 40 and 37.5 minutes for A and B websites respectively. The time spent on Aand B websites by the evaluators while they used the customised method were 72.5 and 50 minutesrespectively. Therefore HE seems to be faster than the customised guidelines method. Both of the methodsperformed well in discovering major problems. HE has been criticised as being an abstract method, whichdoes not provide guidelines for a specific domain, for example education or government websites.Generally, it can be seen that specific guidelines are preferred to the general guidelines as they can helpevaluators to focus on those criteria that are important to a particular type of a website in order to integrate allthe most suitable usability considerations. For instance, in this research, the customised guidelines forelectronic shopping emphasise graphics and security, which may be less important in, for example,educational websites. Moreover, using a checklist in the guidelines maximises the number of usabilityproblems that can be identified for each interface. Furthermore, the use of specific guidelines can break downthe usability problems identified to the lowest level of detail. Also, specific guidelines can provide a morecomprehensive analysis of the interface usability problems.5. CONCLUSIONSIt can be seen clearly from the literature and from the findings of this paper that there is a clear need to havecustomised guidelines for specific domains such as commercial, education or government websites. In thispaper, the customised guidelines for commercial websites performed better and were preferred by most of theevaluators. There are a number of advantages to the customised guidelines method: it is a comprehensive anddetailed method at the same time, and it seems to work as a checklist for the usability guidelines. However, ithas a number of drawbacks: it is limited to commercial websites, and it seems to lead the evaluators to thewebsite aspects rather than leaving them to look over the site in general and think out of the box.173