WWW/Internet - Portal do Software Público Brasileiro

IADIS International Conference WWW/Internet 20103. RESEARCH OBJECTIVES AND THE APPROACH TAKEN:The primary aim of this research is to compare specific customised guidelines with the more traditionalheuristic method; the Nielsen heuristic set is chosen to establish which method can offer better results. Inorder to achieve this aim, there are some objectives that need to be achieved as well. The following is adescription of the research objectives.1. To propose a customised guidelines method for measuring the usability of commercial websites.2. To apply the customised guidelines and the Nielsen heuristic set to targeted websites.3. To compare the customised guidelines vs. heuristic evaluation for commercial websites in order toreveal each method’s performance.Two main experiments were undertaken in order to achieve the research objectives. The first experimentwas to apply heuristic evaluation on two commercial website by using two different groups of evaluators;each group performed HE on a website. The second experiment was to apply the new customised guidelinesmethod upon the same two commercial websites by the same two groups. The group G1 performed HE firston Website A, and then performed the customised guidelines on Website B. The group G2 performed thecustomised guidelines on Website A, and then performed HE on Website B. In more detail, each groupstarted with a different website in order to further control the experiment, rather than performing one methodat the beginning of both studies. Each evaluator had a questionnaire at the end of each evaluation to answersome questions with regard to the method he had used.3.1 Website SelectionThe targeted websites should be live, dynamic, have a number of features, and be of reasonable content size.This selection has only targeted commercial websites because of the limitation of the study. Therefore, theresearchers decided to choose two electronic shopping websites to be the targeted websites that meet theabove conditions.3.2 Number of EvaluatorsSection 2.1 discussed the number of evaluators needed to conduct a usability study. In this research, 8evaluators were needed, where each group had 4 evaluators and they were divided evenly in terms ofexperience and other factors. Therefore, each website has been evaluated by the 8 evaluators but each fourused a different method.3.3 Severity AssessmentUsability evaluation methods depend on evaluator’s judgment when they rate a usability problem. Here inthis study, the rating of usability problems was based on the average rating of all the evaluators. Theirassessment is based on three factors, which are the problem impact upon business goal, its frequency and itspersistence. This should eliminate any deviation or personal judgment, and it is suggested by several studies,such as (Chen and Macredie, 2005) and (Morten Hertzum, 2006).4. DATA ANALYSISThis section presents the data analysis from this experiment. It discusses how well the two groups performedwhile they were evaluating Website A. It then also discusses the results from the second website (B) after itwas evaluated by both groups.171