pdf 820Kb - INSEAD CALT

More documents

Recommendations

Info

Evaluation report of the use of Onto-Logging platform in the user site Deliverable ID: D8b Page : 14 of 110 Version: 1.0 Date: 27 january 2004 Status: Final Confid.: Public particular topic of interest. Each group session typically lasts 1 to 1 ½ hours and should be moderated by a professionally trained facilitator. Groups should always be audiotaped or videotaped. 2.3.4 Interpretative methods The interpretative evaluation consists in deriving, from the observation of the users in natural settings (working on at least partly controlled tasks), the structure and thinking behind the visible behaviours. Cooperative evaluation. Cooperative evaluation uses the think aloud protocol, which allows the user to ask questions, comments and suggest appropriate alternatives of the evaluators, and the evaluators to prompt the user. Here, the user is encouraged to act as a collaborator in the evaluation to identify usability problems and their solutions. Ethnography. Ethnography is an approach developed by anthropologist, which is used in HCI to inform system design. This method is used "in order to look at aspects of social interaction and work practices in natura l work environments. By doing so, they attempt to reveal the detailed and systematic interaction between members in an organization and therefore explicate the underlying work practices". Scenario -based design. The phrase “scenario-based design” represents a diverse field of design practice (Carroll, 1995). The various techniques are connected by the use of scenarios to ground the design process in the situated tasks of users. These scenarios describe sequences of actions taken by a user with a specific goal in mind. These methods are meant to make explicit those assumptions and activities of the design process which are usually implicit. Dialogue can for instance be used as to conduct the inquiry (Erskine, Carter-Tod, and Burton, 1997). Practically, the user participants are asked in a dialogue to provide a context (who they are), a specific goal (for instance they may want to capitalize the knowledge of a just finished project), and an action (a detail description of how they would proceed to achieve this goal using the system that is being tested). Contextual inquiry. Contextual Inquiry (Beyer and Holtzblatt, 1998) is a field study method where usability specialists observe people performing their usual job tasks in the context of their actual work situations. Contextual inquiry is basically a structured interviewing method based on a few core principles (context, partner, focus): (1) the users are observed in their own working situation; (2) the researcher and the users are equal partners in investigating and understanding the usage of a system; (3) a focus is defined, which can be seen as a number of assumptions and beliefs concerning what needs to be accomplished and how to accomplish it. 2.3.5 Predictive methods Experts separately review a system and categorize and justify problems. Predictive evaluation can be based on a specific theory (for instance in cognitive psychology) or on a short set of heuristics (rules of thumb).
Evaluation report of the use of Onto-Logging platform in the user site Deliverable ID: D8b Page : 15 of 110 Version: 1.0 Date: 27 january 2004 Status: Final Confid.: Public Cognitive walkthrough. Cognitive walkthrough is an evaluation technique that considers psychology theory into the informal and subjective walkthrough technique, proposed by (Polson et al., 1992). It aims to evaluate the design in terms how well its supports the user as he learns how to perform the required task. The designer or an expert in cognitive psychology performs the walkthrough. The expert works through the design for a particular task, step by step, identifying potential problems against psychological criteria. The analysis is focused on the users goals and knowledge. The cognitive walkthrough must show if, and how the interface will guide the user to generate the correct goal to perform the required task, and to select the necessary action to fulfil a goal. Heuristic evaluation. Heuristic evaluation (proposed by Nielsen and Molic h) involves experts assessing the design. In this approach a set of usability criteria or heuristic is identified to guide design decision. Evaluators independently run through the performance of the task set with the design and assesses its conformance to the criteria at each stage. Nielsen suggests that around five evaluators found about 75% of potential usability evaluation. 2.3.6 Test, Experiment and Usability Engineering methods Tests, experiment and assessments evaluation methods consist in setting up a more formal setting for the evaluation, and in particular the organization of evaluation sessions with clear evaluation objectives. Tests and assessments can be useful tools in evaluation to measure the impact of the system on the participants’ processes. The tests are conducted successively without and with the use of the system, and the results are compared. Self-reports participants write reports on how much the system has contributed to improve their work process (more than an opinion, users are asked for an explanation). Usability engineering. Usability engineering is an approach to system design where the usability issues of a system are specified quantitatively in advance. The development of the system is based in these metrics, which can be used as cr iteria for testing the usability of a system or prototype. Examples of such metrics are 'percentage of errors', and 'number of commands used'. Experiments. Experiments are the most formal methods of usability testing that requires preparation and knowledge to evaluate the design of a system or prototype by quantifying user performance such as 'time on task', 'number of errors', etc.
Page 1 and 2: The ONTO-LOGGING Consortium consist
Page 3 and 4: Evaluation report of the use of Ont
Page 5 and 6: Executive Summary Evaluation report
Page 13: Evaluation report of the use of Ont
Page 53 and 54: • List of connected users to the
Page 65 and 66:
Evaluation report of the use of Ont
Page 67 and 68:
5.5.1 Description of the evaluation
Page 69 and 70:
6 Discussion and Conclusions Evalua
Page 71 and 72:
Page 73 and 74:
Page 75 and 76:
8 Annex Evaluation report of the us
Page 77 and 78:
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
esolver consultas. C42 Existen axio
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
1 The functionalities provided by D
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
show all

pdf 820Kb - INSEAD CALT

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?