A Review of the Event Analysis of Systemic Teamwork Methodology
A Review of the Event Analysis of Systemic Teamwork Methodology
A Review of the Event Analysis of Systemic Teamwork Methodology
- No tags were found...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
HFIDTC/WP1.1.3/10<br />
Version 2/ 31 October 2005<br />
Related methods<br />
• Cognitive aspects <strong>of</strong> <strong>the</strong> task under analysis are not elicited using<br />
observational techniques. Verbal protocol analysis is more suited for<br />
collecting data on <strong>the</strong> cognitive aspects <strong>of</strong> task performance.<br />
• An observational study can be both difficult and expensive to set up and<br />
conduct. Gaining access to <strong>the</strong> required establishment is <strong>of</strong>ten extremely<br />
difficult and very time consuming. Observational techniques are also costly,<br />
as <strong>the</strong>y require <strong>the</strong> use <strong>of</strong> expensive recording equipment (digital video<br />
camera, audio recording devices).<br />
• Causality is a problem. Errors can be observed and recorded during an<br />
observation but why <strong>the</strong> errors occur may not always be clear.<br />
• The analyst has a very low level <strong>of</strong> experimental control.<br />
• In most cases, a team <strong>of</strong> analysts is required to perform an observation<br />
study. It is <strong>of</strong>ten difficult to acquire a suitable team with sufficient<br />
experience in conducting observational studies.<br />
There are a number <strong>of</strong> different observational techniques, including indirect observation,<br />
participant observation and remote observation. O<strong>the</strong>r related techniques include verbal<br />
protocol analysis, critical decision method, applied cognitive task analysis, walkthroughs<br />
and cognitive walkthroughs. All <strong>of</strong> <strong>the</strong>se techniques require some sort <strong>of</strong> task<br />
observation. Observational data is also used as an input to numerous HF techniques, such<br />
as task analysis, human error identification, and charting techniques.<br />
Approximate training and application times<br />
Whilst <strong>the</strong> training time for an observational analysis is low (Stanton & Young 1999), <strong>the</strong><br />
application time is normally high. The data analysis stage can be particularly time<br />
consuming. Kirwan & Ainsworth (1992) suggest that in <strong>the</strong> transcription process, 1 hour<br />
<strong>of</strong> audio recorded data would take approximately 8 hours to transcribe.<br />
Reliability and validity<br />
Observational analysis is beset by a number <strong>of</strong> problems that can potentially affect <strong>the</strong><br />
reliability and validity <strong>of</strong> <strong>the</strong> technique. According to Baber & Stanton (1996) problems<br />
with causality, bias (in a number <strong>of</strong> forms), construct validity, external validity and<br />
internal validity can all arise unless <strong>the</strong> correct precautions are taken. Whilst<br />
observational techniques possess a high level <strong>of</strong> face validity (Drury 1990) and ecological<br />
validity (Baber & Stanton 1996), analyst or participant bias can adversely affect <strong>the</strong><br />
reliability and validity <strong>of</strong> <strong>the</strong> techniques.<br />
Tools needed<br />
For a thorough observational analysis, <strong>the</strong> appropriate visual and audio recording<br />
equipment is necessary. Observational studies can be conducted using pen and paper,<br />
33