A Review of the Event Analysis of Systemic Teamwork Methodology

More documents

Recommendations

Info

HFIDTC/WP1.1.3/10 Version 2/ 31 October 2005 Disadvantages • The methods within the EAST framework are conceptually simple, ensuring that the methodology requires minimal training and is relatively easy to apply. • The provision of WESTT and Agna software packages reduces the EAST application time considerably. • The EAST methodology is generic, allowing it to be applied in any domain where C4i activity takes place. To date, the EAST methodology has been used to analyse C4i activity in a number of diverse scenarios, including fire service training scenario’s (Baber et al 2004), military aviation scenarios (Stewart et al 2004a), railway signalling tasks (Walker et al 2004a), air traffic control scenarios (Walker et al 2004b), energy distribution scenarios (Salmon et al 2004) and naval warfare scenarios (Stewart et al 2004b). • The methodologies that make up EAST share a common theoretical background. This makes integration viable (and embodies a form of congruent validity). • There are possibilities for further recombination of methods to explore issues outside of the EAST framework (Walker et al 2005a). • All the methods have a validation history and prior context of use. • Due to the techniques exhaustiveness it is time consuming to apply. The initial observation, construction of the HTA and OSD are particularly time consuming elements of the methodology. • The data obtained during the CDM analysis a function of the analysts skill and the quality of the SME used. • Reliability of the technique in some areas is questionable. Much of the analysis is based upon the subjective judgement of the analyst involved, and so intra-analyst and inter analyst reliability may suffer. • For the technique to work properly, access to the system and personnel under analysis is required. It is often difficult to gain sufficient access to the system under analysis. Further, additional access to SME’s for data validation purposes is also difficult to obtain. • The methodology is still evolving and as a consequence, some EAST outputs may differ in format and presentation. • Without the provision of the appropriate software, an EAST analysis may become laborious to perform for larger, complex scenarios. • Some prior knowledge of HF methods is required. 23
HFIDTC/WP1.1.3/10 Version 2/ 31 October 2005 Related methods The EAST methodology is comprised of a number of HF methods, including HTA, observation, CDA, CUD, OSD’s, SNA, CDM and propositional networks. The authors have also developed the WESTT methodology, which can be used to automate a large portion of the EAST analysis procedure. Approximate training and application times The EAST methodology is conceptually simple, however due to the number of component methods involved, requires a lengthy training session. Some prior experience in the application of HF methods is required. For the recent applications of EAST described in this report, a training session lasting approximately 8 hours was given to the analysts involved. Due to the exhaustive nature of the EAST methodology, and the complex nature of C4i activity, the typical application time is high. The construction of the HTA and the OSD for C4i scenarios is a particularly lengthy process. Conducting an EAST analysis manually can take up to two weeks for large, complex scenarios. The application time is significantly reduced by the provision of the WESTT software package, which automates a large portion of the EAST analysis process. Reliability and validity Intra and inter reliability may be questionable in some instances. For example, different analysts may make different interpretations of the OSD symbols in the OSD glossary. The use of multiple methods within the EAST framework heightens the potential for unreliability. It is difficult to assess the validity of the data obtained during an EAST analysis. Typically, SMEs are used during data collection, and so a high level of validity is assumed. Congruent validity is favourable due to effective methods integration. Construct validity also good, as component methods relate well to the core descriptive constructs of C4i (Walker et al 2005b). Construct and face validity are also good because many of the features of interest are also observable and manifest in the environment of interest. The CDM complements this approach by tapping into the ‘implicit’ parts of the decision making process. Tools needed Due to the various component analyses involved in an EAST analysis, a number of different tools and software packages are required. For the observation and CDM phases of an EAST analysis, video and audio recording equipment is required. A word processing software package such as Microsoft Word is required to transcribe the observation and CDM data. The CDA is typically conducted using Microsoft Excel. A large part (e.g. OSD and SNA) of the EAST procedure can be automated by using the WESTT software package. Alternatively, the OSD CUD, and propositional networks are typically drawn using Microsoft Visio. The SNA is conducted using the Agna SNA software package. 24
Page 1 and 2: Analysing C4i Activity: A Review of
Page 3 and 4: HFIDTC/WP1.1.3/10 Version 2/ 31 Oct
Page 25: HFIDTC/WP1.1.3/10 Version 2/ 31 Oct
Page 77 and 78:
HFIDTC/WP1.1.3/10 Version 2/ 31 Oct
Page 79 and 80:
Page 81 and 82:
Page 83 and 84:
Page 85 and 86:
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113:
show all

A Review of the Event Analysis of Systemic Teamwork Methodology

Create successful ePaper yourself

Delete template?

Save as template?