MAP-01-011 HFI Technical Guide - Human Factors Integration ...

Annex 3 – HF Techniques, Methods and Tools• Disadvantages• Questions may be asked regarding the construct validity of the technique.It could certainly be argued that rather than measuring SA itself, MARS isactually rating the difficulty in acquiring and maintaining SA.• The technique has limited validation evidence associated with it. Thetechnique certainly requires further validation in military or infantry settings.• As the MARS questionnaire is administered and completed post-trial, it issubject to problems such as poor recall of events and forgetting on the partof the participants. It is apparent that participants are limited in theaccurate recall of mental operations. For lengthy scenarios, participantsmay not be able to recall events whereby they were finding it difficult oreasy to perceive mission critical cues.• Similar to the above problem, the completion of the MARS questionnairepost-trial may result in a correlation of SA ratings with performance. Thoseparticipants who have performed optimally during the task may rate SAachievement as easy.• Only an overall rating is acquired, rather than a rating at different points inthe task. It may be that the output of the technique is of limited use. Forexample, a design concept may only acquire an overall rating associatedwith SA, rather than numerous SA ratings throughout the task, some ofwhich would potentially pinpoint specific problems with the new design.A3.7.4Situation Awareness Rating Scales (SARS)The situation awareness rating scales technique (SARS) is a subjective rating SAmeasurement technique that was developed for the military aviation domain andwas developed in order to define the SA construct, to determine how well pilotscan assess other pilots SA and also to examine the relationship between pilotjudgements of SA and actual performance.When using the SARS technique, participants subjectively rate their performanceon a six-point rating scale (from acceptable to outstanding) for 31 facets of fighterpilot SA. The SARS SA categories and associated behaviours were developedfrom interviews with experienced F-15 pilots. The 31 SARS behaviours aredivided into 8 categories representing phases of mission performance. The eightcategories are: general traits, tactical game plan, communication, informationinterpretation, and tactical employment beyond visual range, tactical employmentvisual and tactical employment general.• Advantages• The 31 dimensions appear to offer an exhaustive account of fighter pilotSA.• The technique goes further than other SA techniques such as SAGAT inthat it assesses other facets of SA, such as decision-making,communication and plan development.May 2006 Page A3-59 Issue 4