MAP-01-011 HFI Technical Guide - Human Factors Integration ...

MAP-01-011 – HFI Technical Guide (STGP 11)• The scale is generic and so the scale can be used in any domain.• The SUS scale is very useful when comparing two or more devices interms of usability.• Its simplicity and speed of use mean that it is a very useful technique touse in conjunction with other usability assessment techniques.• Very quick in its application.• The scale can be adapted to make it more suitable for other domains.• Disadvantages• The output of the SUS is very limited.• Requires an operational version of the device or system under analysis.• Unsophisticated.A3.12.5User TrialsThe most fundamental and highest-integrity HMI Evaluation technique is theconduct of User Trials or Experimentation. Actual system users, or an appropriateset of SMEs test the real HMI under realistic, controlled conditions.In addition to proving HMI functionality, User Trials explore aspects such asadequacy of the HMI in terms usability and efficiency of use, but can also beused to examine the appropriateness of operating procedures and the actualmethod of HMI use under simulated, but realistic operating conditions.User Trials can also provide an opportunity to capture objective data such asoperating times, operator workload and other performance metrics, human errorrates and human communication flows.User Trials should be conducted against a suitable trials plan, with clearly predefinedoperational scenarios and tasks, together with appropriate objectives andassessment criteria.Suitable, non-intrusive measuring equipment and an adequate number of trainedobservers are required. Observers should observe, and not intervene in theconduct of the trial. Post-trial subject interviews and de-briefing should be carriedout.User trials may require significant investment to access the final HMI and itsrelated systems, and assemble adequate numbers and types of userrepresentatives, measuring equipment and sufficient trained observers in theright place at the right time. Sound management of the trial is important. Largeamounts of data may be collected and thus attract significant data processingresource needs. The trial should therefore be based on a sound design, so thatonly relevant data is collected.May 2006 Page A3-122 Issue 4