ISSUE 182 : Jul/Aug - 2010 - Australian Defence Force Journal

Central to both QA and QC is the concept of ‘fit(ness)-for-purpose’, an expression used in a legalsense with respect to the sale of goods and consumer law and in the manufacturing industryto assess the utility of a product against its stated purpose. The term is, however, being usedmore widely and may be viewed as equating quality with the fulfilment of a specification orexpected outcome. For OR studies, fitness-for-purpose would then be judged in relation to therelevance, timeliness and usefulness of the results provided. In the case of Fast OR, this couldbe judged with respect to the particular OPSTSR to which the study relates; that is, whetherthe results from the study met its prescribed time-scales and stated objectives. Fitness-forpurposeappears deceptively straightforward but it should be kept in mind that the issues of‘whose purpose?’ and ‘how to assess fitness?’ may be multifaceted with no simple, obviousanswer. In some situations, there can be a multitude of clients, stakeholders and users withcompeting or at least divergent purposes or perspectives.For evaluating S&T quality of Fast OR studies, perhaps the aim should be to establish anintegrated but streamlined quality process for both undertaking Fast OR studies and reviewingand certifying their results? Such a process could be included with the evaluation of thetimeliness, relevance and usefulness of the study to the client but should have a primaryfocus on addressing the technical veracity of the work undertaken. The difficulty, however, isnot with deciding what is needed but rather how to do it! To provide a quick appraisal of thequality of a Fast OR study, it could be useful to identify and agree on a few basic hallmarks thatcapture the essence of good studies. At a basic level, any Fast OR study should be:• Coherent (clear and structured). The processes used in, and the results of, the assessmentsshould be intelligible and able to be readily followed.• Consistent. Assessments made should be independent of the assessor and take note ofdomain knowledge and experience.• Checkable (traceable and able to be audited). The approach taken and data used should betraceable and able to be audited or ratified.• Credible. Results should be both plausible and defendable. While the results may notalways be able to be formally verified and validated, they should be able to be ratified usingstakeholders’ and users’ expertise and experience, historical examples or case studies.A suitable (and suitably short) set of pertinent questions could then be constructed to examinewhether a study meets these hallmarks. Constructing a simple checklist based on suchquestions could then provide a framework for reviewing the quality of Fast OR studies in theshort time frame available. The form of this checklist of questions would also depend on themeans used for reporting the findings of the fast OR studies undertaken.Reporting findings of deployed teamsOptions for deployed OA teams to report their findings span a spectrum from informalmechanisms, such as face-to-face briefings, faxes and email, through video-conferencing andworkshops, to informal client reports and more formal, peer-reviewed technical reports. Thetime frames of a Fast OR study will usually preclude some of these options for initial reporting67