ISSUE 152 : Jan/Feb - 2002 - Australian Defence Force Journal

1990/91. In this trial, evaluation resources werevery limited and, to gather as muchinformation as possible, key tasks wereconducted by an M113 troop and by a LAV-25troop. Teams of experts observed the conductof these tasks. These observations and themilitary judgement (opinions) of theparticipants formed the basis of the reports.An important aspect was that, despite alarge number of questions to be answered inRTA (of the order of 200), there were relativelyfew OTE style tests focussed on singlequestions. The vast majority of the informationwas from observations of tasks.A second key innovation is the systematicrefinement of the problem space throughmultiple iterations and layers of analytic andtraining tools, Figure 5. The pivotal tool was awar gaming environment of sufficient fidelityto challenge existing organisational anddoctrinal paradigms — in effect the building ofa synthetic experience base, supported bycarefully selected field experiments and realworld operational experience as validation forthe synthetic experience. Each level of analysisprovided three outputs: answers to specificquestions, the context for the next level ofanalysis and confirmation of the results ofprevious layers.As we have suggested earlier, the choice oftest scenarios is of critical importance to properrefinement of the problem and indeedresolution of the issues. Arguably threeprimary aspects must be considered in thechoice of test scenario. The most important isthat the test scenario produces circumstancesin which the identified and required tasks arein fact tested. That is, the required experiencebase is established. In addition, the testscenario chosen must have sufficient credibilityand detail to engage the participants in a“challenging problem”. Throughout the RTAPhase 1 work and in HE99 and HE00, the testscenarios were deliberately chosen to engage“stress” the force elements nominated againstrequired tasks. The choice of “stress” as a basisINNOVATION AND EXPLORATION 59for test reflects the OTE genesis of the RTAprocess (OTE methods require tests againstmost stressful, least stressful and most likelycomponents of the performance envelope) anda limit on the resources and time forconducting the study. The approach explicitlyassumed that a Task Force “optimised” forwarfighting against a basket of stressfulscenarios could adapt across the performanceenvelope. 7The AEF framework accordingly places asignificant emphasis on problem definition andthe choice of so called “stressful scenarios”.The experiments that have been conducted todate have received some criticism that thechoice of scenarios was unrealistic - inparticular the notion that Australia mightcommit as a lead nation in a mechanised waragainst a large army with a powerful armouredcapability (modelled as T80 tanks in the wargames). While this is indeed unrealistic andunlikely, the experiments were designed tostress the concepts and the ORBATs in order toinvestigate the limits of applicability toimprove the concepts and capabilities. Toachieve this analytic requirement the conceptsproposed were examined out of the “normal”contexts in which they were conceived. This isan acknowledged weakness of the experimentdesign, and caused some difficulty in assessingthe concepts against the BOS requirements, butin principle provided a powerful analyticconstruct. Because of the criticism, scenariodesigns are now tailored to provide “stresswithin context” for the concepts andcapabilities under consideration. This is likelyto be a design feature for the experimentsconducted under the evolving DefenceExperimental Framework.Figure 5, and the “stressful scenarios” inwhich to test the concepts, was developed fromthe build-up training conducted by 1st Brigadein preparation for Exercise Predators Advance90. In preparation for this exercise thecommander trained the brigade staffs firstlywith a TEWT; the “best” solution to the TEWT