2003 IMTA Proceedings - International Military Testing Association

476
Phase I: Initial Job Interest Elements Production
To create a completely new set of data, all available job descriptions for all Navy enlisted
Ratings were downloaded from the Enlisted Community Manager’s website. The author printed,
compiled, and reviewed each of these job descriptions, and highlighted words reflecting process
(action verbs, e.g., make, maintain), content (direct object nouns, e.g., electronics, documents),
environment (e.g., indoor vs. outdoor), and community (e.g., surface, submarine, aviation) to
construct interest job elements (see also, Hindelang, Michael & Watson, 2002, for
complementary description of Methods). From the collections of highlighted extracts, the author
developed common statements for each of the Ratings containing: 1) Between one and five
Process-Content (PC) pairings 2) For each PC pairing, between one and five parenthesized
examples. 3) At least one community statement. 4) At least one environment statement. 5) At
least one ‘style’ statement. An example of these statements is shown in Figure 1.
Rating Community Process Content Work Style
ABE Surface Maintain mechanical (hydraulic systems, steam systems, pneumatic) outdoor
ABE Operate mechanical (hydraulic systems, steam systems, pneumatic) physical
ABE Operate heavy equipment (crane, tractor)
ABE Respond emergency (fire, crash, flood)
ABE Aviation Direct aircraft
Figure 1. Example of job interest elements for the Aviation Boatswain Mate-Launch/Recovery
(ABE) rating built from analysis of ECM documents, and revised in iterative SME interviews.
Phase II: Job Interest Element Revision
The collection of job interest elements (see Figure 2) developed in Phase I were used in
semi-structured interviews with Subject Matter Experts (SME’s). These SME’s were Enlisted
Community Managers (ECM’s; typically the officer supervisors of enlisted personnel, with over
10 years of experience), and the Enlisted Community Technical Advisors (TECHADS; typically,
enlisted personnel with 10-30 years of experience in the interviewed rating). In these small group
interviews (two to four participants) the author systematically presented elements from the
relevant pre-conceived descriptions, and interviewees were asked to assess and comment on the
appropriateness of elements, and modify elements as necessary. Interviewees were also allowed
to add words from the “precoordinated” (Fleishman & Quaintance, 1984) list of terms, which
had been created by pooling all terms developed in Phase I. Also, natural language terms
considered by the interviewer to be key to the description, occurred frequently, and were not
represented in the precoordinated vocabulary, were added to the pool.
The development of the final interest element collection for each rating continued from
this point as a process of iterative interviews, refining and modifying the collection of job
interest elements. Upon completion, all Ratings were described by some collection of interest
elements, and all interest elements occurred for multiple (but not all) Ratings, and could be used
to successfully differentiate between all but 7 Navy Ratings. Ultimately, 27 PC interest elements,
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003