2003 IMTA Proceedings - International Military Testing Association

524
response tendency errors (e.g., halo, leniency), and comparing Soldiers to one another. Our goal
is to develop scales and rater training that will encourage raters to use the scales as a standard
against which to measure performance. This is a continuing challenge.
Common Task Performance
Performs Army-wide Common Tasks
Exhibits Safety Consciousness
MOS-Specific Task Performance
Performs MOS-Specific Technical Tasks
Uses Computer
Communication Performance
Communicates in Writing
Communicates Orally
Information Management Performance
Manages Information
Problem Solving and Decision Making
Performance
Solves Problems and Makes Decisions
Exhibits Tolerance
Exhibits Cultural Tolerance
Supports Peers
Relates to and Supports Peers
Demonstrates Teamwork
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003
Adaptation to Changes in Missions/Locations,
Assignments, Situations
Adapts to Changing Situations
Exhibits Level of Effort and Initiative on the Job
Exhibits Effort and Initiative on the Job
Demonstrates Professionalism and Personal
Discipline on the Job
Adheres to Regulations, Policies, and
Procedures
Exhibits Integrity and Discipline on the Job
Exhibits a Selfless Service Orientation
Personal and Professional Development
Exhibits Self-Management
Exhibits Self-Directed Learning
Demonstrates Physical Fitness
Demonstrates Physical Fitness
Demonstrates Military Presence
Figure 1. Organization of performance dimensions for Army-wide observed performance scales.
In their final versions, each of the OPS has four sections: (a) title of the dimension, (b)
three summary performance paragraphs, (c) behavioral examples, and (d) a 7-point rating scale.
The summary paragraphs (anchors) provide a snapshot description of Soldiers’ behavior
representing three levels of performance: Exceeds Expectations, Meets Expectations, and Fails to
Meet Expectations. The behavioral examples are designed to provide additional pieces of
information about Soldiers' behavior at the various levels of effective performance to improve rater
accuracy. Fig. 2 provides an example of a rating scale. The 7-point rating scale allows raters to
differentiate between rating levels to provide ratings that are an accurate reflection of performance.
The scales looked rather different at the beginning of the project. Keeping in mind our
goal to have raters rely on the scales as the measurement standard, we revised the format of the
scales based on the results of practice rating exercises done with SMEs. The original scales
contained a title, a lead-in question about how effective the Soldier is at performing in that
dimension, and designations of High, Moderate, or Low performance for the three columns. We
discovered from discussion with the SMEs that some of them just read the question and decided
524