Asking Questions - The Definitive Guide To Questionnaire Design ...

ASKING QUESTIONS THAT EVALUATE PERFORMANCE 221
the date and details of the incident on the scale. Scaling the effectiveness
level of the observation is facilitated by a comparison with
the series of illustrative behavioral “anchors” that defined the highest,
lowest, and midpoint of each scale. Each rater was to decide
what had been observed in relation to these specific examples. The
rater could then use the examples as benchmarks when briefly noting
what behavior had been observed. These anchoring illustrations
were to be concrete, specific, and located at irregular intervals along
the relevant scale according to effectiveness. The dimensions themselves
would have been picked only after considerable discussion of
organizational goals and objectives. (See the earlier section in this
chapter on developing a rating system.) After a period of observing
and recording incidents, the rater could, if needed, make a summary
rating. This summary, plus the notes, could serve as a basis for discussion
with the person being rated. It could also serve to measure
performance.
Thus, to summarize the original BARS procedure, the sequence
was as follows:
1. Observation
2. Inference
3. Scaling
4. Recording
5. Summary rating
The procedure is intended to define, to clarify, and to operationalize
the implicit evaluative theory of the rater. In doing so, the
BARS system encourages an evaluator to observe and to explicitly
acknowledge the implications and interpretations of behavior.
What sets the BARS method apart is that it can improve future
observations, such as forced-choice or summated scales (Bernardin
and Smith, 1981). The BARS method is effective because it creates
a common frame of reference so that evaluators look for the
same kinds of behaviors and interpret them more consistently and
objectively.