Asking Questions - The Definitive Guide To Questionnaire Design ...

ASKING QUESTIONS THAT EVALUATE PERFORMANCE 217
in a study sponsored by the National League for Nursing (Smith
and Kendall, 1963). The original BARS approach combined the
Fels parent-behavior rating scales (Guilford, 1954, pp. 266–267)
and Thurstone’s attitude scales (Guilford, 1954, pp. 456–459). It
uses graphic rating scales that incorporate specific behavioral descriptions
using various points along each scale. (See Figure 7.2.)
Each scale represents a dimension or factor considered to be an
important part of work performance, and both raters and those
being evaluated are typically involved in developing the dimensions
and generating behavioral descriptions.
Although many variations of the BARS procedure have been
introduced since its inception, the procedures that follow are true
to the original ideals of BARS. A complete discussion of the various
appraisal formats that have been introduced under the guise of
BARS can be found in Bernardin and Smith (1981).
The BARS procedure was originally an iterative process that
began by having a sample of the evaluating population (the raters)
identify, define, and propose a first draft of scales they believed captured
ideal behavior. This draft was then reviewed and modified by
other raters until a final version was agreed on (Bernardin, 1977).
At this point, dimension-clarification statements are developed
to anchor the high, middle, and low parts of the scale. Behavioral
examples are then written for the high, medium, and low effectiveness
for each dimension. Behavioral examples are then “retranslated”
by a second group of people who are given a randomly
ordered list of these high, medium, and low effectiveness examples
and asked to choose the dimension to which each example is
related. (An 80 percent successful retranslation is typically used.)
Next, 7-point scales (1 = low effectiveness, 7 = high effectiveness)
are used to rate the effectiveness of each behavioral example on the
dimension for which it was written. Behavioral examples with large
variances (in excess of 2.0) are eliminated in order to remove examples
that are unreliable.
When using the BARS method, raters are instructed to record
observed behaviors throughout the appraisal period and to indicate