Developmental psychology.pdf

354 Individual Differences
Test items also can be developed on the basis of theory. Tests of personality
and intelligence are commonly constructed in this way, for the concepts being measured
are broad and difficult to define. Inkblots and other ambiguous figures, for example,
are used to assess the so-called depth factors in personality.
Figure 13.7
Assigning Partial Credit. In
answering Figure 13.3, a subject
might arrange the pictures as follows:
F, C, A, E, B, D. Thus, picture D is
incorrectly placed at the end of the
sequence instead of the beginning.
Some subjects explain that the man's
dinner has been stolen and now he
wants to leave the restaurant before
losing anything else. They describe
him as taking his hat and coat off the
hook rather than placing them there
before eating. This line of reasoning
greatly lessens the joke and makes a
less concise story, but it is not clearly
contradicted by evidence in the
drawing. Hence, this sequence might
receive partial credit, or a change
could be made in the test materials
(Drawing by Alain, © 1943, 1971 The
New Yorker Magazine, Inc.).
Answers to Figure 13.6
1. Alexander Fleming
2. Approximately 3,000 miles
3. _ 4. 5. 6.
7. Weight В 8. Equal
9. 10.
Refining the Test
The first draft of any test usually contains more items than are intended for the final
version. The reason is that some test procedures and items will prove to be defective.
They must be revised or discarded, and these decisions are made by some early trials.
Pretesting the Procedures A small group of subjects is used for the first trial. It
is called a pretest because the purpose is simply to try out the test, determining how
it is understood and how it functions. The subjects are selected for convenience and
willingness to cooperate, but representativeness must be considered, too. If the sample
is not reasonably close to the population for which the test is intended, the value of the
pretest is greatly diminished.
This administration of the test follows specific procedures, and there are detailed
guidelines for recording and scoring each response. One purpose of the pretest
is to check the clarity of the instructions, test items, and scoring procedures (Figure
13.7).
Conducting an Item Analysis Another purpose of pretesting, which would require
an appropriate sample of pilot trainees in this instance,, is to examine the value of the
test items. This procedure is called item analysis, for the aim is to discover which items
are satisfactory and which items are not serving the purposes of the test. Then the
unsatisfactory test items are improved or discarded.
Basically, item analysis involves a study of how often each item is answered
correctly and by whom. For selection purposes, an item that everyone passes or everyone
fails is useless. It does not indicate which candidates are more promising than
others.
Instead, an item has greatest potential when approximately half of the subjects
answer correctly. But which half? When the subjects answering correctly are
those with the greatest flying aptitude, and those answering incorrectly have the least
aptitude, then the item has considerable value. The ideal test item elicits one type of
response from candidates with much ability and a different response from those with
little ability. In other words, it has high power for discriminating among potentially
good and poor pilots.
Suppose one of the items on mechanical comprehension is answered correctly
by about half the subjects. Is it a valuable item? Not necessarily. For each item, those
subjects answering correctly and those answering incorrectly must be compared on
some external criterion, such as past successes in similar training programs. If the most
successful candidates answer correctly and those with the least ability answer incorrectly,
then we would feel that the item discriminated successfully (Figure 13.8).
Sometimes no external criteria are available, for there has been no previous
assessment of the subjects. Then the only recourse for evaluating a test item is to use
the total test score as the criterion of flying ability. The premise here is that the test
as a whole, even in an early version, is a better measure of flying ability than any one
of its items alone. Thus the subjects are considered to be high or low in flying ability
according to their total scores on the test. Any item answered correctly by those in the
high group and incorrectly by those in the low group is considered to be an effective
item, insofar as it can be assessed by this less desirable procedure.
When all of the test items have been retained, revised, or discarded, as required
by the item analysis, and the procedures for test administration changed accordingly,
a final version of the test is ready. Actually, the test is never in final form,
for it can always be improved. But eventually a point is reached at which the test must
be evaluated and used.