I__. - International Military Testing Association

-.
_.---___----
Data from this comparison sample were scored using the sanmprocedures that were used in the applicant
sample. Mean item-level scores from the NROTC student sample were compared with those from
the pilot-test sample in order to identify items with substantially different base rates. If an item’s mean
score in the applicant group is slanted considerably more in the socially desirable direction than that of
the student sample, it suggests that the item is relatively easily distorted by applicants.
Refining the PEC
Results from the pilot test data analyses, along with the information from the retranslation exercise,
were used to revise the composition of the PEC constructs and their definitions. The inventory was then
refined and shortened for future administrations. Descriptive statistics, internal consistency reliabilities,
and scale score intercorrelations were computed for the final shortened scales.
Evaluating the PEC
Retranslation Results
Results and Discussion
Seventy-seven percent of the PEC items were sorted into the same predictor construct scale by five
or more of the seven researchers who participated in the retranslation exercise. Seven of the remaining
items were from the Unlikely Virtues (response validity) scale. It is not particularly surprising that some
researchers sorted the Unlikely Virtues items into the construct categories. The Unlikely Virtues items
were specifically written to resemble the eight original construct categories (so they would be subtle).
The fact that some of the judges mistakenly sorted the Unlikely Virtues items into the construct categories
suggests that the items are indeed subtle. In general, however, there was good agreement among the
researchers concerning the placement of PEC items on constructs.
Pilot-Test Results
Frequency counts revealed that the vast majority of the PEC items had an adequate spread of responses
across the response alternatives. Only a few items had response distributions that were considered
unacceptable (e.g., over 90 percent of the respondents chose the most desirable response alternative).
However, for some items the response distributions were much better than for others. This information
was taken into account in refining the PEC, particularly in making decisions concerning which items to
drop.
The item-level intercorrelations were factor analyzed, and rotated principal factor solutions containing
from 2 to 12 factors were examined. Based on a parallel analysis (Montanelli & Humphreys, 1976),
the amount of variance accounted for by each factor, and the interpretability of the solutions, the eight
factor solution was selected for further consideration,
The amount of overlap between the results of the retranslation exercise and the factor analysis was
encouraging. Items from the Leadership Orientation scale defined a factor, and nearly all of the items
that were retranslated into this scale had their highest loading on that factor (8 of 11). Similarly, Organization
and Planning and NROTC/Military Interest and Motivation also defined their own factors.
Achievement Motivation defined a factor, but most of the Responsibility (7 or 12) items also loaded on
this factor. Dominance and Team Orientation each defined a factor, and the Sociability items were split
between these two factors, with the Sociability items involving friendliness loading on the Team Orientation
factor and those involving talkativeness and assertiveness loading on the Dominance factor. Clearly the
retranslation and the factor analysis results converged on very similar sets of constructs.
500