2003 IMTA Proceedings - International Military Testing Association

10
military context. We are of course not trying to make sweeping generalisations about the
results. Often attitudes raised in survey research require further investigation especially if a
sub set of the population has a different view point to the mainstream. Findings are caveated
because attitudes may not result in direct behaviour or action, or in the direction proposed.
Additionally, the findings may tell us something about a tendency towards a situation but
nothing about that the attitude itself. However, our results will tell us something about
individual or group tendencies and the prevalence of such tendencies.
Measurement of poorly defined concepts
So if there are compelling reasons for trying to measure poorly defined concepts, how
can we achieve this reliably and validly? The solution to this would create enough material
for several additional discussion papers. However, there are a few key points that are worth
higlighting.
Anderson, Herriot and Hodgkinson (2001) believe that the basis for and majority of
occupational/industrial psychology should be in grounded in what they call ‘pragmatic
science’, that is psychology should have “both practical relevance and methodological rigour”
(Anderson et al, p394). It has been suggested that only well defined concepts can be
measured. However, in reality many attitudinal instruments have been developed to measure
poorly defined terms. The relevance and motivation for attitude measures has already been
outlined so the next question relates to ‘methodological rigour’.
Hinkin (1998) sets out the constituents of a good measure of a construct ie ensuring
construct validation. Identify the construct domain (clear definition); develop items (based on
previous research, subject matter experts and pilot studies); determine how items measure the
construct domain (content validity); and assess antecedents, predictors, correlates and
consequences of the concept (convergent, discriminant and criterion validity). Therefore,
measurement can be made reliable and valid although this does not necessarily help define the
concepts in the first place. So what have other researchers done?
Citing morale yet again, as a concept it has been measured as a single entity eg “How
would you rate your own morale?” (Schumm & Bell, 2000) and this might reflect the lack of
operational definition (Liefooghe et al, 2003). Is this the best approach when there is no
agreed definition? Alternatively, should we employ multi-item measures, for example Paulus,
Nagar, Larey and Camacho (1996) who used seven items relating to feelings about being in
the Army, unpleasant experiences in the Army, helpful Army experiences, relationships with
other soldiers, satisfaction with leadership, reenlistment, desire to leave the Army? The
second option assumes that we are confident with the components of a given concept and that
we can construct a ‘morale scale/score’ from the results.
One approach might be to explicitly define terms for respondents ie use stipulative
definitions where we indicate how we will want to use the term. Here a new definition may
be created and one which is specific to a given environment like the military. If a definition
is too prescriptive, examples of attributes and characteristics which comprise the term could
be used; eg for morale terms: ‘dedication’, ‘willingness to sacrifice’, ‘motivation’,
‘confidence’ ‘commitment’. There is a note of caution though, even when a definition is
provided it does not mean that the respondents will use it (Oppenheim, 1992).
45 th Annual Conference of the International Military Testing Association
Pensacola, Florida, 3-6 November 2003