Valuing Our Natural Environment Final Report ... - HM Treasury

Valuing Our Natural Environment – Final Report - Annex1
But since the idea is to cover all points of view, and not to derive a representative sample,
it may be reasonable to expect that repeated environmental applications to slightly
different resources/issues (within a specific social/cultural context) could give similar
results, in the sense of identifying similar general viewpoints on environmental,
technological, economic, political spectra. This will depend on the similarity of issues
covered in the statements though. In addition it may be less obvious precisely how any
specific issue plays out in these spectra, suggesting that transferability across issues must
be considered low.
A1.2.5.9 Key uses – decision-making context(s)
The Q-methodology approach is useful for initial analysis flagging up common issues or
views amongst the public. This can help establish which policies are likely to meet with
wide support or resistance. As described here, it is essentially a consultation technique,
taking respondents' rankings, then analysing and presenting them independently of the
respondents. It may be possible to implement Q-methodology in ways more under control of
participants, for example, as part of participatory group work, allowing participants to
write descriptions of identified typical sorts.
A1.2.5.10 Key uses – coverage of natural environment
Q-methodology can potentially be applied to any resource or issue and statements tend to
range from very general to quite specific to a topic.
A1.2.5.11 Discussion
In applying Q-methodology it is likely to be unavoidable that researcher bias comes in
through the distillation and choice of statements for sorting. Whether or not this has a
strong influence on results may be uncertain. More significant bias may come in the choice
of title and description for each of the ‘typical’ groups identified, which may carry
unintended or unwarranted information cues to those using the results.
It is also possible that the factor analysis is sensitive to precisely which sorts are included.
Barry and Proops (1999) state that as few as 12 participants can generate statistically
meaningful results, however the method in effect considers a large number of choices from
each individual, and might be sensitive to which individuals are included or to small
changes in individuals' sorts (e.g. swapping two statements could involve changing a large
number of pairwise comparisons).
A1.2.5.12 Consideration of distributional impacts
With Q-methodology there is no inherent bias towards certain stakeholders; the idea is to
cover all viewpoints and show “typical” types of view, without attempting to reveal what
the relative frequencies of such views are in the population. However, some stakeholders
(or points of view) might find it harder than others to produce rankings, which could lead to
bias (e.g. if some stakeholders drop out/reject the method).
A1.2.5.13 Advantages and disadvantages
In applying in Q-methodology it is possible to carry out more than one Q-sort with the same
sample, on different topics, thus improving the efficiency of the interviewing process. But
most of the intensity rests with the researcher in any case. In particular, deriving
statements from initial interviews, and distilling these down to the short list for Q-sorting,
can be very time consuming. Barry and Proops (1999) note an advantage of Q-methodology
is that all (or most) statements analysed come from (first-stage) participants, so the
process is largely driven by participants' responses while still generating statistically
rigorous results. However, from the point of view of participation, respondents do not
control how the results are used, or even how they are described. Secondly, there is no
direct measurement of the relative proportions of different ‘types’ in the population,
although this can be achieved to some extent by comparing back to the respondents (but
note that then the low sample size becomes statistically problematic, especially if sampling
has not been random).
eftec A48
December 2006