Valuing Our Natural Environment Final Report ... - HM Treasury

More documents

Recommendations

Info

Valuing Our Natural Environment – Final Report - Annex1 A1.2.1.7 Principal outputs The basic outputs of questionnaires and interviews will take the form of proportions of population responding in a particular way to a particular question. Beyond this, the interest is primarily in the statistical analysis of responses, in particular (a) explaining response proportions in terms of demographics and (b) correlating responses across different questions, for example, linking particular stated behaviours to particular stated attitudes or values. Usually the results will be written up in the form of a report, which may be ‘glossy’, and will often make substantial use of graphs, charts, tables and other ways of presenting the data. This may be combined with other outputs (e.g. focus group results) relating to the same topic. A1.2.1.8 Transferability of outputs Questionnaires will usually produce transferable results if the sample size is adequate and the sample has been randomly selected to fit a demographic quota. It may even be possible to transfer to other populations within the same culture by correcting for demographic differences. A1.2.1.9 Key uses – decision-making context(s) The questionnaire approach is suitable for meeting any requirement for basic data on views, behaviours and so on as well as for research exploring underlying reasons and correlations. However the approach it is not participatory and can not really be implemented within a deliberative setting (but the results of surveys might provide background data for deliberative processes). Therefore questionnaires may be useful either at an early stage in decision making, or where it is unlikely that full participatory approaches will be used but there is nevertheless a desire to take citizens' views into account. Questions can also be directed at estimating policy impacts/effectiveness, for example trying to find out how respondents might alter their behaviour under a particular policy (for example, how behaviour might change if an eco-tax were introduced). However, there is always a problem here going from stated (hypothetical) intentions to actual responses, and further concern about the potential for strategic responses. A1.2.1.10 Key uses – coverage of natural environment Questionnaires can be applied to cover any natural resource or issue. However, where the resource in question is unfamiliar or poorly understood, or where the issues relate to complex human-environment interactions where the general public is unlikely to be well informed or to have considered the issues, there may be limited value in a questionnaire approach. This is because questionnaire time is expensive, and the level of attention/patience is likely to be much less than in a group discussion, so there are limits on the amount of information that can be provided and the amount of thinking time respondents can be allowed (or will take). Hence the technique is best suited to ‘taking the pulse’ of existing attitudes on environmental issues and behaviours, rather than engaging the public in complex reflection on new and challenging topics. Because of time/attention constraints, there is a limit on the number of questions and therefore the number/depth of topics covered. One way of reducing this problem can be to use split sample designs, where different questionnaires are used for each sub-sample, with some questions in common to all versions. This can give maximum statistical representativeness for the questions of key interest, while allowing a wider exploration of secondary issues with lower sample sizes. Careful design is needed in such cases to ensure that all statistical analyses of interest will be possible with the combinations of survey version/sub-sample achieved. A1.2.1.11 Discussion If done well, questionnaires can produce reliable and statistically significant results. However, the methodology can be abused. For example, short questionnaires with leading questions can be used to pad out press releases with ‘information’ of the "8 out of 10 cats prefer..." variety. Hence the results of surveys should be interpreted with care, in conjunction with their questions. Reporting should always include the exact questions, and eftec A32 December 2006
Valuing Our Natural Environment – Final Report - Annex1 it is important to avoid interpreting results out of context. Misleading graphical presentations are quite common and must be avoided. In planning research, the questionnaire pre-testing phase is crucial. Ideally, this can be combined with some form of focus group or other debriefing in which the researchers can explore how the respondent dealt with the question, allowing any ambiguities or other misunderstandings to be identified and addressed. More generally, focus groups or other discursive methods can be a useful part of the early design process, helping to shape ideas of the key features of the topic which people find interesting or relevant, or are concerned about; with questions then being aimed at investigating these features in the population at large. A1.2.1.12 Consideration of distributional impacts Non-response bias is potentially a significant problem with questionnaires, both in terms of questionnaire non-response (where certain groups in the population may be more likely than others to refuse to take part) and item non-response (where certain groups may be more likely than others to refuse to answer particular questions, for example, many people refuse to give their approximate incomes). This can be a particular problem if non-response is related to the ‘true’ answer, for instance, if people whose true view of a question is ‘No’ are more likely to refuse to answer it than those who think ‘Yes’. To some extent, bias (especially questionnaire non-response) can be corrected for by weighting the results according to comparison between the sample and population demographics. But care still needs to be taken in interpreting weighted figures where the “base” of actual respondents in a particular category (of weighting “type”, of a given opinion in a question) is small. A1.2.1.13 Advantages and disadvantages Whilst face-to-face surveys are usually more are expensive they will typically give better quality data and also allow for visual information to be used. Postal and internet surveys can be cheaper to conduct but it is hard to obtain a good response rate or an unbiased sample. Also, it can be hard to control the order in which questions are asked. A possible exception is an internet survey which controls the question and information order quite precisely (and even to vary randomly across the sample to give different ‘treatments’), and is able to make specific measurements which can help explain answers, check engagement and so on. For example, alongside answers, additional information can be logged such as exactly how long a respondent spent looking at a question, whether or not ‘pop-up’ links for addition information were clicked on, and so on. With questionnaires, it is not possible to revise the questions once the survey starts. ‘Fixing’ a problem identified halfway through the survey stage means different respondents will have answered different questions, and so comparison of responses is flawed. Again this emphasises the essential element of comprehensive pre-testing. Questionnaires are very suitable for tracking changing attitudes over time. The same questions can be repeated at suitable intervals with a new sample from the population. Since sample sizes should be large enough to give statistically representative results, the results from different surveys can be compared. Care may be needed to control for the possibility of question order effects and so on if the questionnaires include some of the same questions from previous research, but some are dropped and new ones added (as will often be necessary to keep the survey “up to date” with new developments in policy, knowledge and so on). A1.2.1.14 Conflicts and synergies with other methods Questionnaires and interviews have clear synergies with economic valuation methods, where a survey is often an essential part of the method (e.g. travel cost method, stated preference techniques). How effective or accurate a questionnaire can be in eliciting a particular type of value may vary from case to case, and much of the research in nonmarket valuation relates to ways of improving the survey techniques used for eliciting information from respondents. eftec A33 December 2006
Page 1 and 2: In association with Environmental F
Page 3 and 4: Valuing Our Natural Environment - F
Page 33: Valuing Our Natural Environment - F
Page 81 and 82: 38. Crime 40. Employment 41. Workle
Page 83 and 84: Annex 1 References Valuing Our Natu
Page 85 and 86:
Valuing Our Natural Environment - F
Page 87 and 88:
Page 89 and 90:
Page 91 and 92:
Page 93 and 94:
Page 95:
Page 98 and 99:
Page 100 and 101:
A2.2 References Valuing Our Natural
Page 102 and 103:
A study based in Devon. Valuing Our
Page 104 and 105:
Page 106 and 107:
Page 108 and 109:
Page 110 and 111:
Page 112 and 113:
Page 114 and 115:
Specific resources Valuing Our Natu
Page 116 and 117:
Page 118 and 119:
Page 120 and 121:
Page 122 and 123:
Page 124 and 125:
Page 126 and 127:
Page 128 and 129:
Page 130 and 131:
Page 132 and 133:
Page 134 and 135:
Page 136 and 137:
Page 138 and 139:
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
show all

Valuing Our Natural Environment Final Report ... - HM Treasury

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?