Research Methods Handbook.pdf - CLES

Can the population be counted? Some populations willbe easy to count, in a given geographical area there will besecondary data sources that will give you a population count(Census), in a membership organisation there may be a list of allmembers, however in a newly arrived ethnic community such asthe recent arrivals of Polish and Eastern European communitiesthere is less chance that you can obtain a reliable count of thepopulation. A bias in your survey results can occur if the surveysample does not accurately represent the population. Having acount of the population is also important in order to establishthe significance of your results to allow a generalisation to thepopulation as a whole.Are there language issues? Respondents may have varyingcapacities for being able to complete written surveys orquestionnaires. While telephone and street surveys do notrequire the respondent to be able to read or write in English,postal surveys involve respondents completing the survey orquestionnaire themselves. You should consider the offer of helpin self-administered surveys for respondents to complete a formeither in person or over the telephone, this will help addresspotential language or basic skills issues. If surveying an ethnicminority population you may wish to translate questionnairesinto community languages, or have people who speak the communities’language to assist where necessary.What are the geographic restrictions? The geographic spreadof the population to be surveyed will determine the methodused for collecting your data. If you are surveying people from aparticular location or organisation it may be possible to conducta survey using an interviewer, however if you have a populationsample that is geographically dispersed then you would look touse a different method, such as a telephone or postal survey.SamplingThe sample is the section of the wider population that will be engagedin the survey and sampling is the process of identifying who you willaim to contact from that population. The word ‘population’ is used todescribe the target group, and while this may be the national populationas a whole, it may also be a smaller group such as lone parents, orbusiness members of a Chambers of Commerce in a particular location.Detailed consideration of sampling needs to be made to ensure thevalidity of your results, and the following issues need consideration:Who is the respondent? The first thing you need to understandis who your respondent is going to be. This is the personthat will provide the data you are asking for. If the survey isdistributed amongst households, who in particular will be fillingin the survey? Do you want to specify who the survey is to becompleted by? And do you understand why you are specifyingthis person? The same is true when surveying organisations orgroups. A survey will have much greater success if it is directedto the right respondent. Identifying the person best suited tocompleting a survey will help to increase the response rate andgenerate more accurate data.What is your sampling frame? A sampling frame is a list ofmembers of a population from which members of a sample arethen selected. A sampling frame needs to be accurate, complete,up-to-date and relevant to the purposes of the survey forwhich it is to be used. Once you have an established samplingframe, depending on its size you may need to adopt a samplingtechnique to extract your final sample. For example randomsampling, simple random sampling or stratified sampling (seefurther reading for more details on sampling techniques).Are response rates likely to be a problem? With any survey,you need to look at the profile of the people who did respondedand satisfy yourself that they are about the same as the peoplewho didn’t respond – and also, that they’re about the same asthe overall population that you’re sampling. If you send out asurvey to a population, which is 50% male, and 50% female, butyour responses are 80% from females then your findings willnot represent your target population. Response rates can below for surveys, under 20% for a postal survey is not uncommon.However, all the considerations in this section can help toimprove your response rate.Statistical significance: Understanding your population,sample size, and response rates are important for calculating12 13