Watershed Conservation Plan - Destination Erie

Methodology
! Studying the attitudes of residents of a watershed is a complicated endeavor. Unlike states,
counties or cities, a watershed follows the natural geographic features of the region. Neither
the US Census Bureau, Environmental Protection Agency, nor any state or local organization,
collect population data for particular watersheds. As a result, population data is not easily
acquired and must be approximated.
! Cognizant of this challenge, our study utilized population data supplied by the US Census
Bureau for Zip Code Tabulation Areas (ZCTAs) as a means of approximating the population
of the Pennsylvania-Lake Erie Watershed. The Erie County portion of the watershed contains
all or part of 20 ZCTAs, which include approximately 174,204 residents over seventeen years
of age.
! Once the population was identified, we needed to identify a sample of residents from which to
collect data. To ensure that the results of the study would reflect the views of the residents of
the watershed the sample had to be representative of its population. This meant we needed
to obtain data from a sufficient number of individuals and these individuals had to look like the
population. The desire to be 95% confident that the results of our study were accurate within
+/-5%, required us to obtain a sample of approximately 400 individuals. In addition, the
sample had to be demographically, socially and economically similar to the population as a
whole.
! In order to ensure that these requirements were met, we used a method called “systematic
random sampling.” Beginning with a list of all residential addresses within the ZCTA’s that
comprise the Watershed (compiled by the US Postal Service), 3000 addresses were selected
at predetermined intervals. The number of addresses selected was based on the number of
responses necessary for an accurate assessment of the population’s attitudes and the
expected number of individuals that would participate in the study (estimated response rate of
26%).
Rivers Conservation Planning Grant--Public Opinion Survey Page 4
Methodology (cont’d)
! On March 24, 2003, a four-page questionnaire was mailed to each of the 3000 addresses
comprising the sample. Between March 27 and April 11, approximately 270 individuals
responded to the questionnaire. The response rate for the first mailing was a meager 9%,
which was substantially lower than expected. On April 12, a second questionnaire was sent to
all individuals who failed to respond to the first mailing. Between April 14 and April 26,
approximately 208 additional individuals returned a completed questionnaire. The combined
response rate for the first and second mailing was 16%, or 478 of the 3000 individuals selected.
Again, this response rate was far lower than expected.
! The low response rate warns against over confidence in the study’s findings. Response bias,
which poses a substantial threat to the validity of survey research, is likely to be present
anytime the response rate is below 50%.
! However, individuals who responded to our inquiry looked fairly similar to the population.
Overall, the age of respondents was similar to the watershed’s population, although young
adults (20-24 years) were under-represented when compared to the area’s residents (2% v.
10%); and older adults (65-74 years) were over-represented (15% v. 10%). Likewise,
respondents were substantially more educated than the population, with 46% reporting that they
had an associates degree or higher compared to 27% of the population. However, the race and
income of respondents closely approximated the population, with less than a 3% discrepancy in
any income category and a nearly perfect comparison of race.
Rivers Conservation Planning Grant--Public Opinion Survey Page 5
4