National Survey of Fishing, Hunting, and Wildlife ... - All About Birds

Appendix C.Significant Methodological Changes FromPrevious Surveys and Regional TrendsThe 2011 National Survey of Fishing,Hunting, and Wildlife-AssociatedRecreation (FHWAR) was designedto continue the data collection ofthe 1955 to 2006 Surveys. Whilecomplete comparability between anytwo Surveys cannot be achieved, thisappendix compares major findings ofall the Surveys and presents trends forthe major categories of wildlife-relatedrecreation where feasible. Differencesamong the Surveys are discussed in thefollowing two sections.The principal characteristics of the1955 to 2011 Surveys are summarizedin Table C-1. The table shows the scopeand design of all 12 Surveys.This appendix provides trend informationin two sections (1991 to 2011 and1955 to 1985). A significant changewas made in 1991 in the recall periodused in the detailed phase of theFHWAR Surveys. The recall periodin 1991 was shortened from the 12months used in previous Surveys to 4months in order to improve the accuracyof the data collected. As a resultof that change, the Surveys conductedsince 1991 cannot be compared withthose conducted earlier.The 1955 to 1985 Surveys requiredrespondents to recall their recreationactivities for the survey year at thebeginning of the following year. The1991 to 2011 Surveys went to therespondents two or three times duringthe survey year to get their activityinformation. The change in the recallperiod was due to a study 1 of theeffect of the respondent recall length1Investigation of Possible Recall/Reference Period Biasin National Surveys of Fishing, Hunting and Wildlife-Associated Recreation, December 1989, Westat, Inc.on survey estimates. The study foundsignificant differences in FHWARSurvey results using annual recallperiods versus shorter recall periods.Longer recall periods lead to higherestimates. Even when everything elsewas held constant, such as questionnairecontent and sample design,increasing the respondent’s recallperiod resulted in significantly higherestimates for the same phenomenon.The recall study also found that theextent of recall bias varied for differenttypes of fishing and hunting participationand expenditures. For example,annual recall respondents gave anestimate of average annual days of saltwaterfishing that was 46 percent higherthan the trimester recall estimate, whilethe annual recall estimate of averageannual saltwater fishing trips was 30percent higher than the trimester recallestimate. This means there is no singlecorrection factor for all survey estimateswhen calculating trends fromSurveys using different recall periods.Reliable trends analysis needs to usedata compiled from surveys in whichthe important elements, such as thesample design and recall period, are notsignificantly different.1991 to 2011 SignificantMethodological DifferencesThe most significant design differencesin the five Surveys are as follows:1. The 1991 Survey data wascollected by interviewers fillingout paper questionnaires. The dataentries were keyed in a separateoperation after the interview. The1996, 2001, 2006, and 2011 Surveydata were collected by the use ofcomputer-assisted interviews. Thequestionnaires were programmedinto computers, and the interviewerkeyed in the responses at the timeof the interview.2. The 1991 Survey screeningphase was conducted in Januaryand February of 1991, when ahousehold member of the samplehouseholds was interviewed onbehalf of the entire household.The screening interviews for the1996, 2001, and 2006 Surveys wereconducted April through June oftheir survey years in conjunctionwith the first wave of the detailedinterviews. The 2011 Survey alsoconducted screening interviews andthe first detailed interviews Aprilthrough June of 2011, but furthermorehad an additional screeningand detailed effort from February2012 to the end of May 2012. TheApril–June 2011 screening efforthad a high noncontact rate becauseof poor results using sample telephonenumbers obtained from aprivate firm. Census went back tothe noncontacted component of theoriginal sample in February-May2012 and interviewed a subsample,requiring annual recall for thoserespondents. The Wave 3 screensample was 12,484 of the total48,600 household screen sample. Amodification of the 2011 samplingscheme was to oversample countiesthat had relatively high proportionsof hunting license purchases.The screening interviews for all fiveSurveys consisted primarily of demographicquestions and wildlife-relatedrecreation questions concerning activityin the previous year (1990, 1995, etc.)and intentions for recreating in thesurvey year.108 2011 National Survey of Fishing, Hunting, and Wildlife-Associated Recreation U.S. Fish and Wildlife Service and U.S. Census Bureau