Skin cancer campaign evaluation research: technical report (PDF ...

SKIN CANCER CAMPAIGN EVALUATION RESEARCH:TECHNICAL REPORTThe National Skin Cancer Awareness Campaign, which ran during Summer 06/07, Summer07/08, Summer 08/09, and Summer 09/10 aimed to educate Australians about the importanceof protecting themselves from skin cancer. The primary target audience was teenagers aged14-17 years and young adults aged 18-24 years. Ipsos-Eureka was commissioned to conductresearch to evaluate the effectiveness of this campaign. The Summer 09/10 campaign focusedon 14-17 year olds.The findings from the research have been provided in separate reports. This document is anupdated version of the technical report (originally produced in 2008) that details the researchmethodology.1.1 Online survey methodologyRationale for online surveyA quantitative survey was conducted over the internet using a sample sourced from onlinepanels. This methodology was chosen for a number of reasons:An online survey methodology allows access to a very large sample cost-effectively.An online survey methodology means participants can be presented with visual andaudio stimulus material. This is advantageous, because it allows videos, sound bytesand still images from the campaign to be presented to participants.The timeframe required to conduct an online survey is relatively short compared toother methodologies.It provides an environment where the participant is free to speak their mind, which canbe important when asking questions which have a degree of social sensitivity.Convenience and better access to participants (i.e. the survey arrives at theparticipant’s desktop and can be completed any time of day or night, at theirconvenience).There are no interviewer, data entry or data editing errors or bias through third partiesprocessing surveys.The fieldwork was conducted by ACNielsen, drawing from a number of panels in order to meetthe quotas. Those who qualified and completed the survey received an incentive for theirparticipation, redeemable for goods and services and approximately equivalent to two dollars.The panels used were ACNielsen’s own Your Voice Panel, and two additional online panels(Empowered and GMI), which have comparable structure and incentives schemes and whichDepartment of Health and AgeingEvaluation of Skin Cancer Campaign: Technical report | 2008 | PAGE 1

There was some difficulty recruiting 14-17 year olds in February 2010, and therefore selectedteenagers had to be recruited through their parents. Parents of 14-17 year olds on the YourVoice panel were sent an emailing asking them to have their child complete the survey. Anadditional question was added to the start of the survey to ensure that the child wascompleting the survey. A total of 409 (18.7%) of 14-17 year olds were recruited through theirparents in 2010.Details of Your Voice panelThe initial base of the Your Voice panel, consisting of 10,000 panellists, was sourced fromGeospend. (Geospend is Australia Post’s consumer market analysis and list rental business.Before the Your Voice panel was established, Nielsen sourced sample from them. This sampleeventually became part of the Your Voice panel). Since then, a variety of other methodologieshave been used to recruit for the panel. These include:Opt-in permission to recontact respondents on the back of other online projects, whererespondents would click on banner ads on web pages to complete a survey.Opt-in permission to recontact respondents on the back of other offline projects(including telephone, face-to-face and mail surveys).Your Voice invitation information printed on the back of delivery notes for door-to-doorand central location projects.Nielsen Australia website.Online Recruitment Campaigns (e.g. through banner ads on web portals).Magazines from a large range of demographic groups (eg. parents with kids, males,teenagers).Further offline methodologies, including referrals, face-to-face, telephone, and directmail.Currently, the panel is recruited on a 66% online capacity. The average number of surveysthat active panellists were invited to participate in during the past 12 months was 10.7surveys. The average number of survey completions from these invitations is 2.3 surveyscompleted. Your Voice currently has an active panel of over 96,800 panellists.Details of Empowered paneEmpowered has used a number of methods to recruit online panellists, including (viral) memberget member, online search, affiliate, and television. Approximately 98% of current activepanellists were recruited online. Data is not available on the average number of surveysparticipated in by active panellists in the past 12 months. Empowered has an active panel ofover 448,000 panellists.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 3

Details of GMI panelGMI uses a diverse recruiting process that draws from a wide range of sources, including:Web advertising.Permission-based databases.Public relations.Telephone recruiting.Alliances with major portals.Approximately 70% of the current active panel was recruited online. The survey-takingaverage of the GMI panel at large is 1.7 completed surveys per month. GMI has 71,000panellists who have participated in at least one study in the last 6 months.Open-ended questions in online surveysAs mentioned above, there are numerous advantages associated with an online survey, and forthe current project, these outweighed any disadvantages. However, it is worth noting onedisadvantage of an online survey relative to a telephone survey. Specifically, one tends to getless detailed answers to open-ended questions when using an online survey. When questionedby an interviewer during a telephone survey, participants feel a greater obligation to provide amore detailed answer and there are more opportunities to prompt for further details orclarification. This can assist in eliciting an answer that is more easily able to be classified.Hence, the online methodology may have meant that unprompted recall was lower than mighthave been found using a telephone methodology. 1Invitation textBelow is the invitation text used for this survey for each panel.YourVoiceIf this email displays incorrectly, cut and paste this link into your browser.http://mail.yourvoice.net/aspxInterface/pview.aspx?id=g4nYZFQbrTcMkU8krpcX5w%3d%3dPlease note that this is a system generated email, therefore please do not reply tothis email. If you have questions, please click the following link or paste it into a1 Based on presentations given by members of the Campaign Reference Group, some state-based researchconducted over the telephone has suggested that prompted recall of the TVC for the National Skin CancerAwareness Campaign is around 30-40%, whereas the online evaluation measured prompted awareness tobe around 11-12%.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 4

new browser: http://www.yourvoice.com.au/au/page.asp?E_Page=74330Your current account balance is:4985 e-pointsCLICK HERE to see what rewards your e-points can be redeemed for.Dear Mercury2,As a registered member of the Your Voice Panel, you are invited to participate in asurvey which is now waiting for you on the Your Voice website.Please be sure to take full advantage of our incentives by completing the surveybefore it closes.Completion :30 e-points + prize draw entrySurvey Length : 15 minutesSurvey Topic : Personal CareClosing date : 11:59PM Sunday 15th February 2009(AEDT)You can also access the survey by logging in to the Your Voice websitewww.yourvoice.com.au and proceeding to the My Surveys page.In order to participate in this survey the computer you are planning to use shouldhave:* Adobe Flash PlayerIf you do not have the software, you can download it for free from the Adobe websiteDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 5

, just click on the "GET Adobe Flash Player"button under the READERS AND PLAYERS. It should only take you 2-3 minutes.* Internet Browser - one of the following:Internet Explorer (Version 5.0 minimum, Version 7.0 recommended)Firefox (PC or MAC) (Version 1.0 minimum, Version 2.0 or higher recommended)Apple-MAC Safari (Version 1.0 minimum, Version 2.0 or higher recommended)If you do not have a recent version of the internet browser software, they can bedownloaded for free from one of the links below:1) Microsoft Internet Explorer http://www.microsoft.com/downloads/Internet Explorer will either be listed in the "Popular Downloads" list,or on the "Internet" link under the Download categories list on the left of thescreen.2) Firefox http://en.www.mozilla.com/en/firefox/3) Safari http://www.apple.com/safari/download/Participate in our weekly polls andquizzes for your chance to win 100 e-points.Check out the latest poll at our Newssection here.Regards,E-points can be exchanged for many prizes includingWISH, Coles Group & Myer and dStore gift vouchers.Sandy EdwardsPanel Coordinatoryourvoice.au@nielsen.comwww.yourvoice.com.auACNIELSEN(HOLDINGS) PTY LTD.ABN 35 008 417 84711 Talavera Road,North Ryde NSW 2113.Please note, all members that complete our surveys are expected to read all questionscarefully and consider their responses. All survey lengths and the logical flow of responses aremonitored to ensure our panelists are taking their time to read each question carefully andDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 6

consider each response. Members who do not adhere to these requirements may not receivethe e-points.If you have a problem accessing Your Voice surveys, open a separate browser window andpaste the link from the very top of this email into the address field. This will open an Internetversion of this email in a browser. Then click on the link below from the new window.Important: Don't forget to add yourvoice.net to your email safe list so that our messages arenot sent to your junk mail folder and you don't miss out on the great offers we send you.Privacy Policy | Terms and Conditions | Contact Us | Resign asPanelistDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 7

EmpoweredDear Dylan,You are invited to participate in a survey for a communications company,working on a project to help them understand people's thoughts onhealth issues affecting Australians.The survey should take about 10 minutes of your time.You will be awarded up to 5 SMSPup points* for completing the survey.Please note that most surveys have time limits and quotas. There is apossibility that the survey will be closed by the time you click through. Inthis case, you will not be able to earn any SMSPup points.In the event that you do not meet the criteria to complete this survey,you will be awarded 0 SMSPup point.Regards,SMSPup Team*Please allow up to 10 working days for your SMSPup points to be awarded.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 8

GMISTUDY.CUSTOM_MESSAGE}A new survey is availableSurvey Number: {CUSTOM.ID_STUDY}MarketPoints for survey: {STUDY.SCREEN_POINTS}-{STUDY.COMPLETE_POINTS}Start Survey »You can also access the survey by copying the following URL into your browser:{link}Best regards,GlobalTestMarketGlobalTestMarket validates data by analyzing the quality of responses at several key points within the surveyprocess. Additionally, there are automatic checks and alerts built into its systems that limit membership and checkfor multiple accounts upon redemption of awards. As a participant, you agree to abide by our terms & conditions.If for some reason you no longer wish to participate, please click here. If you would like to contact us in regards toyour account, please click here.GlobalTestMarket . 1100 112th Avenue NE . Suite 200 . Bellevue, WA 98004 . USA1.2 Fieldwork datesFieldwork was conducted in five survey waves:a pre-campaign survey prior to the launch of the first season of the campaign inNovember 2006 (fieldwork dates 13 th to 17 th November 2006); after the conclusion of the TV advertising over Summer 06/07 in February 2007(fieldwork dates 12 th to 16 th February 2007);prior to the second season of the campaign in November 2007 (fieldwork dates 13 thto 19 th November 2007);after the conclusion of the free-to-air television advertising over Summer 07/08, inFebruary 2008 (11 th to 15 th February 2008);Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 9

after the conclusion of the free-to-air television advertising over Summer 08/09, inFebruary 2009 (9 th to 19 th February 2008); andafter the conclusion of the cinema advertising over Summer 09/10, in January toFebruary 2010 (29 th January to 12 th February 2010).With the exception of the November 2007, February 2009, and February 2010 fieldwork, wheresome surveys were completed over the weekend, all of the fieldwork was conducted onweekdays and during school term. Each of the November and February surveys wereconducted on comparable dates.The invitations were sent out in batches. The following are the numbers, dates and times forthe invitations to potential respondents for the November 2007, February 2008 and February2009 surveys:November 200732,164 on 13/11/2007 at 5.00pm2,210 on 14/11/2007 at 2.00pmFebruary 200820,899 on 11/02/2008 at 12.40pm1,863 on 12/02/2008 at 10.24am340 on 13/02/2008 at 11.30am2,200 on 13/02/2008 at 1.00pm157 on 15/02/2008 at 9.33amFebruary 20092,125 on 9/02/2009 at 2:30pm150 on 9/02/2009 at 4:00pm17,650 on 9/02/2009 at 4:20pm1,341 on 10/02/2009 at 12:00pm2,359 on 11/02/2009 at 2:30pm919 on 11/02/2009 at 3:00pm1,795 on 13/02/2009 at 2:40pm1,204 on 13/02/2009 at 2:50pm171 on 17/02/2009 at 3:20pm2,000 on 17/02/2009 at 5:00pmDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 10

2,000 on 19/02/2009 at 10:00amFebruary 2010200 invites sent on 29/01/2010 at 2:17pm14,671 invites sent on 29/1/2010 at 5:31pm230 invites sent on 30/01/2010 at 02:08am450 invites sent on 30/1/2010 at 2:52am740 invites sent on 31/01/2010 at 9:08pm400 invites sent on 21/01/2010 at 9:36pm1 invites sent on 1/2/2010 at 2:47am1 invites sent on 1/2/2010 at 2:49am1 invites sent on 1/2/2010 at 2:51am105 invites sent on 1/2/2010 at 2:53am100 invites sent on 1/2/2010 at 6:34am100 invites sent on 1/2/2010 at 8:08am100 invites sent on 1/2/2010 at 10:41am656 invites sent on 1/2/2010 at 10:45am1,117 invites sent on 1/2/2010 at 5:21pm2000 invites sent on 2/2/2010 at 12:15am1 invites sent on 2/2/2010 at 1:03am2,000 invites sent on 2/2/2010 at 1:09am2,000 invites sent on 3/2/2010 at 3:57am1,416 invites sent on 3/2/2010 at 11:47am2,265 invites sent on 3/2/2010 at 11:01pm1 invites sent on 4/2/2010 at 12:38am1 invites sent on 4/10/2010 at 12:39am1 invites sent on 4/2/2010 at 12:40am1 invites sent on 4/2/2010 at 12:46am2,000 invites sent on 4/2/2010 at 12:47am2,000 invites sent on 4/2/2010 at 12:48am2,000 invites sent on 4/2/2010 at 5:42amDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 11

2,000 invites sent on 4/2/2010 at 12:07pm1,791 invites sent on 4/2/2010 at 6:17pm2,000 invites sent on 4/2/2010 at 11:08pm2,000 invites sent on 5/2/2010 at 1:55am773 invites sent on 5/2/2010 at 9:41pm1 invites sent on 7/2/2010 at 11:35pm1 invites sent on 7/2/2010 at 11:41pm2,000 invites sent on 7/2/2010 at 11:42pm2,000 invites sent on 7/2/2010 at 11:42pm2,866 invites sent on 8/2/2010 at 12:11pm622 invites sent on 8/2/2010 at 12:24pm13,010 invites sent on 8/2/2010 at 6:10pm6,612 invites sent on 10/2/2010 at 01:04am2,000 invites sent on 10/2/2010 at 06:46am2,000 invites sent on 10/2/2010 at 06:46amInterviews achieved per dayBelow are the interviews achieved per day for the November 2007, February 2008 andFebruary 2009 surveys.November 2007DateInterviews achieved13 November 2007 1,20614 November 2007 58015 November 2007 13116 November 2007 3017 November 2007 2418 November 2007 2719 November 2007 1Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 12

February 2008DateInterviews achieved11 February 2008 1,10912 February 2008 41213 February 2008 31114 February 2008 9015 February 2008 99There were 1,994 reminders sent on 15 February 2008 at 9:30am.February 2009DateInterviews achieved9 February 2009 55410 February 2009 27211 February 2009 43812 February 2009 12613 February 2009 38114 February 2009 6015 February 2009 3816 February 2009 1817 February 2009 3218 February 2009 1519 February 2009 80Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 13

A number of reminder emails were also sent out in February 2009, as outlined below.February 2009DateReminders sent11 February 2009 3,09913 February 2009 2,79016 February 2009 74717 February 2009 37518 February 2009 151February 2010DateInterviews achieved29 January 2010 130 January 2010 5931 January 2010 131 February 2010 2932 February 2010 2253 February 2010 2694 February 2010 3215 February 2010 2486 February 2010 877 February 2010 828 February 2010 3139 February 2010 12210 February 2010 10811 February 2010 3412 February 2010 10Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 14

A number of reminder emails were also sent out in February 2010, as outlined below.February 2009DateReminders sent1 February 2010 14,1902 February 2010 3823 February 2010 8094 February 2010 8825 February 2010 1,1598 February 2010 1,40710 February 2010 13,58511 February 2010 3951.3 Influence of weatherMethodological limitationsWhen measuring sun protection behaviours, it is likely that the prevailing weather conditionswill be very influential. That is, the adoption of sun protection behaviours will vary dependingon the time of year (i.e. seasonal variation), and also on the weather. Similarly, awarenessand salience of sun protection, and recall of what precautions one would ordinarily take on atypical summer day, might also be expected to be influenced by these factors. Tan-seekingand burning are also likely to be influenced by time of year and weather conditions. Given theneed to collect data at different times, and monitor the progress of the campaign over time,these problems are unavoidable. However, the impact of weather variation needs to be takeninto account when interpreting the results.Prevailing weather conditions in each fieldwork periodThe questionnaire asked participants about whether they had deliberately sought a tan in thelast fortnight, and also whether they had experienced any reddening of the skin after being inthe sun during the last two weeks. Accordingly, not only was weather information recorded forthe fieldwork periods themselves, but also for the two weeks prior.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 15

The average temperatures during and two weeks prior to the fieldwork periods are shown inTable 1.3.1. The (unweighted) average temperature across capital cities during the November2006 fieldwork, and the two weeks prior, was 19.0˚C. This was similar to the November 2007average national temperature, which was 20.5˚C. Unsurprisingly, average nationaltemperatures were higher for the February surveys, which were 23.4˚C in 2007, 23.0˚C in2008, and 24.3˚C in 2009 and 2010.Table 1.3.1 Average temperature (°C) across capital cities during fieldwork periods andpreceding fortnightsNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Canberra 15.7 22.3 17.1 22.6 22.6 23.2Sydney 19.0 23.1 20.0 22.1 23.0 24.4Melbourne 15.1 22.5 18.8 20.4 24.3 22.4Brisbane 22.3 25.3 22.4 25.0 25.8 26.5Adelaide 17.6 24.9 21.0 22.2 27.8 24.8Perth 21.1 24.0 20.7 27.0 25.4 25.5Hobart 11.7 16.9 14.6 16.6 17.7 18.4Darwin 29.8 28.4 29.2 28.4 27.8 29.0Average acrosscities19.0 23.4 20.5 23.0 24.3 24.3The average amount of rainfall and the proportion of wet days that occurred during each of thefieldwork periods and the preceding fortnights are shown in Tables 1.3.2 and 1.3.3,respectively. In November 2006, the average national rainfall across Australian capital citieswas 1.8mm per day during and just prior to the fieldwork period. The following year, theDepartment of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 16

average was 2.4mm per day during the November fieldwork and the two weeks prior.However, November 2007 had a lower proportion of rainy days compared to November 2006.Thus, overall, there does not appear to have been a great difference between November 2006and November 2007 in terms of wet weather and the corresponding opportunities for exposureto the sun.In contrast, both February 2008 and February 2009 were substantially wetter than February2007 and February 2010. In February 2007, the average rainfall across all capital cities was2.5mm and in February 2010 the average was 2.8mm, compared with 5.6mm in February 2008and 4.4mm in February 2009. However, there was a higher proportion of wet days during andjust prior to the fieldwork in February 2008 than there were in either the previous orsubsequent Februarys. The fact that it was much wetter in February 2008, compared withFebruary 2007, February 2009 and February 2010, needs to be taken into account wheninterpreting the results.Table 1.3.2 Average rainfall (mm) across capital cities during fieldwork periods and precedingfortnightsNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Canberra 2.1 0.8 3.2 2.7 0.2 1.0Sydney 2.3 3.1 4.6 10.5 5.7 5.4Melbourne 1.3 0.1 1.8 0.8 0.1 2.2Brisbane 4.5 3.6 2.4 11.5 5.3 4.7Adelaide 0.8 0.0 1.5 0.0 0.03 0.2Perth 0.3 1.7 0.1 2.5 0.02 0.01Hobart 1.3 0.0 0.2 1.1 0.3 1.0Darwin 1.6 10.8 5.8 15.9 23.8 8.0Average acrosscities1.8 2.5 2.4 5.6 4.4 2.8Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 17

Table 1.3.3 Average proportion of wet days across capital cities during fieldwork periods andpreceding fortnightsNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Canberra 37% 32% 38% 47% 21% 21%Sydney 63% 32% 43% 63% 42% 48%Melbourne 42% 11% 24% 42% 11% 21%Brisbane 32% 47% 57% 68% 42% 41%Adelaide 32% 0% 19% 5% 5% 14%Perth 26% 16% 10% 11% 11% 3%Hobart 68% 11% 19% 32% 32% 21%Darwin 32% 84% 67% 95% 100% 55%Average acrosscities41% 29% 35% 45% 33% 28%Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 18

As raised by a project stakeholder in 2008, ‘oktas’ or cloud cover is an alternative indicator ofweather, and one that is perhaps more likely to influence sun protection behaviours. Cloudcover is measured in units of ‘okta’, where a day that has an okta level of over 6 is regarded asa cloudy day. Table 1.3.4 reports the proportion of cloudy days across capital cities during andjust prior to the research fieldwork. The data shows that cloud cover during the November2006 and November 2007 fieldwork was reasonably consistent with approximately a third ofdays (36% and 33% respectively) during the research period being cloudy. There was agreater proportion of cloudy days during the February 2008 fieldwork period, with a nationalaverage of almost one in two days being cloudy (48%) compared to the corresponding periodsin 2007 (28%), 2009 (43%) and 2010 (39%). This is unsurprising considering the higherproportion of wet days and levels of rainfall over this period noted above.Table 1.3.4 Average proportion of cloudy days across capital cities during fieldwork periods andpreceding fortnightsNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Canberra 42% 15% 38% 50% 40% N/ASydney 42% 35% 38% 65% 52% 55%Melbourne 58% 20% 29% 55% 20% 28%Brisbane 32% 25% 48% 55% 44% 38%Adelaide 42% 10% 24% 15% N/A N/APerth 26% 10% 0% 25% 16% 21%Hobart 37% 40% 48% 40% 36% 28%Darwin 11% 65% 43% 80% 92% 66%Average acrosscities36% 28% 33% 48% 43% 39%Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 19

The average UV indices during and just prior to the fieldwork periods are reported in Table1.3.5. The UV index levels were relatively stable across capital cities during the correspondingfieldwork cycles. The UV index, averaged across all capital cities, for the November researchwas 8.9 in 2006 and 9.1 in 2007. For the February cycles of fieldwork, the national averageswere 10.1 in both 2007 and 2008, 9.7 in 2009 and 11.3 in 2010.Table 1.3.5 Average UV indices across capital cities during fieldwork periods and precedingfortnightsNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Canberra N/A N/A N/A N/A N/A N/ASydney 8.3 8.9 8.7 7.8 9.0 11.4Melbourne 7.5 9.0 6.9 9.2 7.2 9.8Brisbane 9.9 10.9 10.1 10.4 10.3 11.2Adelaide 8.6 9.8 7.8 10.3 9.7 11.6Perth 8.5 9.6 9.1 11.0 11.1 11.5Hobart N/A N/A N/A N/A N/A N/ADarwin 10.5 12.2 12.1 12.0 10.9 12.0Average acrosscities8.9 10.1 9.1 10.1 9.7 11.3In summary, the number of wet, rainy and cloudy days during the fieldwork period is expectedto have the greatest influence on respondents’ behaviour. In this context then, the greaterproportion of wet and rainy days in February 2008, compared to the corresponding periods in2007, 2009 and 2010 is expected to have some impact on the results of the research.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 20

1.4 Participants and sample sizeThe pre-campaign and February 2007 surveys included both the primary and secondary targetaudiences. The November 2007, February 2008, February 2009 and February 2010 surveysdid not include parents, and so only included teenagers aged 14-17 years and young adultsaged 18-24 years. The sample was drawn to be nationally representative by state/territory andmetropolitan/non-metropolitan areas 2 .The final sample for each survey wave is illustrated in the Table 1.4.1.Table 1.4.1 Sample sizeNov 06 Feb 07 Nov 07 Feb 08 Feb 09 Feb 10Total samplesize3,082 3,097 1,999 2,021 2,013 2,185Teenagersaged 14-17years1,105 1,096 993 1,010 1,009 1,060Young adultsaged 18-24years1,120(including120parents)*1,123(including123parents)*1,006 1,010 1,004 1,125Parents ofchildren aged0-17 years(parents 18years+)977 1,001 Nil Nil Nil NilIt is worth noting that using sub-samples of 1,000 yields a 95% confidence interval of, at most,just over ±3% for a stand-alone survey. When comparing between surveys or segments, asample of 1,000 yields a maximum confidence interval of ±4.4% at the 0.05 level. Theseconfidence intervals are conservative, because they are based on a proportion of 50%. Largeror smaller proportions will yield a narrower confidence level.2 While this was achieved in the last four of the five waves, there was oversampling of participants fromMelbourne and undersampling of participants from Sydney in the baseline survey. Hence, the data wasweighted to correct for this.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 21

QuotasQuotas were applied to age and gender: n=500 14-15 years, split by gender (50:50). n=500 16-17 years, split by gender (50:50). n=1,000 18-24 years, split by gender (50:50.)In earlier waves the quota of 250 males aged 14-15 years was not always reached (as reportedin Section 1.9), and so additional females within this age category were used to make up forthe slight shortfall. There was also some difficulty reaching the quotas of 250 16-17 year oldsmales and 500 18-24 year olds males in the February 2010 wave. In this case, rather thanusing additional females, recruitment simply continued until the quotas were filled. Thisresulted in an over-sampling of females 16-17 (n=307) and females 18-24 (n=624).Precise details of the number of interviews achieved with males and females within the 14-15,16-17 and 18-24 year age bands, for each survey wave, are shown in the Table 1.4.2.Table 1.4.2 Interviews achieved by wave for 14-15, 16-17 and 18-24 year oldsSurvey wave Males Females Total14-15 year oldsNov 06 199 310 509Feb 07 252 326 578Nov 07 161 290 451Feb 08 251 256 507Feb 09 250 254 504Feb 10 251 251 50216-17 year oldsNov 06 288 308 596Feb 07 249 269 518Nov 07 278 264 542Feb 08 250 254 504Feb 09 250 255 505Feb 10 251 307 558Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 22

Survey wave Males Females Total18-24 year oldsNov 06 538 582 1120Feb 07 540 583 1123Nov 07 501 505 1006Feb 08 504 506 1010Feb 09 499 505 1004Feb 10 501 624 11251.5 QuestionnaireQuestionnaire development, revisions and learningsThe questionnaire (included at Appendix A) was developed in close consultation with theDepartment of Health and Ageing, to ensure that information needs were prioritised andappropriately addressed. There was no piloting of the survey, although its development wasbased on the findings from the qualitative research, and was also informed by other surveyswithin the field of sun protection.Care was taken to ensure that any changes in the questionnaire from one survey to the nextwere minimised, to allow comparability over time. Specifically, the questions for parents werenot included for the November 2007, February 2008, February 2009 and February 2010surveys, and the stimuli were changed to reflect the inclusion of new executions in phase twoof the campaign. The questionnaire in Appendix A makes it clear which questions and stimuliwere included in each of the surveys.Participants were asked where they were, and what they were doing, last time theyexperienced any reddening of the skin after being in the sun. Despite being asked to answerboth parts of the question, in previous years some participants only answered one component.In addition, some responses were not useful, such as people responding with a suburb locationwhen asked “Where were you?”. As such, in the 2010 survey participants were provided with aseparate response box for each answer, one of which was labelled ‘location’ and one of whichwas labelled ‘activity’. While this increased the number of people answering both parts of thequestion, some of the responses were still not very useful. Most notably, respondentscontinued to give the name of a suburb in the ‘location’ box.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 23

Average survey lengthThe average survey durations were 8.8 minutes for the pre-campaign survey and around 14-15minutes for each of the subsequent surveys, the extra time needed for recall of, and reactionsto, the elements of the campaign. The average survey length for the Feb 10 wave was 15.9minutes.1.6 Response ratesThe response rate for each survey is shown in the Table 1.6.1.Table 1.6.1 Response ratesSurvey waveResponse rateNov 06 12.9%Feb 07 10.3%Nov 07 8.4%Feb 08 11.6%Feb 09 10.9%Feb 10 6.2%These rates are lower than a typical response rate for online surveys. As an indication, thetypical response rate for a 15 minute survey on a health topic is generally 20% or higher foradults. That said, in all forms of survey research, the response rate among younger agecategories is generally lower than for older age categories. Also, a higher response rate couldpotentially be achieved with a longer field period.It is also worth noting that, when response rates are calculated for telephone surveys, theseare based on the number of people who agreed to participate in the survey as a proportion ofall those with whom contact was made. It is impossible to calculate the analogous figure foronline research. The response rate represents the number of people who participated beforethe quotas were filled, as a percentage of all those who were sent an invitation. Some of theseinvitation emails were likely to have not been opened before the quotas were filled. This is notto say that these people would necessarily have refused to participate in the survey. (Thecomparable percentage in telephone surveys would be the number who agreed to participate asa percentage of all numbers dialled.)Finally, it should be noted that the response rate for the Feb 2010 survey was much lower thanin previous years. The reason for this lower response rate is two fold:Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 24

Unlike in previous years, the parents of 14-17 year olds were contacted in an effort to fillquotas. This approach had a very low response rate as a large number of parents werecontacted to achieve a small number of interviews.Respondents who had participated in any of the previous survey waves were not recontacted.As such, at least some of the panellists contacted this year were less responsivethan the ones contacted in previous years.The proportion of questionnaires opened, questionnaires completed, and the percentage withinthe target group is shown in Table 1.6.2.Table 1.6.2. Percentage of invitations opened, questionnaires completed, and the percentage ofparticipants in the target groupNov 07 Feb 08 Feb 09 Feb 10Invitations opened 12.1% 14.7% 12.5% 6.4%Completions from allinvitations5.8% 7.9% 6.4% 2.5%Participants in targetgroup from allinvitations opened 92.3% 91.1% 91.0%85% (not includinginvites sent to parentsof 14-17 year olds)All three panels used conduct on-going data quality controls that ensure all panellists’ detailsare valid. In addition, panellists join via double opt-in process in which their email address isverified before entering the panel. Every survey is subjected to validation checks. Thisvalidation process includes cross-checking survey data against panellist recruitment data andchecking the time taken to complete the survey and each of its sections.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 25

1.7 Statistical analysisStatistical comparisonsStatistical tests were undertaken to compare the results across the five surveys. Threeseparate sets of comparisons have been conducted in 2009, as illustrated in Figure 1.7.1, whichfocuses on the final phase of campaign activity.Figure 1.7.1 Statistical comparisons undertaken after all six surveysNovember2006February2007November2007February2008February2009February2010Media ActivityMedia ActivityMedia ActivityMedia ActivityProgress to dateCumulative effect of Summer 07/08, 08/09 and 09/10Effect of Summer 09/10Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 26

Previous reports contain the results of other statistical comparisons conducted across the firstfour surveys.Figure 1.7.2 Statistical comparisons reported previouslyNovember2006February 2007November2007February 2008February2009MediaMediaMediaImpact of first wave of mediaResidual effect of first wave, controlling forProgress made by FebruaryEffect of second wave of mediaIncremental effect of second wave of media activity, controlling for seasonProgress made by FebruaryEffect of third wave of mediaIncremental effect of third wave of media activity, controlling for seasonFor categorical data, chi-square tests have been used. For ordinal data, Kendall’s tau-b hasbeen employed. In the case of interval data, ANOVA tests have been undertaken to comparemeans, as appropriate.WeightingDuring the analysis, the data were weighted to reflect the geographical distribution of Australia,and also to correct for any gender bias within each target audience. Specifically, there was:1. a weight calculated for gender 3 :The total number of males aged 14-17 years, and the total number of females aged14-17 years, in the Australian population was used to determine how the sample of14-17 year olds should be weighted.The total number of males aged 18-24 years, and the total number of females aged18-24 years, in the Australian population was used to determine how the sample of18-24 year olds should be weighted.2. a weight calculated for location 4 :3 Based on 2001 Australian Bureau of Statistics census data.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 27

The percentage of the population within NSW/ACT, VIC, QLD, SA/NT, WA, and TASwas used to determine how the sample should be weighted to correct for locationskews within each age band (i.e. 14-17 years and 18-24 years).A combined weight for location and gender was then used for the analysis.Note on tables and figuresIt should also be noted that, in tables and figures, the sum of the component items may notappear to add to 100% due to the effects of rounding.1.8 Notes on the data fileSPSS has been used for the statistical analysis, and the Department has been provided with acopy of the SPSS data file, as well as data tables. All variables have been clearly labelled,although there are two things that need to be noted:1. For questions regarding use of sun protection methods in specific contexts, someparticipants indicated that one or more of the specific situations was not relevant tothem (e.g. when asked what sun protection behaviours they adopt at the beach,some said “not relevant”). Data were classified as “system missing” for thesepeople, meaning that the data is presented and analysed only for those for whomthis situation was relevant.2. There are two location variables, called "state" and "metro_non-metro". In the precampaignsurvey, only postcode data was collected. Because there were one ortwo responses that could not be classified, there was a question included insubsequent waves at the beginning of the survey, which asked participants wherethey live. The "state" variable is based on the postcode data for survey 1, but isbased on this additional question used in surveys 2 to 6.1.9 Sample characteristicsThe following figures show the sample characteristics of the unweighted sample.Figure 1.9.1 shows that the sample was somewhat unbalanced in terms of gender for eachtarget audience, particularly in the earlier and most recent waves. The analyses were weightedto correct for the gender imbalance.4 Based on 2004 Australian Bureau of Statistics data.Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 28

Figure 1.9.1 Age and gender100MaleFemale80%6056 54 5644 46 4450 50 50 50 53 52 5247 48 48 50 50 50 50 50 50455540200Nov'06Feb'07Nov'07Feb'08Feb'09Feb'10Nov'06Feb'07Nov'07Feb'08Feb'09Feb'1014-17 yrs 18-24 yrs14-17 years, Wave 1: n=1,105; Wave 2: n=1,096; Wave 3: n=993; Wave 4: n=1,011; Wave 5: n=1,009; Wave 6: n=1,06018-24 years, Wave 1: n=1,120; Wave 2: n=1,123; Wave 3: n=1,006; Wave 4: n=1,010; Wave 5: n=1,004; Wave 6: n=1,125Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 29

Figure 1.9.2 shows the distribution of the five survey samples, across metropolitan and nonmetropolitanareas. In the pre-campaign survey, there was an overrepresentation of peopleliving in Melbourne, and an under-representation of those from Sydney. Accordingly, the datahave been weighted to correct for this imbalance, and to reflect the distribution of theAustralian population.Figure 1.9.2 Location40Non-metroMetro309 11 1010 10 5%20745 5 6527 25 26 28 28 257 66 66 721 22 22 22 24 103171 2 213 14 111 12 10 10 8 7 7 7 702 12 12119 867 76 7 031 111 2 1 1Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10NSW/ACT VIC QLD SA/NT WA Tas14-24 year olds, Wave 1: n=2,225; Wave 2: n=2,219; Wave 3: n=1,999; Wave 4: n=2,021; Wave 5: n=2,013; Wave 6: n=2,185Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 30

The income distribution of the sample is shown in Figure 1.9.3. A large proportion ofparticipants were unable to estimate their household’s income, reflecting the fact that aroundhalf of the sample was comprised of teenagers.Figure 1.9.3 Household income4037 3836 35 353025%2010201816 15 14 15 1516 16 15 15 1610 10 11 11 12 12 139 9 9 8 8 8 9 9 8 86 6 6 6 6 660Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Up to $30,000 $30,001-60,000 $60,001-$100,000$100,001-$150,000Over $150,000 Refused Unable toestimate14-24 year olds, Wave 1: n=2,225; Wave 2: n=2,219; Wave 3: n=1,999; Wave 4: n=2,021; Wave 5: n=2,013; Wave 6: n=2,185Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 31

Those aged 16 years and above were asked about their employment status. The results areshown in Figure 1.9.4.Figure 1.9.4 Employment status10080%60405653 52 50 51 512022 25 20 22 21 24 17 17 19 18 16 154 4 4 4 5 5 3 2 3 2 2 3 0.30.30.20.2 0 0 1 2 2 2 1 10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10StudentWorking fulltimeWorking parttimeor as acasualUnemployedor looking forworkHome duties Retired Other14-24 year olds, Wave 1: n=2,225; Wave 2: n=2,219; Wave 3: n=1,999; Wave 4: n=2,021; Wave 5: n=2,013; Wave 6: n=2,185Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 32

Figure 1.9.5 shows that just under half of those in the sample described themselves as very fairor fair.Figure 1.9.5 Skin colour when not tanned50%4030201010 8 9 9 10 11 36 38 38 3937413331 3133 33 3217 19 19 16 17 143 3 2 3 3 20.40.30.30.40.30.30Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Nov '06Feb '07Nov '07Feb '08Feb '09Feb '10Very fair Fair Medium Olive Dark Black14-24 year olds, Wave 1: n=2,225; Wave 2: n=2,219; Wave 3: n=1,999; Wave 4: n=2,021; Wave 5: n=2,013; Wave 6: n=2,185Department of Health and AgeingEvaluation of Skin Cancer Campaign | 2010 | PAGE 33