Information security awareness initiatives: Current practice and the ...

Information security awareness initiatives:Current practice and themeasurement of successJuly 2007

PrefaceThe European Network and Information Security Agency(ENISA) is a European Union Agency created to advancethe functioning of the Internal Market. The Agency’smission is to achieve a high and effective level of networkand information security within the European Union.ENISA commissioned PricewaterhouseCoopers LLP(PwC) to develop this report to offer a perspective onwhat governments and private companies are currentlydoing for assessing the impact and success of awarenessraising activities.This study is intended to be used by professionals withinorganisations and public bodies that are tasked withplanning, organising, and delivering information securityawareness initiatives.The study has focused on cultural change, the ways inwhich sets of metrics and key performance indicators(KPIs) can pay off, and how assessing methods(qualitative and quantitative) can contribute to thedevelopment of a wider culture of security.This involved gathering information on the currentpractices of a number of European governmentdepartments and companies, to:• Provide an outline analysis of recommendedsecurity awareness practice and metrics to measureawareness;• Provide an outline of key metrics that can be used toeffectively assess awareness, as well as some highlevel;• Convey the results of the survey to assess what entitiesare doing with regards information security awareness;• Provide case studies of good practice for awarenessand measurement of effectiveness or to highlightinformation of benefit; andThe research was carried out during May to July 2007using a structured questionnaire. This was made availableon a self-select basis to people responsible for informationsecurity in European government departments andcompanies. In total, 67 organisations headquartered innine different European countries responded. Many ofthese had operations in several European countries. Thesize of the organisations varied from less than 50 staff tomore than 10,000 staff. There was a spread of responsesacross all industry sectors. PwC then interviewed 12 ofthe 67 respondents in depth and wrote these interviewsup as case studies.This report, therefore, gives an indication of whatEuropean organisations are currently doing to measureand improve information security awareness. Becauseof the self-select nature of this study and limited samplesize, the results should not be interpreted as statisticallyrepresentative of European businesses and governmentdepartments as a whole.About ENISAENISA is a European Union Agency created to advancethe functioning of the Internal Market by advising andassisting Member States, EU bodies and the businesscommunity on how to ensure a high and effective level ofnetwork and information security. ENISA also serves as acentre of expertise for Member States and EU institutionsthat facilitates information exchange and cooperation.Contact detailsIsabella Santae-mail: awareness@enisa.europa.euInternet http://www.enisa.europa.eu• Contribute to the development of an informationsecurity culture in Member States by encouragingorganisations to act responsibly and thus operate moresecurely.Research carried out for ENISA by:The member firms of the PricewaterhouseCoopers network (www.pwc.com/uk) provide industry-focused assurance, taxand advisory services to build public trust and enhance value for its clients and their stakeholders. More than 140,000people in 149 countries share their thinking, experience and solutions to develop fresh perspectives and practical advice.Unless otherwise indicated, ‘PricewaterhouseCoopers’ refers to PricewaterhouseCoopers LLP a limited liability partnershipincorporated in England. PricewaterhouseCoopers LLP is a member firm of PricewaterhouseCoopers International Limited.

ExecutivesummaryThis report analyses how organisations and governmentswithin the European Union (EU) are approachinginformation security awareness and the measurement ofeffectiveness. The report covers three main areas.The first part of the study looks at the importance ofinformation security awareness and specific topics torespondents (see pages 3 to 7). The main findings are:• Information security is seen as a high or very highpriority in four fifths of respondents;• Much of this is driven by a need to provide assuranceto customers that their sensitive data is protected.Identity theft is a significant concern;• There is also widespread recognition that respondentsare now heavily dependent on technology, and theInternet in particular. This leaves companies moreexposed to information security threats than ever;• In addition, there is increased regulatory focus on thisarea, both inside the EU and beyond;• The consensus is that the most important topics forstaff awareness are email, physical access, passwordsand the Internet; and• Instant messaging and clear desk policies are the leastfavoured topics.The second part considers techniques to raise informationsecurity awareness (see pages 8 to 13). The main findingsare:• Almost every respondent has defined their securitypolicies, either in their staff handbook or a separatesecurity policy. 85% of respondents have set upan intranet site that provides guidance to staff oninformation security matters. These techniques areseen as low cost basic disciplines. However, alone theyare not effective ways to change staff behaviour;• Respondents find training to be the most effectivetechnique. 72% include security messages in inductiontraining for new staff. Ongoing training for existingstaff is much more patchy; the cost makes manyrespondents reluctant;• Half of respondents are using computer-based training(CBT), and two thirds of these have mandated it;benefits cited are cost-effectiveness, consistency ofdelivery and ability to measure results;• Despite the high priority given to security, manyrespondents find it difficult to justify significant spend onawareness programmes. Only a third of respondentsbuild a formal business case to justify this expenditure;of these, only half attempt to quantify the benefits thattheir awareness programmes will achieve, and very fewevaluate return on investment (ROI); and• Most respondents instead think of security awarenesstraining as something they just have to do, i.e. acompliance requirement. As such, their budget istreated as an overhead rather than an investment.Current practice and the measurement of success

Executive summaryThe final part reviews the mechanisms and techniquesthat are used to measure information security awarenessinitiatives (see pages 14 to 20). The main findings are:• A wide variety of different methods are used to measurethe effectiveness of information security awarenessinitiatives. Organisations appear to find it very difficult toput effective quantitative metrics in place;• There is little consensus on the most effectivemeasures. This is clearly an area where good practiceis evolving;• Ideally, respondents would like to be able to measureactual changes in staff behaviours resulting from theawareness activities. As a consequence, relatively fewrespondents find input metrics (e.g. number of visitorsto intranet site, number of leaflets distributed) helpful;• The most popular source of information on actualbehaviours is audit (internal or external); two thirds ofrespondents use policy breaches highlighted in auditreports as a measure. The auditors’ objective andsystematic approach was felt to make these reportsreliable sources of information;• Many respondents use their experience of securityincidents as a metric. The most common metrics arethe number of incidents caused by human behaviourand root cause analysis of the most serious incidents;more than half of respondents use each of these. Manyother respondents, however, have abandoned securityincident statistics as a measure of security awareness,since there are many other factors involved;• A third of respondents include questions on securityawareness in staff surveys. They then measureawareness levels before and after initiatives take place.However, some respondents highlight issues with thecomplexity of collecting and processing this data; and• Some metrics are used because they provide insightinto actual behaviours (e.g. scans or tests). Othersare adopted because they resonate with the seniormanagement that sponsor awareness programmes(e.g. cost of incidents).Each organisation needs to find the right balance forthem; there is no “one size fits all” solution. Keeping theapproach simple tends to keep it cost-effective. Manycurrently struggle with quantifying security awareness;however, provided simple mistakes are avoided, abalanced set of key performance indicators (KPIs) andmetrics can provide real insight into the effectivenessof awareness programmes. Only with this insight areorganisations able to change their programmes froma compliance activity to one that really benefits theiroperations.Overall, an iterative approach to security awarenessprogrammes appears most effective, as illustrated below:Inputs:Information security policy; strategy; business case; riskassessment, budget; aims and objectives; legislation/compliancerequirementsSuccess factors:Executive sponsorship; whole business involvement; user buy-in;access to resources and time; cultural sensitivityTechniques:Face-face training; induction training; policy; intranet sites; CBTs;tests and quizzesSuccess factors:Relevance of material; ease of access and use; mandatoryover voluntary; targeted risk focused training; key managementinvolvementTypes:Security incidents/root cause; Results of audits; Survey of business;Test users' behaviours, Number of staff completing trainingSuccess factors:Know what you can measure; relevance of measurement; regulartimely assessmentENISA – Information security awareness initiatives:

Importanceof informationsecurityawarenessOrganisations, whether private or public, are increasinglystoring and making more information availableelectronically. There is a broad increase in reliance on ITsystems.This is coupled with an extraordinary increase in the useof Internet services. This is becoming an increasinglyimportant part of doing business. Lack of an Internetpresence can be detrimental to organisations’ businessobjectives.The increasing use of IT systems to store and processinformation makes keeping this information secure moreimportant. One of the key undertakings an organisationhas is to ensure that staff act in an appropriate manner.This includes staff acting to keep sensitive informationsecure.The Information Security Forum (ISF) is one of the world’sleading independent authorities on information security.Through surveys and research, the ISF have definedinformation security awareness as:‘an ongoing process of learning that ismeaningful to recipients, and delivers measurablebenefits to the organisation from lastingbehavioural change.’This information security awareness is a major componentwithin industry good practice for security. Severalinternational standards refer to this as a prerequisite:• ISO 27001;• COBIT;• Payment Card Industries – Data Security Standard; and• ISO 9001:2000.Some of the key drivers increasing the emphasis oninformation security awareness are:• Business requirements are changing, as use oftechnology (such as podcasts) evolves;• Foreign regulators (e.g. the US and Singapore) areexpecting staff to receive awareness training;• The focus on security from regulatory bodies within EUMember States is increasing. A recent example is theUK information commissioner’s comments to UK ChiefExecutive Officers on “unacceptable privacy breaches”;• The threat from organised crime is on the rise. A recentreport on Internet security highlighted high levels ofmalicious activity across the Internet, with increases inphishing, spam, ‘bot’ networks, Trojans, and zero-daythreats. In the past, these threats were usually distinctand could be addressed separately. Attackers arenow refining their methods, so attacks tend to involvemultiple attack vectors. They are also consolidatingCurrent practice and the measurement of success

Importance of information security awarenesstheir assets to create global networks that support coordinatedcriminal activity;• Customers are more sensitive to security issues thanin the past. Adverse press coverage can cause majorimpact to an organisation’s reputation; and• Identity theft is an increasingly prevalent securityissue. Organisations that store and manage personalidentification information must take care to ensurethe confidentiality and integrity of such data. Anycompromise that results in the leakage of personalidentity data could cause loss of public confidence,legal liability, and/or costly litigation.Given these drivers, it is not a surprise that four fifths ofrespondents rate information security as a high or veryhigh priority to their senior management. This is similar tothe proportion noted in other recent security surveys.Importance of information securityNeither highnor lowpriority16%Low priority4%50%High priority30%Very high priorityInformation security is wide ranging and has manyvaried topics. Their importance to different organisationsdepends on the nature of the risks they face. Forexample, financial services and technology respondentsshare concerns over passwords; however, phishing ismore of a concern in financial services and patching moreimportant to technology companies.The priority given to information security appears to relatemore to the attitude of senior management than the sectorin which the organisation operates (hence the risks towhich it is exposed). For example, most governmentdepartments responding say security is a very highpriority to their senior management; one, however, ratedit as a low priority and seems to be carrying out the bareminimum necessary to comply with mandatory guidance.Investment bank – to changebehaviours, training needs to beinteractiveAn investment bank explained that its primaryobjective is to achieve regulatory compliance in acost-effective way.This is not possible without the creation of clearpolicies that set out what individuals shouldand should not do. Without this foundation,enforcement and discipline become hard if thingsbreak down. The bank has, as far as possible,included information security points in existingpolicies and training, rather than creating newones.Policies themselves are not effective unless staffunderstand them. The bank’s security team givesinduction presentations to all new joiners thatexplain the bank’s security policies. This face-tofacecontact gives staff an opportunity to discusspossible issues with the security team. Feedbackfrom the training shows that interaction is criticalto challenging people’s attitudes and helping themlearn. If people are asking questions, they arethinking and considering the information. A roomfull of silent people is unlikely to be learning much.Sharing war stories and relevant experiences helpsstaff see how security threats might affect them.The bank has found that induction training aloneis not enough. It is important that staff receivefrequent reminders that reinforce key messagesin a coherent way. Critical to this reinforcementhas been getting senior management to lead byexample; they, rather than the security team, arethe best people to promote the importance of themessages.The security team uses a variety of techniquesto reinforce awareness messages on an ongoingbasis. Quizzes and prizes get a good responselevel from staff; they get people thinking, andare well received within the business. Again,interaction with staff is vital. For example, postersthat are passive reminders and ultimately requireno individual action are often ignored in practice.Intranet articles and sites are good ways topromote messages to those that already activelyuse them. However, for people who do not visitthem (the majority of staff), they are not aneffective mechanism.4 ENISA – Information security awareness initiatives:

How important or unimportant is it to your business to ensure that staff are awareof each of the following information security topics or risks?Very importantImportantNot very importantNot at all importantEmail and electronic communicationsPhysical security/access to buildingsPasswordsInternet securityResponsibilities for information securityVirusesSoftware licensing and copyrightSecurity incident reportingSecurity updates and patchesMobile phones and PDAsSecurity out of the officePersonal use of corporate equipmentPhishingClear desk policyInstant messaging-20 -10 0 10 20 30 40 50 60 70Number of respondentsTraditionally, financial services companies have beenleaders in security practice. Our respondents confirm thatsecurity remains a high priority to most financial servicesboards; however, in some security still seems to be drivenbottom-up rather than top-down.One company rating security as a low priority sums upthe attitude of their senior management as taking the viewthat nothing bad has happened yet and so why spendmoney.In contrast, those at the other end of the spectrum areprincipally motivated by customer perception and thedamage to their reputation that a breach might cause.Overall, most respondents agree that four topics are veryimportant for staff to understand:• Email and electronic communications;• Physical security/access to buildings;• Passwords; and• Internet security.For each of these, more than half of respondents ratethem as very important; roughly nine tenths rate them asvery important or important.Passwords remain the primary authentication methodto IT systems. Given concerns about potential privacybreaches, the need for staff to adopt good passworddisciplines is high. There have been many recent pressstories involving inappropriate emails or Internet usage.The reputations of many companies and governmentdepartments have suffered as a result. Making sure thatstaff are aware of what the organisation considers to beacceptable usage is critical here.Avoiding easily guessable passwords, keeping passwordssecure and not sharing them are all important elements ofawareness training.In the light of the rise in terrorist activity over the lastdecade, it is perhaps unsurprising that physical securityis so high up the list. Particular issues here include tailgating,escorting visitors and granting/rescinding accessto temporary staff.Interestingly, the respondents that rate email and physicalsecurity as not very important are from the public sector.Attitudes to these specific areas may be more relaxedhere.For most of the other topics, roughly four fifths ofrespondents rate them as either very important orimportant.Some of these topics (responsibilities for informationsecurity within the organisation and security incidentreporting) are seen as basic information that staff neededto know.Viruses and patching are a particular concern in thetechnology sector. The days of indiscriminate Internetworms are past. Businesses today are subject tosophisticated targeted attacks by programs that hide fromdetection and gather confidential information. Staff needto be aware of the changing nature of this threat.Current practice and the measurement of success 5

Importance of information security awarenessMobile Phones and PDAs (Personal Digital Assistants e.g.Blackberries) are a particular issue for financial servicesrespondents. Organisations in this sector can make andlose money in short timeframes. Information tends to bemore time critical to them and their staff. They, therefore,tend to be leaders in adopting technologies that provideinformation to staff right now.There are two clear topics that are of perceived leastimportance to organisations. These are promotion of aclear desk policy and instant messaging.The low priority given to awareness of clear desk policiesis, perhaps, understandable. Many companies simply donot adopt or enforce such policies. They feel their physicalaccess controls mitigate the risks sufficiently.The low priority given to the use of instant messaging ismore of a paradox given the high importance attributed toemail. Both provide a mechanism for people to connectdirectly with external parties and to transfer information tothem. They would appear to be very similar in nature andrisk.There is a clear risk of uncontrolled distribution ofconfidential information through both media. Indeed,it could be argued that instant messaging poses ahigher risk than the use of email, since email filters areoften more sophisticated. It may simply be that somerespondents have blocked instant messaging technologyfrom working in their organisation, so do not feel theyhave to make staff aware of the risks.International insurer – senior management commitment makesa big differenceAn insurance company explained why information securityis important to their business. They collect, store, andprocess significant amounts of financial, medical, andpersonal information. This information is their number oneasset; confidentiality breaches could put their reputationat risk, as well as exposing them to harmful litigation.Unfortunately, the threats (such as identity theft andscams) are rising; this makes staff awareness vital.The main challenge has been to develop an approachthat is suitable for over 10,000 employees speaking manydifferent languages. To counteract this, the companyengaged an external provider to help them build suitabletraining plans and materials. To create the greatest impactwith staff, training materials were translated into the localmother tongues of the countries concerned.There is a continual programme to adjust and promote thekey messages. The objectives of this are to try to changepeople’s behaviour and perception of risk. Numeroustechniques are used to reach the audience, since differentpeople learn by different mechanisms.The most effective technique has been face-to-facetime with staff through workshops and training sessions.Being able to put a face to a name or function is morepersonable and people are more receptive to messagesbeing face-to-face. The training is mandatory. Seniormanagement actively support the awareness schemes,making sure training events are at convenient times forthe business and promoting them to staff. There is goodattendance at sessions since missing the events resultsin escalation to the employee’s manager. This seniormanagement support across the business has proved tobe critical to the success of the awareness programme.Other non-interactive mechanisms, such as intranetarticles, emails, posters and publications, are used toreinforce important messages. However, it has proveddifficult to gauge how many people have read orunderstood the messages and people can easily ignorethem. So, they are used as a complement to, rather than asubstitute for, classroom training.The main measure of the impact of the awareness trainingis feedback and questionnaires completed on or shortlyafter training sessions. This feedback gives a good insightinto the impact of the training on the individual. Generallythis has been positive, with the vast majority saying thatthey have learned something new and will try to changetheir behaviours.Other ways to test awareness, such as checking thestrength of passwords or mocking up social engineeringtype situations to gauge responses, have beenconsidered. However, these are not used, due to concernsabout dependence on other variables (such as the moodof the person), privacy and entrapment.The company is now focused on ensuring that trainingcontinues to engage people; e-learning modules are beingdeveloped to add variety. A continual process is underwayto enhance the relevance of the material to staff, so theycan see the benefits and understand the risks moreclearly.ENISA – Information security awareness initiatives:

International financial services group – changing times drivechanging needsA large international financial services group explainedwhy a new approach to information security awarenesshas been implemented. The firm’s objective is forcustomers and staff to view the firm as the safest placeto do business. The firm believes good security is goodbusiness.Given its size and the diversity of its operations, the firmand its customers are subject to continually changingthreats. Fraudsters have always targeted banks, but theincreasing use of the Internet has changed the nature ofthese fraud risks; keeping losses to customers and thefirm under control is a strong driver for security.There also appears to be a shift in the regulatory andcultural environment. Countries outside the EU (such asthe US and Singapore) already have more prescriptiverequirements for information security training. The climatewithin the EU appears to be changing. Informationsecurity and privacy are becoming more important onpeople’s agendas. In this changing environment, the bankwants to make sure it is ahead of the curve.This has driven some changes to their global awarenessstrategy over the last year. Corporate information securitypolicy has been altered and awareness and trainingare now mandatory. Job descriptions and individuals’objectives are being tailored to include informationsecurity responsibilities.A challenge is the size and scope of the different divisionsof the company. A centralised team is now in place toco-ordinate the awareness and training strategy and settraining standards for information security awarenessacross the firm. Individual business units are thenresponsible for implementing the policy and standards intheir local operations.The firm has found that the most important thing is to havea structured approach, and not just do things in an ad-hocfashion. In this vein, the firm uses a variety of techniquesto keep the messages and media channels fresh,including a security web portal. Keeping the materialrelevant and up-to-date has helped the effectiveness ofthe message. Currently, there is not much face-to-facetraining, although there are plans to include more ofthis later in the programme. This will be initially targetedat the key influencers and managers, so that it has thebiggest impact on the culture. If management buy intothe importance of security awareness, they will drive andpromote it within their business units.While some business units use computer-based training(CBT), they are not as widespread as was initiallyplanned. There were plans for a centralised global CBTsystem. However, due to the diversity of the business andthe cost of updating material, this was not implemented.Other techniques they have found to be ineffective are“free stationery”; pens, pencils, etc.Despite the very structured and clearly defined approachadopted, quantitative assessment of the impact andeffectiveness has proved problematic. An informationsecurity specific self assessment used to be carriedout regularly to gauge the level of awareness with staff.However, this was discontinued since it required a largeamount of resources to co-ordinate and analyse, andit was found that some of the results were misleading.People will answer surveys with the answers that theythink you want to hear and not what is actually goingon. The survey suggested staff knew procedures well;however, the results of internal and external auditsshowed that this was not always correct.The firm is now focusing on measuring and reporting ontraining, as well as watching the results of internal andexternal audits closely. Now that information securityawareness and training requirements are set in policy, thecentral team can review audits and compliance measuresto monitor the levels of awareness and the effectivenessof training.Current practice and the measurement of success

Approachesto raiseawarenessThe foundation for any framework for information securityawareness is a formal security policy. Without an outline‘law’ covering the use of systems and information,enforcing good behaviour is very hard.Good practice standards place a strong emphasis onhaving an organisation-wide security policy. For example,ISO 27001 suggests that organisations implement trainingand awareness programmes. There is a requirement ofmanagement to ensure that people working for them applysecurity according to polices. To accomplish this they arerequired to provide appropriate awareness training andregular updates in organisational policies and procedures,as relevant for the job function of all employees of theorganisation and, where relevant, contractors and thirdparty users.Incidentally, many standards also suggest or require thata company’s security policy should also include userawareness training.Recent surveys suggest that the number of companieswith a formal security policy in place has never beenhigher. Among our respondents, 88% have a specificsecurity policy, and a further 76% refer to securityrequirements in their staff handbook.A key component of any information security policy andawareness training is to analyse the threats and risks thatthe business faces. This analysis should drive the areasthat the policy and training need to cover.Every organisation faces changing environments, threatsand risks. To be effective, any awareness initiativesshould be supported by senior management. Ideally, itshould have board or executive level endorsement, toenhance the importance of the topic with staff. If seniormanagement do not treat awareness as important, it isunlikely that training will be successful.Most standards recommend that a formalised approachis adopted to information security awareness. A virtuouscircle involves three reinforcing elements:1. Requirements analysis: Management need to identifywhat topics staff need to understand. Users should bemade aware of the sections of the security policy thatare relevant (to their job function). Many standardssuggest topics to consider, such as spyware, virusoutbreaks and strong passwords.2. Training tailored to role: Both contractors andemployees should receive training, appropriatelygeared towards their role. They should also be regularlyupdated with any relevant changes to the securitypolicies or procedures. Training needs to addresshow staff can implement security in their day-to-dayprocedures.ENISA – Information security awareness initiatives:

There appear to be certain basic disciplines that everyorganisation should adopt. Almost every respondenthas defined their security policies, either in theirstaff handbook or a separate security policy. 85% of 9.31respondents have set up an intranet site that providesguidance to staff on information security matters. These 8.76techniques are low cost and so there is no reason not touse them.7.12However, many respondents believe policies, 6.57handbooksand guidance alone are not an effective way to improveawareness. It is simply unrealistic to 5.66 expect most staff toread and absorb all the information they are bombardedwith. These techniques serve a useful 5.47role in underpinningand reinforcing other awareness raising activities.4.93However, alone they are not effective ways to change staffbehaviour.4.38Respondents find classroom training to be the most4.38effective technique to change the way people behave.72% include security messages 4.38in induction training fornew staff. This reaches the highest risk people (newjoiners) and is relatively low 4.01 cost, since security aspectscan be incorporated into existing events.3.833.28While 10.77 classroom training is considered highly effective,relatively few respondents carry out ongoing training for10.4existing staff. This could be due to the perceived cost ofarranging and running these courses. Time is a preciouscommodity to busy business people. Getting sufficienttime to cover training needs may be very difficult. Themost effective awareness programmes appear to bethose that target their limited classroom training budget atthe highest risk populations. Blanket classroom trainingappears unlikely to be cost-effective.Instead, half of respondents are using CBT, and twothirds of these have mandated it to all staff. While there isan investment cost in setting up CBT, once it is running,the delivery costs are very low. It, therefore, lends itselfwell to ongoing training to a large population of existingusers. Consistency of delivery is usually better than withlarge classroom training programmes. Building tests intothe CBT also allows some measurement of how wellrecipients have absorbed the training.What techniques have you used to make staff aware of information security issues and their obligations?1.462.742.55A formally documented security policy has been published outlining securitysafeguardsIntranet site provides guidance on information security mattersSecurity requirements are covered in staff handbook or procedures manualsSecurity awareness training is built into the induction process when new staffjoin the organisationSecurity responsibilities are included in contract or letter of appointment fornew staffA specific document/leaflet (that covers information security policy) isdistributed to staffPoster campaigns on information security topicsFormal communication plan (i.e. how you will communicate with staff oninformation security awareness)Regular email or newsletter distributed to staffFormal analysis of target groups (i.e. which staff it is important to ensue havegood information security awareness)Other promotional material (e.g. screensavers, pens, mouse mats)Security messages are integrated into existing business training courses thatstaff attendOptional computer-based security awareness trainingMandatory computer-based security awareness trainingOptional classroom security awareness training27%36%36%33%31%40%36%46%45%54%58%72%76%88%85%Quizzes on security matters (e.g. offering prizes) in staff magazinesUse of external expertise (e.g. security awareness training vendors)Mandatory classroom security awareness training12%22%21%Current practice and the measurement of success 11

Approaches to raise awarenessRetailer – fitting in with the cultureA large retailer explained why being flexible in theapproach to information security awareness is important.They handle large volumes of information aboutcustomers, such as their financial and credit card details.However, the retail sector does not have as strong acompliance culture as many other industry sectorsalthough Data Protection and PCI compliance are key.The diverse nature of the work force makes delivering aneffective awareness programme challenging. The level ofcomputer literacy varies widely and the age of staff rangesfrom school leavers to retirement age. Messages needto be tailored accordingly. Staff broadly comprise of threedifferent groups. Firstly, in shops and outlets, staff dealwith customers and use tills and stock systems and havegenerally less IT experience. Secondly, most back officestaff and Head Office staff are ordinary users of computerequipment. Finally, the technical teams within IT that havepowerful access rights.A risk-based approach is used to define messages. Thekey risks that are present for each group of users areanalysed. Based on this, key messages for each year areidentified and communication plans put in place. Eachgroup faces different risks, so the messages for eachgroup are different.A wide range of techniques are used due to the diversityof the staff. Information security is built into staff inductiontraining; this ensures that people are informed of theirresponsibilities as they join. In store outlets posters havebeen particularly effective with good feedback from staff.In the last campaign, security messages were tied intoanother campaign running at the time and this approachwas effective. Using similar presentation of the informationfor both campaigns helped get a consistent messageacross to staff. Security needs to be part of, not separatefrom, the rest of the business.Face-to-face methods to promote awareness are notwidely used to raise security awareness within theorganisation especially within the store outlets. Given thelarge number of staff who use computers relatively little inthese locations, classroom training tends not to be costeffective.Cultural issues also have a big impact on the techniquesused to raise awareness. People do not expect to readlong policies or complex handbooks. What works insteadis delivering the key messages in a short snappy stylefor example via leaflets or posters or computer-basedtraining (CBT). Being sensitive and aware of what isappropriate for the organisation has improved delivery ofthe messages.Baseline awareness is reinforced by mandatory CBTwithin the Head Offices. The CBT includes tests; thetest scores are monitored. Surveys have been used tomeasure the success of campaigns; the number of staffknowing key messages is measured before and afterthe campaign. This information is then used to refine theprogramme.What techniques have proved effective at raisingMost effectiveinformation security awareness?Least effectiveClassroom training-214Induction process/appointment letterSecurity policy/staff handbookPoster campaignsRegular email or newsletterComputer-based trainingLeafletsIntranet siteQuizzes-9Promotional material (e.g. pens)-5-5Part of running an effective programme is targetingthe right messages to the right people. This involvesunderstanding each group’s current information securityissues and the extent to which they are aware of them.Surprisingly, only 36% of respondents have any formal-4-1-3-1-3-222233469Most effectiveLeast effectivemechanism for doing this. This happens more often withinfinancial services than other sectors. Many financialservices providers have learned the hard way that it ispossible to spend large sums of money on indiscriminateawareness activities without having much impact on theoverall risk profile. They now use a combination of blanketcoverage basic disciplines and target extra activity on theareas of greatest risk.Poster campaigns, promotional materials (such as pens)and blanket emails are each used by a significant numberof respondents. Many respondents had used thesetechniques in the past but have now abandoned or scaledback their use. They have a relatively short shelf-life andcan be expensive to distribute across the organisation.There is also a limit to how much information they canconvey to the reader, and many people simply ignore themcompletely.One in five respondents use surveys and quizzes to drumup interest and raise awareness. Of those that have triedthem in the past, more respondents found them effectivethan not. The proper use of incentives can achieve hightake-up and can really get people thinking about theirbehaviours.Implementing a successful security awareness programmecan be a difficult task. There may be some large or12 ENISA – Information security awareness initiatives:

seemingly insurmountable barriers along the way.What is most effective in the long term is being able torecognise any particular limitations in your efforts, suchas a lack of senior management buy-in or a culturalresistance within the organisation. Recognising potentialhurdles beforehand will enable plans to be put in place toovercome these obstacles.Law enforcement agency – ISOstandards can helpA law enforcement agency explained why putting inplace information security controls in line with ISO27001 has helped with awareness.They store and process information that could result inpeople’s lives being put at risk if compromised. As useof IT systems has increased over the years, the needfor information security has risen.Putting in place a structured approach from the startwas very important. Government guidelines andindustry good practice was combined to create newpolicies. A dedicated information security function wascreated.The initial adoption of new policies and proceduresbrought into focus areas where staff were not awareof the security risks. People started asking questions:Why change? Were the new procedures necessary?This showed that the existing training in these areaswas not sufficient. Based on this, the training andawareness programmes with HR were reviewed. Morerigorous risk focused inductions and training were putin place.Changes since the initial rollout are creatingnew challenges. Public sector organisations areincreasingly networked. Information is being sharedwith other departments. This makes security evenmore important. The agency’s primary focus is tocomply with government legislation and guidelines.The agency also reports on compliance with the ISO27001 standard to their regulatory body.Formal face-to-face training sessions have proved tobe the most effective way to raise awareness. Theyallow staff to get to know people within the informationsecurity function; they also enable the material to beput across in a more relevant way. Discussion getspeople to think about risks and different situations;this has been very useful in challenging and ultimatelychanging behaviours.Recently, computer-based training has beencommissioned. This will be mandatory - all peoplewithin the agency will have to complete it. Testsbuilt into the training will check users’ understandingand the delivery of key messages. Results will beanalysed; awareness training strategies and materialswill then be tailored to address any knowledge gapsidentified.Financial services group –reducing the training burden onstaffA large financial services company explained thatinformation security awareness has a high priority. Itis on the board’s agenda; they see it as important toretaining the trust of customers.One challenge is the high percentage of parttime staff and contractors. Another is the existingmandatory training burden on staff (anti-moneylaundering, data protection, anti-fraud, etc.).Linking information security awareness traininginto other on-the-job training activities has provedvital. The company has recently restructured itssecurity function to bring together physical security,information security and fraud prevention. The keyawareness issues from each of these aspects arecombined and distilled into a single set of trainingmessages.Staff show good understanding of some securityissues, such as email and mobile devices (phonesand lap top computers etc.). Getting messagesacross in other areas (such as Internet-relatedthreats and instant messaging) has proved harder.The awareness training clearly explains eachindividual’s personal responsibilities for informationsecurity. It then provides guidance on goodpractices the individual can adopt to discharge thoseresponsibilities.The business demands training to be availableas required and in a cost-effective way. To meetthese demands, there is a drive to deliver securityawareness through on-line systems and self training.Completion of computer-based training (CBT) is nowmandatory. Quizzes in the CBT provide statisticsthat measure the levels of awareness; the CBT itselfrecords the extent to which staff have been trained.The speed, ease of use and consistency of the onlinetraining programme are seen as key benefits.While the set-up has involved some investment,the efficiency of training delivery achieved hasmaximised the return on this investment.Other measures that have proved helpful in trackingstaff awareness include the number of mobiledevices lost, and the number of concerns andsecurity-related incidents reported.The content of CBT training is continually reviewed,so that it reflects emerging risks and staff continueto see the benefits. The next stage will be to targethigh risk groups for additional face-to-face securityawareness training.Current practice and the measurement of success 13

Measuring theeffectivenessof awarenessprogrammesMany business leaders have observed that “what getsmeasured gets done”. Ultimately, information securityawareness is about people’s behaviours. These arealways hard to measure, so this is a challenging area formost organisations.Different organisations adopt different methods ofassessing the effectiveness of information securityawareness activities. These include both quantitative andqualitative approaches. In general, there are four mainapproaches, each with different performance indicators:1. Process improvementThis approach assesses the effectiveness of theprogramme by looking at its activities. In other words,these measures are around the effort put into theprogramme; they do not directly measure whether the endresult has improved security.Possible performance indicators include:• The extent of development of security guidelines.For example, people can assess how well securityguidelines address the main security risks ortechnology platforms;• The extent to which the guidance is disseminated.Typical metrics are the number of leaflets distributed,visitors to the intranet site, or staff receiving awarenesstraining;• The efficiency of the awareness process. The normalmeasure is the cost of delivery, e.g. cost (in time andexpenses) per person trained;• The relevance of the awareness material. A simplemeasure here is the frequency with which it is updated;and• The effectiveness of the deployment of the securityguidelines. Surveys that ask staff whether they areaware of guidelines and know what procedures tofollow are one way to measure this.14 ENISA – Information security awareness initiatives:

The advantage of process improvement measures is thatthey are easy to define and to gather.The disadvantage is that they provide only indirect comfortas to whether the programme is making the organisationany more secure.2. Attack resistanceThis approach focuses on measuring how resistant staffare to a potential attack. Possible performance indicatorsinclude:• The extent to which staff recognise attacks. Thisnormally involves asking specific questions in a staffsurvey, quiz or computer-based test; and• The extent to which staff fall prey to attacks. Simulatedattacks, such as emails containing executables orpeople phoning up to ask staff for their passwords, arehelpful here.The advantage of attack resistance measures is that theyprovide some direct evidence of the actual state of staffawareness. They tend to be good for impressing seniormanagement on the need for investment in securityawareness.The main disadvantage is that there are potentially manyattack scenarios; any individual measure will be quitespecific to the scenario it is testing. Simulated testscan also be relatively expensive to set up. A risk-basedapproach can help overcome these issues.3. Efficiency and effectivenessThis approach focuses on the actual experience ofsecurity incidents within the organisation. Possibleperformance indicators include:• The extent of security incidents arising from humanbehaviour. Typical metrics include the number and costof those incidents. Some organisations also considerthe proportion of security incidents arising from humanbehaviour;• The extent of downtime arising from human behaviour.This is a particular concern in sectors where availabilityof systems is critical; and• The extent to which human behaviour caused theorganisation’s most severe incidents. Root causeanalysis into serious incidents provides this data; themeasure is normally then expressed as a proportion ofthe total number of serious incidents.Airport operator – the role ofmetrics and auditThe operator of an airport is subject to anincreasing threat from terrorists and other maliciousattacks. They regularly transfer large volumesof information between their systems and thirdparties. Their key control systems are networked.All of this means information security is of criticalimportance to their business.They employ a large number of staff from diversebackgrounds, including lots of temporary andcontracting staff. As a result, they choose touse a wide range of techniques to raise securityawareness. Since some of the staff are not verycomputer literate, regular topical emails andcommunications have proved very effective.Monitoring incidents within and outside theorganisation allows staff to provide up-to-dateguidance.They have implemented polices and proceduresin line with ISO standards. This has not, on itsown, improved awareness. Policy is a necessarycomponent of the framework for control, but issimply not very exciting to staff.Where practical, requirements from the policieshave been built into electronic or automatedprocesses. These help staff comply with policy, andproduce better activity logs than equivalent manualprocesses. Reviewing these logs is a quick way tocheck people’s behaviours against policy.Internal and external audits have played amajor role in examining behaviour and checkingadherence to process and policy. The audits havesuccessfully highlighted areas where awarenessof good practice or policy has been lacking. Sinceaudit reports go to senior management deficienciesare taken seriously. This makes the approval ofnew security initiatives and awareness training gomore smoothly.Tracking incidents also sheds light on awarenesslevels. Investigating the root causes of incidentsand downtime has highlighted trends in behaviours.These are then analysed to identify any particulargaps in awareness or training and then addressedin the planning of future awareness initiatives.Current practice and the measurement of success 15

Measuring the effectiveness of awareness programmesGovernment – get safe onlineA government department explained how on-line systemsare increasingly used to deliver public services. Security isessential to maintaining citizens’ trust in the continued useof these and future technologies. The government wantsto ensure that the country is a ‘secure’ place to be onlineand so is keen that people are aware of the associatedsecurity threats. Good information security is viewed asbeing increasingly important to the success and stabilityof the country as a whole.The threats are growing rapidly, with e-crime doublingroughly every 18 months. A recent survey showed thatone in ten people had suffered Internet fraud losing 1,200euros each on average. While often banks rather thancitizens take these losses, individuals affected by identitytheft suffer a great deal of disruption to their personallife. To reduce this, the government aims to make peoplemore aware of the risks to their electronic information, beit their credit card or social security details.Within the general public, there is a diverse range ofpeople to reach. Different techniques work well ondifferent audience groups. The level of prior knowledgeand age are useful ways of categorising the audience.Overall, many different techniques are used to increaseawareness. Some of the most successful campaignelements have been websites, phone-ins, conductingonline and offline quizzes and email newsletters. Thesehave been very good at grabbing people’s attention andgetting across key messages.Measurement is critical to ensuring the campaigns aredelivering the right messages and working as intended.Surveys measure behaviours and perceptions beforeand after the campaigns. These immediately highlightdifferences and shed light on the effectiveness and impactof the campaign.A big challenge is retaining the right balance in thecontent. The purpose of awareness is not to scare people,but to educate them and change their behaviour. Thecontent and distribution methods also need to remainrelevant in the face of a rapidly changing environment.It is important that there is a joint government and industryapproach to promoting Internet safety. Government isnot solely responsible for keeping people safe online– industry must also accept responsibility for the safetyof their customers to assure the continuing growth of e-commerce.The advantage of these metrics is twofold: firstly, the datacan be gathered through the overall security incidentmonitoring that most information security groups doanyway; secondly, these statistics are usually of greatinterest to senior management.The disadvantage is that they do not necessarily give atrue reflection of security awareness. It is not just securityawareness that determines whether incidents occur; theextent to which attacks actually occur is the main factor.In the long term, the trend can be a good indicator ofawareness. In practice, however, people often take actionbased on individual incidents; this may not be the mosteffective approach.4. Internal ProtectionsThis category is concerned with how well an individual isprotected against potential threats. In other words, hasthe individual’s awareness resulted in secure behaviour?Possible performance indicators include:• The extent to which individuals incorporate security intothe development and acquisition of systems. This canbe measured by reviewing a sample of business casesand requirements specifications;• The extent to which individuals protect their data files.Scanning tools can be used to build up a picture of this;• The extent to which individuals have allowed theirsystems to be infected by viruses or other malicioussoftware. Normally anti-virus activities can providestatistics on this; and• The extent to which individuals have allowed theirsystems to harbour inappropriate (e.g. pornographic)material or unauthorised (e.g. pirated) software. Thereare specific scanning tools that can quickly measurethis.The advantage of these measures is that they providedirect evidence of staff behaviours. They assess whetherawareness is making the organisation more secure andavoid hypotheses or extrapolation. In addition, existingaudits (by internal or external auditors) may providefeedback here, effectively for free.The disadvantage is that any individual measure isquite specific to the behaviour it is measuring. Often, anawareness programme aims to change many behaviours.This can result in many potential metrics. Each, in turn,may require investment in scanning tools or audits. A riskbasedor rotational approach can help reduce the ongoingcost.Most organisations use a combination of several of the16 ENISA – Information security awareness initiatives:

four approaches. Blending different metrics enables themto build up a balanced scorecard for their awarenessprogramme. Decisions are based on the overall picture,rather than on any single measure.The respondents in this study use a wide variety ofdifferent methods to measure the effectiveness of theirinformation security awareness initiatives. All of themeasures prompted in our questionnaire have someadvocates.Measures of internal protection are the most popularoverall. Two thirds of respondents use policy breacheshighlighted in (external or internal) audit reports as ameasure. The audits can be undertaken by membersof internal teams or may be as a result of external orthird party audits. The auditors’ objective and systematicapproach is felt to make these reports reliable sourcesof information. In addition, nearly a third of respondentsuse the results of software scans as a metric for theeffectiveness of their awareness programme. Somepossible metrics (such as the proportion of systems thatare made with security in mind) are hardly used.Efficiency and effectiveness measures are the next mostpopular. Many respondents use their experience of securityincidents. The most common metrics are the number ofincidents caused by human behaviour and root causeanalysis of the most serious incidents; more than half ofrespondents use each of these. A third also consider theproportion of incidents caused by human behaviour. Fewerrespondents track cost of incidents, but many of those thatdo believe this is one of their most important metrics.A significant minority of respondents use some form ofattack resistance metrics. A third include questions onsecurity awareness in staff surveys. They then measureawareness levels before and after initiatives take place.However, some respondents highlight issues with thecomplexity of collecting and processing this data. A quarterof all respondents carry out tests to check whether staffbehave in the right way when presented with a possiblethreat.International commercial bank – measuring is critical to targeting effortsA large commercial bank has a central informationsecurity function. This team is responsible for drivingawareness training across the world. They aim toget basic messages about security across to a large,geographically dispersed audience. They also need tosend specific messages to smaller groups of staff withkey roles in systems or security.A big challenge faced by the bank has been how tomeasure awareness levels and the effectiveness ofits awareness programme. Ideally, the bank wantsto measure the change in people’s behaviours.This is difficult to assess quantitatively. However,measurement is critical to targeting training efforts atweak areas, so the bank has invested in identifyingpractical metrics and key performance indicators.A particularly successful technique has been the useof computer-based training (CBT). A centralised CBTlibrary includes training courses and captures testresults from the automated testing of staff. All newemployees must complete the training as part of theirinduction. The training is updated regularly, and all staffmust complete the updated training. Reports analysethe extent of completion of CBT training and the scoresin tests; the central team monitor these and act on anysignificant trends.Password scans provide a useful direct quantitativemeasure of the attitude and behaviour of staff. Thebank periodically runs software that scans passwordfiles on key systems and analyses the strength ofindividual passwords. The number of staff using easilyguessable passwords is a key indicator of securityawareness.Other techniques that have proved effective includesimulated phishing emails and competitions. Thesehave made the targeted staff think carefully about whythey are asked to be secure. They have also providedhelpful statistics for trend analysis.There are plans to introduce a new survey to gauge thelevel of security awareness and behaviours within thebank. An independent third party will gather responsesfrom a random sample of staff (rather than self-select).This will enable the bank to use the survey results todraw statistically valid conclusions across the business.Initially, the bank monitored incidents to assess securityawareness. However, root cause analysis has shownthere are many different factors behind each incident,so the number of incidents is not a true reflectionof security awareness. In addition, the frequency ofincidents is so low that trend analysis is not meaningful.For these reasons, incident statistics are no longerused to measure awareness.Current practice and the measurement of success 17

Measuring the effectiveness of awareness programmesPublic department – blocking and monitoringA government department explained why enforcingpolicy and measuring people’s behaviours are critical togood security awareness. Security of the personal datathey store and process is vital to maintaining the publictrust. The time sensitivity of this information is higherthan in most private companies. Threats to it, fromforeign governments, criminals, and journalists, arenumerous. To maintain security, the department needsa rigorous, comprehensive control and awarenessframework.One challenge they face is a high turnover of staff, 30-40% per year. With so many transient staff, maintainingeffective awareness is difficult. A key technique to dothis is the use of comprehensive induction training.This covers important topics including data privacy andinformation security. Staff have to sign to confirm theyunderstand and will abide by the department’s policies.Cost effectiveness drives the approach adopted to raiseawareness. The use of intranet sites and emails havebeen effective. These are instantly available to staff;they can convey important messages quickly and to awide audience. Surveys were used in the past to gaugeawareness; however, response was low and the resultsdid not always provide the required information.Where possible, information security requirements areautomated, i.e. built into systems. Manual spot checksare also used, targeted at high risk systems and areas.Penetration testing and social engineering are usedto assess people’s actual behaviours. In addition, toensure staff are following policy, a random sample ofusers’ emails are audited. Based on these, a ‘securityleague table’ report is sent to management. Thisencourages people to improve their areas. Wherespecific weaknesses are found, they are addressedwith targeted training.Incidents are also tracked. Due to the relatively lownumbers of events, slight increases can be easily seenand analysed. The results are used to target furtherawareness training if any trends are found. Previousanalysis showed that new joiners had lower awarenessand changes were made to induction training toaddress this. Further monitoring of the awareness isaccomplished through yearly audits for compliance withISO 27001.How do you measure the level of information security awareness in your organisation?Results of audits (by internal or external auditors)66%Number of security incidents due to human behaviourThe root causes of the most serious incidents54%58%Proportion of security incidents due to human behaviourNumber of staff successfully completing awareness trainingcoursesResults of staff security surveysResults of scans for viruses and unauthorised softwareQualitative feedback from focus groups or staffCost of security incidents reported due to human behaviourProportion of downtime due to human behaviourTesting whether staff follow correct proceduresNumber of visitors to security intranet websiteResults of periodic self-assessments by managementProportion of purchases that have been made with security inmindDistribution of security policy or leaflets6%4%37%36%34%31%28%27%27%25%22%19%18 ENISA – Information security awareness initiatives:

Given the ease with which process improvement measurescan be captured, the number of respondents using them islow. Ideally, respondents would like to be able to measureactual changes in staff behaviours resulting from theawareness activities. As a consequence, relatively fewrespondents find input metrics (e.g. number of visitors tointranet site, number of leaflets distributed) helpful. Themost used measures of this type are the number of staffreceiving training and qualitative feedback from staff on theprogramme; roughly a third of respondents used each ofthese metrics.There is little consensus among respondents on the mosteffective measures. This is clearly an area where goodpractice is evolving.Even the most popular metrics had been found wantingin some organisations. For example, many respondentshave abandoned security incident statistics as a measureof security awareness. One reason is that there are manyother factors driving the number of security incidents.Another is that the volume is (mercifully) low and hencespiky in nature; this makes trend analysis difficult.Overall, there was a good correlation between the metricsthat were highlighted as most effective and the mostpopular metrics in actual use across all respondents. As agroup, they seem to have learned a lot over the years; theirpast experience of what works well has shaped what theydo today. Most respondents acknowledge that they arecontinuing to improve their approach, but there is much tolearn from what they do today.Generally the results did not show significant differencesbetween the sectors of respondents. This indicates thatwhat people have found to be effective across industriesis broadly similar. Although one particular item of note wasthat financial services organisations were less likely to usemetrics related to the costs of incidents than government,retail, telecommunication and utilities.Many respondents have encountered problems in thepast, putting effective quantitative measures in place. Itis important that the method any organisation uses toproduce and collect awareness indicators addresses thesecommon issues. The main concerns raised by respondentsin this study include:• Quality and comparability issues. A particular issuehere is with staff surveys, where the exacting wordingand the placement of the question in the survey canaffect the answers given. Often staff tell surveys whatthey think management want to hear, not necessarilywhat they really think. Compliance returns from seniormanagement (e.g. self assessments) can also bemisleading; the people signing the returns are oftendivorced from the detail of their operations and so reportwhat they are told by their teams;• Relevance. It is important not to take the wronginference from measures. For example, an increase invirus infection rates may indicate a problem with staffawareness, but it could equally be an issue with the antivirussoftware. A rise in security incidents could indicatea problem with awareness (more actual breaches),What metrics have proved effective at measuring the success of information security awareness activities?Number of security incidents due to human behaviour-416Audit findings6Results of staff surveys-45Tests of whether staff follow correct procedures-15Number of staff completing training5Qualitative feedback from staff3Cost of Security incidents due to human behaviour-12Number of visitors to Security Intranet site-12Proportion of downtime due to human behaviour2Results of scans for viruses and unauthorised software-21Number of policies/Leaflets distributedReturn on investment-2-2Most effectiveLeast effectiveCurrent practice and the measurement of success 19

Measuring the effectiveness of awareness programmesor improved awareness (more reporting of the samebreaches). The number of leaflets or emails sent outdoes not mean anyone has necessarily read them.Using a portfolio of measures enables sense to be madeout of what can be a confusing picture;• Availability of specific indicators. Some measuresare simply too hard to gather for the payback they give.While in principle, many respondents think return oninvestment is a sensible approach, most find it hardto quantify the benefits of better staff awareness. In anon-sales environment, estimating the cost of securitybreaches can be hard; and• Processing. Once data has been collected, it isimportant to process this and turn it into meaningfulinformation. The information may need to be editedto remove suspect results (for example, if there is aproblem with a particular training course). Data mayneed to be weighted to reflect better the overall staffprofile of the organisation. A general rule of thumb isthat the less processing the better. Some respondents,for example, have abandoned using before and aftercomparisons of survey data because of the complexityof the processing required.In conclusion, there appear to be many reasons whyindividual metrics might be helpful. Some metrics are usedbecause they provide insight into actual behaviours (e.g.scans or tests). Others are adopted because they resonatewith the senior management that sponsor awarenessprogrammes (e.g. cost of incidents). Others are simplyeasily to hand and require little effort (e.g. results ofaudits).Each organisation needs to find the right balance forthem; there is no “one size fits all” solution. Keeping theapproach simple tends to keep it cost-effective. Manycurrently struggle with quantifying security awareness;however, provided simple mistakes are avoided, abalanced set of metrics can provide real insight into theeffectiveness of awareness programmes. Only with thisinsight are organisations able to change their programmesfrom a compliance activity to one that really benefits theiroperations.An example of a balanced set of key performanceindicators is provided in the following table. This combinesthe five most popular measures used by respondents intoan overall security awareness dashboard. Also listed arecase studies in the report where these particular metricsare being used; these include more information about howto use them effectively to assess the level of awareness.Metric Points to consider Case studiesNumber of Can quickly show trends and deviations in behaviour. Financial services group – page 13security incidentsCan help understand root causes and estimate costs to Airport operator – page 15due to humanthe business.behaviourPublic department – page 18May not be enough incidents to draw meaningful results.May be other factors that affect the incidents.Audit findings Generally conducted by independent and knowledgeable International finance services group – page 7people who can provide third party assurance onbehaviours.Airport operator – page 15May be significant areas of awareness not reviewed.13.8% Results of staff If used before and after specific training, can be used to International insurer – page 6surveysgauge the effectiveness of campaigns.International airline – page 9If sufficiently large, can provide statistical conclusions onTelecommunications provider – page 10staff behaviours.Retailer – page 12Need to be targeted at verifying key messages.Government – page 16Have to be carefully designed since staff may respond with‘expected’ answers and not true behaviours.Tests ofVery good way of actually measuring behaviours and International commercial bank – page 17whether staff highlighting changes after training.Public department – page 18follow correctHave to be carefully planned and carried out since couldproceduresbe breaches of employment and data protection laws.Need a big enough sample if results are to be meaningful.Number of staffcompletingtrainingNeed to decide what combination of classroom andcomputer-based training to use.Have to consider what training to make mandatory.May need to be tailored for different areas or regions.May need regular and potentially costly updates.International finance services group – page 7Retailer – page 12Law enforcement agency – page 13International commercial bank – page 1720 ENISA – Information security awareness initiatives:

Legal NoticeNotice must be taken that this publication represents the views and interpretations of the authors and editors, unless it isstated otherwise. This publication should not be construed to be an action of ENISA or the ENISA bodies unless adoptedpursuant to the ENISA Regulation (EC) No 460/2004. This publication does not necessarily represent state-of the-art and itmight be updated from time to time.Third party sources are quoted as appropriate. ENISA and PricewaterhouseCoopers LLP, their members, employees andagents are not responsible for the content of the external sources including external web sites referenced in this publication.This publication is intended for educational and information purposes only. No representation or warranty (express orimplied) is given as to the accuracy or completeness of the information contained in this article, and, to the extent permittedby law, ENISA and PricewaterhouseCoopers LLP, their members, employees and agents accept no liability, and disclaimall responsibility, for the consequences of you and anyone else acting, refraining from acting, in reliance on the informationcontained in this article of for any decision based on it.Reproduction is authorised provided that the source is acknowledged.© European Network and Information Security Agency (ENISA), 2007.

European Network and Information Security AgencyP.O. Box 130971001 HeraklionGreeceTel: +30 2810 39 1280http://www.enisa.europa.eu