Measuring Up: Facilitators guide - Monitoring and Evaluation NEWS

MEASURING UPA GUIDE FOR FACILITATORSHIV-related advocacy evaluationtraining for civil societyorganisationsINTRODUCTION TO THE GUIDEWhat is the guide and how can it be used?The guide has two main components:1. a stand-alone learner’s guide for anyoneinterested in evaluating HIV-related advocacy2. a facilitator’s guide that describes how to runa two-day skills-building workshop to coverthe content of the learner’s guide and applythe skills learned. Accompanying the learner’sguide (the main handout and referencematerial for sessions), there are ten PowerPointpresentations and a document for facilitatorscalled Measuring up: workshop supplementaryfacilitation material.Who is the guide written for?The guide is a resource for leaders, advocacy andmonitoring and evaluation staff of civil societyorganisations (including networks) who areinvolved in designing, implementing and assessingadvocacy projects at different levels – international,national and sub-national. Leaders of networksof key populations are likely to find this guideparticularly relevant and helpful.What is the purpose of the guide?The overall purpose is to increase users’ capacity toevaluate the progress and results of their advocacywork.The guide aims to:1. help users to identify and confront thechallenges faced by community-basedorganisations evaluating HIV-related advocacy2. introduce new thinking for designing advocacyevaluations3. give users the opportunity to apply someaspects of the evaluation design process to theirspecific contexts4. make users aware that advocacy evaluation isa fast-growing and evolving field, with a largenumber of publications on advocacy evaluationdesign, approaches and methods available viathe Internet and summarised in the resourcessection of the learner’s guide.Why has the guide been developed?During the XVII International AIDS Conferencein Mexico in August 2008, the Alliance, ICASOand Constella Futures jointly organised a skillsbuildingsession on the challenges of monitoringand evaluating advocacy work. There was a highlevel of interest, and participants wanted to acquirethe practical skills necessary to improve theirmonitoring and evaluation. So the facilitation teamagreed to explore different ways of responding tothis demand and follow up on lessons learned fromMexico.Until recently, few resources existed to guideevaluation in this area. However, in the lastfew years advocacy evaluation has become aburgeoning field, and several approaches andtools for evaluating advocacy and policy workhave been developed. Therefore, the Alliance andICASO decided to invest in producing a guide toevaluating advocacy work, drawing on the differentapproaches and tools now available.Who contributed to the content of the guide?The guide includes numerous extracts frompublications developed by leading organisationsin the field of advocacy evaluation, includingOrganizational Research Services, Harvard FamilyResearch Project and Innovation Network. Anyunintended misrepresentation of information fromthese sources is the fault of the guide developers.Other key contributors to the development ofthe guide were ICASO and Alliance staff, ICASOand Alliance grantees and network memberorganisations, and consultant Nicky Davies(www.aidsdev.com).1 MEASURING UP \\ A GUIDE FOR FACILITATORS

You will need the following resources foryour workshop:● Measuring up facilitator’s guide – one perfacilitator and one spare● Measuring up learner’s guide – one perparticipant and facilitator, and a few spare● ten Measuring up workshop PowerPointpresentations● printed content of the Measuring up: workshopsupplementary facilitation material as describedin the session instructions● ‘Evaluation design framework’ flip chart showingthe diagram of the seven components of theadvocacy evaluation design framework, frompage 11 of the learner’s guide (see also thefacilitator’s notes for the intoductory session toModule 3, page 18)● computer with Adobe Acrobat Reader andPowerPoint packages, projector (check it ‘talks’to your computer), projector screen, extensioncables and plug adapters● access to a printer to print off informationfrom Measuring up: workshop supplementaryfacilitation material● notepads, pens and workshop timetables (oneset for each participant’s chair at the start ofthe workshop) – the workshop timetable canbe adapted and amended using the templatein Measuring up: workshop supplementaryfacilitation material● sticky labels or name tags with participants’ fullname and organisation – encourage participantsto add or underline the name they would like tobe used during the workshop● flip chart stands (preferably two), plenty of flipchart paper and 30 flip chart pens (mainly blackand blue, plus four to five red)● any props or preparation needed for energisers(red boxes with ideas for an energiser areincluded at relevant points in the facilitator’sguide)● make a ‘car park’ flip chart for any questionsasked by participants during the workshop thatyou intend to address later● refreshments (including healthy snacks and drinkoptions to maintain energy levels and supportthose taking medication), water available atall times, meals, accommodation and dailyallowances for participants and facilitators● semi-circle of chairs for plenary sessions, withtables and chairs around the edge of the roomand breakaway spaces to accommodate allparticipants for group work.Important noteIf you are facilitating a one-day workshopusing the instructions provided in this guide,you will need to add information to someof the existing PowerPoint presentationsor create flip charts of information. All theinformation you will need can be drawn fromthe learner’s guide. Details of when you willneed to do this are included in the one-dayworkshop instructions box at the end of eachsession described in this guide.3 MEASURING UP \\ A GUIDE FOR FACILITATORS

Suggested three-day workshop timetableMeasuring up: HIV-related advocacy training for civil society organisationsTimingActivityDAY ONEDAY TWO09:00 – 09:30 Ice-breaker and workshop expectations09:30 – 10:00 Workshop introduction, ground rules and pre-workshop questionnaire10:00 – 10:05 Session 1: Module 1 introduction10:05 – 10:30 Session 1.1: Defining advocacy10:30 – 11:00 Refreshments break11:00 – 12:00 Session 1.2: Defining key elements of monitoring and evaluation12:00 – 12:05 Session 2: Module 2 introduction12:05 – 12:40 Session 2.1: Understanding why it is important to evaluate advocacy12:40 – 13:00 Flexible session13:00 – 14:00 Lunch break14:00 – 15:05 Session 2.2: Identifying the challenges faced by civil society organisationsevaluating HIV-related advocacy15:05 – 15:30 Session 2.3: Exploring how evaluating advocacy can be different fromprogramme evaluations15:30 – 16:00 Refreshments break16:00 – 16:15 Session 3: Module 3 introduction16:15 – 17:00 Session 3.1: Determining evaluation users and uses09:00 – 09:30 Day one recap09:30 – 10:30 Session 3.2(a): Mapping your advocacy work10:30 – 11:00 Refreshments break11:00 – 12:00 Session 3.2(b): Mapping your advocacy work12:00 – 12:05 Energiser12:05 – 13.00 Session 3.3: Prioritising what to evaluate13:00 – 14:00 Lunch break14:00 – 15:05 Session 3.4: Developing your evaluation questions15:05 – 15:30 Session 3.5: Deciding on an approach to measurement15:30 – 16:00 Refreshments break16:00 – 17:00 Session 3.6: Selecting indicators09:00 – 09:30 Day two recap09:30 – 10:30 Session 3.7: Identifying and choosing data collection methods relevant toadvocacy evaluation10:30 – 11:00 Refreshments break11:00 – 12:00 Session 3.7: continuedDAY THREE12:00 – 12:05 Session 4: Module 4 introduction12:05 – 13:00 Session 4.1: Consolidating learning on how to evaluate HIV-related advocacy13:00 – 14:00 Lunch break14:00 – 16:00 Session 4.1: continued16:00 – 16:15 Session 4.2: Reflecting on the key messages and learning points from theworkshop to share with our colleagues16:15 – 17:00 Workshop summary, post-workshop questionnaire and workshop close4 MEASURING UP \\ A GUIDE FOR FACILITATORS

Suggested one-day workshop timetableMeasuring up: HIV-related advocacy training for civil society organisationsTimingActivity09:00 – 09:20 Workshop introduction09:20 – 09:30 Session 1: Module 1 introduction and Session 1.1: Defining advocacy09:30 – 09:50 Session 1.2: Defining key elements of monitoring and evaluation09:50 – 10:00 Session 2: Module 2 introduction and Session 2.1: Understanding why it isimportant to evaluate advocacy10:00 – 10:15 Session 2.2: Identifying the challenges faced by civil society organisationsevaluating HIV-related advocacy10:15 – 10:30 Session 2.3: Exploring how evaluating advocacy can be different fromprogramme evaluations10:30 – 10:50 Refreshments break10:50 – 11:00 Session 3: Module 3 introduction11:00 – 11:05 Session 3.1: Determining evaluation users and uses11:05 – 11:35 Session 3.2: Mapping your advocacy work11:35 – 11:45 Session 3.3: Prioritising what to evaluate11:45 – 11:55 Quick energiser11:55 – 12:05 Session 3.4: Developing your evaluation questions12:05 – 12:15 Session 3.5: Deciding on an approach to measurement12:15 – 12:35 Session 3.6: Selecting indicators12:35 – 13:15 Session 3.7: Identifying and choosing data collection methods relevant toadvocacy evaluation13:15 – 14:00 Lunch break14:00 – 16:30 Session 4: Module 4 introduction and 4.1: Consolidating learning on how toevaluate HIV-related advocacy16:30 – 16:50 Session 4.2: Reflecting on the key messages and learning points from theworkshop to share with our colleagues16:50 – 17:00 Workshop summary, post-workshop questionnaire and workshop close5 MEASURING UP \\ A GUIDE FOR FACILITATORS

CONTENTSIce-breaker and workshop expectations 7Workshop introduction, ground rules and pre-workshop questionnaire 7Module 1 Defining advocacy and key elements of monitoring and evaluation 9Session 1.1 Defining advocacy 10Session 1.2 Defining key elements of monitoring and evaluation 11Module 2 Understanding why evaluating HIV-related advocacy is important, 12its challenges and differencesSession 2.1 Understanding why it is important to evaluate advocacy 13Flexible session 14Session 2.2 Identifying the challenges faced by civil society organisations 15evaluating HIV-related advocacySession 2.3Exploring how evaluating advocacy is different fromprogramme evaluations 17Module 3 Introducing a framework for designing evaluations of 18HIV-related advocacySession 3.1 Determining evaluation users and uses 19Session 3.2 Mapping your advocacy work 20Session 3.3 Prioritising what to evaluate 23Session 3.4 Developing your evaluation questions 24Session 3.5 Deciding on an approach to measurement 25Session 3.6 Selecting indicators 26Session 3.7 Identifying and choosing data collection methods relevant to 27advocacy evaluationModule 4 Putting it all into action 29Session 4.1 Consolidating learning on how to evaluate HIV-related advocacy 29Session 4.2 Reflecting on the key messages and learning points from the 34workshop to share with our colleaguesClosing session Workshop summary, post-workshop questionnaire and 36workshop close6 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONEOPENING SESSIONIce-breaker and workshop expectationsFacilitator’s notesA simple and quick ice-breakeris to ask participants to movearound the room and greeteach other, sharing their names,organisation and somethingdifferent, interesting or sillyabout themselves (such as away to remember their name).You can also do this sitting in theplenary chairs, which is quickerbut more serious.MaterialsA small piece of paper and apen, one for each participantTime30 minutesInstructions1. Welcome everyone to the workshop.2. Facilitate a 15-minute Ice-breaker to help everyone to get toknow each other.3. Facilitate a 15-minute activity to understand participants’expectations of the workshop. Hand out small pieces of paperand marker pens to each participant. Ask participants to spendfive minutes brainstorming their expectations of the workshopand write each one on a separate small piece of paper. Afterfive minutes ask participants to volunteer to read out theirexpectations. Each participant can offer one expectation at atime until they have all been read out. As each expectation isread aloud, collect the piece of paper and stick it up for all tosee. If others have similar expectations, ask if you can collectthem and stick them in a group. Once all the expectations havebeen collected, review them and explain which ones will or willnot be addressed during the workshop and why. If realistic andif you feel confident to do so, explain which expectations youwill try to add to the workshop sessions or timetable.Workshop introduction, ground rules and pre-workshopquestionnaireSession informationThe purpose of the guide,and therefore this workshop,can be found on page 3 of theintroduction to this facilitator’sguide, and the suggestedworkshop timetables for threeorone-day workshops can befound on pages 4 and 5.Materials● PowerPoint 1● Workshop timetable, one foreach participant● Copies of the pre- and postworkshopquestionnaire fromMeasuring up: workshopsupplementary facilitationmaterialFacilitator’s notes● You can use Measuring up workshop PowerPoint 1 –Measuring up workshop introduction for this workshopintroduction. It is also useful to have flip charts with theworkshop’s purpose, aims and ground rules up on theworkshop room walls. Make sure that participants havethe workshop timetable on their chairs at the start. You canfacilitate developing the ground rules if you think this isnecessary and you can accommodate the time needed todo so.● If you feel that you have time, you could facilitate a briefexercise (5–10 minutes) to gather the expectations ofparticipants. Remember to give participants feedback on whichexpectations you aim to meet during the workshop and why.● Flip charts with workshopaims and ground rulesTime30 minutes7 MEASURING UP \\A GUIDE FOR FACILITATORS

DAY ONEInstructions1. Give a 10-minute introduction to the training pack and its twocomponents; the workshop purpose and aims; and provide asummary review of the workshop timetable.2. Spend ten minutes brainstorming and agreeing some basicground rules for participants and facilitator on flip chart paperand stick them to the workshop wall.3. Ask participants to spend five minutes completing the firstquestions on the pre- and post-workshop questionnaire.Explain that the questionnaire will be repeated at the endof the workshop. The data will be anonymous and thequestionnaire will be kept safe and not read in the meantime.Then ask them to fold the questionnaire in half and draw asymbol or little picture on the back (not their name) so thatthey can identify their own questionnaire at the end of theworkshop. Collect the questionnaires.4. Hand out the learner’s guide to each participant and ask themto write either their own name or their organisation’s name onthe guide. Tell them to have this guide always available duringsession time.Suggested ground rules for participants and the facilitator during the workshop include:●●●●●Help each other to start and finish sessions and breaks on time.Switch off mobile phones or leave them on vibrate (take phone calls during breaks wherepossible).Respect each other by listening actively, providing constructive criticism and participatingfully to support collective learning.Recognise that there is a lot to learn and share; stay focused on the topic and objectives ofthe sessions rather than be drawn into the details of broader issues.Avoid using too much evaluation jargon – there are differing levels of knowledge andexperience of evaluation work among participants and jargon may not be familiar toeveryone.ONE-DAY WORKSHOPTime20 minutesInstructions1. Welcome everyone to the workshop.2. Ask the participants and facilitator to say their name and the organisation they are from.3. Give a 10-minute presentation introducing the guide and its two components; the workshoppurpose and aims; the workshop timetable; and basic ground rules for participants and facilitator.(Note you will need to add basic ground rules to Measuring up workshop PowerPoint 1 –Measuring up: workshop introduction or make a flip chart.)4. Hand out the learner’s guide to each participant and the pre- and post-workshop questionnaire.5. Ask participants to spend a few minutes completing the first questions on the pre-and postworkshopquestionnaire. Since this will be an anonymous process, ask them to fold thequestionnaire in half and draw a symbol or little picture on the back (not their name) so that theycan identify their own questionnaire at the end of the workshop.8 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONEMODULE 1Defining advocacy and key elements of monitoringand evaluation – introduction to Module 1Session informationThere is rarely one correct or‘best’ definition for differentaspects of our work, andthere are many definitions ofadvocacy and monitoring andevaluation. Here we introduceone advocacy definitionand important elements ofmonitoring and evaluation,along with some examples tohelp clarify key terms.Facilitator’s notesYou can use Measuring upworkshop PowerPoint 2 –Module 1: Defining advocacyand key elements of monitoringand evaluation for thisintroduction, although it is alsouseful to have a flip chart of themodule objectives taped to thewall during sessions.InstructionsGive a brief introduction to this module, explaining the objectivesfor the session.Objectives for Module 1● To provide a useful definition of advocacy.● To provide ‘refresher’ information on thedefinition of monitoring and evaluation, thedistinction between outputs and outcomes,and the principles underlying how they can bemeasured.Materials● PowerPoint 2● Flip chartTime5 minutes9 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESESSION 1.1Defining advocacySession informationA definition of advocacy and anexplanation of its key elementsor characteristics are includedon page 5 of the learner’s guide.Facilitator’s notesYou can use Measuring upworkshop PowerPoint 2 –Module 1: Defining advocacyand key elements of monitoringand evaluation to present theadvocacy definition and keyelements.MaterialsPowerPoint 2Time25 minutesInstructions1. Introduce the name of the session and its place in theworkshop timetable.2. Ask participants to brainstorm the key points for definingadvocacy.3. Present the advocacy definition and its key elements orcharacteristics.4. Ask participants if they know of other good advocacydefinitions.5. Conclude the session by repeating that there is not one correctdefinition and that there is further guidance available in theAPCASO toolkit.6. Tell participants that there is now a 30-minute break forrefreshments and what time they should be back in theirchairs.ONE-DAY WORKSHOPTime10 minutesInstructions1. Present the advocacy definition and its key elements or characteristics.2. Ask participants if they know of other good advocacy definitions.3. Conclude the session by repeating that there is not one correct definition and that there is furtherguidance available in the APCASO toolkit.10 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESESSION 2.1Understanding why it is important to evaluate advocacySession informationThe introductory text below hasbeen extracted from page 7 ofthe learner’s guide.Facilitator’s notesCirculate among the groups tocheck they are not stuck on thetask.MaterialsFlip chartTime35 minutesInstructions1. Introduce the name of the session and its place in theworkshop timetable.2. Ask participants to form mini groups of three people with theirneighbours by adjusting their chairs slightly. Ask groups: “Canyou identify the main reasons for evaluating advocacy work?”Tell them they have five minutes to consider the question intheir groups.3. After five minutes ask participants to move their chairsback. Tell them you would like to collect their ideas – ask fordifferent contributions from participants. Write the ideas on aflip chart.4. Check that the main reasons identified below (see page 7 ofthe learner’s guide) have been considered.There is growing interest in advocacy evaluation, both from advocates and funders. The reasons forthis interest may differ between the two groups, but in some cases there may be overlap. The mainreasons for evaluating advocacy include a desire to:● learn how to improve the capacity of advocates● learn how to adjust advocacy strategies as they are being implemented● inform the planning of future advocacy work, including funding cycles/proposals● demonstrate relative worth and specific areas of added value (for example, for networks of keypopulations 1 that are competing for funding with development non-governmental organisationsand academic institutions)● demonstrate evidence-based approaches to advocacy work (increasingly demanded by funders,influential policymakers and communities to encourage engagement)● account for funding and demonstrate results● demonstrate results to mobilise more resources for future advocacy work, including monitoringand evaluation of advocacy.1. The term key populations refers to groups of individuals particularly affected by HIV, including people living with HIV, sex workers, men who havesex with men, people with disabilities, people who use drugs, prisoners, etc.ONE-DAY WORKSHOPTime10 minutesInstructions1. Present the main reasons why it is important to conduct advocacy evaluation, provided in thesession information above.2. Ask if the participants have anything to add.13 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONEFLEXIBLE SESSIONSession informationThis session can be titledand used as necessary by thefacilitator. For example, thiscould be left blank to allow forsession overrun, be titled ‘carpark’ to address any questionsthat have not be addressed, orbe titled ‘administration session’and used to hand out per diems,address logistics and anyissues concerning participants.Alternatively you could conducta fun energiser if appropriate.Time20 minutesInstructions1. To be determined by the facilitator.2. Tell participants that there is now a one-hour break for lunchand what time they should be back in their chairs.Idea for an energiserSimon saysThe facilitator tells the group that they should follow anyinstructions that begin “Simon says”. If the facilitator doesnot begin an instruction with the words ‘Simon says’ thenthe group should not follow the instruction. The facilitatorbegins by saying something like, “Simon says clap yourhands”, while clapping her hands. The participants follow.The facilitator speeds up the actions, always beginningwith the words “Simon says”. After a short while the words“Simon says” are omitted. Those participants who continueto follow the instructions are out of the game. The game cancarry on for as long as it remains fun.Source: 100 ways to energise groups: games to use in workshops, meetings and thecommunity (International HIV/AIDS Alliance, 2003)This session is not relevant for a one-day workshop.14 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESESSION 2.2Identifying the challenges faced by civil societyorganisations evaluating HIV-related advocacySession informationKey challenges identified byguide contributors can be foundon pages 7 and 8 of the learner’sguide.MaterialsOne set of the below for eachgroup:● Challenges from the learner’sguide, printed off from theMeasuring up: workshopsupplementary facilitationmaterial and cut into strips(make sure you cut the blankends off the strips so thatthey can fit side by side onthe scale)● Tape (Stickie Stuff or Blu-Taketc.)● A few ordinary (not flip chart)pens● 20 spare strips of paper thesame size as the challenges(enough for additionalchallenges and ways to avoidor overcome challenges)● Instructions for the activityprinted off from the workshopsupplementary facilitationmaterials● Flip chart paper with thesliding-scale presentedhorizontally at the top andacross the full width of thepaper so that the challengescan be placed to the rightTime65 minutesParticipants conducting this activity at the Measuring up field-test workshop, India 2010Facilitator’s notes● Prepare resources for group work in advance – don’t leave thisto break-time.● Prepare group work spaces in advance, including tables andcorrect number of chairs, to avoid wasting time during thesession.● Circulate among the groups to help them be productive,focused and not get stuck.● Keep to time, even if participants haven’t completed all theactivity instructions.● The learner’s guide notes that challenges that are more withinour control are easier to overcome than those more out of ourcontrol. This could be a useful point of discussion after theactivity.Instructions1. Introduce the name of the session and its place in theworkshop timetable.2. Explain to participants that the learner’s guide lists challengesidentified by training pack contributors to evaluating HIVrelatedadvocacy work in resource-limited settings. Some ofthese challenges are specific to HIV; others to the capacity ofcivil society organisations and the context of resource-limitedsettings. The list is not exhaustive or in any particular order.3. Explain that you would like participants to work in groups ofapproximately four people from any organisation. Read outthe instructions on the next page and ask each group to collectfrom you a flip chart, pens, a set of strips of challenges, sparestrips, tape and activity instructions printed off for each group.Indicate where the group work spaces are.15 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESliding scale1 2 3 4Although perceivedas a challenge it canbe easily avoided orovercomeA challenge sogreat it means thatevaluating advocacyis almost impossible,meaningless ordamagingONE-DAY WORKSHOPGroup work instructions● You have 35 minutes to complete this activity.● Read through as a group the challenges identified by theguide contributors.● Write any other key challenges on additional strips of paper(don’t spend too long on this or go into detail; you can alwaysadd challenges later).● Organise the challenges on the flip chart using the slidingscaleat the top of the page, sticking the challenges along thesliding-scale. Where you disagree among yourselves, you willneed to average the score when placing the challenge, or useyour own rule such as ‘how most civil society organisationsworking on HIV would feel’. Remember, this is a tool tofacilitate discussion, so complete the activity rather than getstuck. Note: please write the names of your group memberson your flip chart.● If you have time, write down on spare strips of paper ways toavoid or overcome the challenges that you have put in or nearthis end of the sliding scale.4. After 35 minutes ask the groups to place their group work flipcharts together on the central area of the floor.5. Facilitate a discussion for the remainder of the sessiontime around the outputs from the groups, noting additionalchallenges, placement of challenges and ways to avoid orovercome challenges if there are any. Draw some conclusionsabout which challenges are easier to overcome and why, andwhich are more difficult to overcome and why.Time15 minutesInstructions1. Ask the participants to form ‘buzz pairs’ with the person sitting next to them. Ask them to spend acouple of minutes sharing a few challenges they face in evaluating their HIV-related advocacy.2. Present a flip chart or PowerPoint presentation with the challenges faced by civil societyorganisations evaluating HIV-related advocacy from pages 7 and 8 of the learner’s guide (you willneed to prepare this flip chart or PowerPoint presentation).3. Ask the participants to share any additional challenges they identified in their buzz pairs.16 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESESSION 2.3Exploring how evaluating advocacy can be different fromprogramme evaluationsSession informationInformation for this session canbe found on pages 8 to 10 of thelearner’s guide.MaterialsPowerPoint 3Time25 minutesFacilitator’s notes● You may want to use or adapt Measuring up workshopPowerPoint 3 – Module 2: Exploring how evaluating advocacywork is different from programme evaluations to give thispresentation. You will need to know the page number of box 3in the learner’s guide.● If the discussion is productive for this session, you couldbreak for refreshments late or continue the discussion afterrefreshments, as session 3.1 can be shortened (see thefacilitator’s notes for session 3.1).Instructions1. Introduce the name of the session and its place in theworkshop timetable.2. Explain to participants that you will be giving a presentationexploring how evaluating advocacy work is different fromevaluating programmes. Give the presentation.3. Ask if participants have any questions about the presentationand facilitate a discussion.4. Tell participants that there is now a 30-minute break forrefreshments and what time they should be back in theirchairs.ONE-DAY WORKSHOPTime15 minutesInstructions1. Present how evaluating advocacy can be different from evaluating programmes.2. Ask if the participants have anyg questions or comments about the presentation. Discuss these.17 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONEMODULE 3Introducing a framework for designing evaluations ofHIV-related advocacy – introduction to Module 3Session informationInformation for this sessioncan be found on page 11 of thelearner’s guide.Facilitator’s notes● You can use Measuringup workshop PowerPoint4 – Module 3: Module 3Introduction to give thispresentation.● Prepare an ‘Evaluation designframework’ flip chart showingthe diagram of the sevencomponents of the advocacyevaluation design frameworkfrom page 11 of the learner’sguide.Time15 minutesInstructions1. Introduce the name of the session and its place in theworkshop timetable.2. Give a brief introduction to this module, explaining theobjectives, the seven components of the frameworkrepresented by the topic headings, and why it is importantnot to consider these components as steps. Highlight theinformation used to develop this section of the learner’s guideand suggest that they might find it helpful to refer to thesesources in the future.Objectives for module 3● To provide information and tools for the sevenkey components of a framework for decisionmakingon how to evaluate HIV-relatedadvocacy.● To practise developing a theory of change,developing evaluation questions andidentifying indicators.ONE-DAY WORKSHOPTime10 minutesInstructions1. Present Measuring up workshop PowerPoint 4 – Module 3: Module 3 Introduction.2. Ask if the participants have any questions or comments about the presentation. Discuss.18 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY ONESESSION 3.1Determining evaluation users and usesSession informationInformation for this sessioncan be found on page 12 of thelearner’s guide.Facilitator’s notesIf more discussion time wasneeded for session 2.3 you canmiss out instruction 3.MaterialsFlip chartTime40 minutesInstructions1. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Use the flip charts developed during session 2.1 (and/or page7 of the learner’s guide) to review the main reasons for, oruses of evaluations.3. Ask participants to identify the main audiences for evaluations.Write them on a flip chart. Make sure that they include themain audiences identified in the learner’s guide.4. Place this flip chart next to the flip chart developed duringsession 2.1 and briefly facilitate participants to match themain reasons for evaluations with these different audiences.Comment on the extent of the overlap.5. Explain the importance of considering who the primaryaudience for the evaluation will be, and the value of workingwith other audiences (particularly funders) to develop designevaluations.6. Close day one of the workshop by reminding participants whattime they should be back in the workshop room tomorrowmorning. Tell them about evening logistics, and that theyshould remember to bring their learner’s guides with them inthe morning.ONE-DAY WORKSHOPTime15 minutesInstructions1. Use the flip charts developed during session 2.1 (and/or page 7 of the learner’s guide) to reviewthe main reasons for, or uses of evaluations.2. Ask participants to identify the main audiences for evaluations. Write them on a flip chart. Makesure that they include the main audiences identified in the learner’s guide.19 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWODAY TWORecap of day oneFacilitator’s notesIt is useful to use a combinationof the workshop timetableand flip charts from output orplenary brainstorms to reviewkey points from the previousday.Time30 minutesInstructions1. Facilitate an energiser.2. Provide a recap of the previous day’s sessions.SESSION 3.2Mapping your advocacy workSession informationInformation for this session canbe found on pages 12 to 17 ofthe learner’s guide.MaterialsOne set of the following for eachgroup:● Approximately 30 rectanglesof blank paper (about aneighth of the size of a piece ofA4; i.e. fold completely in halfthree times, unfold and cut)● A few ordinary (not flip chart)pens● A piece of flip chart paper● Tape (Stickie Stuff, Blu-Tak,etc.)● Instructions for the activityprinted off from theworkshop supplementaryfacilitation materialTime2 hoursFacilitator’s notes – session 3.2(a)● You may want to use or adapt Measuring up workshopPowerPoint 5 – Module 3: Mapping your advocacy work togive this presentation. You have a maximum of 30 minutes.● Leave plenty of time for the group work activity.● Prepare the group work resources in advance and ensurethere is space to work, with a table and correct number ofchairs (or floor, if appropriate).● Make sure that all facilitators are constantly moving fromone group to another to provide support and answerquestions. It is a good idea to allocate specific groups toindividual facilitators so that only one facilitator works witheach group. Share any problems you are having (as groupsupport facilitators) with each other and support each other asnecessary.Instructions – session 3.2(a)1. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Explain to participants that the aim of this activity is tofamiliarise them with the content of the learner’s guide,which tells them how to map their work and provides threeframeworks to help them do so. The aim is also to allow thema chance to internalise the information and practise usingit. Explain that a strategic view of our advocacy work, whichbreaks the goals down into interim outcomes or incrementalmeasures, helps to address the challenges and differencesconcerning HIV-related advocacy evaluation made in Module 2.3. Ask participants: “How many of you, or your civil societyorganisation colleagues, have developed a theory of changefor your advocacy work?”20 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOTheory of change map4. Explain to participants that you will give a presentation on howto develop a theory of change, including a set of instructions,a completed example and list of example advocacy outcomesthat they can use to help develop their own theory of changefor their advocacy work. Explain that this is a fairly longpresentation that covers the information in section 3.2 (pages12 to 17) of the learner’s guide. Tell participants they will needto refer to their learner’s guide during the presentation. Givethe presentation.5. After the presentation, consider how much time you haveleft for this session before the refreshments break. Adjust thetimings for the remainder of this session as appropriate.6. Explain that for the following learning activity you will askthem to work with colleagues from the same organisation.Explain that they should follow the instructions that will behanded to them.Activity instructions:● You have 70 minutes to complete this activity.● Identify one real advocacy goal or impact statement fromyour advocacy work.● Using your learner’s guides, have the following threeframeworks for developing a theory of change open forreference:● Six steps for developing a theory of change visual map(box 3, page 14)● an example theory of change (figure 3, page 15)● Categories and menu of outcomes for advocacy andpolicy work (table 2, pages 16 and 17).● Develop a theory of change outcome map similar to figure3 using the blank rectangles of paper and an ordinary (notflip chart) pen. You can place the rectangles on a piece of flipchart paper to create a transportable map. Don’t be temptedto copy figure 3; instead use the tools to develop your ownmap. You do not necessarily need to finish mapping; theactivity aims to familiarise you with the content of the threeframeworks and to practise using them. For example, youmay only develop a ‘so that’ chain for one strategy on youroutcome map. Remember to focus on outcomes in your ‘sothat’ chain’ – don’t end up listing activities or strategies.7. Ask participants to share what they are learning fromdeveloping their theory of change.8. Break for 30 minutes for refreshments.Time90 minutesFacilitator’s notes – session 3.2(b)● Make sure that all facilitators are constantly moving from onegroup to another to provide support and answer questions.● At the end of the session, or when convenient, collect togetherall the maps and store ready for session 4.1 on day three.21 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOInstructions – session 3.2(b)1. Explain to participants that because it is so important to havea good theory of change to be able to design an advocacyevaluation, we will continue to map out the theories of changeand also add some assumptions.2. Introduce the activity instructions below.Activity instructions:● Continue to work on your theory of change for the next 80minutes.● Before the end of the group work time, stand back from youroutcome map and identify assumptions you have made aboutthe context of the advocacy work. Some examples are given inFigure 3 on page 15 of the learners’ guide.● Save at least ten minutes to reflect on the process ofdeveloping the theories of change. Use the opportunityto reinforce the key benefits of mapping out a theory ofchange (both of the process itself and of the resulting map),particularly if participants have found the activity difficult. Asummary of the key benefits of a theory of change includes:● important for agreeing high-level strategies and outcomes,and common agreement about impact for your evaluationwork● an opportunity to learn what others are doing inside andoutside of the organisation, from the different perspectivesof mapping participants● a useful decision-making tool for ongoing learning andimprovement of strategies● a visual map to include in fundraising proposals.ONE-DAY WORKSHOPTime30 minutesInstructions1. Explain to participants that you will give a presentation outlining some tips for developingevaluation questions and some examples.2. Ask participants if they have any questions. Discuss as necessary.22 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOSESSION 3.3Prioritising what to evaluateSession informationInformation for this sessioncan be found on page 18 of thelearner’s guide.Facilitator’s notesYou may want to use or adaptMeasuring up workshopPowerPoint 6 – Module 3:Prioritising what to evaluate togive this presentation.Time55 minutesInstructions1. Conduct an energiser for five minutes.2. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Give a presentation covering why it is important to prioritisewhat to evaluate and provide some guidance in making thisdecision.3. Ask if there are any questions and discuss as necessary.4. Ask participants to go back into their organisation groups anddecide who would be the end users for an evaluation of theiradvocacy work and what they would prioritise for evaluation.5. Ask participants if they found this a straightforward exercise.ONE-DAY WORKSHOPTime10 minutesInstructions1. Explain to participants that you will give a presentation on how to prioritise what to evaluate.2. Ask participants if they have any questions. Discuss as necessary.23 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOSESSION 3.4Developing your evaluation questionsSession informationInformation for this sessioncan be found on page 15 of thelearner’s guide.Facilitator’s notesYou may want to use or adaptMeasuring up workshopPowerPoint 7 – Module 3:developing your evaluationquestions for the presentationsduring this session.Time65 minutesInstructions1. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Ask participants to share their experience of setting evaluationquestions. Write on a flip chart any points that will be useful torefer back to later.3. Explain to participants that you will give a presentationoutlining some tips for developing evaluation questions andsome examples.4. Ask participants if they have any questions. Discuss asnecessary.5. Ask participants to work in their organisation groups tobrainstorm what their evaluation questions might be.6. Bring the groups back together to share what they havelearned from the activity.7. Facilitate an energiser or give the participants a short break.Idea for an energiserFruit saladChoose five fruits (for example, apple, orange, pear,pineapple and banana). Point to the first person in theplenary line of chairs and say “apple”, move to the secondand say “orange” and so on until you are back to “apple”for the sixth participant. Repeat until everyone has anassigned fruit.Tell the participants to swap places with others with thesame assigned fruits when you call out the name of theirfruit. Shout “apples” and everyone must run to swap placeswith someone else. Repeat for all fruits randomly. Theparticipants can then form five groups based on thefive fruits.ONE-DAY WORKSHOPTime10 minutesInstructions1. Explain to participants that you will give a presentation outlining some tips for developingevaluation questions and some examples.2. Ask participants if they have any questions. Discuss as necessary.24 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOSESSION 3.5Deciding on an approach to measurementSession informationInformation for this session canbe found on pages 20 and 21 ofthe learner’s guide.Facilitator’s notesYou may want to use or adaptMeasuring up workshopPowerPoint 8 – Module 3:Deciding on an approachto measurement for thepresentations during thissession.Time25 minutesInstructions1. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Explain to participants that you will give a presentationoutlining five practical and strategic approaches tomeasurement put forward by Organizational ResearchServices for advocacy evaluation. Give the presentation.3. Ask participants if they have any questions about the fivepractical and strategic approaches to measurement.ONE-DAY WORKSHOPTime10 minutesInstructions1. Explain to participants that you will give a presentation outlining some different approaches tomeasuring.2. Ask participants if they have any questions. Discuss as necessary.25 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY TWOSESSION 3.6Selecting indicatorsSession informationInformation for this session canbe found on pages 22 and 25 ofthe learner’s guide.MaterialsFor each pair of participants:● an Indicator selectionworksheet and copy of theExample indicators from theMeasuring up: workshopsupplementary facilitationmaterial● have some extra copies of theworksheet available in caseparticipants ask for a cleansheetTime1 hourFacilitator’s notes● You may want to use or adapt Measuring up workshopPowerPoint 9 – Module 3: Selecting indicators for thepresentations during this session. You have a maximum of 30minutes.● Prepare resources for each pair of participants in advance.Instructions1. Introduce the name of the session and its place in theevaluation design framework flip chart.2. Remind participants that they will need their learner’s guideduring the presentation. Give the presentation.3. Explain to participants that a quick learning activity will followto help familiarise them with the indicators in the learner’sguide and give them practice in writing tailored indicators.The activity will be done in ‘buzz pairs’ (people from differentorganisations). Hand each pair an indicator selectionworksheet and a copy of the example indicators from theMeasuring up: workshop supplementary facilitation material.Ask the buzz pairs to complete the table with examples ofoutput and outcome indicators. Ask them not to cheat bylooking in the learner’s guide! Tell participants they have 20minutes to work in pairs.4. Circulate around the pairs to help and also to identify goodexamples to share in plenary. Point out these good examplesto the pairs so that you can draw on them in the plenary.5. After 20 minutes ask everyone to stop writing indicatorsand share some examples. Encourage those with the goodexamples you identified to contribute. Look at the examples intable 5 on page 25 of the learner’s guide.6. Hold a brief discussion or ask for any questions.ONE-DAY WORKSHOPTime20 minutesInstructions1. Explain to participants that you will give a presentation on selecting indicators.2. Ask participants if they have any questions. Discuss as necessary.26 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEDAY THREERecap of day twoFacilitator’s notesIt is useful to use a combinationof the workshop timetableand flip charts from output orplenary brainstorms to reviewkey points from the previousday.Instructions1. Facilitate an energiser.2. Provide a recap of the previous day’s sessions.Time30 minutesSESSION 3.7Identifying and choosing data collection methods relevantto advocacy evaluationSession informationInformation for this session canbe found on pages 26 to 36 ofthe learner’s guide.Materials● PowerPoint 10● Copies of the contentsfrom A handbook of datacollection tools: a companionto ‘A guide to measuringadvocacy and policy’ fromthe Measuring up: workshopsupplementary facilitationmaterial. One copy for eachparticipant and the facilitator.Time2 hours (not including therefreshments break)Facilitator’s notes● If participants were not already aware of the seven datacollection methods before the workshop, they may have readthe learner’s guide overnight and now be familiar with theseapproaches.● You may want to use or adapt Measuring up workshopPowerPoint 10 – Module 3: Identifying and choosing datacollection methods relevant to advocacy evaluation for thepresentations during this session.● Keep an eye on the time and make sure that you stop for therefreshments break.● Participants are likely to ask where to access more informationabout these methods. There is limited information availableat the moment, but they should keep searching the Internetas this situation may change quickly. Recommend thatparticipants who wish to track the work of key authorssubscribe to The evaluation exchange periodical, which is freeof charge (google ‘The evaluation exchange’). This is a forumfor sharing new and innovative data collection methods,including for advocacy and policy evaluation.Instructions1. Introduce the name of the session and its place in theevaluation design framework flip chart. Explain that thesession will be divided by a refreshments break.2. Explain that data collection methods are the processes orapproaches used for collecting data to measure our indicators.3. Ask participants: “Can you identify some common datacollection methods? Let’s brainstorm them.”27 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEONE-DAY WORKSHOPTime40 minutesInstructions1. Ask participants: “Can youidentify some common datacollection methods? Let’sbrainstorm them.”2. Ask participants: “Do youknow of any new datacollection methods designedfor advocacy evaluation?”3. Explain to participants thatyou will give a presentationoutlining seven data collectionmethods for advocacyevaluation.4. Ask participants if they haveany questions. Discuss asnecessary.4. Ask participants: “Do you know of any new data collectionmethods designed for advocacy evaluation?”5. Explain to participants that you will present data collectionmethods for advocacy work. This will include an introductionto publications that include relevant tools, an overview ofcommon data collection methods and an explanation of sevendata collection methods useful for advocacy evaluation. Tellthem that all the information for this presentation can befound in the learner’s guide in section 3.7, on pages 26 to 36,except for the contents of A handbook of data collection tools:a companion to ‘A guide to measuring advocacy and policy’,which will be handed out now. Explain that we will be lookingat the seven data collection methods in more detail in groupwork, so participants should write down any questions theyhave during the presentation. Give the presentation.6. After the presentation, facilitate a discussion with theparticipants about the seven evaluation methods based on thefollowing kinds of questions:● “Has anyone in your group used this data collectionmethod before?”● “What do you think are some of the key strengths ofthis method?” (Don’t forget to point out any particularstrengths for evaluating HIV-related advocacy)● “What aspects of this method may be challenging?” (Don’tforget to point out any particular challenges for evaluatingHIV-related advocacy.)Additional examples of the use of specific methods*Bellwether methodology can be used at thenational level in southern African countries“International HIV/AIDS Alliance partners isSouthern Africa were interested in whetherthe Bellwether methodology is appropriatefor evaluating advocacy for treatment access.The answer is ‘it depends’ – on the context,method and indicators. For instance, if there waspolitical and health system support for accessand advocacy was directed around pricing andsuppliers, then the best indicator would simplybe the number of people enrolled in treatment,which could be verified from clinic data. However,if there was an affordable treatment supply, butproblems of stock-outs and lack of commitmentfrom the health system or health minister toaddress this, the advocacy intervention might beto try to raise awareness of the consequencesof stock-outs among other influential peoplewho could persuade the minister to act. Thenbellwether could be a helpful evaluationtechnique”. – Measuring up contributor.The intense period debrief method can be used atthe national level in KenyaSee the example provided for the partnershipanalysis tool (learner’s guide, page 33).KANCO and its partners could ask the followingintensive period debrief questions:● What events triggered the need for thisintense period of advocacy?● How would you describe the political contextof this period concerning the HIV and AIDSPrevention and Control Act?● What was the general public mood concerningthe HIV and AIDS Prevention and Control Actwhen the advocacy action began?● What did KANCO do well? What did not workwell and could have been improved?● What aspects of the advocacy partnershipworked well and what did not? How couldsuch an advocacy partnership be improved inthe future?● Which advocacy strategies employed were themost and least effective in achieving change?● As a result of this intense period of advocacyactivity, what new opportunities have arisenfor the legislative campaign for the HIV Actgoing forward?● What insight will you take away from theexperience that could inform advocacystrategy development in the future?* Not included in the learner’s guide or PowerPoint presentations28 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEMODULE 4Putting it all into action – introduction to Module 4Time5 minutesInstructions1. Introduce the name of the session and its place in theworkshop timetable.2. Give a brief introduction to this module, including the title andobjectives.Objectives for Module 4● To practise using the seven key componentsof the advocacy evaluation design frameworkfor decision-making on how to evaluate HIVrelatedadvocacy.● To reflect on the key messages and learningpoints from the workshop to share withcolleagues.SESSION 4.1Consolidating learning on how to evaluate HIV-relatedadvocacySession informationInformation for this sessioncan be found on page 37 of thelearner’s guide.Materials● Participants’ previous groupwork activity outputs fromsessions 3.2 and 3.3● A copy for each participantof the group work activityinstructions on page 29● Flip chart paper, flip chartpens and tape (Stickie Stuff,Blu-Tak, etc.)Time3 hours (not including the lunchbreak)Facilitator’s notes● Make sure that you read through the example theory ofchange carefully before facilitating this session.● Keep an eye on the time and make sure that you stop for thelunch break● Prepare resources for each group and ensure there is a spaceto work, with a table and correct number of chairs (or floor, ifappropriate). Make sure you do this in advance.● Circulate around the groups to help them get started with theactivity and then monitor their progress periodically to helpthem keep time and address any questions they might have.● Identify which group you would like to present their work andwarn them in advance.Instructions1. Introduce the name of the session and its place in theworkshop timetable.2. Explain to participants that they will work in their organisationgroup using their previous work on developing a theoryof change to help them practise designing an advocacyevaluation in line with the last four components of theadvocacy evaluation design framework. (Use the advocacyevaluation framework flip chart to point to these lastfour components – evaluation questions, approaches tomeasurement, indicator selection and selection of datacollection methods.)29 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREE3. Introduce the group work instructions to participants using aprepared flip chart.4. Hand out a copy of the activity instructions.5. After two hours ask participants to return to plenary. Facilitatea 45-minute discussion about the activity by asking one groupto present its work. Invite other participants to comment ontheir experience of the activity, including what they learnedfrom the process.Group activity instructions:1. You have two hours to complete this activity.2. Answer the following questions to help you to practise designing anadvocacy evaluation related to your work:● What assumptions have been made in the design of your advocacyeffort? (Learner’s guide, Module 3, section 3.2, pages 12 to 17)● What evaluation questions would be informative for this advocacywork? (Learner’s guide, Module 3, section 3.4. page 19)● What approach(es) to measurement do you think would be appropriatefor this advocacy work? (Learner’s guide, Module 3, section 5, pages 20and 21)● What would you prioritise for this evaluation and what evaluationquestions would be useful for the end users? (Learner’s guide, module3, sections 3.3 and 3.4 pages 18 and 19)● What key indicators will allow you to measure the results of thisadvocacy work? (Learner’s guide, Module 3, section 3.5, pages 22 to 25)● What are some sources and/or means of data collection for theseindicators? (Learner’s guide, Module 3, section 3.7, pages 26 to 36.)3. You may be asked to present your group work, so write the answers to yourquestions on flip chart paper.30 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEONE-DAY WORKSHOPTime2 hours 30 minutesFacilitator’s notes● Make sure that you read through the example theory of change carefully before facilitating thissession.● You could use the Lifeboats energiser, see page 34, to create the groups of four people.● Each group will need the following resources for the group work (make sure you prepare these inadvance):● one copy for each participant (four copies per group) of the group work activity instructions,case study and example theory of change printed off from the Measuring up: workshopsupplementary facilitation material● flip chart paper, flip chart pens and tape (Stickie Stuff, Blu-Tak, etc.)● a space to work, with a table and correct number of chairs (or floor, if appropriate).● Circulate around the groups to help them get started with the activity and then monitor theirprogress periodically to help them keep time and address any questions they might have. Identifywhich group you would like to present its work and warn the group participants in advance.Instructions1. Introduce the names of sessions 4 and 4.1 and their place in the workshop timetable.2. Introduce the objectives for Module 4.3. Explain to participants that they will work in groups of approximately four people to practiseapplying the advocacy evaluation design framework to an example.4. Review the case study example and related theory of change for improved access to harmreduction services for injecting drug users included in the Measuring up: workshop supplementaryfacilitation material.5. Introduce the group work instructions to participants using a prepared flip chart.6. Hand a copy of the activity instructions, case study and example theory of change to eachparticipant.7. Use an energiser to create the groups of four.Group work instructionsa) You have 1 hour 30 minutes to complete this activity.b) Read through the case study and example theory of change as a group. Do not debate thedesign of the advocacy effort. The evaluation will be conducted after three years of this fiveyearfunded advocacy initiative. The primary users of the evaluation will be the advocatesthemselves and donors, and the secondary users will be other advocates wishing to advocatefor harm reduction. The evaluation will be used to (i) demonstrate the effectiveness of theadvocacy coalition, which is funded through a lead partner; and (ii) provide an opportunity forthe advocates to learn more about the effectiveness of one or more approaches for improvedstrategy development and evidence-based learning for future advocacy work.31 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEONE-DAY WORKSHOP (CONTINUED)c) Answer the following questions to help you to practise applying the information coveredduring the workshop:● What assumptions have been made in the design of the advocacy effort?(Learner’s guide, Module 3, section 2, pages 12 to 17)● What elements of the advocacy work would you prioritise for evaluation?(Learner’s guide, Module 3, section 3.3, page 18)● What evaluation questions would be informative for this advocacy work?(Learner’s guide, Module 3, section 3.4, page 19)● What approach to measurement do you think would be appropriate for this advocacywork?’ (Learner’s guide, Module 3, section 5, pages 20 and 21)● What key indicators will allow you to measure results of this advocacy work?(Learner’s guide, Module 3, section 3.6, pages 22 to 25)● What are some sources and/or means of data collection for these indicators?(Learner’s guide, Module 3, section 3.7, pages 26 to 36)d) You may be asked to present your group work, so write the answers to your questions on flipchart paper.Please note that there are no right or wrong answers. We ask that you think back to what hasbeen presented so far, as well as to your experience on the ground.8. After 1 hour 30 minutes ask participants to return to plenary. Facilitate a 40-minute discussionabout the activity by asking one group to present their work and inviting the other participantsto comment and suggest amendments, additions, etc. Make sure that you and the participantscritique the ideas sufficiently in order to create a good example.Activity example output: using the people who use drugs case study from the supplementaryfacilitation materials to consolidate learning on how to evaluate HIV-related advocacyWhat assumptions have been made in the designof the advocacy effort?● The government officials and police authorityrepresentatives that are accessible fortargeting with advocacy messages andtraining are those who have enough influenceto achieve change.● Education concerning the public healthbenefits of harm reduction services will allowindividuals from the government, police andpublic to overcome their prejudices and thebarriers/obstacles put in place of access toharm reduction services.What elements of the advocacy work would youprioritise for evaluation?● Were the right government officials andpolice authority representatives targeted withadvocacy messages and training?● Effective joint actions and aligned messagingof the coalition.● Changes in political will/commitment to adoptpolicies that are sensitive to the needs ofpeople who use drugs.● Changes in the position of harm reduction onthe policy agenda for HIV prevention.32 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEActivity example output: (continued)What evaluation questions would be informativefor this advocacy work?● Were the government officials/policeauthority representatives invited to roundtablediscussions/training and targeted withcoalition messages well selected to affect thenecessary change?● How effective has the coalition been in itsmessage development and targeting forgovernment officials and the police authority?● Has the coalition been able to contribute toincreased political will to support improvedaccess to harm reduction services andpolicies or legislation that supports this? If so,how? If not, why not?● Has the coalition been able to contribute toimproved policies or legislation that supportsthis improved access to harm reductionservices? If so, how? If not, why not?● To what extent has the coalition’s mediacampaign contributed to progress to improveaccess to harm reduction services?What approach to measurement do you thinkwould be appropriate for this advocacy work?Approach A: identification and measurement ofcore outcome areas related to social or policychange.What key indicators will allow you to measurethe results of this advocacy work?● Number of civil society organisationstargeted to join the advocacy coalition.● Percentage of civil society organisationstargeted to join the advocacy coalition thathave joined the coalition.● Number and type of organisations that havejoined the coalition.● Number of incidences that collation keymessages are used favourably in the media.● Number of incidences that collationkey messages are used favourably bygovernment officials.● Improved political will of government officialsto improve access and reduce barriers toharm reduction services for those that needthem.● Number of policy proposal documentsaccepted for consideration by governmentauthorities.● Increased position of harm reduction servicesfor people who use drugs on the policyagenda for HIV prevention.What are some sources and/or means of datacollection for these indicators?The following data collection methods can beused:● Policymaker rating to assess increase inpolitical will for selected government officials(data collection every three months).● Bellwether methodology to determine theplacement of harm reduction on the policyagenda for HIV prevention (data collectionevery six months).● Use of a systems map to identifyrelationships between government officials,police authorities, service providers and thepublic in terms of the barriers to access ofharm reduction services.● Media monitoring for inclusion of coalitionmessages in the media.● Media monitoring, meeting observationchecklist and review of documentationof parliamentary committee proceedingsto identify use of coalition messages bygovernment officials.What challenges do you anticipate?● Turnover of government officials.● Opposition to messaging from counterlobbyinggroups.● Gaining access to decision-makers within thegovernment and police authority.● Number of training events/round tablesfor government officials/police authoritiesprovided on the issue.● Number of government officials/policeauthority representatives attending roundtable discussions/training events on the issue.33 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREEIdea for an energiser: lifeboatsGather everyone into a space. Explain that they are on a boat in a thunderstorm (mimebeing tossed about by the waves). Suddenly the boat capsizes and they must get to thelifeboats with a specified number of people. Anyone not in the correct-sized group drowns!Shout:”Lifeboats of three” (participants must rush to huddle in groups of three). Then repeat,changing the number for the lifeboats. Finish with the number of people you want in eachgroup for the group work activity.SESSION 4.2Reflecting on the key messages and learning points fromthe workshop to share with our colleaguesSession informationSince we often attendworkshops as a singlerepresentative, we know thatnothing will change unless wetransfer the knowledge we nowhave back to our colleagues atour civil society organisations. Itis important that we do not leavea workshop without thinkinghow to achieve this so that thelearning and problem-solving isnot lost.Facilitator’s notesParticipants should be able tostay where they are seated todo the task. However, some mayprefer to move to the edge ofthe room or nearby where thereis more space to work and think.Time40 minutesInstructions1. Introduce the name of the session and its place in theworkshop timetable.2. Explain to participants that you would like them to completea task in the next 30 minutes. They should focus on theirown organisation for the task and work alone or with othercolleagues from their civil society organisation if they arepresent. If participants are on their own, they can sit withsomeone from another civil society organisation to swap notesand get advice and support, but they should focus on theirown organisation. Remind participants that this is valuablequiet time for them to think about the future. Read out thetasks and then hand them out to participants.TaskUsing your learner’s guide and extra handouts, write adetailed bullet-point plan of what information from thisworkshop you should share with colleagues back at your civilsociety organisation (or coalition) and how. Consider someof the insights and new ideas you have learned from theworkshop and how your organisation can respond to them.3. With two minutes to spare before the end of the session,call participants back to plenary and ask: “What kinds ofinformation or key points will you share with your colleaguesback at your civil society organisation?”34 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREETask example: Reflecting on the key messages and learning points from the workshop to share withour colleagues● Why advocacy evaluation is different and some ways to get around the challenges.● The concept of the theory of change and its usefulness for:● strategic planning of our advocacy work● sorting out our strategies, outcomes and goals and how they are related● acknowledging our assumptions and considering them seriously● prioritising what to focus our evaluation resources on.● The difference between output and outcome indicators for advocacy efforts at different levels andover different time frames.● The different kinds of indicators we can use around interim outcomes, such as political will andthe placement of issues on a policy agenda. Also the importance of evaluating our own network’sability to respond to advocacy opportunities effectively.● All the methods of data collection, but particularly the systems mapping and policymaker ratingwhich we can use internally.These points will be covered during a special session with our organisation and throughquarterly coordination meetings with our local partner organisations.Source: Measuring up field-test workshop, India, 2010ONE-DAY WORKSHOPTime20 minutesInstructionsImplement the activity in session 4.2 but only allowing 15–20 minutes for the task.35 MEASURING UP \\ A GUIDE FOR FACILITATORS

DAY THREECLOSING SESSIONWorkshop summary, post-workshop questionnaire andworkshop closeTime45 minutesInstructions1. Provide a 20-minute summary of the workshop content andhow this relates to the content of the Measuring up guide as awhole.2. Thank participants for taking part in the workshop and makeother complimentary remarks as relevant.3. Explain that you have one final task for them before youwrap up the workshop: completion of the post-workshopquestionnaire. Ask participants to re-score the pre- and postworkshopquestions on the questionnaire and answer thepost-workshop summary questions at the bottom. Allow theparticipants to collect their own questionnaires using thesymbols or pictures that they drew on the back of them. Givethe participants ten minutes to fill them in. Assign a place forall the questionnaires to be left with the symbol/picture facingdown. Make sure all the questionnaires are handed in.4. Ask participants: “Can you share a few highlights from theworkshop (they can be anything)?”5. Close the workshop with any administrative information andthanks.ONE-DAY WORKSHOPTime10 minutesInstructions1. Thank participants for taking part in the workshop and make other complimentary remarks asrelevant.2. Explain that you have one final task for them before you wrap up the workshop: completion of thepost-workshop questionnaire. Ask participants to re-score the pre- and post-workshop questionson the questionnaire and answer the post-workshop summary questions at the bottom. Allowthe participants to collect their own questionnaires using the symbols or pictures that they drewon the back of them. Give the participants ten minutes to fill them in. Assign a place for all thequestionnaires to be left with the symbol/picture facing down. Make sure all the questionnaires arehanded in.3. Ask participants: “Can you share a few highlights from the workshop? They can be anything.”36 MEASURING UP \\ A GUIDE FOR FACILITATORS

International HIV/AIDS Alliance(International secretariat)Preece House91-101 Davigdor RoadHove, BN3 1REUKTelephone: +44(0)1273 718900Fax: +44(0)1273 718901mail@aidsalliance.orgwww.aidsalliance.orgInternational Council of AIDSService Organizations (ICASO)65 Wellesley Street EastSuite 403Toronto, OntarioM4Y 1G7 CanadaTelephone: +1(416)921 0018Fax: +1(416)921 9979icaso@icaso.orgwww.icaso.orgAVFG 06/10