46<strong>Creating</strong> a <strong>Mentoring</strong> <strong>Programme</strong> <strong>for</strong> <strong>Sport</strong>7CollectionData collection is about obtaining useful data, not simply collecting data! It is about establishing a factual basis<strong>for</strong> making in<strong>for</strong>med decisions – what may have started as ‘I think the problem is...’ becomes ‘The data indicatethat the problem is...’. Data collection helps you assess the health of the programme, and it helps if you havesome quality standards against which to compare your findings. The following checklist should help you answerthe question ‘How do we obtain useful data?’.Question Example What Will I Do?Why am I undertakingan evaluation?• To satisfy management/budget holders• To see if aims are being met• To improve the service we offer• To measure outcomes/outputs• To build ownership within menteesand mentorsWhat evidence will I collect? • Mentee and mentor views• Management team views• Changes in knowledge, behaviours,skills of mentee• Success of the induction andtraining via mentor reflections• Changes to process andprogramme delivery• Frequency of meetings/activitiesHow will I collect data? • Baseline assessment /reflection tools• Questionnaires• Interviews• Observation• Document review• Focus groupWhen will I collect this evidence? • At the start of the programme• Quarterly (periodic reviews)• At the conclusion of the project• Ongoing ad hoc feedbackWho will be responsible <strong>for</strong>collecting the evidence?• <strong>Programme</strong> coordinator• Administration/business support• Mentors• Mentees• External parties
<strong>Creating</strong> a <strong>Mentoring</strong> <strong>Programme</strong> <strong>for</strong> <strong>Sport</strong>47Demonstrating impactIf your project is to continue, it is important that youare able to demonstrate its impact. You will need tobe able to evidence that its outcomes have beenachieved and that the participants involved havebenefited. This process will involve setting successmeasures (the criteria against which the effectivenessof your programme can be judged) and assessingagainst these. Outcomes are the difference a projectmakes to the lives of those involved, <strong>for</strong> example:• changes in attitudes or behaviours• development of skills/expertise• achievement of specific qualifications• access to new opportunities• attainment of specific goals/targets.You should refer back to the planning andprogramme design stage and consider the initialreasons <strong>for</strong> implementing the programme. What wasthe problem your programme was trying to address?What activities did you want to undertake? Whatoutputs did you hope <strong>for</strong>? Did these lead to thedesired outcomes/benefits you hoped theprogramme would bring about <strong>for</strong> mentors andmentees? These must have been identified prior tothe commencement of the programme to ensureyou can collect the right ‘useful in<strong>for</strong>mation’.Activities and outputsIf you remember from the planning stage, outputsare the things that mentees and mentors get as aconsequence of being involved in the programme –often, tangible things they can get their hands on,such as products and resources, or experiences/events, such as orientation, training and additionalcontinuous professional development (CPD). Theseare what you provide in order to bring about thedesired outcomes/programme benefits.OutcomesOutcomes may be split into those that are shortterm (ie the immediate effect of the programme),intermediate (ie the positive changes you hope willtake place as a result of the service you offer) andlong term (ie the lasting significant results of yourprogramme). Demonstrating impact isn’t somethingthat can be rushed or done quickly as changes inbehaviour and attitudes often take years to embed.However, in reality, we do not always have years todemonstrate that the work we are doing is having apositive effect and meeting its aims. There<strong>for</strong>e, inorder to begin to demonstrate the impact of yourwork, it is good practice to focus your attention onmeasuring the intermediate outcomes of yourprogramme, with a view to tracking some elementsof the programme/relationships over a longer periodof time. This process will allow you to gather data toevidence that you are meeting the aims andobjectives in the shorter term, which will contributeto a measure of impact in the longer term.When assessing your outcomes, it is important tonote that not all outcomes will have been plannedor expected. You may find that there are a variety ofunexpected outcomes. These may provide addedvalue and are evidence of additional benefits of theprogramme (previously not considered). However,they may include negative elements that will need tobe dealt with to avoid their impact on futureprogramme delivery.Bringing activities, outputs andoutcomes togetherIt is important that you are able to link the activitiesand outputs with the outcomes as this will helpprovide evidence that changes in behaviour,knowledge, skills etc are as a consequence of theservice you have been providing. Ensure that eachoutput is linked to a specific outcome (ie everyactivity has a purpose and contributes to theend goal).Note: While this process may help add weight to theview that it is your programme that has broughtabout change, a direct cause and effect link is difficultto make.Success indicatorsWith clear aims <strong>for</strong> the project in place, you shouldbe able to identify the indicators of success. Successindicators are the observable characteristics of yourprogramme that demonstrate the achievements ofyour programme goal/aim.Where output indicators provide evidence ofactivity, outcome indicators provide evidence thatchange is occurring. See the examples below:OutputOutput: Training eventIndicator:Attendance figuresOutput: SupportingresourceIndicator:Usage numbersOutcomeOutcome: Increasedskill levelIndicator: More proficient<strong>coach</strong>ing observedOutcome: ImprovedconfidenceIndicator: Increasedindependentactivity/decision making7