Dr Viv Wylie, Consultant, Action on Access ... - Universities UK

Lessons fromAimhigherEvaluation<strong>Viv</strong> <strong>Wylie</strong>

Aimhigher:what was it?• National initiative delivered at local level• Aim to increase numbers of disadvantagedlearners entering HE, and learners withdisabilities• Target group largely young learners on C/Dborderline re GCSE achievement – not fordisadvantaged highfliers• Focus on outreach, partnerships, learnerprogression, integrated programmes

Evaluation• Annual monitoring from outset• Annual evaluation from 2008 in accordance with nationalmodel.• Rigorous targeting using IMD/NSSEC data and engagingLAs and teachers• Use of KPIs - GCSE results, continuation and progressionrates• Use of qualitative data – focus groups, questionnairesinvolving learners, teachers, parents to informquantitative data and demonstrate attitudinal shift• Increasing focus on VFM as KPI

What happened?• Difficult to impose standard model on diversity ofpartnerships with differential funding and locallyinflected provision• Data collection was onerous. Data sharing was frequentlydifficult. Criteria for disadvantage were problematic.Many but not all partnerships worked through the issues.• Systems had to be developed quickly especially fortracking – probably most robust in larger partnershipsand at regional level.• Partnerships found that good evaluation shared withpartners was a powerful tool for improvement andencouraged participation in the process

Outcomes• Aimhigher evidence focused on the student rather than theactivity• Tracking seemed to show that the more students participatedthe more they improved their achievement at GCSE.Continuation and progression rates also improved.• Participation in intensive activities (mentoring, summerschools) was most effective in raising aspiration (cf individualevaluations)• Qualitative evidence appeared to show that Aimhigheractivities built confidence, increased knowledge andunderstanding of HE, improved motivation and so impactedon achievement.• Some evidence showed a positive impact on school culturesand a greater understanding among teachers of the potentialof the target group.

But….• The discrete “Aimhigher effect” could not be proved butthere was a strong association between improvedoutcomes for learners and Aimhigher programmes.• Methodologically there were flaws – no control groupand too much diversity within generic activities• Where integrated programmes were effective it wasimpossible to tell what improvements were due toschools’ interventions and what to Aimhigher activities.• What could be asserted was that there was evidencethat the Aimhigher cohort performed better on a rangeof KPIs than their peers and that stakeholders attributedchanged behaviours to Aimhigher support.

Now?• Different context – more diversified, smaller scaleoperations, fewer partnerships• Competitive environment• Widening participation addresses student lifecycle• Evaluation smaller scale, more focused on manageableoutcomes, and with methodologies devised by HEIsrather than to national models• But national framework for evaluation proposed ininterim report by HEFCE and OFFA

Lessons forOutreach• Build in evaluation from the outset: define what you want toknow and what evidence you will require• Focus on the learner and on attitudinal change together witheffectiveness of activities. Track where possible. Usequalitative evidence to link interventions with raisedaspirations and attainment.• Target stringently – enlist help of schools to refine targeting atpupil level• Evaluate partnerships and positive change within schoolcultures• Explore the Aimhigher thesis that more participation leads togreater improvement in attainment• Remember that evaluation takes time to deliver useful results.