Through a glass darkly Measuring the social value of universities

Part 311. A Socially Modified Economic Valuation approach for highereducation (SMEV)11.1 Parts 1 and 2 of this report gave an overview of the current situation regarding valuation ofuniversity activities in the UK. We considered why valuation is necessary and gave a ‘helicopter view’ ofthe current evidence base as well as discussing a range of approaches adopted in the third sector andpublic sector, including the concept of public value generation. In this section we go on to propose a new,holistic, approach to measuring the value of higher education which we have called ‘Socially modifiedeconomic valuation’ (SMEV).11.2 It is important to emphasize that SMEV has robust theoretical foundations, which are rooted inuniversally accepted economic theory. 57 Essentially SMEV is based on the application of economic costbenefitanalysis techniques to higher education outputs and follows accepted theory and practice foreconomic appraisals and programme evaluation – as recommended by, for example, the HM TreasuryGreen Book. Our development of SMEV for higher education has drawn on recognised economicsource documents such as the System of National Accounts (SNA 93), 58 Eurostat 59 and World BankProgramme Evaluation Manuals to ensure rigorous compatibility with internationally recognised theoryand practice in economic analysis. Therefore the methodology is not ‘novel’ but well tried and tested bymajor international agencies and used for programme evaluation across the world; all that is new is itsapplication to higher education.11.3 We have tried to make the explanation of SMEV in this report as approachable as possible for thegeneralist reader. The fuller technical detail and explanation of the underlying economic theory can befound in earlier project reports, which also include extensive analysis of how the SMEV approach to thevaluation of outputs is compatible with that recommended by the Office of National Statistics AtkinsonReview of 2006. 6011.4 The SMEV is an output-based measurement approach, which is focussed on the higher educationinstitutions, what they do and what they deliver.11.5 As we have seen from the discussion in Parts 1 and 2, there are numerous other measurementapproaches which are focussed on societal outcomes, or the broader desired ‘impact’ of a development,a policy or an initiative.11.6 The difficulty with many ‘outcomes-based’ approaches is not only that the measurement ofoutcomes can be extremely difficult but, crucially, that they are not necessarily the most appropriateperformance measures for universities. Broader outcomes (e.g. a healthier society) are the main pointof interest for government policymakers looking from an overall planning perspective. However the HMTreasury Green Book itself acknowledges:“ Outcomes are the eventual benefits to society that proposals are intended to achieve…butoutcomes sometimes cannot be directly measured, in which case it will often be appropriate tospecify outputs, as intermediate steps along the way…”11.7 Outcomes (the eventual benefits to society) are often the result of a complex mix of factors; theyare not always easily attributable to a single influence. Universities cannot deliver direct outcomes (e.g.a ‘healthier society’), rather they can deliver outputs (e.g. ‘deliver medical training’), which may or maynot contribute towards the desired societal outcomes. Policy developed to encourage particular outcomestherefore needs to identify the actual outputs that are thought to contribute towards the eventualoutcomes and encourage the production of those outputs.11.8 This is illustrated in Figure 8 below, which presents the component elements of the chain ofprocesses from resources and inputs to outcomes and impact. This is sometimes known as a ‘theoryof change’ or ‘logic model.’ Universities can use resources to produce outputs – but the final outcomesarising cannot be guaranteed and other factors (possibly entirely unrelated) can be involved in theeventual outcome.24