University Rankings, Diversity, and the New ... - Sense Publishers

More documents

Recommendations

Info

UNIVERSITY RANKINGS AND LEAGUE TABLES and rankings by selecting a restricted number of indicators and asking the web site’s database to provide comparative institutional information on that basis. In so doing, the CHE approach effectively cedes the power of defining ‘quality’ – which, as we have seen, is one of the key roles arrogated by the authors of ranking schemes – to consumers of the ranking system (i.e. prospective university students and their parents or sponsors). Though it is not a logical corollary of the CHE approach, it is instructive to note that the three main interactive rankings are also fairly distinctive when it comes to data sources and the types of indicators used. In all three cases, massive numbers of students are surveyed about their study experiences and levels of satisfaction at their university. In Germany, the CHE conducts regular surveys of approximately 130,000 students and 16,000 faculty, covering nearly 250 higher education institutes. In Canada, about 25,000 students are surveyed each year. In all three cases, between half and two-thirds of indicators are based on surveys of students and – as is the case in the Elsevier and Keuzegids Hoger Onderwijs rankings – the result is a reasonably strong emphasis on Learning Experience variables. These are shown below in table 2. Table 2. Distribution of Indicators in Interactive ‘CHE-Style’ Rankings Beginning characteristics Learning inputs – staff Learning inputs – resources Learning Environment Learning outputs CHE – Germany (2007) – 21 indicators** 1 1 10 5 0 0 2 2 SwissUp – Switzerland (2007) – 36 indicators** 3 4 9 9 1 2 4 4 University Navigator – Canada (2007) – 27 indicators 2 5 12 4 0 0 4 0 ** in both the CHE and SwissUp rankings, the exact indicators used differ somewhat from subject to subject. For SwissUp we have excluded four indicators which appeared to be used only for medical faculties; in the case of CHE, where there is quite significant variation from one subject to another, we took the social sciences categories as a base. In both cases, we have excluded an indicator relating to rent levels in towns surrounding the university as there was no way to fit these into the overall model. Final outcomes Research Reputation It is tempting to see the CHE-style rankings as being a distinct alternative to League Tables and in many ways they are. But on the other hand, we also see that a number of rankings – especially the two Dutch rankings – share a great deal in common with the CHE-style rankings while still superficially resembling US News-style League Tables. Clearly, there is not a single immutable model of university rankings; many approaches are possible and indeed many approaches are 15
USHER AND MEDOW currently being undertaken. In all cases, the basic tools – indicators populated by quantitative measurements – are the same. The biggest difference between rankers is philosophical; traditional rankings with weightings aggregation by definition are trying to find an institution which can be called ‘the best’; CHE-style rankers are using the power of the internet to help consumers find the institutions that is ‘best for them’. CONCLUSIONS Based on this survey of league tables, we can conclude a number of things, notably: – There are vast differences between university league tables in terms of what they measure, how they measure it and how they implicitly define ‘quality’. – Some of these differences appear to be geographic or cultural in nature. There is notable clustering of certain types of indicators and certain types of data sources. Whether this reflects genuine differences in opinion about the definition of what constitutes ‘quality’ in universities or cross-national differences in the collection and availability of data is unclear, although we lean towards the former explanation. The lack of common indicators across countries explains why the large international league tables (Shanghai Jiao Tong and THES) are so reliant on measures of publication outputs and on reputational surveys (respectively), as they are the only indicators that do not rely on governments or institutions to first collect and process the data. – An increasing number of rankings are using student survey data to populate their rankings and as a result there is an increasing bias towards looking at the student experience. While this may seem ideal in some quarters, it is worth stating that very little research has been done in order to ensure that answers to these responses are in fact apples-to-apples. Perception of quality is a very different metric than quantitative measurements such as ratios of professors to students or size of research budgets. Most significantly, if students at school A have significantly higher expectations of service and quality than students at school B, then it is very difficult to interpret results on any questions related to satisfaction. While data about the student experience of the learning environment are surly very useful and important to potential applicants, the comparability of these data across institutions for the purpose of objective ranking is still unclear. – Despite major inconsistencies in the methodologies used to rank universities, there is a surprising level of agreement between ranking systems as to which universities in a given country are ‘the best’. To the extent that different methodologies give differing opinions about the quality of an institution, the variance between observations grows as one moves down the ordinal rankings. – Although the definition of ‘quality’ is contested, league tables by definition impose a ‘one-size-fits-all’ approach to the matter; this is precisely why they are so controversial. As the CHE approach shows, however, league tables are not the only way to approach rankings. Indeed, the spread of the World Wide Web provides collectors of institutional data with an opportunity to democratize rankings 16
Page 1 and 2: G L O B A L P E R S P E C T I V E S
Page 3 and 4: GLOBAL PERSPECTIVES ON HIGHER EDUCA
Page 5 and 6: A C.I.P. record for this book is av
Page 7 and 8: TABLE OF CONTENTS Towards Unified a
Page 9 and 10: STENSAKER AND KEHM There is growing
Page 11 and 12: STENSAKER AND KEHM of higher educat
Page 13 and 14: STENSAKER AND KEHM perspective, the
Page 15 and 16: STENSAKER AND KEHM about their view
Page 17 and 18: STENSAKER AND KEHM according to Mar
Page 19 and 20: STENSAKER AND KEHM Second, and very
Page 22: PART I: METHODOLOGY
Page 25 and 26: USHER AND MEDOW WHAT ARE UNIVERSITY
Page 27 and 28: USHER AND MEDOW With only a few exc
Page 29 and 30: USHER AND MEDOW data from universit
Page 31 and 32: USHER AND MEDOW Table 1. League Tab
Page 33 and 34: USHER AND MEDOW Other regional patt
Page 35: USHER AND MEDOW RANKINGS WITHOUT LE
Page 39 and 40: USHER AND MEDOW National Accreditat
Page 41 and 42: FEDERKEIL REPUTATION INDICATORS IN
Page 43 and 44: FEDERKEIL In particular, reputation
Page 45 and 46: FEDERKEIL Discipline Reputation sco
Page 47 and 48: FEDERKEIL The relationship between
Page 49 and 50: FEDERKEIL 41% 40% percent of cases
Page 51 and 52: FEDERKEIL 100 THES 2005: repu
Page 53 and 54: FEDERKEIL universities. According t

University Rankings, Diversity, and the New ... - Sense Publishers

Create successful ePaper yourself

Delete template?

Save as template?