University Rankings, Diversity, and the New ... - Sense Publishers

REPUTATION INDICATORS IN RANKINGS OF HIGHER EDUCATION INSTITUTIONS
100
75
reputation score 2005
50
25
0
0 25 50 75 100
reputation score 2004
Figure 4. THES ranking – reputation scores 2004–2005
The problems of transforming such a skew distribution into league tables can be
illustrated by changes in reputational ranking over time in the THES ranking. If we
conceive reputation as an expression of social networks we can expect reputation
scores/hierarchies to remain rather stable over time – in particular with regard to
the one-year intervals of many rankings. By comparing the original reputation
scores of the THES ranking in 2004 and 2005 we find a high correlation (r = .93;
cf. fig. 4). Similar results could be shown with CHE ranking data.
Transforming those original reputation scores into league table positions, the
fallacies of the league table approach become evident: In a sample of 200 universities
almost two-third moved by more than ten rank positions (and 41% even more
than 20 positions) within a single year when original scores are strongly correlated.
This example indicates that league table approaches tend to exaggerate the differences
between universities and changes over time and produce artefacts.
27