Mapping Diversity: Developing a European Classification of ... - U-Map

78 Reliability
The indicators selected differ in status. Some indicators are already used in different contexts and
build on standard data, whereas others are ‘experimental’ and use information that is not in the set
of commonly reported data. This has consequences for the perceived validity of the indicators (see
above) but it may also have consequences for the perceived reliability of the information reported.
In most higher education systems the defi nitions and data collection procedures for standard data,
like the number of students enrolled or the number of staff, are harmonized. Because of this, it is
more than likely that the data reported are not infl uenced by the person or department that provides
the data. As long as the procedures and defi nitions are followed, the data will be trustworthy. For the
‘experimental’ indicators, defi nitions and procedures are not (yet) harmonized. For these indicators
it might be that the data reported depend on the person or department that reports the data. To
fi nd out whether this reliability problem is perceived to exist by the responding higher education
institutions, they were asked to respond to the statement: ‘the information is reliable’.
The responses are very positive about the reliability of the information provided. For 25 indicators
at least fi ve out of six responding higher education institutions reported that they (strongly) agreed
with the statement ‘information is reliable’. The indicators on which slightly more responding higher
education institutions had some doubts regarding the reliability are 3a and 3b (orientation of
degrees), 6d (revenues from private contracts) and 14b and 14c (regional engagement).
This very positive result may be biased because the person giving the opinion has most likely put
in a lot of effort to collect the information. Avoiding cognitive dissonance may lead the respondent
to an ‘over-positive’ assessment.
MAPPING DIVERSITY