A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Grades
Another common source of already-existing data in the learning environment is grade point
average (GPA). Although GPA is generally thought to be a reliable and valid measure of
academic achievement, and therefore a good proxy for learning, that may not necessarily be
the case (Imose & Barber, 2015). The literature has established that GPA has good internal
consistency, although the test-retest reliability and the comparison of GPA standards across
schools have not been established. For example, in one study focused on business majors,
four-year cumulative GPA was reliable (Cronbach’s alpha = .94), showing excellent internal
consistency, but they were not able to assess test-retest reliability or compare across
institutions (Bacon & Bean, 2006). The different standards employed by universities also
impact comparability. For instance, a 3.5 GPA at one university may make the student in the
top 10% of the class, whereas that same GPA would be in the top 25% at another university
(Imose & Barber, 2015). In addition, although GPA may be measuring general cognitive ability
and conscientiousness, GPA has not been established as a valid measure of learning. For
instance, overall GPA had strong predictive validity of individual measures of academic
performance, but mainly when that performance was reliably measured and was based on the
individual student’s ability and effort, not necessarily their learning (Bacon & Bean, 2006).
Student ratings of instructors and GPA are easily accessible outcome measures in SoTL because
these variables are already being assessed as part of the university class experience. Because of
their accessibility, using already-measured variables reduces the burden on students. However,
the accessibility of these sources of data does not mean that they are the best measures of
learning. Continuing to establish the reliability and validity of these commonly used measures
in SoTL in diverse situations is crucial to ensure confidence that the measures are high quality
and are tapping into the constructs we intend to measure.
Conclusion
Reliability and validity are paramount to researchers across fields to insure that information
gathered and reported is measuring what the researcher wants to measure and measuring it
well. In the SoTL literature in psychology, internal consistency has been assessed and reported
very consistently, while other forms of reliability and validity have not. This is a call for SoTL
researchers to be more thorough in their reporting of psychometrics. Publishing the reliability
and validity of scales for every new study will elevate the quality of the body of research in
SoTL. Assessing the psychometric properties of newly developed scales is also important to
provide researchers with novel, reliable, and valid ways to measure SoTL outcomes. Continuing
to report the reliability and validity of previously established scales also engenders more
confidence that the ways in which we measure SoTL outcomes are psychometrically sound in
varied environments and with diverse samples. Particularly important is increasing the focus on
establishing the validity of scales, especially due to the finding that the reporting of validity was
significantly lower than reliability in recent SoTL in psychology. Lastly, in looking at some of the
common ways we measure learning, particularly student ratings of instructors and GPA, we
must do better in standardizing and measuring reliability and validity so that we can build a
solid track record for the most commonly used measures in the field. This would include
studies across institutions with standardized SRI and GPA standards to establish reliability and
49