A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Psychology, 36 of the 47 articles (76.6%) where scales were used to assess SoTL outcomes,
internal reliability was reported. Thirty-one studies (66.0%) reported Cronbach’s alpha values
that were all acceptable (> .70; Cronbach, 1951; DeVellis, 1991), whereas five (10.6%) reported
at least one alpha value that was below the acceptable .70 value. One example of reporting
internal consistency is Maybury’s (2013) study of the influence of a positive psychology course
on several measures of student well-being using previously established scales and reporting the
Cronbach’s alpha of each scale and subscale. The alphas in this study ranged from .72 to .89, all
above the .70 minimum for acceptable reliability (see Chapter 11 in this e-book by Layous,
Nelson, & Legg for more detail of this study). Another example of reporting internal
consistency is a study of a team approach to undergraduate research that used items that the
researchers developed, rather than a previously established scale (Woodzicka, Ford, Caudill, &
Ohanmamooreni, 2015). In this study, the researchers developed a 19-item, 5-point Likertscale
online survey with four subscales. Each subscale had a Cronbach’s alpha above
acceptable, even with a relatively small sample size (Woodzicka et al., 2015). Reporting the
reliability of newly developed items used for the first time is particularly important because of
the lack of established history.
Although, the majority of internal consistency values of the scales published in SoTL in
psychology examined were above the acceptable minimum (83.3% of the studies reporting
acceptable Cronbach’s alphas), there were several instances where at least one scale assessed
was not. One particular study addressed this issue directly, by assessing reliability and noting
that it was not acceptable for one of the subscales and then giving an appropriate rationale for
still using the subscale in the research (Boysen, 2015). Specifically, the rationale was that there
was a strong theoretical relation of the items in the subscale and that the items showed equal
patterns of results; therefore, the researcher kept the subscale in the model even though it was
found to be unreliable (Boysen, 2015). Another example of the use of a scale with poor
reliability is Boysen, Richmond, and Gurung’s (2015) study of model teaching criteria using the
Ten-Item Personality Inventory (TIPI; Gosling, Rentfrow, & Swann, 2003). Although the
Cronbach’s alpha reliability coefficients for four of the five subscales were below .70, the
authors justified their use because of poor reliability reported in previous studies and the fact
that each subscale only had two items, making good reliability less likely (Boysen et al., 2015).
These are good examples of how to handle situations when data that have already been
collected are not reliable or valid.
Another common occurrence in the reporting of internal consistency is listing the reliability of
an established scale from previously published studies rather than reporting the reliability of
that scale in the present study. Of the 47 studies that reported scales, 30 studies reported
Cronbach’s alphas from the current study and six reported internal consistency values from
previously published studies, but not the current sample. An example of using previously
published internal consistency values is a study looking at changing stereotypes of older adults
(Wurtele & Maruyama, 2013), as measured by the Fraboni Scale of Ageism (FSA; Fraboni,
Saltstone, & Hughes, 1990). The authors reported support for internal consistency, construct
validity, and test-retest reliability from previously published work but did not report any
sources of reliability from the current study (Wurtele & Maruyama, 2013). Although using
45