A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

esearchers should modify Cullen and Harris’ rubric to at least have an interval or better scale.
That is, convert the 1-4 categories into Likert-type questions. For instance, the factor of
Evaluation/Assessment and the subfactor of feedback mechanisms, scale question would read,
"The syllabus describes summative and formative evaluations including written and oral
presentations, group work, self-evaluation and peer evaluation" anchored in 1 (strongly agree)
to 5 (strongly disagree) or could have anchors of frequency, such as 1 (always) to 6 (never). This
would allow SoTL researchers to conduct factor analyses, test-retest reliability, split-half
reliability, and convergent, construct, content, and predictive validity studies of this SoTL scale.
Second, other forms of SoTL scales that assess the efficacy of syllabi are needed. It has been
argued that exemplar syllabi should serve as a contract to students, a permanent record, a
cognitive map, a learning tool, and a communication device (Matejka & Kurke, 1994; Parkes &
Harris, 2002). As such, SoTL researchers should devise scales that assess the degree to which
syllabi contain these elements. For example, researchers could use these scales to assess
differences between college and university teachers’ pedagogy. Or, researchers could use these
scales to assess the efficacy of the syllabus design and how it may affect student learning. In the
end, let’s demystify the syllabus (aka Nesse the Lochness Monster) because not only is syllabus
research under-studied in SoTL, there is great room and need to develop and validate SoTL
scales which attempt to assess the efficacy of syllabi.
Prove that Bigfoot is Real! Self-Report SoTL Scales Need Matching Behavioral Measures
Do you know someone who believes in Bigfoot? How do they know? Did they see the elusive
creature? Or did they say, “My cousin once saw Bigfoot” or “My neighbor Billy lost three goats
to Bigfoot”? Unlike the TV show MonsterQuest by the beloved History Channel, it is likely that
most people who believe in Bigfoot do so because of self-report or the report of others. Herein
lies the rub. Somewhat like cryptozoology, many SoTL scales rely heavily on self-report and not
enough on actual behavior. As psychologists, we know all to well the pitfalls of self-report
scales. That is, issues of honesty, introspection, social desirability, understanding and
comprehension, response bias, response set, and on and on. As Baumeister, Vohs, and Funder
(2007) so poignantly said,
the eclipse of behavior…in which direct observation of behavior has been
increasingly supplanted by introspective self reports, hypothetical scenarios, and
questionnaire ratings. We advocate a renewed commitment to including direct
observation of behavior whenever possible and in at least a healthy minority of
research projects. (p. 396)
However, this does not mean that self-report SoTL scales are worthless. Rather, I’m here to
suggest that there are three primary methodological solutions.
First, when using self-report SoTL scales, it is important to have matching or complementing
behavioral measures that support and have consistent results with the self-report scales. Let’s
illustrate this issue through a common measure of metacognitive awareness, the Metacognitive
Awareness Inventory (MAI; Schraw & Dennison, 1994). The MAI has been used and cited in over
1000 studies and is a 52-item inventory that attempts to measure the metacognitive
176