A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Chapter 2: Best Practices in Scale Use in SoTL
Regan A. R. Gurung
University of Wisconsin-Green Bay
"Measure what is measurable, and make measurable what is not so" (Galileo Galilee)
Measurement is at the core of robust scholarship of teaching and learning (SoTL, Gurung &
Landrum, 2012). Is my teaching effective? Are my students learning? What predicts student
learning? Most empirical projects and quantitative approaches to pedagogical research involve
measurement and if there is an existing scale to measure what you want to, why reinvent the
wheel? In this chapter I will review some key issues to keep in mind when picking and using
scales. I will overview construct validity so pedagogical researchers remember what to look for
in a scale and I will touch on best practices in writing your own items.
Before diving in, it is important to clarify some terminology usage. In doing descriptive studies
researchers measure many different variables and use a number of different research designs
(Bartch, 2013; Schwartz & Gurung, 2012). In descriptive studies, we want to get a picture of
what is going on in classroom or teaching (e.g., How many students can list the major
approaches to psychology?). In correlational studies we want to measure associations between
variables (e.g., Are the students who took more practice quizzes on the chapter better at listing
the major approaches to psychology?). In experimental designs we want to collect evidence to
see if the changes we implemented, to assignments, lectures, or design, resulted in increases in
learning (e.g. Did the group of students who watched my special Intro to Psych video list more
of the major approaches to psychology?). For each of these major designs, descriptive,
correlational, and experimental, there are a number of ways to measure variables.
Aside from observation or conducting focus groups, which require different forms of
measurement, the majority of pedagogical research in psychology involves assessing students’
attitudes and behaviors with surveys and measuring their learning with exams, quizzes, and
other performance outcomes. Surveys or questionnaires are general terms and each survey
can contain many items or questions. Sometimes the researcher generates the questions
themselves. Often the researcher uses a preexisting published scale. A survey can hence
comprise of many scales, many questions (i.e., never before been used or published), or a
combination of both. There are best practices to both the selection and use of scales, and the
creation of your own items or questions. I will first cover some best practices regarding scale
use and choice and then briefly review pointers for item construction.
Measure the Usual Suspects: Key Variables in Learning
There is a long history of research on teaching and learning (Gurung & Schwartz, 2012). A wide
range of disciplines study the factors influencing learning, with psychology and education
playing a major role, although there are also significant studies of the neuroscience of learning
(Doyle & Zakrajsek, 2013). The average pedagogical researcher will not be popping students