A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Chapter 4: A Primer on Scale Development
Heather D. Hussey and Tara J. Lehan
Northcentral University
Introduction
One of the primary features of contemporary academic, professional, public, and personal life is
a reliance on information and arguments involving numbers. This chapter includes an
introduction to the steps involved in developing and validating instruments to collect numerical
data for use in the study of teaching and learning (SoTL) research. Independent scholars must
be able to evaluate quantitative evidence thoughtfully and critically as well as employ
quantitative skills to make contributions in the workplace and to society. To become competent
creators and consumers of quantitative research, readers must understand how numerical
information is generated, summarized, evaluated, and represented. With these goals in mind,
classical and modern test theories, reliability, validity, factor analysis, and other topics are
covered in some detail in this chapter. However, it is crucial to note that this chapter is only a
starting point for those who are interested in better understanding the use and creation of
instruments in SoTL research. Further, many of the discussions lack the depth necessary to
provide readers with the knowledge needed to begin to develop their own instruments. Only
the most relevant topics are covered at a superficial level due to space limitations.
Although the process of scale creation frequently is iterative, this chapter includes a description
of the general steps involved in developing an instrument, including searching the scholarly
literature to locate and evaluate preexisting instruments and then determining whether they
are adequate or need modification (see Figure 1). If researchers must modify a preexisting
instrument or create an entirely new one, they should return to the scholarly literature to
ensure that they acquire all of the relevant knowledge on the construct(s) of interest. Next, in
conjunction with measurement theories and/or models, researchers use this information to
develop a pool of potential items. Following the creation of this pool, researchers often have a
panel of experts review the items for clarity and conduct an initial screening of the degree to
which they align with construct(s) of interest. Some researchers also conduct a pilot or field test
with their instrument with a small sample. They then implement all suggested changes and
administer the instrument to a larger sample. Following data collection, researchers analyze the
responses. This process often involves factor analysis and assessments of reliability and validity.
Based on the findings, they make necessary modifications and repeat the process and/or
conduct studies to replicate and/or generalize their findings.
29