A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Recommendations

Info

Review Scholarly Literature Administer Survey Develop Pool of Items Pilot/Field Test Expert Review Figure 1. Scale creation process depicted as an iterative process. Note: this process should be interpreted loosely, as one might return to any step at any time, depending on the outcome. For example, comments on an expert review might result in returning back to the scholarly literature and so on.) Use of Existing Scales Researchers commonly employ questionnaires to collect data, especially in applied research (Boynton & Greenhalgh, 2004). Whenever possible, researchers should locate and employ previously developed scales with evidence of validity and reliability. One benefit of using existing scales is that there are generally published studies with data that researchers can compare to the data obtained in subsequent studies. When researchers conduct multiple studies using the same scale, they can build a body of literature and theory. Further, they need to devote fewer resources, especially time and energy, when using existing scales. Many of the chapters in this e-book are devoted to sharing such scales that researchers can use in their SoTL research. For example, the Teacher Behaviors Checklist (TBC; Buskist, Sikorski, Buckley, & Saville, 2002) is a popular measure used in SoTL research (see Kirk, Busler, Keeley, & Buskist, 2015). It is discussed further in Chapter 13 of this e-book. However, researchers might find that preexisting scales do not measure certain elements of a construct. For example, Wilson, Ryan, and Pugh (2010) noted that, although there were existing measures of rapport, none specifically addressed professor-student rapport. As a result, they sought to develop such a measure. Also discussed further in Chapter 3 of this e-book, researchers sometimes use scales before there is sufficient research to support their validity and reliability (Christopher, 2015). As such, they might draw seemingly significant conclusions from the application of new scales, only to have the findings of subsequent studies contradict their findings (Cook, Hepworth, Wall, & Warr, 1981). In such cases, it might be appropriate for the researchers to modify existing 30
scales or develop new ones. In either case, it is important for them to return to the scholarly literature to collect as much relevant information as possible on the construct of interest to inform scale development. Measurement Theories There are two major underlying theories upon which quantitative measurement is founded: classical test theory (CTT) and item response theory (IRT). Representing two distinct frameworks, they provide the foundation from which researchers often build instruments. Using these theories, researchers can obtain information about the quality of the items that they develop. Given that there are entire chapters and books devoted to these topics, the purpose of this section is to provide a general overview (see Hambleton & Jones, 1993, and de Champlain, 2010, for an overview and comparison of these theories). CTT CTT describes a set of psychometric procedures used to test scale reliability, item difficulty, and item discrimination (i.e., the extent to which an item helps the research to differentiate between respondents with higher and lower abilities). Many widely available statistical packages produce these statistics. There are several classical test theories, all of which are founded upon the assumption that a raw score is comprised of a true score and a random error (Kline, 2005). In the context of teaching and learning, a true score might reflect a student’s true ability. In this case, the unsystematic influence of any factors on the measurement of that ability would be referred to as random error. In addition, the overriding concern of CTT is to manage the random error. The less random error there is, the more the raw score reflects the true score. Due to its relatively weak assumptions, CTT can be used in a wide variety of testing situations (Hambleton & Jones, 1993). However, the main criticism is that statistics that are generated in terms of the observed score, item difficulty, and item discrimination are dependent upon the sample (Fan, 1998). IRT Also known as modern test theory, IRT allows researchers to overcome many of the limitations of CTT. It can be used to model respondent ability using item-level performance, rather than aggregate test-level performance. Many times, the variable that researchers want to study, such as intelligence, cannot be measured directly; therefore, they create a number of items that they believe capture it (Noar, 2009). Such variables are referred to as latent variables. IRT involves establishing a model that specifies the probability of observing each response to an item as a function of the latent trait being measured, which is often knowledge, a skill, or an ability (DeVellis, 2012). The item response function illustrates the relationship between a latent variable and the probability of endorsing an item. It can then be converted into an item characteristic curve, which shows respondent ability as a function of the probability of endorsing the item. 31
Page 1 and 2: A COMPENDIUM OF SCALES for use in t
Page 3 and 4: Table of Contents Chapter 1: Introd
Page 5 and 6: Chapter 1: Introduction Rajiv S. Jh
Page 7 and 8: and boredom; whereas the former inc
Page 9 and 10: References Bishop-Clark, C. & Dietz
Page 11 and 12: Chapter 2: Best Practices in Scale
Page 13 and 14: o Participants respond to items suc
Page 15 and 16: - Be aware of whether the scale is
Page 17 and 18: References References marked with a
Page 19 and 20: *Wielkiewicz, R. M. & Meuwissen, A.
Page 21 and 22: example, McCabe (2015) examined the
Page 23 and 24: psychometrically sound tool. Howeve
Page 25 and 26: a history of the authors’ work do
Page 27 and 28: *Oleson, K. C., Poehlmann, K. M., Y
Page 29: Chapter 4: A Primer on Scale Develo
Page 33 and 34: found their 40-item measure could b
Page 35 and 36: eflect overall teacher behaviors or
Page 37 and 38: Although there is some variability
Page 39 and 40: and/or do not correlate with instru
Page 41 and 42: Fraley, R. C., & Vazire, S. (2014).
Page 43 and 44: Streiner, D. L. (2003). Starting at
Page 45 and 46: Psychology, 36 of the 47 articles (
Page 47 and 48: Assessing and Reporting Validity In
Page 49 and 50: Grades Another common source of alr
Page 53 and 54: model of collaborative undergraduat
Page 55 and 56: Chapter 6: Measuring Learning and S
Page 57 and 58: joins a new team to teach them abou
Page 59 and 60: Summative Assessments Summative Rub
Page 61 and 62: programs. In a national survey of p
Page 63 and 64: Course Outcomes Scale Centra and Ga
Page 65 and 66: discuss The Motivated Strategies fo
Page 67 and 68: The Motivated Strategies for Learni
Page 71 and 72: Hacker, D. J., Bol, L., & Bahbabani
Page 73 and 74: Stoloff, M. L., & Feeney, K. J. (20
Page 75 and 76: interpretation of analogies is an a
Page 77 and 78: Similar difficulties in accurately
Page 79 and 80: How do we balance the need for effi
Page 81 and 82:
References References marked with a
Page 83 and 84:
*Halpern, D. F. (2010). Halpern Cri
Page 85 and 86:
Appendix: A Compendium of General C
Page 87 and 88:
Chapter 8: Student Engagement Towar
Page 89 and 90:
particular items. Indeed, a point o
Page 91 and 92:
employing the SCEQ have shown other
Page 93 and 94:
one of eight achievement emotions m
Page 95 and 96:
Recent efforts (Saddler, Sonnert, H
Page 97 and 98:
Page 99 and 100:
64, 847-860. doi:10.1037/0022-3514.
Page 101 and 102:
*Schaufeli, W.B., Salanova, M., Gon
Page 103 and 104:
course. Pretest and posttest survey
Page 105 and 106:
Perceptions 7 Problem-Solving 1 Sat
Page 107 and 108:
Culture and Issues Knowledge of Cur
Page 109 and 110:
Civic Attitudes and Skills Question
Page 111 and 112:
Scale of Service-Learning Involveme
Page 113 and 114:
engagement and psychological measur
Page 115 and 116:
Giles, D. E., & Eyler, J. (2013). T
Page 117 and 118:
Appendix A: Research Measures for M
Page 119 and 120:
Robby, M.A. -CASQ -SAT/9 Scores -¹
Page 121 and 122:
Steinberg, K. S., Hatcher, J. A., B
Page 123 and 124:
Chapter 10: Measuring Individual Di
Page 125 and 126:
either unnecessary or is referenced
Page 127 and 128:
Park, 2005). The merit of the instr
Page 129 and 130:
Similar factor structures were prod
Page 131 and 132:
Page 133 and 134:
Chapter 11: Measuring Well-Being in
Page 135 and 136:
items, and the PANAS 44 items, and
Page 137 and 138:
Meaning in Life The Meaning in Life
Page 139 and 140:
future,” using a 5-point scale (1
Page 141 and 142:
Domain-Specific Stress and Anxiety
Page 143 and 144:
demonstrate peoples' varying levels
Page 145 and 146:
Gehlbach, H. (2015). Seven survey s
Page 147 and 148:
Ritchie, R. A., Meca, A., Madrazo,
Page 149 and 150:
Chapter 12: Assessing Professor-Stu
Page 151 and 152:
ends with some unanswered questions
Page 153 and 154:
positively about their professors
Page 155 and 156:
The PSRS underwent further validati
Page 157 and 158:
help or hinder the graduate student
Page 159 and 160:
Komarraju, M. (2013). Ideal teacher
Page 161 and 162:
Chapter 13: Effective Tools for Ass
Page 163 and 164:
offer guidance based on their exper
Page 165 and 166:
The Original TBC Study Our approach
Page 167 and 168:
student/faculty rating with critica
Page 169 and 170:
eveals that five qualities (knowled
Page 171 and 172:
Page 173 and 174:
*Wann, P.D. (2001, January). Facult
Page 175 and 176:
Nessie the Mythical Loch Ness Monst
Page 177 and 178:
components of knowledge of cognitio
Page 179 and 180:
a cognitive activity (Cacioppo et a
Page 181 and 182:
teacher? In other words, how do we
Page 183 and 184:
Christopher (2015) also suggests th
Page 185 and 186:
Kirk, C., Busler, J., Keeley, J., &
show all

A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?