A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Recommendations

Info

ated on a 5-point Likert scale from 1 (never) to 5 (frequent) on how often the professor exhibits the given teaching behavior. For instance, for the behavior of 'provides constructive feedback,' the question would state, “Writes comments on returned work, answers students’ questions, and gives advice on test taking” (Keeley et al., 2006, p. 85). In the case of SoTL scales such as the TBC (or check out The Teacher Immediacy Scale by Gorham, 1988) these measures are, albeit indirect records of specific behaviors, but allow researchers to approximate specific behavioral outcomes. Third, to mitigate some of the pitfalls with self-report scales is to use SoTL scales that are scenario based. For instance, Berry, West, and Denehey (1989) developed a self-report SoTL scale to measure self-efficacy for memory tasks called the Memory for Self-Efficacy Questionnaire (MSEQ). The MSEQ has been widely used and retains acceptable validity and reliability (Berry et al., 1989; West, Thorn & Bagwell, 2003). The measure is comprised of 40 confidence ratings on 10 memory scenarios. Within each scenario the first question describes the simplest memory task for the scenario then the 4 th question describes the most difficult memory task. Individuals are asked to give their confidence rating (0-100%) on their ability to successfully accomplish the memory task. For example, in the MSEQ individuals are given a scenario about their ability to remember a grocery list. The first question (which is the easiest memory task) would state, “If I heard it twice, I could remember 2 items from a friend’s grocery list of the 12 items, without taking any list with me to the store” (Berry et al., 1989, p. 713). Whereas the most difficult memory task would state, “If I heard it twice, I could remember 12 items from a friend’s grocery list of 12 items, without taking any list with me to the store” (p. 713). Confidence ratings are then totaled for each memory scenario (e.g., grocery list, phone numbers, pictures, location, words, etc.). As you can see, SoTL scales such as the MSEQ are selfreport, however they are rooted in past performance and actual real-life examples of what students or teachers may experience. The benefit of these types of measures is that they attempt to obtain external validity by putting the respondents in real-life scenarios. In SoTL research, self-reports are inevitable and do serve a purpose. However, we are not cryptozoologists, we are SoTL scholars. So in order to debunk the mythical creatures such as Bigfoot, it is important that we complement self-report measures with behavioral measures and/or select and create SoTL scales that are rooted in actual behavior. Measuring Yeti’s Metacognition and Use of Learning Strategies Do you think Yeti, aka the abdominal snowman, thinks about thinking or employs effective learning strategies? Likely—not! But if he did, how would we know? For that matter, how do we know our students’ metacognition and use of learning strategies? We could use the previously described MAI by Schraw and Dennison (1994), but it is 52-items long and a little theoretically dated (e.g., metacognitive awareness is more than knowledge and regulation of cognition). If you are combining this with other measures for a study, you run the risk of getting participant fatigue and consequential response sets (e.g., answering 5 to all questions or playing connect the dots). If you want something shorter you could use the Need for Cognition Scale (NCS) by Cacioppo, Petty, Feinstein, and Jarvis (1996). The NCS is a personality measure of metacognition intended to measure the extent to which an individual might need to engage in 178
a cognitive activity (Cacioppo et al., 1996). The NCS consists of 18 questions rated on a 5-point Likert scale and has demonstrated to be quite reliable. However, many metacognitive researchers would suggest that the NCS does not measure metacognition, rather it measures a personality trait. Or you can use behavioral measures such as calibration (although a little tricky to calculate), or judgments of learning or feelings of knowing that are measured on confidence ratings (e.g., 0-100% confident). But this doesn’t measure holistic beliefs about metacognition (i.e., all aspects of metacognition). You may also use the MSEQ, as previously described, but it takes a long time to complete and is hyper-specific to memory and self-efficacy—only one component of metacognition. What truly is needed, is a brief current SoTL measure of metacognition. For example, a researcher could possibly condense the MAI (Schraw & Dennison, 1994) and combine it with behavioral measures such as the MSEQ (Berry et al., 1989) and/or personality measures such as the NCS (Cacioppo et al., 1996). Future SoTL researchers should take up this endeavor to allow SoTL a quick, reliable, and valid metacognition measure that can be easily administered. What if the Yeti was a teacher and you wanted to measure whether or not he was using metacognition in his teaching. How would you measure this? There is a new SoTL area that refers to this process as meta-teaching (Chen, 2013). Chen describes meta-teaching as, Like meta-cognition and meta-learning, meta-teaching, as ‘teaching about teaching’, can serve to design, examine and reflect on teaching. From practiceorientation, it defines what teaching activity is and what it is for, under which theoretical framework it is being carried out, and what experience and rules can be applied to it. Meanwhile, meta-teaching can assist teachers in discovering drawbacks in the teaching system and solving problems. This demonstrates that meta-teaching contains such functions such as understanding teaching, changing teaching and reflecting on teaching. (p. S64) Furthermore, Spring (1985) argued that effective college and university teachers should use the meta-teaching strategies of proper lesson planning and goal setting, reflecting critically on appropriate use of effective instructional strategies to achieve the instructional goals, both formally and informally monitor student learning, and constantly evaluate the efficacy of chosen instructional strategies. To date, there are no SoTL scales that attempt to measure meta-teaching. As such, SoTL researchers and psychometricians should focus on building a meta-teaching inventory (both self-report and behavioral measure), which assess how often teachers engage in these practices. What about measuring how the Yeti (aka students) uses learning strategies? The hallmark and stalwart measure for learning strategies is the Motivated Strategies for Learning Questionnaire (MSLQ) by Pintrich, Smith, Garcia, and McKeachie (1991). The MSLQ is a well-documented measure of motivation and metacognition, with consistently high reliability and validity. See Table 2 for a list of scales and sub scales. The MSLQ is comprised of 15 separate measures with 81 questions on a 7-point Likert scale. The MSLQ has been used in many SoTL studies and can be broken into subscales. Or you could use a SoTL scale developed by Gurung, Weidert, and 179
Page 1 and 2:
A COMPENDIUM OF SCALES for use in t
Page 3 and 4:
Table of Contents Chapter 1: Introd
Page 5 and 6:
Chapter 1: Introduction Rajiv S. Jh
Page 7 and 8:
and boredom; whereas the former inc
Page 9 and 10:
References Bishop-Clark, C. & Dietz
Page 11 and 12:
Chapter 2: Best Practices in Scale
Page 13 and 14:
o Participants respond to items suc
Page 15 and 16:
- Be aware of whether the scale is
Page 17 and 18:
References References marked with a
Page 19 and 20:
*Wielkiewicz, R. M. & Meuwissen, A.
Page 21 and 22:
example, McCabe (2015) examined the
Page 23 and 24:
psychometrically sound tool. Howeve
Page 25 and 26:
a history of the authors’ work do
Page 27 and 28:
*Oleson, K. C., Poehlmann, K. M., Y
Page 29 and 30:
Chapter 4: A Primer on Scale Develo
Page 31 and 32:
scales or develop new ones. In eith
Page 33 and 34:
found their 40-item measure could b
Page 35 and 36:
eflect overall teacher behaviors or
Page 37 and 38:
Although there is some variability
Page 39 and 40:
and/or do not correlate with instru
Page 41 and 42:
Fraley, R. C., & Vazire, S. (2014).
Page 43 and 44:
Streiner, D. L. (2003). Starting at
Page 45 and 46:
Psychology, 36 of the 47 articles (
Page 47 and 48:
Assessing and Reporting Validity In
Page 49 and 50:
Grades Another common source of alr
Page 51 and 52:
Page 53 and 54:
model of collaborative undergraduat
Page 55 and 56:
Chapter 6: Measuring Learning and S
Page 57 and 58:
joins a new team to teach them abou
Page 59 and 60:
Summative Assessments Summative Rub
Page 61 and 62:
programs. In a national survey of p
Page 63 and 64:
Course Outcomes Scale Centra and Ga
Page 65 and 66:
discuss The Motivated Strategies fo
Page 67 and 68:
The Motivated Strategies for Learni
Page 69 and 70:
Page 71 and 72:
Hacker, D. J., Bol, L., & Bahbabani
Page 73 and 74:
Stoloff, M. L., & Feeney, K. J. (20
Page 75 and 76:
interpretation of analogies is an a
Page 77 and 78:
Similar difficulties in accurately
Page 79 and 80:
How do we balance the need for effi
Page 81 and 82:
Page 83 and 84:
*Halpern, D. F. (2010). Halpern Cri
Page 85 and 86:
Appendix: A Compendium of General C
Page 87 and 88:
Chapter 8: Student Engagement Towar
Page 89 and 90:
particular items. Indeed, a point o
Page 91 and 92:
employing the SCEQ have shown other
Page 93 and 94:
one of eight achievement emotions m
Page 95 and 96:
Recent efforts (Saddler, Sonnert, H
Page 97 and 98:
Page 99 and 100:
64, 847-860. doi:10.1037/0022-3514.
Page 101 and 102:
*Schaufeli, W.B., Salanova, M., Gon
Page 103 and 104:
course. Pretest and posttest survey
Page 105 and 106:
Perceptions 7 Problem-Solving 1 Sat
Page 107 and 108:
Culture and Issues Knowledge of Cur
Page 109 and 110:
Civic Attitudes and Skills Question
Page 111 and 112:
Scale of Service-Learning Involveme
Page 113 and 114:
engagement and psychological measur
Page 115 and 116:
Giles, D. E., & Eyler, J. (2013). T
Page 117 and 118:
Appendix A: Research Measures for M
Page 119 and 120:
Robby, M.A. -CASQ -SAT/9 Scores -¹
Page 121 and 122:
Steinberg, K. S., Hatcher, J. A., B
Page 123 and 124:
Chapter 10: Measuring Individual Di
Page 125 and 126:
either unnecessary or is referenced
Page 127 and 128: Park, 2005). The merit of the instr
Page 129 and 130: Similar factor structures were prod
Page 131 and 132: References References marked with a
Page 133 and 134: Chapter 11: Measuring Well-Being in
Page 135 and 136: items, and the PANAS 44 items, and
Page 137 and 138: Meaning in Life The Meaning in Life
Page 139 and 140: future,” using a 5-point scale (1
Page 141 and 142: Domain-Specific Stress and Anxiety
Page 143 and 144: demonstrate peoples' varying levels
Page 145 and 146: Gehlbach, H. (2015). Seven survey s
Page 147 and 148: Ritchie, R. A., Meca, A., Madrazo,
Page 149 and 150: Chapter 12: Assessing Professor-Stu
Page 151 and 152: ends with some unanswered questions
Page 153 and 154: positively about their professors
Page 155 and 156: The PSRS underwent further validati
Page 157 and 158: help or hinder the graduate student
Page 159 and 160: Komarraju, M. (2013). Ideal teacher
Page 161 and 162: Chapter 13: Effective Tools for Ass
Page 163 and 164: offer guidance based on their exper
Page 165 and 166: The Original TBC Study Our approach
Page 167 and 168: student/faculty rating with critica
Page 169 and 170: eveals that five qualities (knowled
Page 171 and 172: References References marked with a
Page 173 and 174: *Wann, P.D. (2001, January). Facult
Page 175 and 176: Nessie the Mythical Loch Ness Monst
Page 177: components of knowledge of cognitio
Page 181 and 182: teacher? In other words, how do we
Page 183 and 184: Christopher (2015) also suggests th
Page 185 and 186: Kirk, C., Busler, J., Keeley, J., &
show all

A COMPENDIUM OF SCALES for use in the SCHOLARSHIP OF TEACHING AND LEARNING

Create successful ePaper yourself

Delete template?

Save as template?