Abstracts - Earli

K 131 August 2007 08:30 - 10:30Room: KonferenciaSIG Invited SymposiumThe assessment of metacognition.Chair: Marcel V. J. Veenman, University of Amsterdam/ Leiden University, NetherlandsOrganiser: Marcel V. J. Veenman, University of Amsterdam/ Leiden University, NetherlandsDiscussant: Christa van Kraayenoord, The University of Queensland, School of Education,AustraliaIn educational research a serious problem pertains to the disparity amongst definitions of concepts,and, hence, to the lack of congruence in operationalizations of concepts and their assessmentmethods. This is particularly the case for the field of metacognition, i.e., the knowledge about ourcognitive system and the skills for regulating that system. Even as definitions appear to convergeto some extent, until recently a proliferation of assessment methods occurred without thoroughvalidation studies. Questionnaires administered either prior to or retrospective to task performance,observations and thinking aloud during task performance, eye-movement or computer-logfileregistrations during task performance, and post hoc interviews or video-stimulated recallafterwards, belong to the broad range of assessment tools for metacognition (Veenman, Van Hout-Wolters, & Afflerbach, 2006). All methods suffer from a trade-off between advantages anddisadvantages. Some may be easy to administer, but raise doubts about what is actually beingmeasured; others are hard to obtain, but do reflect on-line metacognitive activities during taskperformance. Quite often, only the internal consistency of metacognition measures (reliability) isreported, rather than the convergent validity with other assessment methods that intend to assessthe same construct (Veenman, 2005). Contributions to this symposium will show that multimethoddesigns are essential for establishing construct validity. Moreover, a second criterion forevaluating assessment methods is their external validity: Do assessment methods of metacognitionconverge in their prediction of learning performance to the same extent as expected? Thesevalidity issues will reoccur during the symposium.An overview of assessment methods for metacognitive skills: Their internal consistency,concurrent validity, and external validity.Marcel V. J. Veenman, University of Amsterdam/ Leiden University, NetherlandsBernadette H. A. M. Van Hout-Wolters, University of Amsterdam, NetherlandsMetacognitive skillfulness refers to the repertoire of skills and strategies for the regulation of andcontrol over one’s learning behavior. Task analysis, activating prior knowledge, planning,monitoring one’s activities, evaluating outcomes, and reflecting on one’s learning processes areactivities representative of metacognitive skillfulness. Moreover, metacognition appears to be oneof the most profound predictors of learning outcomes (Wang, Haertel, & Walberg, 1991). Ineducational research a variety of methods for assessing metacognitive skills, self-regulation, andmetacognitive strategy usage is employed. Questionnaires, interviews, systematical observation,protocol analyses, log-file registrations, and stimulated recall are the most frequently usedmethods. The validity of assessment methods, however, is rarely investigated in a systematicalway (Veenman, Van Hout-Wolters, & Afflerbach, 2006). Drawing from the literature, an up-todatereview of the pros and cons of assessment methods will be presented. A distinction will bemade between off-line and on-line assessments: Off-line methods are administered either prior or– 579 –