Research in Engineering Education Symposium 2011 - rees2009

Universidad Politécnica de Madrid (UPM) Página 562 de 957
The Development of Assessment Tools Using Phenomenography
Carla B. Zoltowski
cbz@purdue.edu
Purdue University, West Lafayette
United States of America
Monica E. Cardella
cardella@purdue.edu
Purdue University, West Lafayette
United States of America
William C. Oakes
oakes@purdue.edu
Purdue University, West Lafayette
United States of America
Abstract: A major challenge in Engineering Education is assessing students’ understanding
of complex concepts and/or the strategies that are proposed to develop them. Often
appropriate assessment tools do not already exist for the concept of interest, motivating
researchers and educators to develop and validate their own instruments. This process
can be further complicated if the underlying frameworks have not yet been established.
This paper describes a new model of assessment development which uses
phenomenography as a starting point for the development of the assessment tool. In
phenomenographic studies, the focus of the research is the variation in how a
phenomenon is experienced which yields empirical information that identifies both
different dimensions of and different levels of sophistication in the target populations’
understanding of the phenomenon. These findings can be used to construct an underlying
framework of the concept of interest, providing a research-informed approach to
assessment tool development.
Introduction
One of the major challenges in Engineering Education is assessing understanding of
complex concepts. Having reliable and valid assessment tools is important because
assessment of the students’ understanding helps students to learn, as students receive
feedback on their current level of proficiency or understanding, and helps instructors to
teach, as they are able to use this information to shape the instruction so that it better
matches students’ current state (Bransford et al., 2000). Furthermore, assessment tools
provide a mechanism for evaluating the effectiveness of different teaching strategies.
In Engineering Education, there are many concepts where appropriate assessment tools
do not already exist, motivating researchers and educators to develop and validate their
own assessment instruments. However, this process can be further complicated if the
underlying theoretical frameworks have not yet been established. For example, what does
it mean for a student to have a more comprehensive understanding of human-centered
design? Although the design literature describes variations in design thinking and
Proceedings of Research in Engineering Education Symposium 2011
Madrid, 4 th - 7 th October 2011