Research in Engineering Education Symposium 2011 - rees2009

Universidad Politécnica de Madrid (UPM) Página 726 de 957
CIs. In this context, the application of modern methods of psychometrics and diagnostic
modeling has the potential to determine the diagnostic capacity of CIs and to suggest ways
to improve the CIs. Diagnostic feedback is a key component for formative use of CIs in
classrooms.
Recommendations for Engineering Education
A substantial instructional challenge within the engineering disciplines is to find practical
methods for instructors to ensure that students develop deep conceptual understandings
within their disciplines and to identify ready assessment methods for determining
whether students have achieved these deeper understandings along with necessary topic
knowledge and problem solving expertise. CIs provide a particular type of assessment that
could materially improve engineering education in just these ways.
A rigorous and systematic application of a framework for evaluating the validity of
assessments intended for classroom use has the potential to identify and guide needed
improvements. Early findings from our examinations of existing CIs clearly indicate that
CIs can benefit from standardization of methods and approaches for CI development and
validation, and from application of modern psychometric methods for profiling students’
conceptual understanding, including misconceptions. Expanding the range of CI uses to
include formative assessment within classes could provide a substantial improvement in
engineering education practice.
Support for this Work
This work has been supported by the following grants from the U.S. National Science
Foundation: CCLI Project #0920242, REESE Project #0918552 and REESE Project
#0815065.
References
Allen, K., Stone, A., Rhoads, T. R., & Murphy, T. J. (2004). The Statistics Concepts Inventory:
Developing a Valid and Reliable Instrument. In Proceedings of the ASEE Annual
Conference and Exposition.
American Educational Research Association, American Psychological Association, &
National Council on Measurement in Education. (1999). Standards for educational
and psychological testing. Washington, DC: American Educational Research
Association.
Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Assessment for Learning –
putting it into practice. Buckingham: Open University Press.
DiBello, L., Roussos, L. & Stout, W. (2007). Review of cognitively diagnostic assessment and
a summary of psychometric models (979-1030). Invited chapter in Handbook of
Statistics, Volume on Psychometrics, Rao, C.V., and Sinharay, S. (Eds.).
Proceedings of Research in Engineering Education Symposium 2011
Madrid, 4 th - 7 th October 2011