Research in Engineering Education Symposium 2011 - rees2009

Universidad Politécnica de Madrid (UPM) Página 675 de 957
helping students to understand them, the instructor prepares students to present quality
evidence that is tailored to the subject’s learning outcomes. Through writing and
discussing the grading rubric, the instructor helps differentiate the knowledge and skill
that are essential to progression from content that is useful but not essential. These two
artefacts are introduced to students within the first two weeks of the term, making the
assessment process as clear as possible while creating opportunities for deeper
exploration of the subject learning outcomes.
Implementation and evaluation of the assessment framework
Currently, we are piloting the assessment framework in engineering subjects at four
Australian universities in the second half of 2011. Academic staff participants (11
instructors across the four Australian institutions) are writing the performance standards
and the grading rubric and introducing them within their subjects. To better frame the
evaluation of this piloting, we have turned to design research, as constructed by Ann
Brown (1992) and Allan Collins (2004). This research approach attempts to better
understand teaching and learning practices in naturalistic settings, rather than the
controlled environment of a laboratory. In our approach, we have created our framework
and related artefacts and we are using their implementation in the classroom to explore
the following:
• Can our academic staff participants use the founding principles, assessment
framework, and artefacts to assess individual students’ learning in a manner that is
both acceptable and useful?
o What challenges do they encounter?
o What elements in the framework do they embrace and what elements do
they ignore?
• How do students experience this assessment framework and do they report any
challenges or opportunities in comparison to other approaches?
• Our different participants will adapt the framework to their own teaching styles
and the practicalities of their own teaching situations. How do the varying
instantiations impact the perceived value and the intended purpose of the
framework, artefacts, and principles
At this beginning point in our Term 2 2011 pilots, we are happy with the diversity of our
academic participants and the engineering subjects they are using to pilot the framework.
Our academic staff participants range from relatively new instructors to those with more
than two decades of teaching experience. The subjects range from small class sizes to
those with more than 100 students. Academic staff participants are expressing great
interest in the two artefacts and their use, suggesting that the assessment framework
holds great promise for improvements in assessment in general, as well as for more
effective assessment of individual students’ learning in team-based subjects.
In the next few months, we will collect data on academic staff participants via online
surveys, observational notes by research team members, and final interviews. In addition,
we will collect data from students about their impressions about this approach to
assessment.
Proceedings of Research in Engineering Education Symposium 2011
Madrid, 4 th - 7 th October 2011