Research in Engineering Education Symposium 2011 - rees2009

Universidad Politécnica de Madrid (UPM) Página 912 de 957
groups reported to the room. Participants were encouraged to talk with people from other
tables during the break to enhance the integration.
Group facilitators were prepped before the workshop. Meyer (personal communication)
had reported that at similar workshops in other disciplines, theoretical discussion
occurred outside the purpose of the workshop. Table facilitators focused discussion on the
required tasks, asked probing questions, engaged all participants, and limited discussion
about threshold concept theory by “parking” theoretical issues until they could be
addressed by an overarching facilitator. Disciplinary facilitators ensured that discussions
were summarised on the handouts. Process facilitators took notes on the progress of the
discussion, including participants’ responses to the method and the theory.
Data collection and analysis throughout both phases
Data were collected as part of the methodology, and also to evaluate the methodology.
This section discusses data collected as part of the methodology. During interviews and
focus groups the researcher made observations and notes. Discussions were recorded if
consent was granted. At the student and student-staff workshops, the conversations
involving the whole room were recorded. At the regional workshops, all discussions were
recorded with a device on every table. Recordings were transcribed, unless background
noise was excessive. Workshop groups also responded to questions on handouts.
The evolution of a threshold concept inventory is central to the methodology. Content
based analysis identified threshold concepts, their transformative and troublesome
features, and possible ways to help students overcome them. Consequently an inventory of
threshold concepts, their features and possible ways to help students overcome them was
developed iteratively. Each item includes sufficient description for engineers to identify
the concept, relevance in the course or engineering practice, ways the concept can be
troublesome, and ways to help students understand the concept.
At each stage, relevant parts of the inventory to date were presented to each new group of
participants. This ensured increasing depth and effective use of participants’ and
researchers’ time. For example, academics were shown potential threshold concepts
identified in their fields early in their interviews.
As far as practical, participants were given opportunities to correct misunderstandings
evident in iterations following their contributions. This improved validity, and helped
maintain trust and support. For example, academics were sent the relevant potential
thresholds they had identified, or iterations made in response to their comments.
Similarly, students were sent the concepts they had raised in the student workshop, before
they were discussed in the student-staff workshop.
Evaluation of the methodology
Data collected to evaluate the methodology included responses on workshop evaluation
forms, and process notes taken by the process facilitators at the regional workshops. The
evaluator, John Bowden, took a formative approach, attending project meetings, attending
Proceedings of Research in Engineering Education Symposium 2011
Madrid, 4 th - 7 th October 2011