Research in Engineering Education Symposium 2011 - rees2009

Universidad Politécnica de Madrid (UPM) Página 855 de 957
referred specifically to personal experience to inform their decisions. “This perhaps is
another instance where I need to make an assumption. I don‘t have the area for this beam,
but I can estimate a reasonable area... Assume 6 inches by 6 inches and this is another
guess based on, I guess partially an experience of, based on what would be a reasonable
width per beam based on that length because I‘ve worked with beams before.”
Some students responded to the ambiguity in the problems by providing ambiguous
answers. Andrew simply specified “steel” as his material selection for one problem,
explaining, “Could be made of any structural steel as they all have high modulus, it‘s going
to be fine for that.” Alex provided a range of answers for one problem, explaining, “And in
that case I would recommend aluminium, and this is with a diameter of 1 mm, but we‘ll
give a range, 1 mm to 1 cm. These will all work depending on what they want by “thin”.
Students added constraints, or evaluation criteria, to the problems to aid in the decision
making process. Cost was a common criterion that students used to evaluate their
solutions even when cost was not specified as a criterion in the problem statements. Alex
described his material choice for one of the problems in this way, “Aluminium fits the
criteria without going too overboard which means the cost should be best for it as well.”
Another student, Ryan, added several constraints to one of the problems including
material “formability”, the location of the bridge that he was designing (“outdoor project”),
and “fatigue characteristics” of the material that he would choose.
Discussion
Qualitative results of this study appear to be in opposition to quantitative ones with
students scoring higher on open-ended problems than on closed-ended ones. Students
who participated in think aloud problem solving sessions, however, described
considerable discomfort with the open-ended problems. The higher scores on the openended
problems may be a result of both the way in which problems were written and the
way in which they were assessed. Few constraints and/or evaluation criteria were
provided in the problem statements for the open-ended problems. In addition, the rubric
that was used to evaluate students’ solutions assessed solutions based only on criteria that
were supplied in the problem statements. As a result students could receive full credit for
solutions that met the problem criteria but were not reasonable in a real-world context.
The problems and the rubric were written to mimic problems and grading practices that
students might encounter in their courses. Larger and more complex design projects
would likely place more emphasis on the reasonableness of solutions. Nevertheless,
results reveal that a solution score may not accurately reflect the difficulties that students
face in solving open-ended problems. It may be necessary for instructors to develop
alternate assessment methods that take into account barriers students’ face while solving
open-ended problems.
Many of the students in this study struggled with the lack of constraints provided by the
open-ended problems, feeling that the problem did not provide them with adequate
details or context. When students solve exam or textbook problems they have the benefit
of knowing that these problems will cover the material from a particular course or chapter
Proceedings of Research in Engineering Education Symposium 2011
Madrid, 4 th - 7 th October 2011