Guenter Hilmer - Phd Thesis - Glamorgan Dspace - University of ...

5.1 Analysis of alternative approaches 5 EVALUATION METHODOLOGY
high flexibility regarding the members of the evaluation group, which means that
the evaluation candidates can be selected as they are needed for the evaluation.
Additionally, the single session approach provides highly individualised feedback
from the test candidates, due to the fact that the test candidates can be interviewed
in a very individual way.
However, the structure of the evaluation must be well designed because it has
to cover all the topics that are needed for the project. Therefore, the evaluation
can become rather large and can take a lot of time. Besides, long-term effects
cannot be tested because the single session approach takes place only once in one
particular way.
Secondly, the multiple session approach has to be regarded as an option for this
project.
First of all, it has to be stated that long-term effects of the project, for example a
learning curve with the HEAT, can be examined only in a multiple session inter-
view evaluation. Furthermore, the evaluation can be adapted to any problems that
could possibly appear during the interviews.
However, the multiple session approach is very time-consuming and the evalua-
tion candidates must consistently be the same to produce significant results.
5.1.3 Selected approach
For this PhD evaluation the one-to-one approach was selected. This selection
was mainly based on the most important advantage of the one-to-one approach,
the high flexibility during the evaluation regarding the members of the evaluation
group. Another reason was that the evaluation does not aim at a large number of
test subjects. Nielsen suggests around ten evaluation subjects to get significant
evaluation results [182]. For this research project it was decided to undertake the
evaluation with nine test participants, but to select the nine test participants from
a broad range of educational domains (cf. section 6.1.2).
Furthermore, it was decided to use structured interviews during the evaluation as
a way to gain a qualitative insight into the test participants’ experiences.
128