Guenter Hilmer - Phd Thesis - Glamorgan Dspace - University of ...

5.2 Design of the interview evaluation 5 EVALUATION METHODOLOGY
give their comments about the 3-Layer-Model that had not been covered by
the previous questions.
The interview was undertaken as a face-to-face interview. The participants’ an-
swers were recorded on paper and on audio tape and transcribed (cf. appendix
A.7.2).
The desired outcome of the mid-test interview was an evaluation of the compre-
hensibility of the 3-Layer-Model to the test participants and the impression they
got from it. Additionally, the feedback concerning the 3-Layer-Model was sepa-
rated from the feedback on the HEAT, which gave the possibility of evaluating the
two tasks separately.
Evaluation of the HEAT
To evaluate the created framework of separating technical from educational con-
tent in educational hypermedia and the practical application of the 3-Layer-Model
the test participants were introduced to the HEAT and a brief demonstration of the
system was given to them. After this introduction they were asked to undertake
the following tasks with the HEAT:
• Creation of a QA-TLS
It was important for the evaluation of the HEAT to undertake tasks that are
similar to the ones that had to be done in the manual markup tasks since
then the EFTECS and the HEAT can be compared. The exact design of the
tasks during the markup based on the HEAT can be found in the appendices
in Appendix A.1.
• Practical conversion of the QA-TLS from a Multiple-Choice-QA into a
True-or-False-QA
To evaluate the functionality and the practicability of the HEAT it was im-
portant to test the possibility of converting material that has already been
authored as easily as possible.
138