AMEE Berlin 2002 Programme

Session 5C Curriculum evaluation
5C1 A course programme evaluation:
contribution of students’ selfevaluation
Carlota Saldanha*, Jorge Lima and Joao Martins-Silva
Institute of Biochemistry, Faculty of Medicine of Lisbon, Av Prof
Egas Moniz, 1649-028 Lisbon, PORTUGAL
A course programme evaluation (CPE) is useful to
highlight issues for introducing further changes. The
aim of this work was to compare the results of the CPE
performed in the discipline of Cellular Biochemistry
in three consecutive years. The first year students in
1999 (n=199), in 2000 (n=189) and in 2001 (n=173)
of the Faculty of Medicine of Lisbon were required to
answer a questionnaire anonymously with 19 closed
questions. Phi and Cramer’s VC statistical analysis were
used. The answers profile for the majority of the
questions was similar in the three consecutive years.
The main differences were those related to the influence
of the teaching process in the assessment methods and
in students’ self-confidence. The majority of students
were not motivated with their self-evaluation. It is
concluded that the course programme evaluation
depends on the student population and not exclusively
on the changes introduced by the teaching staff.
5C2 A national evaluation of the reforms
to higher specialist training in the
UK: methodological issues
Janet Grant*, Rodney Gale, Mairead Beirne and
Heather Owen
Open University Centre for Education in Medicine, The Open
University, Walton Hall, Milton Keynes MK7 6AA, UK
Over the past 5 years, wholesale reform of higher
specialist training in the UK has occurred with changes
in training structure, processes, infrastructure,
assessment, records, post types, endpoint qualification,
curricula, selection, standard setting, and monitoring.
The Department of Health commissioned a national
evaluation, conducted over three years. The evaluation
was designed and conducted as policy research,
involving 15 researchers with expertise in postgraduate
medical education, qualitative and quantitative research
methods, management, selection, public sector
economics, assessment, and pedagogy. Data collection
methods included surveys, consultative groups,
interviews, site visits, case studies, documentary
analysis and in-depth reference studies. Seven focal
specialties were studied. Data for the 20 separate reports
of the evaluation were gathered from upwards of 5,000
participants. The evaluation covered costs, organisation,
management, funding, change management,
assessment, appraisal, educational processes and
outcomes, new roles and responsibilities, selection, and
effects on service and workload. Methodological issues
will be discussed.
Tuesday 4 September
- 4.32 -
5C3 Evaluation of PME courses in
General Practice in Germany
S Wilm*, R Jahromi, S Krause and B Hemming
Heinrich-Heine University, Department of General Practice,
PO Box 10 10 07, D-40001 Duesseldorf, GERMANY
Vocational training in general practice in Germany
requires 240 hours of accompanying courses, aiming
at reinforcing professional knowledge, problem solving
abilities and at training in interdisciplinary teamwork
and audit. For the participants it has the character of an
expensive obligation, with possibly negative effects on
intrinsic motivation and demands. The continuous builtin
evaluation of courses in Duesseldorf/North-Rhine
covers written interviews with participants and teachers,
participatory observation, oral interviews and focus
groups. In 1999 and 2000 71 seminars were evaluated.
935 questionnaires of participants (response rate 70%),
46 data sheets for participatory observation, 33
questionnaires of 29 teachers and 10 focus groups show
that the educational quality increased, while the overall
satisfaction of the trainees decreased. Possible reasons
for this complex phenomenon are discussed.
Organizational aspects influence participants’
satisfaction much more than content of seminars.
5C4 The Matrix: a visual presentation of
registrar term evaluation
Richard Tarala and Alistair Vickery
Department of Postgraduate Medical Education, Royal Perth
Hospital, GPO Box X2213, Perth, Western Australia 6847,
AUSTRALIA
Evaluation of junior medical officer posts can be
hindered by the need to distribute questionnaires, the
time required for their completion and by low response
rates. Collation and assessment can be time-consuming
and the resultant evaluation difficult to interpret. We
used a simple evaluation tool for the assessment of
registrar terms. Trainees were asked to assess on a fivepoint
scale the workload and the training/teaching value
of each rotation they had experienced over the previous
two years. We have undertaken a pilot to provide a
‘snapshot’ or audit of the trainees’ perception of the
rotations. The evaluation can be performed quickly and
the collated results when plotted in a graphical
(“matrix”) form provide a powerful visual assessment
of the trainee’s perspective of each post. This evaluation
can be used to assess the workload of each position to
assist in reallocation of workload and to optimise
training in each post.
5C5 Site visits as a method to assess the
educational quality of clerkships
Janke Cohen-Schotanus* and Rein P Zwierstra
Institute for Medical Education (OWI-OK), Faculty of Medical
Sciences, University of Groningen, Ant.Deusinglaan 1, 9713
AV Groningen, NETHERLANDS
Clerkships, as part of the training of medical students,
are organized in several hospitals. In general, students
are content with this training. However, what do we