Vol 41 # 3 September 2009 - Kma.org.kw

206
Evaluation of Dermatology Residents Using the Multisource (360-Degree) Assessment ...
September 2009
and applicability of the method in postgraduate
dermatology education was not particularly
investigated. Also its use in Turkey needs to be
evaluated. Studies conducted in non-medical fields
have shown that 360-degree assessment method
may produce different results among cultures [11,12] .
However, there is no similar study conducted in the
field of medicine. Turkish culture is considerably
different from the cultures of countries in which
the previous studies were performed. Therefore,
the present study may address whether the method
may differ in different cultures.
The main purpose of this study was to test the
applicability of the 360-degree assessment method
in the Turkish setting of postgraduate medical
education. An additional aim was to evaluate
the 360-degree assessment of residents in the
department of dermatology and the improvement
of areas where deficiencies were noted.
SUBJECTS AND METHODS
The first step was the preparation of the 360-
degree assessment forms. We used the competencies
framework of the ACGME. With regards to ACGME
competency areas a considerably large (consisting of
71 items) item pool was formed through a literature
review.
In order to ensure the validity of the scale’s
scope, a dermatology specialist concluded whether
the attitudes to be measured are well represented
by the given items.
The competency areas, contents and the items
are described below:
1) Patient care (22 items): Items for evaluating
whether patient care is convenient and efficient
enough to improve health and treat health
problems.
2) Medical Knowledge (6 items): Items for
evaluating the knowledge of the residents in
fields such as biomedicine and clinical practice.
3) Practice-based learning (8 items): Items to
evaluate the applications used in patient care.
4) Professionalism (18 items): These items tend
to evaluate the responsibilities required by
professionalism, the adherence to ethical
principles and the sensitivity against different
populations.
5) Communication Skills (10 items): These items
tend to evaluate the relations of the residents
with their colleagues and patients.
6) Systems Based Practice (7 items): These items
question whether the resources of the system are
used efficiently enough to provide the highest
quality care possible [1] .
The items in the pool were specifically arranged
based on the targeted respondent groups. The faculty
evaluation form consisted of all 71 items whereas,
other residents-peer evaluation form consisted of
38 items, nurse evaluator form 33 items, auxiliary
staff evaluator form 7 items, secretary evaluator
form 15 items, patient evaluator form 9 items and
self-evaluation form 58 items. Each item was asked
to score on a Likert scale of 1 (never) to 5 (always).
Open-ended questions were added to the end of the
assessment forms and in this final part, the evaluators
were asked to write down their opinion about the
residents. Negative statements were also included
in order to attract the attention of the reader and
increase reliability. During the data entry, the positive
items were coded as 5, 4, 3, 2, 1 while the negative
ones were coded as 1, 2, 3, 4, 5 in the database.
The second step was the data collection. When the
study was conducted, there were seven residents in the
Department of Dermatology of Akdeniz University
School of Medicine. These students were on a fiveyear
education program and were keeping records of
their studies in “log books”. Out of these residents,
four were in their 2.5 years of training, two were in
their 1.5 year and one was in the first year. In order
to ensure reliable collection of the data, researchers
focused on this step with high priority. Code numbers
were assigned to the residents who would assess the
performances. The 360-degree questionnaires were
submitted to the evaluators during March 2007 and
all were received during next June. The respondents
were informed in detail about how to complete the
questionnaire. The evaluators consisted of seven
residents, four faculty members, nine nurses, three
auxiliary staffs, three secretaries and ten patients for
each assistant. After scoring the assessment forms,
they were all collected in the closed boxes of each
department. The residents’ identities were blinded to
avoid bias from those who carried out data analyzing
In the third step, the data were analyzed with
SPSS 13.0 computer program and mean scores were
calculated for each resident. Internal consistency
of the scores was measured by Cronbach’s alpha.
Faculty members’, nurses’ and colleagues’ evaluation
scores were compared using Spearman’s correlation
and ANOVA statistics. A significance level of 0.05
was determined for all statistical procedures.
In the last step, feedback on the assessment was
provided both to each of the residents and the head
of the department in order to enhance the quality of
learning environment.
RESULTS
In the present study, a total of 259 forms were
filled out to evaluate seven residents. All the staff
in the dermatology clinic participated throughout
the process.
Internal consistency of scores is given in the
Table 1. The consistent scores were very high for all