Vol 41 # 3 September 2009 - Kma.org.kw

More documents

Recommendations

Info

September 2009 KUWAIT MEDICAL JOURNAL 205 Original Article Evaluation of Dermatology Residents Using the Multisource (360-Degree) Assessment Method Yesim Senol 1 , Ozlem Dicle 2 , Halil Ibrahim Durak 3 1 Department of Medical Education, Akdeniz University Faculty of Medicine, Antalya, Turkey 2 Department of Dermatology, Akdeniz University Faculty of Medicine, Antalya, Turkey 3 Department of Medical Education, Ege University Faculty of Medicine, Turkey ABSTRACT Kuwait Medical Journal 2009; 41 (3): 205-209 Objective: To test the applicability of the 360-degree assessment method of postgraduate medical education (dermatology) in a Turkish setting and discover the possible areas of improvement Design: Cross-sectional research Setting: Akdeniz University Faculty of Medicine, Antalya, Turkey Subjects and Methods: Using the competencies framework of the The American Accreditation Council for Graduate Medical Education (ACGME), a 71 item containing pool was formed. Based on evaluation aspects of possible evaluator groups, the pool was converted into seven different evaluation forms for the following groups: teachers (faculty), nurses, peers, secretaries, auxiliary staff, patients and self. All seven residents and members from each department voluntarily participated in the study. Data was collected over a period of three months. Main Outcome Measures: In the data analysis process, mean scores, and internal consistency scores were measured and evaluator groups’ scores and resident’s ranks in competency areas were compared to find out differences. Results: A total of 259 forms were filled out by seven groups to evaluate seven residents. All the staff in the dermatology clinic participated in the process. The reliability coefficient for the faculty members was 0.99 while it was 0.60 for the auxiliary staff. Low scores were clustered. Conclusions: Our results show that the 360-degree assessment is very well received by the evaluator group and residents and the method is acceptable in the Turkish setting of postgraduate dermatology specialist training. There is a need for larger sample study for other settings. KEY WORDS: 360-degree assessment, medical education, resident INTRODUCTION The American Accreditation Council for Graduate Medical Education (ACGME) endorsed six general competencies that the residents should be expected to meet in 1999. These competencies were in the areas of patient care, medical knowledge, practice-based learning, interpersonal and communication skills, professionalism and system-based practice [1] . Consequently, many medical schools and institutions have been conducting studies that aim to develop proper, valid and reliable assessment tools to assess these competencies. Various assessment tools were developed, tested and used for both formative and summative purposes [2] . One of the most promising among those newly developed methods is the 360- degree assessment. 360-degree assessment, which is also referred to as “360-degree performance assessment”, “360-degree feedback” or “multi-source performance appraisal”, aims to collect the information on the performance of the employee by using different evaluation perspectives [3] . It originates from quality work in business [3] and its use in medical education has been reviewed by Lockyer [4] . This method aims to evaluate the employees in a versatile and continuous manner and collect information about the performance of their employees from different sources that have different type of relations with them [4] . In recent years, the 360-degree assessment has been tested for both undergraduate [5] and postgraduate medical education. Studies have stated that the 360-degree assessment is one of the best methods in the evaluation of professionalism and communication skills [6] while at the same time it is valid, reliable and applicable [7-10] . However, the great majority of these studies have been conducted in North American countries and validity, reliability Address correspondence to: Dr. Yesim Senol, MD, Assistant Professor, Department of Medical Education, Akdeniz University Medical Faculty Campus, Antalya. Turkey 07070. Tel: +902422496189, E-mail: yigiter@akdeniz.edu.tr
206 Evaluation of Dermatology Residents Using the Multisource (360-Degree) Assessment ... September 2009 and applicability of the method in postgraduate dermatology education was not particularly investigated. Also its use in Turkey needs to be evaluated. Studies conducted in non-medical fields have shown that 360-degree assessment method may produce different results among cultures [11,12] . However, there is no similar study conducted in the field of medicine. Turkish culture is considerably different from the cultures of countries in which the previous studies were performed. Therefore, the present study may address whether the method may differ in different cultures. The main purpose of this study was to test the applicability of the 360-degree assessment method in the Turkish setting of postgraduate medical education. An additional aim was to evaluate the 360-degree assessment of residents in the department of dermatology and the improvement of areas where deficiencies were noted. SUBJECTS AND METHODS The first step was the preparation of the 360- degree assessment forms. We used the competencies framework of the ACGME. With regards to ACGME competency areas a considerably large (consisting of 71 items) item pool was formed through a literature review. In order to ensure the validity of the scale’s scope, a dermatology specialist concluded whether the attitudes to be measured are well represented by the given items. The competency areas, contents and the items are described below: 1) Patient care (22 items): Items for evaluating whether patient care is convenient and efficient enough to improve health and treat health problems. 2) Medical Knowledge (6 items): Items for evaluating the knowledge of the residents in fields such as biomedicine and clinical practice. 3) Practice-based learning (8 items): Items to evaluate the applications used in patient care. 4) Professionalism (18 items): These items tend to evaluate the responsibilities required by professionalism, the adherence to ethical principles and the sensitivity against different populations. 5) Communication Skills (10 items): These items tend to evaluate the relations of the residents with their colleagues and patients. 6) Systems Based Practice (7 items): These items question whether the resources of the system are used efficiently enough to provide the highest quality care possible [1] . The items in the pool were specifically arranged based on the targeted respondent groups. The faculty evaluation form consisted of all 71 items whereas, other residents-peer evaluation form consisted of 38 items, nurse evaluator form 33 items, auxiliary staff evaluator form 7 items, secretary evaluator form 15 items, patient evaluator form 9 items and self-evaluation form 58 items. Each item was asked to score on a Likert scale of 1 (never) to 5 (always). Open-ended questions were added to the end of the assessment forms and in this final part, the evaluators were asked to write down their opinion about the residents. Negative statements were also included in order to attract the attention of the reader and increase reliability. During the data entry, the positive items were coded as 5, 4, 3, 2, 1 while the negative ones were coded as 1, 2, 3, 4, 5 in the database. The second step was the data collection. When the study was conducted, there were seven residents in the Department of Dermatology of Akdeniz University School of Medicine. These students were on a fiveyear education program and were keeping records of their studies in “log books”. Out of these residents, four were in their 2.5 years of training, two were in their 1.5 year and one was in the first year. In order to ensure reliable collection of the data, researchers focused on this step with high priority. Code numbers were assigned to the residents who would assess the performances. The 360-degree questionnaires were submitted to the evaluators during March 2007 and all were received during next June. The respondents were informed in detail about how to complete the questionnaire. The evaluators consisted of seven residents, four faculty members, nine nurses, three auxiliary staffs, three secretaries and ten patients for each assistant. After scoring the assessment forms, they were all collected in the closed boxes of each department. The residents’ identities were blinded to avoid bias from those who carried out data analyzing In the third step, the data were analyzed with SPSS 13.0 computer program and mean scores were calculated for each resident. Internal consistency of the scores was measured by Cronbach’s alpha. Faculty members’, nurses’ and colleagues’ evaluation scores were compared using Spearman’s correlation and ANOVA statistics. A significance level of 0.05 was determined for all statistical procedures. In the last step, feedback on the assessment was provided both to each of the residents and the head of the department in order to enhance the quality of learning environment. RESULTS In the present study, a total of 259 forms were filled out to evaluate seven residents. All the staff in the dermatology clinic participated throughout the process. Internal consistency of scores is given in the Table 1. The consistent scores were very high for all
Page 1 and 2: VOLUME 41 NUMBER 3 KMJ KUWAIT MEDIC
Page 3 and 4: Kuwait Medical Journal (KMJ) Publis
Page 5 and 6: Instructions for Authors Acknowledg
Page 7 and 8: AN ADDED FEATHER TO THE KMJ’S CAP
Page 9 and 10: 186 Swine-origin Influenza A (H1N1)
Page 11 and 12: 188 Population Health Genomics in M
Page 27: 204 Population Health Genomics in M
Page 31 and 32: 208 Evaluation of Dermatology Resid
Page 33 and 34: 210 KUWAIT MEDICAL JOURNAL Septembe
Page 35 and 36: 212 Mode of Detection of Transvenou
Page 37 and 38: 214 Mode of Detection of Transvenou
Page 39 and 40: 216 External Non-Invasive Cardiac P
Page 47 and 48: 224 Clinical Characteristics and Ou
Page 51 and 52: 228 Blunt and Penetrating Thoracic
Page 55 and 56: 232 Small Bowel Tumors: Insidious a
Page 57 and 58: 234 Small Bowel Tumors: Insidious a
Page 61 and 62: 238 Clinical and Radiological Aspec
Page 65 and 66: 242 Severe Diabetic Ketoacidosis Pr
Page 67 and 68: 244 Acquired Hemophilia in a Child:
Page 75 and 76: 252 Neurosarcoidosis as a First Pre
Page 79 and 80:
256 Urosepsis Simulating Congenital
Page 81 and 82:
258 Rituximab in Severe Refractory
Page 83 and 84:
260 Rituximab in Severe Refractory
Page 85 and 86:
262 Selected Abstracts of Articles
Page 87 and 88:
264 Selected Abstracts of Articles
Page 89 and 90:
266 Forthcoming Conferences and Mee
Page 91 and 92:
Page 93 and 94:
Page 95 and 96:
272 KUWAIT MEDICAL JOURNAL Septembe
Page 97 and 98:
274 WHO-Facts Sheet September 2009
Page 99 and 100:
Page 101:
show all

Vol 41 # 3 September 2009 - Kma.org.kw

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?