Click here to view complete journal in pdf-format - IAMSE

More documents

Recommendations

Info

Physiology 84% and Biochemistry 73%). However, overall 82% passed as per national medical examination guidelines. According to University regulations, many students were not able to pass in theory paper alone and passed only when it was combined with the marks of viva voce. CONCLUSIONS A significant number of students passed as per the national guidelines. Majority of the students failed Anatomy I, Physiology II and Biochemistry I. This brings out the trends of medical students focusing more on lighter part of the syllabus and avoiding the difficult parts. A medical student must pay equal attention to the entire syllabus which can be brought by improving both teaching and learning methodologies. Assessment Abstract ID: 111 TEST ITEM ANALYSIS CAN SUPPORT CRITERION REFERENCED TESTING FOR ASSESSMENT OF STUDENT COMPETENCE Edward C. Klatt MD, Mercer University School of Medicine, Savannah, GA 31404, U.S.A. PURPOSE Test item analysis to promote student achievement of mastery for demonstration of competence is described. Test item performance limits can adjust the test average to support competency-based education and yield an acceptable passing rate. METHODS For multiple-choice tests based upon classical test theory (CTT) a student’s observed score is related to the standard error (SE) of the mean. The proportion of students correctly answering a CTT item is the difficulty index, and better students should do better on any item. A discrimination index (DI) is calculated by comparing scores of the top minus the bottom scores of test takers. A higher positive value for DI is desirable. Criterion-referenced testing (CRT) ascertains if students have achieved mastery. The CRT passing level is based upon the minimum acceptable level of competence. CRT accepts many test items with low difficulty and DI at or near zero. The desired CRT outcome for test takers is correctly answering all test items, and for the school passing all students. If the majority of test items match CRT criteria, then a passing level set from 65 to 70% may yield few failures. Test analysis is applied to 300 CTT items with SE of 6.5, and DI (comparing the top and bottom 27%) derived from a single cohort of 100 students. The % of students passing is calculated from the z-score distribution: (Test average - Passing score) / SE. RESULTS Test averages, passing levels, and student failures using increasingly stringent item performance limits applied to 300 test items, discarding questions with higher difficulty and lower discrimination are shown. Test Average Discarded Items % Students Passing at 70% % Students Passing at 65% 67% 0 32 62 70% 20 50 78 73% 42 68 89 76% 63 72 95 80% 100 94 99 CONCLUSIONS As the test average moves positively away from the passing level, with fewer test items of higher difficulty, and fewer minimal or negative discrimination index items, student failures decrease. More stringent limits on test item performance yield a test closer to CRT, so that the key issue is setting the passing score at an acceptable level of confidence. Assessment Abstract ID: 113 ARE VIGNETTE MCQ MORE DIFFICULT OR DISCRIMINATING THAN TRADITIONAL MCQ? Stanley J. Nazian and Frazier T. Stevenson, University of South Florida, Morsani College of Medicine, Tampa, FL USA PURPOSE Methods of instruction have evolved in an attempt to move medical student learning to higher levels of the revised Bloom’s Taxonomy. Licensing exams have also moved away from recall by incorporating vignettes of varying complexity. We have also shifted our testing to include more vignettes. We hypothesized that a vignette based question would be more difficult and more discriminating. METHODS Multiple Choice Questions originated with individual instructors and were edited at question vetting sessions. Questions were assigned to one or more of our Competencies (Structural, Molecular, Functional) and assigned a category: Recall had no context; Pseudo-vignette had a clinical or scientific context not needed to correctly answer the question; Vignette1 or Vignette2 questions required 1or 2 pieces of information to arrive at the correct answer. Difficulty (Diff), Discrimination Index (DI) and Point Biserial Correlation Coefficient (PtBiS) were provided by our testing software. Medical Science Educator © IAMSE 2012 Volume 22(4S) 278
RESULTS To date 587 questions used during the first year have been analyzed. Recall and Pseudo-vignettes constituted 85.6% of Structural, 87.3% of Molecular and 68.4% of Functional questions. We anticipated a shift to more vignettes as the year progressed, but this did not occur. Questions that were categorized as either Pseudo-vignette or Vignette had significantly higher Diff, DI and PtBiS ratings. CONCLUSIONS The presence of a vignette, whether or not it provided necessary information, resulted in questions that were more difficult and more discriminating. The Competency appeared to influence the type of questions that our faculty wrote. These baseline data will help guide future faculty and test development. Assessment Abstract ID: 114 THE RELATIONSHIP BETWEEN CRITICAL THINKING SKILLS AND PERFORMANCE ON THE USMLE STEP I Kevin D. Phelan and Bruce W. Newton, University of Arkansas for Medical Sciences, Little Rock, AR, USA. PURPOSE Critical thinking (CT) skills of freshman medical students were previously reported as significantly correlated with USMLE Step I scores (Scott & Markart, 1994). However, their study was limited to one class of students and confounded by different curricular tracks. The present study explores this relationship further by determining which CT skills are correlated with success on the USMLE Step I and by comparing the performance of the high CT (HCT), middle 3/5s (MCT) and low CT (LCT) quintile groups of students. METHODS Volunteer M1 students in three successive classes (2009-2011) were given the Watson-Glazer Critical Thinking Assessment (WGCTA) in an untimed fashion during orientation. RESULTS Total WGCTA scores exhibited a significant positive correlation with Step I scores (r=0.17, p
Page 2 and 3: Medical Science Educator The Journa
Page 4 and 5: MEDICAL SCIENCE EDUCATOR The Journa
Page 8 and 9: main focus of the collaborative LTR
Page 10 and 11: in 1978 2 The adages that have been
Page 18 and 19: questions were evaluated in rank or
Page 22 and 23: to class attendance, about 80% of 2
Page 24 and 25: and allowed students to become conf
Page 26 and 27: CONCLUSION Data evaluation is in pr
Page 28 and 29: only 4% of students are interested
Page 30 and 31: multiple choice quiz was administer
Page 32 and 33: four students complete a personal i
Page 34 and 35: METHODS The new curriculum began to
Page 36 and 37: signal detection (absorption, scatt
Page 38 and 39: from these encounters include the p
Page 40 and 41: RESULTS Strengths of the program in
Page 42 and 43: self-study to integrate basic scien
Page 44 and 45: E-Learning Abstract ID: 166 DETERMI
Page 46 and 47: E-Learning Abstract ID: 170 eDemo U
Page 48 and 49: performance scores (linear regressi
Page 50 and 51: RESULTS Yammer provided privacy, ea
Page 52 and 53: CONCLUSIONS Requiring students to a
Page 54 and 55: against intracellular and extracell
Page 56 and 57: Programmes, Non Governmental organi
Page 58 and 59: METHODS Literature search was condu
Page 60 and 61: member's change with the patient's
Page 62 and 63: graduates recognized the availabili
Page 64 and 65: CONCLUSIONS This rubric will provid
Page 66 and 67: RESULTS The TBL results, examinatio
Page 68 and 69: etween pre-test and post-test mean
Page 70 and 71:
students, audio-recording their dis
Page 72 and 73:
Individual Journal Subscriptions Av
Page 75:
MEDICAL SCIENCE EDUCATOR The Journa
show all

Click here to view complete journal in pdf-format - IAMSE

Create successful ePaper yourself

Delete template?

Save as template?