Click here to view complete journal in pdf-format - IAMSE

Physiology 84% and Biochemistry 73%). However,
overall 82% passed as per national medical
examination guidelines. According to University
regulations, many students were not able to pass in
theory paper alone and passed only when it was
combined with the marks of viva voce.
CONCLUSIONS
A significant number of students passed as per the
national guidelines. Majority of the students failed
Anatomy I, Physiology II and Biochemistry I. This
brings out the trends of medical students focusing
more on lighter part of the syllabus and avoiding the
difficult parts. A medical student must pay equal
attention to the entire syllabus which can be
brought by improving both teaching and learning
methodologies.
Assessment Abstract ID: 111
TEST ITEM ANALYSIS CAN SUPPORT
CRITERION REFERENCED TESTING FOR
ASSESSMENT OF STUDENT COMPETENCE
Edward C. Klatt MD, Mercer University School of
Medicine, Savannah, GA 31404, U.S.A.
PURPOSE
Test item analysis to promote student achievement
of mastery for demonstration of competence is
described. Test item performance limits can adjust
the test average to support competency-based
education and yield an acceptable passing rate.
METHODS
For multiple-choice tests based upon classical test
theory (CTT) a student’s observed score is related to
the standard error (SE) of the mean. The proportion
of students correctly answering a CTT item is the
difficulty index, and better students should do
better on any item. A discrimination index (DI) is
calculated by comparing scores of the top minus the
bottom scores of test takers. A higher positive value
for DI is desirable. Criterion-referenced testing
(CRT) ascertains if students have achieved mastery.
The CRT passing level is based upon the minimum
acceptable level of competence. CRT accepts many
test items with low difficulty and DI at or near zero.
The desired CRT outcome for test takers is correctly
answering all test items, and for the school passing
all students. If the majority of test items match CRT
criteria, then a passing level set from 65 to 70% may
yield few failures. Test analysis is applied to 300
CTT items with SE of 6.5, and DI (comparing the
top and bottom 27%) derived from a single cohort of
100 students. The % of students passing is
calculated from the z-score distribution: (Test
average - Passing score) / SE.
RESULTS
Test averages, passing levels, and student failures
using increasingly stringent item performance limits
applied to 300 test items, discarding questions with
higher difficulty and lower discrimination are
shown.
Test
Average
Discarded
Items
% Students
Passing at 70%
% Students
Passing at 65%
67% 0 32 62
70% 20 50 78
73% 42 68 89
76% 63 72 95
80% 100 94 99
CONCLUSIONS
As the test average moves positively away from the
passing level, with fewer test items of higher
difficulty, and fewer minimal or negative
discrimination index items, student failures
decrease. More stringent limits on test item
performance yield a test closer to CRT, so that the
key issue is setting the passing score at an
acceptable level of confidence.
Assessment Abstract ID: 113
ARE VIGNETTE MCQ MORE DIFFICULT OR
DISCRIMINATING THAN TRADITIONAL
MCQ?
Stanley J. Nazian and Frazier T. Stevenson,
University of South Florida, Morsani College of
Medicine, Tampa, FL USA
PURPOSE
Methods of instruction have evolved in an attempt
to move medical student learning to higher levels of
the revised Bloom’s Taxonomy. Licensing exams
have also moved away from recall by incorporating
vignettes of varying complexity. We have also
shifted our testing to include more vignettes. We
hypothesized that a vignette based question would
be more difficult and more discriminating.
METHODS
Multiple Choice Questions originated with
individual instructors and were edited at question
vetting sessions. Questions were assigned to one or
more of our Competencies (Structural, Molecular,
Functional) and assigned a category: Recall had no
context; Pseudo-vignette had a clinical or scientific
context not needed to correctly answer the question;
Vignette1 or Vignette2 questions required 1or 2
pieces of information to arrive at the correct answer.
Difficulty (Diff), Discrimination Index (DI) and
Point Biserial Correlation Coefficient (PtBiS) were
provided by our testing software.
Medical Science Educator © IAMSE 2012 Volume 22(4S) 278