Statistical Report of the Part III Clinical Skills Examination April and ...

Statistical Report of the Part III Clinical Skills ExaminationApril and August 2010The Part III Clinical Skills Examination (CSE) is the NBEO’s patient-oriented performance test whichconsists of 18 optometric skills assessed in 4 examination stations. The NBEO held the firstadministration of the restructured, single component Part III across three weekends during the spring of2010. A second, smaller administration was held in Chicago in August 2010.The purpose of this report is to summarize the two Part III examinations that were administered during2010. An overview of the exam and a summary of results is provided.2010 Exam VolumesThe April CSE was administered in 17 clinical test centers to 1432 candidates over three separateweekends in the period from April 9 through May 2, 2010. This examination was conducted by almost450 examiners and over 250 individuals served as “patients.” As the April test is taken by nearly allstudents in the 2010 graduating class, this is referred to as the “targeted” administration.The August administration of Part III served mainly as a retake examination for many of the candidateswho failed the April test. Although the April and August tests are equivalent, the August administrationis referred to as “off-cycle” because of the consistently small number of candidates who are tested. TheAugust CSE was conducted by 35 examiners with over 50 individuals serving as patients.Table 1. Numbers of Examinees taking Part III ExamApril August2004 1372 802005 1319 1042006 1285 952007 1366 612008 1369 742009 1398 962010 1432 59Table 1 contrasts the numbers of examinees for the April and August administrations over the past sevenyears. The number of candidates for the April administration was the largest in the history of the exam;while the August candidate volume is the lowest among these administrations.

ResultsThe CSE includes four 30-minute stations which assess candidates across 18 skills. Total testing time is3.5 hours. The examination results for both 2010 administrations are summarized in Tables 2-4. Table 2presents the final descriptive data; that is, after the candidates with an incomplete for the April CSE scorewere retested in August. There were six candidates with an incomplete because of a statistical orinterrater reliability concern, an irregularity during the test, or a successful appeal. Candidates with anincomplete from the April administration took the August Part III examination with a reduced fee.Table 2. Summary Statistics for Part III, 2010 administrationsApril 2010, N=1432 Raw Score Percent Score Scaled ScoreDescriptive DataScore RangeMean ScoreStandard DeviationPass-Fail DataPass-Fail Cutoff ScoreNumber (%) PassingAugust 2010, N=59Descriptive DataScore RangeMean ScoreStandard DeviationPass-Fail DataPass-Fail Cutoff ScoreNumber (%) Passing349 – 884809.255.2504 – 864743.476.639 – 10091.5100 – 900617.8728.4 82.4 3001373 (95.9%)57 – 9884.1100 – 823398.0728.4 82.4 30036 (61.0%)Table 2 presents summary data for raw, percentage, and scaled scores. The scaled score range for eachPart is based on a 100-900 scale, where 100 represents the lowest score possible, and 900 represents aperfect score. The Part pass-fail cutoff scores are scaled so that they always equal 300. The maximumnumber of weighted points available for the 2010 CSE was 884; the raw pass-fail cutoff was 728.4. In2010, one candidate earned a 100% score and received a scaled score of 900 for this examination.Table 3. Discipline Scores for Part III, 2010 administrationsNumber Number Raw Scores Percent ScoresApril 2010, N=1432 Items Points Range Mean (S.D.) Range Mean (S.D.)Disciplines:CommunicationAffectivePsychomotorClinical Observation &ReportingAugust 2010, N=59Disciplines:CommunicationAffectivePsychomotorClinical Observation &Reporting1107155901107155901858496195185849619560 – 1854 – 8176 – 49643 - 195114 – 1847 – 8251 – 49196 - 18792.2 (6.5)99.5 (3.4)92.2 (5.8)88.9 (7.9)163.7 (17.1)7.8 (0.4)416.9 (53.1)155.0 (21.9)32 – 10050 – 10035 – 10022 – 10062 – 9988 – 10051 – 9949 – 9692.2 (6.5)99.5 (3.4)92.2 (5.8)88.9 (7.9)88.5 (9.2)98.1 (4.5)84.0 (10.7)79.5 (11.2)

In addition to the total Part score, official score reports provide sub-scores for each content discipline.Table 3 presents a summary of discipline scores for the 2010 CSE administration as raw and percentagescores. The scaled scores for each discipline have a maximum range of 0-99, where 0 represents thelowest score possible, and 99 represents the highest score possible (i.e., a perfect score). For those statesthat have subtest requirements, the discipline scores are scaled so that 75 is reported as the cutoff score.The CSE assesses 18 skills performed in 4 stations. Table 4 summarizes percent score statistics for eachskill by administration. Candidates performed best on Skill 2, Near Cover Test, in April with a meanscore of 95.1%. Skill 12, Retinoscopy, had the lowest mean score in April and August, 84% and 73.9%,respectively.Table 4. Percent Scores by Skill in 2010AprilAugustSkill Range Mean (SD) Range Mean (SD)1 Case History / Patient Communication 28.6 – 100 92.0 (8.0) 66.7 – 100 90.5 (8.4)2 Near Cover Test 0 – 100 95.1 (11.1) 12.5 – 100 84.7 (23.2)3 Binocular Extraocular Muscle Motility0 – 100 98.2 (10.1) 46.2 – 100 95.8 (10.5)Evaluation4 Blood Pressure Measurement 0 – 100 88.4 (16.1) 0 – 100 85.6 (21.1)5 Ophthalmic Lens Evaluation 8.3– 100 84.9 (13.5) 29.2 – 100 74.1 (17.3)6 Pupil Testing 0 – 100 92.8 (12.7) 28.9 – 100 92.3 (14.0)7 Biomicroscopy 37.9– 100 92.3 (9.5) 43.9 – 100 85.5 (13.9)8 Goldmann Applanation Tonometry 0 – 100 92.1 (12.3) 26.0 – 100 84.1 (17.4)9 Gonioscopy 0 – 100 88.7 (16.4) 0 – 100 74.6 (28.9)10 Collagen Implant Insertion And Removal 0 – 100 93.6 (14.7) 0 – 100 79.1 (30.8)11 Ocular Disease Management: Patient0 – 100 87.4 (15.0) 0 – 100 77.1 (31.6)Education12 Retinoscopy 16.4 – 100 84.0 (16.6) 30.9 – 100 73.9 (21.9)13 Distance Subjective Refraction 0 – 100 91.2 (11.7) 16.9 – 100 81.7 (19.9)14 Heterophoria And Vergence Testing At 0 – 100 94.4 (11.3) 36.2 – 100 85.7 (15.1)Distance15 Accommodation Testing 0 – 100 90.7 (14.7) 0 – 100 77.0 (23.3)16 Binocular Indirect Ophthalmoscopy 0 – 100 94.1 (11.0) 12.7 – 100 81.0 (21.5)17 Dilated Biomicroscopy And Non-Contact 1.7 – 100 92.4 (11.5) 32.2 – 100 86.0 (15.5)Fundus Lens Evaluation18 Soft and GP Contact Lens Insertion,Evaluation, And Removal0 – 100 93.1 (8.8) 45.6 – 100 88.0 (12.2)The next administration of CSE in April 2011 will be the last examination held at the various optometricinstitutions. Beginning in August 2011, the CSE will be administered at a single site in Charlotte, NC,and will be available throughout the year. The National Board acknowledges the enormous efforts of thehundreds of volunteers, including patients, examiners, and site personnel who have devoted their time,talent, and expertise to the successful development and administration of the Clinical Skills Examinationsince the late 1980s when it was first developed and in 1993 when it was first administered across all ofthe institutions.