Quality: assurance or standards? - Kuwait Institute for Medical ...

BULLETIN OF THE KUWAIT INSTITUTE FOR MEDICAL SPECIALIZATION 2003;2:1-4ReportDiscussion of the issues of quality and safetyin clinical practice go far beyond the definitionsof terms such as ‘competence’ and‘performance’. Certainly an understanding ofthe distinction between these terms in amedical education context is important, butfar more important are “the implications ofthe distinctions between the two areas for thepurpose of assessing doctors in practice”. 3 Thedistinction between competence and performancehas particular relevance for identifyingmethods of assessment. Those methods whichcan measure what a practitioner actually doesin professional practice when dealing with theproblems of patients (i.e. the clinical performance)may be quite different from the methodsneeded to assess what doctors are capableof doing in controlled representations of professionalpractice as possible predictors of futurepractice (i.e. clinical competence). As isclearly true at all levels of professional learning,amongst other variables the context ofpractice, including the nature of professionalteams and systems, impacts dramatically onthe outcomes and quality of patient care. 4Any investigation into the assessment ofclinical performance in practice settingsshows very readily that clinical performanceis an extremely varied and idiosyncratic construct,and therefore although a vast array ofperformance components and tasks may beidentifiable 5 , the matching of appropriate assessmentmethods 4-6 is a complex and difficultprocess. Practitioners, employers and consumerswill obviously need to accept thatsome form of performance assessment is actuallynecessary as a personal and professionalresponsibility 7 no matter what range of contextsand systems are involved. 4 To achievethis, assessment approaches will need to beseen as being fair and defensible 8-10 andclearly relevant to the defined purposes of assessmentand the nature of the clinical performance.Such is the complexity and hugerange of potential combinations of factors involvedthat any single approach to assessmentor single method of assessment wouldbe extremely limited in its application. Whatis necessary is an understanding of principlesand guidelines that allow for their applicationin any well-defined setting.Further, acceptance of any model of learningand practice that builds upon and furtheradvances the notion of CME/CPD will requirean understanding of distinctions between formativeand summative assessment approaches.11 Gathering information and providingfeedback that is largely qualitativeand designed to assist in the improvement ofan individual practitioner’s performance is avastly different activity from gathering ratingsor largely quantitative informationwhich demonstrates characteristics of practicewhich are then used for judgmental orlegislated summative purposes. Any approachthat is designed to provide comparativedata against other practices, standardsor requirements will also require individualpractitioners to accept the need to have a realisticinsight into their own performance 12and accept the need for remediation wheneverindicated. A conflict will arise wheneverthe purposes or intentions of an approach,and thus the outcomes, are used for differingpurposes. A basic starting position requiresthe clear and accepted definition of the purposesof assessment, and the intended use ofthe outcomes or judgments made as a resultof the assessment.An approach recommended for considerationis one that links “performance and assessmentto ensure that ongoing learning andcontinuing competence are, in reality, connected”.11 It needs the practitioner to be ableto identify and accept those aspects of clinicalpractice that are not only adequate and acceptableto himself or herself but also in linewith acceptable professional standards. Assessment,as an educational process, allows aclinician opportunity to compare characteristicsof actual practice with some ideal or desiredpractice, whether defined by the individual,by the profession or by some externalregistration or accreditation authority. Assessmentalso validates good practice whenthere is consistency between what the individualdoes and what is expected of him orher. Any perceived discrepancy between performanceand requirements should initiatefurther consideration and reflection.As clinicians are involved in such a vastrange and variety of tasks and components,assessment through any single approach isunlikely to be able to reflect an overall levelof performance. Typically, the notion of case2Quality: assurance or standards? ● Neil Paget

BULLETIN OF THE KUWAIT INSTITUTE FOR MEDICAL SPECIALIZATION 2003;2:1-4or content specificity indicates that one willdo better at some tasks, in some domains, insome contexts, with some patients than onother occasions. Performance levels are notstatic over time or content, and individualpractitioners will have differing perceptionsof both what standards they apply to theirown performance and what standards are expectedof them. Critical errors may occur, butmost practitioners with adequate insightshould be aware of their own weaknesses anddeficiencies before such unacceptable eventsoccur. Setting personal standards too high ortoo low could have unwelcome consequenceseither in the standard of care provided to patientsor the perception of the practitioner’sown performance and need for remediation.Accepting too low a standard obviously putspatients at risk, whereas recognizing toolarge a discrepancy from an expected standardmay block effective remediation throughappearing to be unrealistic and thus be denied.13An important area to consider in discussingthe issues of quality and standards is thereforethe ability of the individual practitionerto assess the level of their own performanceacross a range of tasks over extended periodsof time and to set the impressions gainedagainst realistic standards and requirements.Single incident or task measures are not sufficientlyrobust to rely upon, nor is the averagingof various differing performances. Froma technical viewpoint, assessment must besufficient to gain an adequate and accurateperception 8 so that some generalization of behavioris possible, i.e. the assessment measuresmust be both valid and reliable.Summative assessments undertaken to allowjudgments on standards of practice, suchas examinations, take a snapshot of performanceacross a range of tasks at a particularpoint in time. Practice audits collect dataabout single conditions or a small group ofconditions, and being largely qualitativemeasures do not easily allow summative comparisonsto be made. Formative assessmentallows for more frequent feedback on tasksand components, but is designed to assist inongoing development and extension of skillsrather than comparison with existing standards.Combining several assessment approachesto provide data for the compilationof a portfolio provides increased flexibility ininterpreting the appropriateness of practice,but requires coverage of the domains of patientcare, personal development and contextmanagement. 6 Further, portfolios have becomemore acceptable in offering greater opportunitiesto identify effective learning activitiesfor formative purposes than in contributingto global judgments of performancefor summative purposes.Whatever approach to assessment of clinicalpractice is taken, the purpose and intendedoutcome of that process must be veryclear and agreed upon by the individual practitioner.9 Assessors from the same or relateddisciplines will often be required. These assessorswill make judgments on the basis of asample of performance that is observed ordocumented, and are key components of anypractice performance assessment programthat is used for certification, recertification oraccreditation purposes. 9 Of particular importanceis the need to use assessors who are appropriatelyselected and trained and who areacceptable to the clinician. Fair and defensibleprograms for performance assessmentmust also use an acceptable and valid samplingof behavior, and be as free of potentialbias and error as is possible. 8DiscussionAlthough there has been a recent shift in attentionaway from the structure and deliveryof CME/CPD programs, they remain at thecore of ongoing maintenance and developmentof clinical knowledge and skills. What is clearis that, in themselves, participation in suchprograms is not likely to meet the increasinglegislative and employment requirementsthat favor clear indicators of professional andpersonal performance which will allow practitionersto continue to practice. From bothsummative and formative aspects of the assessmentof performance in realistic practicecontexts, CME/CPD programs are a necessarycomplement to the assessment processes andallow practitioners to check personal perceptionsand insight. To a large extent these programswill also allow practitioners to validatestandards against professional colleagues andlegislative requirements.The assessment of clinical performance is avery complex and difficult process that needsto be set within individual practice settingsQuality: assurance or standards? ● Neil Paget3

BULLETIN OF THE KUWAIT INSTITUTE FOR MEDICAL SPECIALIZATION 2003;2:1-4and the broader team and system contextsthat are also extremely varied and difficult todefine. 4 To accept the pressures and challengesof future registration and reaccreditationrequirements, clinicians need to understandthe advantages and disadvantages of awide range of assessment approaches, methodsand tools. They need to develop ways ofdemonstrating the quality of their clinicalperformance that are realistic, valid and reliable,and will be acceptable to legislative andemployment authorities.The 10 th Cambridge Conference providessome useful guidelines to practitioners in itsrecent publications. 3-12 The medical educatorsinvolved in the conference had a commonaim: to provide the best evidence on the complexissues investigated and to provide practicalguidance that would apply to all clinicians,no matter their discipline, level of expertiseor practice setting. A closer investigationof all the issues covered in these articlesis highly recommended.References1. Bandaranayake R. Recertification: Is it abouttime? Bull Kuwait Inst Med Spec 2002;1:51-2.2. Paget NS. Competence to practise: Whose responsibility?Bull Kuwait Inst Med Spec2002;1:40-2.3. Rethans J-J, Norcini JJ, Baron-Maldonado M,Blackmore D, Jolly BC, LaDuca T, et al. Therelationship between competence and performance:implications for assessing practice performance.Med Educ 2002:36;901-9.4. Farmer EA, Beard JD, Dauphinee WD, La-Duca T, Mann KV. Assessing the performanceof doctors in teams and systems. Med Educ2002:36;942-8.5. Hays RB, Davies HA, Beard JD, Caldon LJM,Farmer EA, Finucane PM, et al. Selecting performanceassessment methods for experiencedphysicians. Med Educ 2002:36;910-7.6. Wilkinson TJ, Challis M, Hobma SO, NewbleDI, Parboosingh JT, Sibbald GR, et al. The useof portfolios for assessment of the competenceand performance of doctors in practice. MedEduc 2002:36;918-24.7. Finucane PM, Barron SR, Davies HA, Hadfield-JonesRS, Kaigas TM. Towards and acceptanceof performance assessment. MedEduc 2002:36;959-64.8. Schuwirth LWT, Southgate LH, Page GG,Paget NS, Lescop JMJ, Lew SR, et al. Whenenough is enough: a conceptual basis for fairand defensible practice performance assessment.Med Educ 2002:36;925-30.9. Lew SR, Page GG, Schuwirth LWT, Baron-Maldonado M, Lescop JMJ, Paget NS, et al.Procedures for establishing defensible programmesfor assessing practice performance.Med Educ 2002:36;936-41.10. Melnick DE, Asch DA, Blackmore DE, KlassDJ, Norcini JJ. Conceptual challenges in tailoringphysician performance assessment toindividual practice. Med Educ 2002:36;931-5.11. Handfield-Jones RS, Mann KV, Challis ME,Hobma SO, Klass DJ, McManus IC, et al.Linking assessment to learning: a new routeto quality assurance in medical practice. MedEduc 2002:36;949-58.12. Hays RB, Jolly BC, Caldon LJM, McCrorie P,McAvoy PA, McManus IC, et al. Is insight important?Measuring capacity to change performance.Med Educ 2002:36;965-71.13. Fox RD, Bennett NL. Continuing medical education:learning and change: implications forcontinuing medical education. Br Med J1998:316;466-8.4Quality: assurance or standards? ● Neil Paget