Patient Safety Curriculum Module 8: Methods for Measuring ...

NPSF Professional Learning Series presents:Patient Safety Curriculum Module 8:Methods for Measuring Performanceand Clinical OutcomesMaulik Joshi, DrPHPresident, Health Research & Educational TrustSenior Vice President of Research,American Hospital Association© Copyright 2011, National Patient Safety Foundation

ObjectivesUpon completion of this module, participants will be able to:• Identify effective methods for using performancemeasurement• Discuss the major domains of patient safetymeasurement• Identify some approaches to data collection2NPSF Professional Learning Series

UsingPerformance MeasurementEffectively3NPSF Professional Learning Series

Effective uses of performancemeasurement MeasurementImprovementTo improve care and service to patients• Identify opportunities for improvement• Benchmark and compare performance• Track progress to improvement• Spread improvement• Sustain improvementAccountability• Identify and track progress against organizational goals• Governance/fiduciary responsibility• Align to financial incentives for leadership and staff4NPSF Professional Learning Series

National Medicare Readmission Rate forHeart Failure Patients353025National Average=24.7%Percent20151050* 52 students (40+ physicians) from Harvard School of Public Health class for MPH; excludes 8 outliers.5NPSF Professional Learning Series

National Mortality Rate for Pneumonia Patients403530Percent25201510National Average=11.6%50* 52 students (40+ physicians) from Harvard School of Public Health class for MPH; excludes 4 outliers.6NPSF Professional Learning Series

National Average for Heart Failure PatientsGiven Discharge InstructionsPercent10090National Average=88%80706050403020100* 52 students (40+ physicians) from Harvard School of Public Health class for MPH.7NPSF Professional Learning Series

Major Patient SafetyMeasurement Domains8NPSF Professional Learning Series

Institute of Medicine Six Aims• Safe• Timely• Effective• Efficient• Equitable• Patient-centered9NPSF Professional Learning Series

Major Domains• Harm• Mortality• Infections/hospital-acquired conditions• Readmissions• Core measures• Patient satisfaction• Safety culture10NPSF Professional Learning Series

HarmUnintended physical injury resulting from or contributedto by medical care that requires additional monitoring,treatment or hospitalization, or that results in death.11NPSF Professional Learning Series

The Focus on Harm• Overall patient safety goal is to reduce patient injuryor harm• Medical errors are numerous• Many have potential to be harmful• Numerous reports show that error is often not linked to injury• Focus on error tends to focus on individual• Focus on harm tends to focus on systems• Focus on systems more likely to improve care and outcomes• Focus on systems reduces fear of punishment and encouragescooperation with patient safety efforts12NPSF Professional Learning Series

Commission vs. Omission• Harm measures focus on active care (commission)• Excludes omission (substandard care)13NPSF Professional Learning Series

Partnership for Patients:Better Care, Lower CostsLed by the Department of Health and Human Services (HHS)Goal 1: Keep patients from getting injured or sicker. By the end of 2013,preventable hospital-acquired conditions would decrease by 40% compared to2010. Achieving this goal would mean approximately 1.8 million fewer injuries topatients with more than 60,000 lives saved over three years.Goal 2: Help patients heal without complication. By the end of 2013,preventable complications during a transition from one care setting to anotherwould be decreased so that all hospital readmissions would be reduced by 20%compared to 2010. Achieving this goal would mean more than 1.6 millionpatients would recover from illness without suffering a preventablecomplication requiring re-hospitalization within 30 days of discharge.Potential to save up to $35 billion dollars over three years.14NPSF Professional Learning Series

Eliminating All Cause HarmNine areas of focus to start:• Adverse drug events (ADE)• Catheter-associated urinary tract infections (CAUTI)• Central line–associated blood stream infections (CLABSI)• Injuries from falls and immobility• Obstetrical adverse events• Pressure ulcers• Surgical site infections• Venous thromboembolism (VTE)• Ventilator-associated pneumonia (VAP)• Other hospital-acquired conditions15NPSF Professional Learning Series

Health Affairs article, Classen et al.: ‘Global trigger tool’ shows thatadverse events in hospitals may be ten times greater thanpreviously measured.How do you measure Harm?Options include:• Mortality rates• Incident reporting rates• Sentinel events• Never events• Adverse events• Medication errors• Infections• Falls• ……..Classen DC et al. Health Aff. 2011;30(4):581–589.16NPSF Professional Learning Series

Health Affairs article, Classen et al.: ‘Global Trigger Tool’ showsthat adverse events in hospitals may be ten times greater thanpreviously measured.• Evaluated incidence of adverse events forpatients at 3 hospitals• Used 3 different approaches:1. IHI Global Trigger Tool: Medical chart review2. Voluntary event reporting system: Staff reporting3. Agency for Healthcare Research and Quality (AHRQ) PatientSafety Indicators: Electronic review of administrative data“We used the following definition for harm:unintended physical injury resulting from or contributed to bymedical care that requires additional monitoring, treatment, orhospitalization, or that results in death.” (p. 583)Classen DC et al. Health Aff. 2011;30(4):581–589.17NPSF Professional Learning Series

Health Affairs article, Classen et al.: ‘Global Trigger Tool’ showsthat adverse events in hospitals may be ten times greater thanpreviously measured.Classen DC et al. Health Aff. 2011;30(4):581–589.18NPSF Professional Learning Series

Health Affairs article, Classen et al.: ‘Global Trigger Tool’ showsthat adverse events in hospitals may be ten times greater thanpreviously measured.• Global Trigger Tool identified 90% of adverse events• AHRQ Patient Safety Indicators: 9%• Voluntary reporting: 1%• Overall, adverse events occurred in 33.2% of hospitaladmissions• Some patients experienced more than one adverse event;the overall rate was 49 events per 100 admissionsClassen DC et al. Health Aff. 2011;30(4):581–589.19NPSF Professional Learning Series

MortalityDeath is the most tractable outcome of care—• It is easily measured• It is of undisputed importance to everyone• It is common in hospital settingsMortality rates, especially overall hospital mortalityrates, have therefore become the natural focus formeasurement of clinical qualityCheck Engine Light20NPSF Professional Learning Series

Mortality Rate Measurement Issues• Crude Mortality Rate• Mortality rate for selected procedures: AMI, heart failure,and pneumonia on medicare.gov• Risk adjusted mortality rateVariability in the measurement of hospital-wide mortality ratesNEJM. December 22, 2010DM Shahian, RE Wolf, LI Iezzoni, L Kirle, S-LT NormandThis study showed that four commonly used methods to calculatehospitals’ risk-adjusted rates of death produced different results. Thesame hospitals were classified as having higher-than-expectedmortality by one method and lower-than-expected mortality by othermethods• Consistent measurement over time21NPSF Professional Learning Series

Healthcare-Associated Infections(HAIs)• At least 20% of episodes are preventable, perhaps asmuch as 70%. (Harbath et al. J Hosp Infect. 2003)• Medicare no longer reimburses U.S. hospitals for theadditional costs of certain infections.• Preventive practices are variably used.• Infection is understood by all, including consumers.22NPSF Professional Learning Series

Hospital-Acquired Conditions(HACs)Hospital-Acquired Conditions are conditions that arereasonably preventable and were not present oridentified at the time of admission, but are eitherpresent at discharge or documented after admission.23NPSF Professional Learning Series

Hospital-Acquired Conditions• Foreign object retained after surgery• Air embolism• Blood incompatibility• Stage III and IV pressure ulcers• Falls and trauma• Fractures• Dislocations• Intracranial injuries• Crushing injuries• Burns• Electric shock24NPSF Professional Learning Series

Readmissions• Nearly 20% of Medicare hospitalizations are followed by readmissionwithin 30 days• 10% to 90% of rehospitalizations within 30 days MAY BE unplanned,the result of clinical deterioration• MedPAC: 75% of readmissions preventable, adding $12 Bn/yr toMedicare spending• Only half of the patients rehospitalized within 30 days had a physicianvisit before readmission• Unknown if lack of physician visit causes readmissions – but poorcontinuity of care, esp for many chronically ill patients• 19% of Medicare discharges are followed by an adverse event within30 days – 2/3 are drug events, the kind most often judged “preventable”25NPSF Professional Learning Series

Opportunity to Reduce ReadmissionsCMS Payment Policy• Beginning in 2013, acute care hospitals with higher than expectedrisk-adjusted readmissions rates will receive reduced Medicarepayments for every discharge• Payment reduction for individual facilities: Up to 1.0% in 2013,increasing to as much as 3.0% in 2015 and thereafter• The Secretary will have the authority to expand the policy toadditional conditions in future years, including all-cause readmissionsPublic Reporting• CMS Hospital Compare website reports hospital performance on aseries of condition-specific readmission measures26NPSF Professional Learning Series

Core Measures• Core Measures is one way to compare the quality of care thathospitals give. Core measures set national standards of care in fivedistinct categories, and hospitals are then measured on how oftenthey provide recommended treatments known to get the best resultsfor patients with certain medical conditions or surgical procedures.• Core Measure sets include:• Acute Myocardial Infarction (AMI)• Children's Asthma Care (CAC)• Heart Failure (HF)• Hospital Based Inpatient Psychiatric Services (HBIPS)• Hospital Outpatient Department Measures• Perinatal Care (PC)• Pneumonia (PN)• Stroke (STK)• Surgical Care Improvement Project (SCIP)• Venous Thromboembolism (VTE)27NPSF Professional Learning Series

Patient Experience• Patient Experience measures patients’ opinions of thequality of customer service provided to them and theirfamily members/visitors during their stay in the hospital• Under CMS’s “value-based purchasing” proposal, Medicarewill begin withholding 1% of its payments to hospitalsstarting in October 2012. That money – $850 million in thefirst year – will go into a pool to be doled out as bonuses tohospitals that score above average on several measures.Patient scores would determine 30% of the bonuses, whileclinical measures for basic quality care would set the rest.28NPSF Professional Learning Series

Culture• “Safety culture” refers to the shared attitudes, beliefs, values,and assumptions that underlie how people perceive and acton safety issues within their organization• The culture of an organization may be a major asset incontinuous safety improvement or, conversely, a majorobstacle to any meaningful change• Safety culture is a new concept in health care and can bedifficult to assess and change• We know that how people think in an organization – theirvalues, assumptions, and beliefs – has the potential toinfluence how they behave when delivering services29NPSF Professional Learning Series

Safety Culture• A culture that exhibits the following five high-level attributes thathealthcare professionals strive to operationalize through theimplementation of strong safety management systems:• A culture where all workers (including front-line staff, physicians, andadministrators) accept responsibility for the safety of themselves,their coworkers, patients, and visitors• A culture that prioritizes safety above financial and operational goals.• A culture that encourages and rewards the identification,communication, and resolution of safety issues• A culture that provides for organizational learning from accidents• A culture that provides appropriate resources, structure, andaccountability to maintain effective safety systems30NPSF Professional Learning Series

Hospital Survey on Patient SafetyCulture• The Hospital Survey on Patient Safety Culture is designedto measure four overall patient safety outcomes:• Overall perceptions of safety• Frequency of events reported• Number of events reported• Overall patient safety grade• The research survey is also intended to measure tendimensions of culture pertaining to patient safety:• Supervisor/manager expectations andactions promoting patient safety• Organizational learning – continuousimprovement• Teamwork within units• Communication openness• Feedback and communications about errors• Non-punitive response to error• Staffing• Hospital management support forpatient safety• Teamwork across hospital units• Hospital handoffs and transitions31NPSF Professional Learning Series

Hospital Survey on Patient SafetyCultureThe Hospital Survey on PatientSafety Culture 2011 UserComparative Database Reportdisplays results from 1,032hospitals and 472,397 hospitalstaff respondents. The 2011 reportalso includes a chapter on trendingthat presents results showingchange over time for 512 hospitalsthat administered the survey andsubmitted data more than once.32NPSF Professional Learning Series

Major Domains• Harm• Mortality• Infections/hospital-acquired conditions• Readmissions• Core measures• Patient satisfaction• Safety culture33NPSF Professional Learning Series

Data Collection34NPSF Professional Learning Series

Triggers: Assessing Harm• Use of manual chart review to study harm as a resultof active medical care• Use of “trigger” methodology to search for harm• Trigger – event often associated with harm• If trigger is present, chart is reviewed further to determine ifharm occurred35NPSF Professional Learning Series

Global Trigger Tool• The IHI Global Trigger Tool contains six “modules,” or groupings oftriggers. Four of the groupings are designed to reflect adverse eventsthat commonly occur in a particular unit; the Cares and Medicationgroupings are designed to reflect adverse events that can occuranywhere in the hospital. The six modules are:• Cares• Intensive Care• Medication• Surgical• Perinatal• Emergency Department• All patient records should be reviewed for the triggers in the Cares andMedication modules. The other modules should only be used ifapplicable; for example, the Intensive Care module should be usedwhen reviewing a record for a patient who spent any part of the hospitalstay in an intensive care unit.36NPSF Professional Learning Series

Global Trigger Tools – OutpatientSurgery ExampleTrigger NameEmergencyDepartment (ED)Re-AdmitProcedure (Proc)Length of stay(LOS)DescriptionED visits within 21 days of an outpatient surgery arelikely to include visits due to complications from surgeryHospital admissions within 30 days of an outpatientsurgery may be due to complications from surgeryReoperation within 30 days and certain procedures onthe same day as a scheduled same-day surgery areindicative that a complication may have occurredThere is a higher probability of complications in samedaysurgeries that result in longer-than-planned stays inthe hospital37NPSF Professional Learning Series

2.50%2.00%1.50%1.00%0.50%How Can We Measure Mortality?Crude Mortality (% of discharges)Apr-07May-07Jun-07Jul-07Aug-07Sep-07Oct-07Nov-07Dec-07Jan-08Feb-08Mar-08Apr-08May-08Jun-08Jul-08Aug-08Sep-08Oct-08Nov-08Dec-08Jan-09Feb-09Mar-09Apr-09May-09Jun-09Jul-09Aug-09Sep-09Oct-09Nov-09Dec-09Jan-10Feb-10Mar-10Apr-10May-10Jun-10Jul-10Aug-10Sep-10Oct-10Nov-10Dec-10Crude Mortality % of dischargesLower Control LimitMeanUpper Control LimitCrude mortality (IHI Measure A02): Hospital deathsas a percentage of all deaths and discharges.Numerator: The number of hospital deaths [Thenumber of deaths in hospital during the month]Denominator: The number of discharges [Thenumber of discharges including deaths (or completedhospital spells) from hospital during the month. Thisincludes the following admission types: elective, daycases, and emergency, and excludes outpatients.]HSMR (Hospital Standardised Mortality Ratio) is aNational number, calculated as a percentage ofobserved deaths to the expected number, adjusted forcomplexity of case-mix, age, sex, and co-morbidities.Scores greater than 100 indicate worse than expectedperformance, scores under 100 indicate better thanexpected performance. Scores under 75 representsignificantly better than expected outcomes. HSMR isbased on data from only 56 diagnosis groups; theseaccount for 80% of all in-hospital mortality.38NPSF Professional Learning Series

ICU AdmissionYesNoPatients admitted to non-ICU beds for comfortcare only, may also indicate inadequatehospice or other end-of-life care resources inthe community. Work here could involveexamining the resource capacity of thecommunity and/or long-term care facilities, aswell as changing the culture aroundappropriate end-of-life care.IHI 2x2 MatrixComfort Care OnlyYesNoPatients admitted to the ICU forcomfort care only, suggests overuse ofICU beds and corresponding flowissues in the ICU. This mightrepresent, for example, physiciansadmitting patients to the ICU forcomfort care because they trust onlythe ICU nurses to administer effectivepain management. This box highlightsa need to identify more appropriateresources for end-of-life care.Patients admitted for care in the ICU,represents the potential for applyingknown improvement techniques to ICUcare such as ventilator bundling (agroup of processes that provides arobust picture of quality in ventilatorcare) and multidisciplinary rounds.Since the outcome was death, these patientswere in fact high risk but possibly notassessed as such. Work here might involveaddressing core systems issues such as inpatient safety and, specifically, in medicationsafety. Work here might also involve improvingrisk assessment on admission for all patients.39NPSF Professional Learning Series

Hand HygieneDirect Observation – Secret Shopperhttp://www.jointcommission.org/assets/1/18/hh_monograph.pdf40NPSF Professional Learning Series

Patient Experience SurveyHospital Consumer Assessment of HealthcareProviders Survey (HCAHPS)Nurse communicationDoctor communicationResponsiveness of hospital staffPain managementMedication communicationDischarge informationEnvironment items: Cleanliness and quietnessOutcomes: Overall rating and willingness to recommend41NPSF Professional Learning Series

Hospital Survey on Patient SafetyCulture (HSOPSC)Dimensions of culture pertaining to patient safety• Unit-level Safety Areas Covered• Overall perceptions of safety• Frequency of events reported• Supervisor/manager expectations and actions promoting patient safety• Organizational learning – continuous improvement• Communication openness• Feedback and communication about error• Staffing• Hospital-wide Safety Areas Covered• Hospital management support for patient safety• Teamwork across hospital units• Hospital handoffs and transitions• Teamwork within units• Nonpunitive response to error42NPSF Professional Learning Series

HAI Prevention Plan 5-year Targets –Progress ReportTOPICCentral line-assoc. bloodstreaminfection (CLABSI)CLABSI Insert. BundleC. difficile Infection (CDI)Catheter-assoc. UTI (CAUTI)Methicillin Resistant Staph Aureus(MRSA)Surgical Site Infection (SSI)SSIMETRIC & TARGETCLABSI Std Infection Ratio (SIR);50% reductionProportion of insertions using bundle;100% adherenceRate/1000 discharges; 30% reductionCAUTI rate ; 25% reductionRate invasive MRSA/100k pop.; 50% reductionStandardized Infection Ratio (SIR);25% reductionProportion Surgical Care Improvement Project(SCIP) measures; 95% adherence43NPSF Professional Learning Series

Event/Incident Reporting• Report all• Report near misses• Report serious or never events• What does the rate mean?• Are we analyzing all the data?• How are we using the data for improvement?44NPSF Professional Learning Series

A Good Incident Report• Person affected – patient, employee, visitor• Date, time, and location of incident• Date and time report completed• Factual description of incident• Patient’s comments about incident• Observations of incident scene• Eyewitness statements• Identification of medical devices involved• Physician notified and response• Condition of the patient45NPSF Professional Learning Series

Executive Walkarounds• During walkarounds, leaders make an hour-long announced visit to adepartment/unit to talk with staff and physicians in a nonthreateningenvironment about the issues that have caused or could cause harm.Concerns that are raised during these conversations are recorded,entered into a database, and addressed according to their level ofseverity, with results reported back to staff.• The objectives of the walkarounds are to:• Show clinicians and staff the commitment of leadership to patient safety• Clearly articulate leadership commitment to developing the infrastructurenecessary to ensure responsible reporting• Identify opportunities for improvement based on issues identified duringthe walkarounds• Implement changes based on systems issues identified46NPSF Professional Learning Series

Patient Safety Indicators (PSIs)• Defined using hospital admissions as denominator• Inpatient complications of care and potential patientsafety events• Potential for internal quality improvement purposes,monitoring of patient safety events• Novel indicators, based on concepts reported in theliterature47NPSF Professional Learning Series

Patient Safety Indicators (PSIs)Provider-level Patient Safety Indicators• Accidental puncture or laceration duringprocedure• Complications of anesthesia• Death in low mortality DRGs• Decubitus ulcer• Failure to rescue• Foreign body left in during procedure• Iatrogenic pneumothorax• Selected infection due to medical care• Postoperative hemorrhage or hematoma• Postoperative hip fracture• Postoperative physiologic and metabolicderangements• Obstetric trauma – vaginal delivery withinstrument• Obstetric trauma – vaginal delivery withoutinstrument• Obstetric trauma – cesarean sectiondelivery• Postoperative pulmonary embolismor deep vein thrombosis• Postoperative respiratory failure• Postoperative sepsis• Transfusion reaction• Postoperative wound dehiscence inabdominopelvic surgical patients• Birth trauma – injury to neonateArea-level Patient Safety Indicators• Foreign body left in duringprocedure• Iatrogenic pneumothorax• Infection due to medical care• Technical difficulty with medical care• Transfusion reaction• Postoperative wound dehiscence inabdominopelvic surgical patients48NPSF Professional Learning Series

Major Domains/Data SourceCollectionHarmMortalityInfectionsReadmissionsCore MeasuresPatientExperience/SatisfactionSafety CultureChart Review, Administrative,Composite of IndicatorsChart Review, AdministrativeChart Review, Administrative,ObservationAdministrativeAdministrativeSurveySurvey49NPSF Professional Learning Series

Conceptual Model for MeasuringsafetyStructure Process OutcomeHave we reducedthe likelihood ofharm?How often do wedo what we aresupposed to?How often dowe harm?Adapted from DonebedianContextHave we created a culture of safety?50NPSF Professional Learning Series

Sample Strategic Quality MapStrategic Aims1. Preventable death2. Harm3. Infection4. Readmissions5. Core measures6. Patient satisfactionMeasures and Goals (2012 and 2020)1. 2012: Quarterly review of IHI 2x2 table2020: Zero preventable deaths2. 2012: Baseline measure and followupof IHI Global Trigger Tool quarterly2020: Zero preventable deaths3. 2012: Zero Infections4. 2012: 10% decrease in readmissions2012: Zero preventable readmissions5. 2012: Top decile performance in state2020: Top decile in nation6. 2012: 5% improvement2020: Top 5% nationallyKey Strategies (and Projects)1. Teamwork Training (Team Steppstraining in each unit, ….)2. Engaging patients and families (patientstories at Board meetings, linked toadverse events, …)3. Credentialing Best Practice (…)4. Evidence Based Care (Implementationof surgical safety checklist, handhygiene monitoring, bundles, …)5. Quality System/Data AnalysisInfrastructure (participating in nationalimprovement projects – NSQUIP,etc….)51NPSF Professional Learning Series

ObjectivesUpon completion of this module,participants will be able to:• Identify effective methods for using performancemeasurement• Improvement and Accountability• Discuss the major domains of patient safety measurement• Harm, Mortality, Infections, Readmissions, CoreMeasures, Patient Experience, Safety Culture• Identify some approaches to data collection• Chart Review, Incident Reporting, Surveying,Claims/Administrative Data, Electronic Health Records,Walkarounds52NPSF Professional Learning Series