Evaluation - FHOP - University of California, San Francisco

Innovative Strategies forAddressing Disparities: Lessonslearned from Implementationthrough EvaluationJennifer Rienks, PhD,* Virginia Smyly, MPH,**Geraldine Oliva, MD, MPH,* Judith Belfiori, MPH,MA,* Sandra Johnson, BS***Family Health Outcomes Project (FHOP), University of California,San Francisco**Health Promotion and Prevention, San Francisco Dept. of PublicHealth1

Background• San Francisco Department of PublicHealth received a REACH 2010 Grantfrom the Centers for Disease Control tocreate the SevenPrinciples for AfricanAmerican Infant Survival andCommunity Unity Project• Family Health Outcomes Project at theUniversity of California, San Franciscocontracted as the evaluator2

Seven Principles ofKwanza• Used to guide the values andstrategies of the project• Includes unity, self-determination,collective work and responsibility,cooperative economics, purpose,creativity, and faith3

Project Strategies1. Social marketing based communityawareness campaigns2. Cultural competency trainings andUndoing Racism Workshop for healthand social service providers3. Community Action Teams (CATS) toaddress underlying causes ofdisparities, create unity, lead /participate in community change4. Dinner and Movie Events to createcommunity connections and social4support

Awareness Campaigns:1. African American babies in SFare 2 times more likely thanwhite babies to die in the firstyear of life2. Babies sleep best on theirbacks3. Take action to stop AfricanAmerican babies from dying5

Campaigns Mediums• Ads on bus shelters, the backs of busesand inside buses in targetedneighborhoods Bus ads• Posters and hand cards distributed(delete) at San Francisco clinics, daycaresites, doctor offices, agency waitingrooms, delete, put in notes and communityorganizations.• Church fans distributed to 50 places ofworship• Radio spots were on KMEL and KBLX,radio stations popular with AfricanAmericans in the SF6

Evaluation Methodology• Time 1 and Time 2 telephone surveysconducted with African Americansbetween 18-65 years old in 4 targetneighborhoods• Time 1 telephone survey -- 804respondents prior to awarenesscampaigns• Time 2 telephone survey -- 654respondents at conclusion ofawareness campaigns12

Exposure to Mediums by Age70.0%60.0%50.0%65.2%49.3%55.6%43.9%41.8%% Under 40% 40 to 65 years old49.3%45.9%40.0%34.8%30.0%20.0%10.0%0.0%Bus ads* Posters* Handouts,cards, churchfans* Percentages significantly different (p

70.0%60.0%Heard of San Francisco Black InfantMortality Disparity – Time 1 & 2Time 1 (N = 804)Time 2 (N = 654)60.4%50.0%40.0%37.6%37.3%30.0%24.4%20.0%10.0%11.9%7.0% 8.1%13.2%0.0%Repeatedly Often Occassionally Never14

Awareness of Disparity and SleepPosition• Significant increase in % of AfricanAmericans who have ever heard of SFBlack Infant Mortality Disparity (39.6% vs.62.7%, OR = 2.5, CI 2.1-3.2)• No overall significant increases inknowledge of proper sleep position(70.4% vs. 71.7%)• Significantly higher level of awareness ofproper position for those exposed tocampaign vs. not exposed (79.7% vs.64.3%, OR = 2.2, CI 1.5-3.1)15

Undoing Racism Workshop• Conducted by Peoples Institute West• 227 people attended 1 of 7 workshops• Face to face interviews of 14 participants– 80% (11/14) used tools provided atworkshop to make changes in theirworkplace or community– 67% (8/12) of agencies adopted newpractices and made efforts to improveorganizational cultural competency anddecrease institutional racism.– 100% (14/14) of participants reportedtrying to take action to address or undoracism on a personal level.16

Cultural Competency TrainingComponents• Didactic session on disparities• Didactic session on cultural humility:• Small group exercises on cultural awareness• List of resources for continuing informationand education on cultural issues• Packet of client education materials and localreferral resources for clients• Article on components of culture for medicalstudents educationEvaluation• Pre-Post Tests, Baseline Assessments and 3and 6 follow-up assessments17

Training Participants• 193 providers have attended 1 of 5 trainingsover 15-months• Included African-Americans, Latinos, whites,Asian and the others of mixed race/ethnicity• Included MDs (17.1%), NPs (3.1%), RNs andPHNs (19.2%), social workers (23.8%), healtheducators (13.0%), and others such asmental health professionals, administrators,clinic assistants and managers (21.8%)• 82% of participants were very likely to usetools presented in workshop• 85.3% rated the training as good or excellent18

Cultural Competency Pre-PostTest FindingsIncreased confidence in their ability to:• Identify credible medical and social resourcesthat can assist with lifelong learning aboutAfrican American - 72.7%• Describe how the health provider bias of raceand culture can negatively impact the patientencounter - 58.2%• Use communication tools taught in theworkshop in clinical encounters with AfricanAmericans - 64.8%• Describe the elements of culturally competenthealth care - 66.5%• Describe the social conditions that impactrates of African American infant mortality -65.8%19

Preliminary Frequencies for Participants with Baseline and FollowupCultural Competency Assessments% Responding Yes OddsBaselineFollow-up Ratio (LCI - UCI)Does medical clinic have a checklist or tips oncultural competency? (N = 66) 18.2% 37.8% 2.74* 1.23-6.10Culturally appropriate patient education materialsavailable for staff to distribute? (N = 67)70.2% 89.6% 3.64* 1.42-9.35Materials available staff can read to betterunderstand how to help eliminate ethnic healthdisparities like infant mortality? (N = 67) 37.3% 61.2% 2.64* 1.32-5.32Patient brochures relevant to African Americanson display in office or clinic? (N = 67) 62.7% 79.1% 2.25* 1.04-4.86Are posters related to black infant mortality ondisplay where patients can see them? (N = 68) 35.3% 48.5% 1.72 0.87-3.44Did you find the culturally appropriate materialsprovided by the SevenPrinciples Project useful?(N = 66 on follow-up) 92.4%* = Difference statistically significant at p < .0520

Results: Snowball EffectOrganizational Attendence at Undoing Racism and Cultural CompetencyTrainings302520Number ofAttendees151021102788Number of subsequenttraining attendeesNumber of initital trainingattendees7503Black InfantHealth1SFDPHClinics7SF GeneralHospital35UCSF Goodwill HomelessPrenatal7Organizations21

Community Action Teams (CATS)• Mini-grants for CBOs to address African-American community-identified economic,social and/or environmental problems thatcontribute to poor community health.Evaluation Methods:– Process evaluation measures (i.e. numbermeetings attended, documenting activitiesand training through tracking form)– Questionnaires administered at thebeginning and end of the CATS work– Case study of selected CATS that includesinterviews and evaluator observation 22

CAT Findings• 14 CATs have been funded andparticipated in the evaluation (8 youth, 6adult)• 108 Baseline surveys, 79 follow-upsurveys, 50 baseline and follow-ups canbe linked• CAT Experiences - 39% Chaired ameeting; 33% attended Bd of Sup mtg.;84% represented CAT at meeting; 56%lead a special project; 63% did publicspeaking How about bullet these on their own slide?23

CAT Evaluation Challenges• Low response rates make generalizingfindings difficult• Notable variations in CATs based onagency, coordinator, and membershipimpacted success• Difficult to implement a structuredevaluation around communityimprovement activities – hard to getconsistent efforts and buy-in toimplement evaluation tools24

Dinner And A Movie (DAAM):Goal• Previously: African American women andmen of reproductive age and their familieswill be healthy through strengthened socialsupport networks in their neighborhoods.• Present: Develop within the project staff,the Partnership, community and thenetwork of community members andcommunity agencies the skills andcapacity to improve the health of thecommunity.Current evaluation method: Observation25

Evaluation: Progress to DateAttendance at Dinner And A Movie (DAAM)sNumber of Attendees35302520151050BCA Booker T. OMI FRC1 2 3 4Event NumberMean Attendance: Booker T. = 8, BCA = 6, OMI FRC = 2726

Key LessonsAwareness Campaigns• Involving target audience in developingcampaigns and reviewing messagesresulted in culturally appropriatecampaigns that reached a large number ofSan Francisco’s African AmericanresidentsCultural Competency Training and UndoingRacism Workshop• Health and social service providers areopen to trainings on these important topics,will utilize materials provided in theirworkplace, and will often try to take actionto address racism27

Key Lessons (Cont.)CATS• Team members feel a sense of belongingwith their CAT and enjoy participating• Success of CATs highly influenced bycoordinator• Different experiences in each group plus lowresponse rates make generalizations difficultDinner and A Movie• Attendance dependent on venue location andoutreach efforts by agencies• With the right venue and sufficient outreach,people will attend we have no proof of that28

FHOP website and contactinformationhttp://www.ucsf.edu/fhop(415) 476-528329