Conference Program PDF - SIRC : Seattle Implementation Research ...

S I R C 

Seattle Implementation Research Collaborative 

2 nd Biennial Conference: 

Solving Implementation Research Dilemmas 

Seattle, Washington — May 16‐17, 2013 

www.seattleimplementation.org

TABLE OF CONTENTS 

Welcome from the Conference Directors………………………………………………………………………………............................. 2 

Acknowledgements…………………………………………………………………………………………………………………………………………… 2 

Schedule & Hotel Map 

Schedule‐At‐A‐Glance & Hotel Map……………………………………………………………………………………………………………. 4 

Thursday, May 16……................................................................................................…………………………………….. 6 

Friday, May 17……............................................................................................................................................. 8 

Conference Tracks............................................................................................................................................. 10 

Symposia Abstracts — Thursday, May 16 

Should EBPs Be Locally Grown or Factory Farmed?……………………………………………………………………................... 15 

Innovative Approaches for Making EPBs Work................................................................................................. 19 

Leadership & Implementation.......................................................................................................................... 27 

Breakout Sessions 

Breakout A: EBP Champion Symposium: Is My Patient Getting Better?.................................................... 31 

Breakout B: Learning from Implementation Observation......................................................................... 33 

Breakout C: Implementation Through Collaborations with Policymakers................................................ 37 

Breakout D: Advancing Fidelity Measurement.......................................................................................... 41 

Breakout E: Implementing Primary Care Interventions............................................................................. 45 

Breakout F: EBP Champion Symposium: Solving Research Dilemmas Related to Implementation 

Fidelity........................................................................................................................................................ 49 

Breakout G: Sustainability & Adaptation in Social Services....................................................................... 51 

Breakout H: Implementation of Critical Time Intervention....................................................................... 55 

Breakout I: Matching Implementation to Setting...................................................................................... 59 

Breakout J: Research‐Community Relationships....................................................................................... 63 

Leveraging Technology...................................................................................................................................... 67 

Symposia Abstracts — Friday, May 17 

SIRC Instrument Review Taskforce: An Overview of Progress Made & Plans for the Future........................... 71 

Key Findings & Future Paths in Implementation Research............................................................................... 73 

Implementation in Zambia................................................................................................................................ 77 

Breakout Sessions 

Breakout K: Global Models of Implementation......................................................................................... 81 

Breakout L: Innovative Substance Abuse Treatment................................................................................. 85 

Breakout M: Statistical Methods Workshop Part I.................................................................................... 89 

Breakout N: Learning from Scale‐Up.......................................................................................................... 91 

Breakout O: Fidelity of Interventions Across the Age Spectrum................................................................ 95 

Breakout P: EBP Champion Symposium: Implementation of TF‐CBT Across Washington State............... 99 

Breakout Q: Sustainability.......................................................................................................................... 101 

Breakout R: Statistical Methods Workshop Part II..................................................................................... 105 

Breakout S: New Implementation Measures............................................................................................. 107 

Breakout T: Outcomes from New Interventions........................................................................................ 111 

Final Symposium: Interagency Collaborative Teams to Scale‐Up Evidence‐Based Practices……………………….. 115 

Poster Session Abstracts......................................................................................................................................... 119 

Additional Note Pages............................................................................................................................................. 142 

Nearby Restaurants & Map of Downtown Seattle................................................................................................. 147 

Program Evaluation Form........................................................................................................................................ 149 

Continuing Education Form..................................................................................................................................... 151 

2 nd Biennial Conference: Solving Implementation Research Dilemmas | May 16‐17, 2013 1

WELCOME FROM THE CONFERENCE DIRECTORS 

2013 Conference Core 

Planning Committee 

Kate Comtois, PhD, MPH 1 — 

Co‐Director 

Cara C. Lewis, PhD 2 — 

Co‐Director 

Andria K. Pierson, MEd 1 — 

Conference Co‐Coordinator 

Kristin Bumgardner, BS 1 — 

Conference Co‐Coordinator 

Rinad Beidas, PhD 3 

Cameo Borntrager, PhD 4 

Adam Carmel, PhD 1 

Kay Yu Yuan Chai 1 

Doyanne Darnell, PhD 1 

Shannon Dorsey, PhD (Co‐I) 1 

Karin Hendricks, BA 2 

Meghan Keough, PhD 1 

Suzanne Kerns, PhD (Co‐I) 1 

Sara J. Landes, PhD (Co‐I) 5 

Aaron R. Lyon, PhD (Co‐I) 1 

Ruben Martinez, BA 2 

On behalf of the SIRC Core Committee and staff, we welcome you to the 

2 nd Biennial Conference of the newly renamed Seattle Implementation 

Research Collaborative. We have several initiatives beyond the 

conferences themselves, so we decided to change our name to reflect our 

larger mission. We encourage you to check out our initiatives on our 

website (www.seattleimplementation.org) including: 

Instrument Review Project 

Strategic Planning Group (SPG) with its related projects 

o Implementation Research Development Workshop 

(IRDW) and mock grant reviews 

o Junior SPG Mentoring Program 

Impact of Infrastructure on Implementation International Survey 

Dissemination and Implementation Training Opportunities, & 

Dissemination and Implementation Reading Group 

This year’s conference theme is Solving Implementation Research 

Dilemmas and we have many presentations with innovative and effective 

solutions. As with our first conference, mornings will be spent together 

with two symposia, afternoons will have two breakout sessions where you 

can choose between 5 symposia, and we’ll come back together for a final 

symposium in the late afternoon. There are no set assignments for 

attending breakouts; you can attend whichever you like as seating 

permits. (Our survey last month simply allowed us to get a general idea of 

planned attendance for room size accommodations.) 

We would like to call your attention to our conference tracks this year. 

Please see page 10 for a schedule highlighting presentations that focus on 

each of these tracks: 

1 

Maria Monroe‐DeVita, PhD (Co‐I) EBP Champions* Scale‐Up 

1 

Stephen O’Connor, PhD Fidelity Sustainability 

6 

Landon Sach Global Perspectives Technology 

1 

Jennifer Villatte, PhD(c) 

Measurement Training 

1 University of Washington 

*i.e., the presentation includes clinicians or leaders from the practice community 

2 who have successfully implemented EBPs and champion them in their system 

Indiana University 

3 University of Pennsylvania 

We invite all of you to attend our reception and poster session Thursday 

4 University of Montana evening at 5:30 that includes our undergraduate, junior as well as senior 

5 National Center for PTSD, & VA Palo Alto Health 

colleagues’ work. Please enjoy hors d’oeuvres and music, a chance to 

Care System 

socialize, and encourage our junior colleagues as they start their careers 

6 Seattle University 

in implementation research. 

You will find evaluation pages for the conference in the back of this 

program. We encourage you to complete the evaluation to tell us how 

this meeting goes for you, and also to offer suggestions for new ventures 

on which SIRC might embark in the next two years. 

2 Seattle Implementation Research Collaborative (SIRC)

We want to extend our gratitude to the many people who have 

contributed to this conference and to SIRC in general. Thank you to our 

Advisory Committee for their wonderful ideas. Thank you to our Core 

Committee who has met bimonthly for over a year and volunteered an 

enormous amount of time as well as ideas to make the conference work. 

Thank you to our sponsors including NIMH who funded the majority of 

this conference through the R13 mechanism, as well as the in kind 

support of the University of Washington including the CHAMMP and 

WIMIRT centers, Indiana University, and the National Center for PTSD and 

Department of Veterans Affairs. We want to thank our many students 

and SIRC volunteers for their hard work in keeping the collaborative and 

this conference moving. Finally, we want to thank Andria Pierson and 

Kristin Bumgardner for their staff coordination managing all of the 

logistics and creating this setting where we can all learn and enjoy. 

Make the most of these days together and take advantage of the many 

opportunities to meet new colleagues and network. We look forward to 

seeing you at the reception tonight. We hope you will become active in 

SIRC, our dynamic collaborative. 

Welcome! 

2013 Advisory Board 

Eric Bruns, PhD 1 

Paul Ciechanowski, MD, MPH 1 

Dean L. Fixsen, PhD 2 

Dan Fox, MSW, LICSW 3 

David Kolko, PhD, ABBP 4 

Antoinette (Toni) Krupski, PhD 1 

Leif Solberg, MD 5 

Bradley Steinfeld, PhD 6 

Shannon Wiltsey Stirman, PhD 7 

Jürgen Unützer MD, MPH, MA 1 

1 University of Washington 

2 FPG Child Development Institute 

University of North Carolina at Chapel Hill 

3 Lutheran Community Services Northwest 

Kate Comtois, PhD, MPH 

Associate Professor, Department of 

Psychiatry & Behavioral Sciences, 

University of Washington 

Co‐Director, SIRC 

Cara C. Lewis, PhD 

Clinical Assistant Professor, 

Department of Psychological & 

Brain Sciences, Indiana University 

Co‐Director, SIRC 

4 Western Psychiatric Institute & Clinic, 

University of Pittsburgh 

5 HealthPartners Institute for 

Education & Research 

6 Group Health Cooperative 

7 VA Boston Healthcare System, National Center 

for PTSD,, & Boston University 

FUNDED BY: 

Grant No. 5 R13 MH086159 

SPONSORED BY: 


HOTEL DECA MAP 

Mezzanine Level 

Lobby Level 

Registration & Badges 

All SIRC conference attendees must be registered. The conference registration table will be located outside the 

Grand Ballroom. 

Badges are required for admission to all sessions, meals, and receptions. Please wear your badge during the 

conference, and remember to remove it outside the hotel. 

Lunch 

Additional seating for lunch will be available in the Governor Room on the lower level (not shown), across the hall 

from The District Lounge. For additional dining options, please see the list of nearby restaurants on page 147. 


SCHEDULE‐AT‐A‐GLANCE 

7:00 AM 

Wednesday 5/14 Thursday 5/16 Friday 5/17 Saturday 5/18 

Registration & Continental 

Breakfast 

8:00 AM Welcome Ceremony 

(8:00‐8:30) 

9:00 AM 

10:00 AM 

11:00 AM 

Innovative Approaches for 

Making EBPs Work 

(8:30‐9:45) 

Break 

Leadership & 

Implementation 

(10:00‐11:15) 

Registration & Continental 

Breakfast 

Day 2 Welcome & SIRC 

Instrument Review 

Taskforce (8:00‐8:45) 

Understanding 


Scale‐up 

(8:45‐10:00) 

Break 

Implementation in Zambia 

(10:15‐11:30) 

Schedule 

12:00 PM 

Buffet Lunch 

(11:15‐12:30) Buffet Lunch 

(11:30‐12:45) 

Implementation Research 

Development Workshop 

(IRDW) 

(9:00‐4:00) 

1:00 PM 

2:00 PM 

3:00 PM 

Concurrent Breakout 

Sessions: A‐E 

(12:30‐1:45) 

Break 


Sessions: F‐J 

(2:00‐3:15) 

Break 


Sessions: K‐O 

(12:45‐2:00) 

Break 


Sessions: P‐T 

(2:15‐3:30) 

Break 

Strategic Planning Group 

(SPG) Member Invited Only 

4:00 PM 

5:00 PM 

6:00 PM 

7:00 PM 

8:00 PM 

Dinner Excursions 

(Time TBA) 

Leveraging Technology 

(3:30‐4:45) 

Reception & Poster Session 

with Live Music, Hors 

d'oeurvres, & Cash Bar 

(5:30‐7:30) 


(Time TBA) 

Final Symposium, Closing 

Remarks, Presentation of 

Awards, & SIRC Updates 

(3:45‐5:15) 

Strategic Planning Group 

(SPG): Optional Review 

(5:15‐6:00) 

SIRC Fun Run 

(6:15) 


(Time TBA) 


(Time TBA) 


SCHEDULE—THURSDAY, MAY 16 1 

7:00‐8:00 REGISTRATION & CONTINENTAL BREAKFAST 

8:00‐8:15 Welcome to SIRC & Orientation to Conference………………………………………………………………………… Grand Ballroom 


8:15‐8:30 Should EBPs be Locally Grown or Factory Farmed?....................................................................... Grand Ballroom 

Greg Simon, MD, MPH 

8:30‐9:45 Innovative Approaches for Making EBPs Work (MC: Cameo Borntrager, PhD)…………………………. Grand Ballroom 

Bridge Over Troubled Waters: The Interactive Systems Framework for Dissemination & Implementation 

Abraham Wandersman, PhD 

Implementation of School‐Wide Positive Behavior Interventions & Supports: The Influence of Emotion, Self‐ 

Efficacy, & Organizational Commitment 

Zed Kramer, MA, Molly K. McDonald, MA, Brandon Rennie, EdS, & Cameo Borntrager, PhD 

Seeing is Believing: Behavioral Rehearsal Methodology 

Shannon Dorsey, PhD, Rinad Beidas, PhD, & Wendi Cross, PhD 

9:45‐10:00 BREAK 

10:00‐11:15 Leadership & Implementation (MC: Adam Carmel, PhD)…………………………………………………………… Grand Ballroom 

Leadership & Implementation 

Bruce J. Avolio, PhD 

Taking a Lesson from Usual Care: Predictors of Use of Evidence‐Based Practices for Youth 

Charmaine K. Higa‐McMillan, PhD 

11:15‐12:30 BUFFET LUNCH 

12:30‐1:45 Concurrent Breakout Sessions 

Breakout A: EBP Champion Symposium (MC: Sara J. Landes, PhD)…………………………………………… President 

Is My Patient Getting Better? Implementation of Mental Health Progress Monitoring/Outcomes System in an 

Integrated Delivery System 

Bradley Steinfeld, PhD 

Breakout B: Learning from Implementation Observation (MC: Suzanne Kerns, PhD)………………. Chancellor 

Training in Triple P (Positive Parenting Program): Exploring Implementation Outcomes Across Practitioner 

Groups in the United States, Australia, England & Canada 

Suvena Sethi, PhD 

Factors Associated with Adoption of a Mental Health Intervention for Autism Spectrum Disorders 

Colby Chlebowski, PhD 

Observed Barriers to Implementation of Empirically‐Supported Treatments by Clinicians Working with Military 

& Veteran Patients 

Craig J. Bryan, PsyD, ABPP, & David S. Riggs, PhD 

Breakout C: Implementation Through Collaborations with Policymakers (MC: Aaron Lyon PhD) College 

Effective Implementation of EBP Legislation by Engaging Providers in a Coaching Process 

Eric Trupin, PhD, & Gabrielle D’Angelo, MSW 

Negotiating Implementation Science & Evaluation Research: Lessons Learned from a National Teen Pregnancy 

Prevention Implementation Study 

Jacqueline Berman, PhD 

Identifying the Needs of OEF/OIF Veterans with TBI & Co‐Occurring Behavioral Health Issues 

Lisa Brenner, PhD, Jennifer Olson‐Madden, PhD, Bridget Matarazzo, PsyD, & Gina Signoracci, PhD 

Breakout D: Advancing Fidelity Measurement (MC: Jennifer Villatte, PhD(c))….………………………. Grand Ballroom 

Fidelity Measurements in the Real World: Feasibility of BECCI & MITI for Motivational Interviewing in Child & 

Youth Mental Health 

Melissa Kimber, MSW, PhD(c), Raluca Barac, MA, PhD, & Melanie Barwick, PhD, CPsych 

Comparisons Among Six Methods for Measuring Fidelity: Implications for Research & Practice 

Kristin Duppong Hurley, PhD 

An Update on Project BEST (Bringing Evidence‐Supported Treatments to South Carolina Children & Families): 

Challenges to Measuring Provider Fidelity 

Rochelle F. Hanson, PhD 

1 The schedule includes presenters only. Please see the abstracts for the full lists of authors. 


Breakout E: Implementing Primary Care Interventions (MC: Maria Monroe‐DeVita, PhD)……….. Regent 

Transformation & Spread of Primary Care Clinics into Medical Homes: It’s Slow, Hard Work 

Leif I. Solberg, MD 

A Qualitative Study of Fidelity: Understanding Variations in Implementation of the Patient‐ 

Centered Medical Home 

Rosalind Keith, PhD 

CADET: Clinical & Cost Effectiveness of Collaborative Care for Depression in UK Primary 

Care: A Cluster Randomized Controlled Trial 

David A. Richards, PhD 

1:45‐2:00 BREAK 


Breakout F: EBP Champion Symposium (MC: Cara C. Lewis, PhD, & Cameo Borntrager, PhD)..… President 

Solving Research Dilemmas Related to Implementation Fidelity 

Rebecca Selove, PhD, MPH, Kathryn Mathes, BSN, RN, MS, PhD, & Heather Wallace, PhD 

Breakout G: Sustainability & Adaptation in Social Services (MC: Maria Monroe‐DeVita, PhD)…. Chancellor 

Implementation Strategies in Social Service Settings: A Research Agenda 

Byron J. Powell, AM 

DBT Teams in Training 2008‐2011: Implementation Follow‐up in 2012 

Anthony DuBose, PsyD, & André Ivanoff, PhD 

Understanding Modifications to CBT in Community Settings: A Comparison of Providers in Adult & Child Mental 

Health Service Settings 

Shannon Wiltsey Stirman, PhD 

Breakout H: Implementation of Critical Time Intervention (MC: Meghan Keough, PhD)………….. Regent 

From Inception to Practice: Taking an Evidence‐Based Practice from Development to Implementation 

Challenges & Successes in Assessing Fidelity to the CTI Model Over Time 

Assessing the Implementation of the Critical Time Intervention Model Across 20 Homeless‐Service Agencies 

R. Neil Greene, PhD, & Melissa Martin, MSW 

Breakout I: Matching Implementation to Setting (MC: Sara J. Landes, PhD)…..………………………… Grand Ballroom 

Matching Training To Setting: A New Implementation Model For Dialectical Behavior Therapy 

Helen Best, MEd, Kate Comtois, PhD, MPH, & Nancy A. McDonald, MS, CADC, LPC 

User‐Centered Design & the Implementation of Evidence‐Based Interventions 

Aaron R. Lyon, PhD 

Designing an Implementation Strategy to Support the Multi‐Site Scale‐Up of an Evidence‐Based, Culturally 

Appropriate Practice Model for Intensive Family Support Services Across the Northern Territory, Australia 

Robyn Mildon, PhD 

Breakout J: Research‐Community Relationships (MC: Suzanne Kerns, PhD)…………………………….. College 

Evaluating the Success of a Statewide EBP Scale‐Up Project: The Children’s Administration‐University of 

Washington EBP Partnership 

Eric Bruns, PhD, Andrea Negrete, MEd, & Tammy Cordova, MSW 

Reviewing the Use of Research‐Community Partnerships to Facilitate Implementation of Evidence‐Based 

Practices in Children’s Community Services 

Nicole Stadnick, MS, MPH 

Developing the Autism Model of Implementation for ASD Community Providers: Use of Research‐Community 

Partnership 

Amy Drahota, PhD 

3:15‐3:30 BREAK 

3:30‐4:45 Leveraging Technology (MC: Rinad Beidas, PhD)..……………………………………………………………… Grand Ballroom 

Scaling Up Assessment of Therapist Fidelity in Motivational Interviewing: Preliminary Development of the 

AutoMITI 

David C. Atkins, PhD 

PracticeGround: An Online Platform to Help Therapists Learn, Implement, & Measure Impact of EBPs 

Gareth Holman, PhD 

Dialectical Behavior Therapy Implementation Process & Outcomes in VA & Community Settings 

Sara J. Landes , PhD, & Matthew Ditty, MSW 

4:45‐5:30 BREAK 

5:30‐7:30 Reception & Poster Session with Live Music, Hors d’oeuvres, & Cash Bar…….…………………….…… Grand Ballroom 

Featuring Classical Guitarists: Teresa Jaworski & Mark Hilliard Wilson 

Schedule 


SCHEDULE—FRIDAY, MAY 17 

7:00‐8:00 REGISTRATION & CONTINENTAL BREAKFAST 

8:00‐8:15 Day 2 Welcome…………………………………………………………………………………………………………………………. Grand Ballroom 


8:15‐8:45 SIRC Instrument Review Taskforce: An Overview of Progress Made & Plans for the 

Future.............................................................................................................................................. Grand Ballroom 


8:45‐10:00 Understanding Implementation Scale‐Up (MC: Suzanne Kerns, PhD)……………………………………….. Grand Ballroom 

Implementation Science in an Era of Health Reform & Patient‐Centered Comparative Effectiveness Research: 

New Threats, New Expectations, New Opportunities 

Brian Mittman, PhD 

Synthesis of Findings from 3 Lifestyle Behavior Change Program Implementation in the VA 

Laura J. Damschroder, MS, MPH 

Racial/Ethnic Disparities & the Implementation of Evidence‐Based Practices in Public Youth‐Serving Systems 

Antonio R. Garcia, PhD 

10:00‐10:15 BREAK 

10:15‐11:30 Implementation in Zambia (MC: Shannon Dorsey, PhD)…………………………………………………………… Grand Ballroom 

Implementation of TF‐CBT in Zambia: Perspectives from Local Supervisors & Counselors 

Margaret Kasoma 

Organizational Implementation Barriers & Facilitators for Mental Health Programs in Zambia: A Mixed‐Methods 

Study 

Laura Murray, PhD 

Mixed Methods Assessment of Implementation Barriers & Facilitators for Mental Health Programs in Zambia: 

Provider Level Themes 

Rinad Beidas, PhD 

11:30‐12:45 BUFFET LUNCH 


Breakout K: Global Models of Implementation (MC: Rinad Beidas, PhD).………………………………… Chancellor 

Scaling Up Care for Orphans in Tanzania: A Task‐Sharing Approach to Mental Health Treatment 

Shannon Dorsey, PhD 

A Transdiagnostic Mental Health Intervention in Low Resource Countries: An Alternative Solution to Mental 

Health Implementation Challenges 


Implementation of Cognitive Processing Therapy Provided by Community‐Based Paraprofessionals in the 

Democratic Republic of Congo: Influence of Therapist Factors Randomized Clinical Trial 

Debra Kaysen, PhD 

Breakout L: Innovative Substance Abuse Treatment Implementation (MC: Doyanne Darnell, 

PhD).………………………………………………………………………………………………………………………………………….. President 

Scaling Up & Sustaining Alcohol & PTSD Screening & Intervention in US Trauma Care Systems 

Douglas Zatzick, MD 

Lessons Learned from Implementing a Web‐Based Tool for Brief Alcohol Interventions in a Large Integrated 

Health Care System 

Kenneth R. Weingardt, PhD 

Disseminating Contingency Management: A Training & Implementation Trial 

Bryan Hartzler, PhD 

Breakout M: Statistical Methods Workshop Part I (MC: Kate Comtois, PhD, MPH).…….…………… College 

Design & Analysis Challenges with Multilevel Implementation Data 

David C. Atkins, PhD, & Scott A. Baldwin, PhD 

Breakout N: Learning from Scale‐Up (MC: Meghan Keough, PhD)…………………….………………………. Grand Ballroom 

Overcoming Implementation Research Challenges While Studying CPT Training & Implementation Across 

Canada 


Financing & Scaling Up Early Intervention Services 

Howard H. Goldman, MD, MPH 

System Improvement Through Service Collaboratives: Closing Gaps & Improving Access & Coordination 

Brian Rush, PhD 


Breakout O: Fidelity of Interventions Across the Age Spectrum (MC: Suzanne Kerns, PhD)…….. Regent 

Implementation of the Program to Encourage Active & Rewarding Lives for Seniors 

(PEARLS) 

Lesley Steinman, MSW, MPH 

Common Issues with Assessing Fidelity to Complex Multi‐Modal Service Programs: Lessons 

Learned from Assessing Fidelity to the ACT Model 

Maria Monroe‐DeVita, PhD 

Assessing Implementation Fidelity of the Family Check‐Up: Development & Validation of 

the COACH Rating System 

Justin D. Smith, PhD 

2:00‐2:15 BREAK 


Breakout P: EBP Champions Symposium (MC: Shannon Dorsey, PhD)…………………………….………… President 

Implementation of TF‐CBT Across Washington State 

Joe Leroy, MSW, Dan Fox, MSW, Ron Gengler, MS, & Lori Vanderburg, MS 

Breakout Q: Sustainability (MC: Adam Carmel, PhD)………………………………………………………………… Chancellor 

Sustainability of CBT for Youth Anxiety in Community Settings Following Implementation 


Supporting Implementation of the Triple P System: A Standardized Framework 

Jacquie Brown, MES, RSW, & Sara van Driel, PhD 

Research Implementation within a Clinical Practice: Resolving the Science/Practice Dialectic 

Sally A. Moore, PhD 

Breakout R: Statistical Methods Workshop Part II (MC: Kate Comtois, PhD, MPH)…………………… College 

Design & Analysis Challenges with Multilevel Implementation Data 

David C. Atkins, PhD, & Scott A. Baldwin, PhD 

Breakout S: New Implementation Measures (MC: Doyanne Darnell, PhD)……..………………………… Grand Ballroom 

Measuring an Evidence‐Based Model of Implementation: Preliminary Development of a Survey Instrument 

Josef I. Ruzek, PhD 

Solving Measurement Issues in Implementation Science 

Ruben Martinez, BA, & Cara C. Lewis, PhD 

Common Elements for Implementing Evidence‐Based Practices in Children’s Mental Health 

Lisa Saldana, PhD 

Breakout T: Outcomes from New Interventions (MC: Meghan Keough, PhD)…………………………… Regent 

Team‐Based Exposure & Ritual Prevention for Adults with Obsessive Compulsive Disorder: An Open Trial 

Implemented in a Community Mental Health Center 

Maria Mancebo, PhD 

Implementation of the Family Check‐Up in Community Mental Health Agencies: Clinical Effectiveness, Fidelity, 

& Other Outcomes 


Cognitive Retraining (CR) for Attention & Working Memory for Older Adults: What to Train, to Whom, & How 

Long? 

Lee Hyer, PhD, ABPP 

3:30‐3:45 BREAK 

3:45‐5:00 Final Symposium: Interagency Collaborative Teams to Scale‐Up Evidence‐Based Practices: 

Preliminary Results from a Large Scale Implementation (MC: Maria Monroe‐DeVita, PhD)……… Grand Ballroom 

Interagency Collaborative Teams for Capacity Building to Scale‐Up Evidence‐Based Practice 

Michael Hurlburt, PhD, Gregory A. Aarons,PhD, Danielle Fettes, Cathleen Willging, Lawrence A. Palinkas, PhD, & 

Mark J. Chaffin 

Collaboration, Negotiation, & Coalescence for Interagency‐Collaborative Teams to Scale‐up Evidence‐Based 

Practice 

Gregory A. Aarons, PhD, Michael Hurlburt, PhD, Danielle Fettes, Cathleen Willging, Lara Gunderson, MA, Mark 

Chaffin, & Lawrence A. Palinkas, PhD 

Leadership & Practice in the Face of Policy: How Supervisors & Providers Exercise Discretion in Evidence‐Based 

Practice Implementation 

Lara Gunderson, MA, & Cathleen Willging 

5:00‐5:15 Closing Remarks, Presentation of Awards, & Updates to SIRC Website…………………………………... Grand Ballroom 

5:15‐6:00 Strategic Planning Group (SPG) Members: Optional Session to Review Key Themes Identified 

in Conference & Areas on which to Follow‐Up……..….……………………….……………………………………. Grand Ballroom 

6:15 SIRC Fun Run………………………………………………………………………………………………………………………….…. Meet in Lobby 

Schedule 


CONFERENCE TRACKS 

The 2 nd Biennial SIRC Conference identified eight themes related to implementation research represented in the 

presentations. To assist those who would like to see all talks with a specific theme, we have made them into tracks. 

Please find below the talks that relate to each theme with their time and location. 

EBP CHAMPIONS 

Presentations featuring clinicians or leaders from the practice community who have successfully implemented 

EBPs and champion them in their system and beyond. 

Time Room Breakout 2 First Author 3 Title 

THURSDAY 

12:30 President A Steinfeld Is My Patient Getting Better? Implementation of Mental Health 

Progress Monitoring/Outcomes System in an Integrated Care 

Delivery System 

12:50 Chancellor B Bryan Observed Barriers to Implementation of Empirically‐Supported 

Treatments by Clinicians Working with Military & Veteran Patients 

2:00 President F Selove Solving Research Dilemmas Related to Implementation Fidelity 

2:00 Grand I Best Matching Training to Setting: A New Implementation Model for 

Dialectical Behavior Therapy 

FRIDAY 

10:15 Grand Kasoma Implementation of TF‐CBT in Zambia: Perspectives from Local 

Supervisors & Counselors 

2:15 President P Leroy Implementation of TF‐CBT Across Washington State 

2:55 Chancellor Q Moore Research Implementation within a Clinical Practice: Resolving the 

Science/Practice Dialectic 

FIDELITY 

Time Room Breakout First Author Title 

THURSDAY 

8:15 Grand Simon Should EBPs be Locally Grown or Factory Farmed 

8:50 Grand Kramer Implementation of School‐Wide Positive Behavior Interventions & 

Supports 

9:10 Grand Dorsey Seeing is Believing: Behavioral Rehearsal Methodology 

10:00 Grand Avolio Sourcing & Transmitting Leadership to Optimize Organizational 

Change 

12:30 Chancellor D Kimber Fidelity Measurements in the Real World: Feasibility of the BECCI & 

MITI for Motivational Interviewing in Child & Youth Mental Health 

12:50 Grand D Hurley Comparisons Among Six Methods for Measuring Fidelity: 

Implications for Research & Practice 

1:10 Grand D Hanson An Update on Project BEST: Challenges to Measuring Provider 

Fidelity 

12:50 Regent E Keith A Qualitative Study of Fidelity: Understanding Variation in 

Implementation of the Patient‐Centered Medical Home 

1:10 Regent E Richards CADET: Clinical & Cost‐Effectiveness of Collaborative Care for 

Depression in UK Primary Care: A Cluster RCT 

2:00 President F Selove Solving Research Dilemmas Related to Implementation Fidelity 

2 If Breakout is not listed, presentation is part of a symposium presented to the full conference in the Grand Ballroom. 

3 If the First Author is not presenting at the conference, it will be noted with an asterisk. 


FIDELITY 


2:20 Regent H Conover* Challenges & Successes in Assessing Fidelity to the Critical Time 

Intervention (CTI) Model Over Time 

3:30 Grand Atkins Scaling Up Assessment of Therapist Fidelity in Motivational 

Interviewing: Preliminary Development of the AutoMITI 

POSTERS (THURSDAY EVENING RECEPTION) 

5:30 Chancellor Mason Fidelity Assessment of Widely‐Disseminated but Understudied 

Prevention Programs: A Framework & Illustration from the Common 

Sense Parenting Trial 

5:30 Grand White Development & Use of a Fidelity Checklist for Permanency 

Roundtables: A New Child Welfare Intervention 

5:30 Grand Meisel Comparing Self, Clinician, & Observer Reports of Cognitive 

Processing Therapy (CPT) Adherence 

FRIDAY 



1:25 Chancellor K Kaysen Implementation of Cognitive Processing Therapy Provided by 

Community‐Based Paraprofessionals in the Republic of Congo: 

Influence of Therapist Factors Randomized Clinical Trial 

1:25 President L Hartzler Disseminating Contingency Management: A Training & 

Implementation Trial 

12:45 Grand M Atkins Part I: Design & Analysis Challenges with Multilevel Implementation 

Data 

12:45 Regent O Steinman Implementation of the Program to Encourage Active & Rewarding 

Lives for Seniors (PEARLS) 

1:05 Regent O Monroe‐DeVita Common Issues with Assessing Fidelity to Complex Multi‐Modal 

Service Programs: Lessons Learned from Assessing Fidelity to the 

ACT model 

1:25 Regent O Smith Assessing Implementation Fidelity of the Family Check‐Up: 

Development & Validation of the COACH Rating System 


2:15 College R Atkins Part II: Design & Analysis Challenges with Multilevel Implementation 

Data 

2:15 Regent T Mancebo Team‐Based Exposure & Ritual Prevention for Adults with Obsessive 

Compulsive Disorder: An Open Trial Implemented in a CMHC 

Schedule 

GLOBAL PERSPECTIVES 


THURSDAY 

12:30 Chancellor B Sethi Training in Triple P: Exploring Implementation Outcomes Across 

Practitioner Groups in the US, Australia, England, & Canada 

12:30 Grand D Kimber Fidelity Measurements in the Real World: Feasibility of the BECCI & 




2:40 Grand I Mildon Designing & Implementation Strategy to Support the Multi‐Site 

Scale‐Up of an Evidence‐Based, Culturally Appropriate Practice 

Model for Intensive Family Support Services Across the Northern 

Territory, Australia 


GLOBAL PERSPECTIVES 



5:30 Grand Aarons Implementation of an HIV Preventive Intervention in Mexico: The 

Roles of Context, Organizational Structure & Process, & Community 

Violence 

5:30 Grand Spielvogle Utilization of the Hybrid Model to Evaluate an Adolescent Treatment 

Engagement Intervention 

FRIDAY 



10:35 Grand Murray Organizational Implementation Barriers & Facilitators for Mental 

Health Programs in Zambia: A Mixed‐Methods Study 

10:55 Grand Beidas Mixed Methods Assessment of Implementation Barriers & 

Facilitators for Mental Health Programs in Zambia: Provider Level 

Themes 

12:45 Chancellor K Dorsey Scaling Up Care for Orphans in Tanzania: A Task‐Sharing Approach to 

Mental Health Treatment 

1:05 Chancellor K Murray A Transdiagnostic Mental Health Intervention in Low Resource 

Countries: An Alternative Solution to Mental Health Implementation 

Challenges 


Community‐Based Paraprofessionals in the Republic of Congo 

12:45 College N Wiltsey Stirman Overcoming Implementation Research Challenges while Studying 

CPT Training & Implementation Across Canada 

1:25 College N Rush System Improvement Through Service Collaboratives: Closing Gaps & 

Improving Access & Coordination 

3:05 Chancellor Q Brown Supporting Implementation of the Triple P System: A Standardized 

Framework 

MEASUREMENT 


THURSDAY 



Change 

12:30 President A Steinfeld Is My Patient Getting Better? Implementation of Mental Health 

Progress Monitoring/Outcomes System in an Integrated Care 

Delivery System 

12:30 College C Berman Negotiating Implementation Science & Evaluation Research: Lessons 

Learned from a National Teen Pregnancy Prevention Implementation 

Study 

12:30 Chancellor D Kimber Fidelity Measurements in the Real World: Feasibility of the BECCI & 


12:50 Grand D Hurley Comparisons Among Six Methods for Measuring Fidelity 

1:10 Grand D Hanson An Update on Project BEST: Challenges to Measuring Provider 

Fidelity 






MEASUREMENT 





5:30 Grand Martinez A Multi‐Level Framework for Implementation Science 

5:30 Grand White Development of an Assessment of Organizational Readiness for EBP 

Implementation in Public Child Welfare 

FRIDAY 

8:15 Grand Lewis SIRC Instrument Review Taskforce: An Overview of Progress Made & 

Plans for the Future 



12:45 Grand M Atkins Part I: Design & Analysis Challenges with Multilevel Implementation 

Data 

12:45 Regent O Steinman Implementation of the Program to Encourage Active & Rewarding 

Lives for Seniors (PEARLS) 

1:05 Regent O Monroe‐DeVita Common Issues with Assessing Fidelity to Complex Multi‐Modal 

Service Programs: Lessons Learned from Assessing Fidelity to ACT 

1:25 Regent O Smith Assessing Implementation Fidelity of the Family Check‐Up: 

Development & Validation of the COACH Rating System 

2:15 College R Atkins Part II: Design & Analysis Challenges with Multilevel Implementation 

Data 

2:15 Grand S Ruzek Measuring an Evidence‐Based Model of Implementation: Preliminary 

Development of a Survey Instrument 

3:05 Grand S Martinez Measurement Issues in Implementation Science 

3:25 Grand S Saldana Common Elements for Implementing Evidence‐Based Practices in 

Children’s Mental Health 

Schedule 

SCALE‐UP 


THURSDAY 


8:30 Grand Wandersman Bridge Over Troubled Waters: The Interactive Systems Framework for 

Dissemination & Implementation 

12:30 Chancellor C Trupin Effective Implementation of EBP Legislation by Engaging Providers in a 

Coaching Process 

1:10 College C Brenner Identifying the Needs of OEF/OIF Veterans with Traumatic Brain Injury 

(TBI) & Co‐Occurring Behavioral Health Issues & their Families 

1:10 Grand D Hanson An Update on Project BEST: Challenges to Measuring Provider Fidelity 

12:30 Regent E Solberg Transformation & Spread of Primary Care Clinics into Medical Homes: 

It’s Slow, Hard Work 



2:40 Regent H Zerger* Assessing the Implementation of the Critical Time Intervention Across 

20 Homeless‐Service Agencies 

2:40 Grand I Mildon Designing & Implementation Strategy to Support the Multi‐Site Scale‐ 

Up of an Evidence‐Based, Culturally Appropriate Practice Model for 

Intensive Family Support Services Across the Northern Territory, 

Australia 

2:00 College J Bruns Evaluation of the Success of a Statewide EBP Scale‐Up Project: The 

Children’s Administration‐University of Washington EBP Partnership 


SCALE‐UP 




4:10 Grand Landes Dialectical Behavior Therapy Implementation Process & Outcomes in 

VA & Community Settings 

FRIDAY 

9:25 Grand Garcia Racial/Ethnic Disparities & the Implementation of Evidence‐Based 

Practices in Public Youth‐Serving Systems 





1:05 Chancellor K Murray A Transdiagnostic Mental Health Intervention in Low Resource 

Countries: An Alternative Solution to Mental Health Implementation 

Challenges 



12:45 President L Zatzick Scaling Up & Sustaining Alcohol & PTSD Screening & Intervention in 

US Trauma Care Systems 

1:05 President L Weingardt Lessons Learned from Implementing a Web‐Based Tool for Brief 

Alcohol Interventions in a Large Integrated Health Care System 

12:45 College N Wiltsey Stirman Overcoming Implementation Research Challenges while Studying CPT 

Training & Implementation Across Canada 

1:05 College N Goldman Financing & Scaling Up Early Intervention Services 

1:25 College N Rush System Improvement Through Service Collaboratives: Closing Gaps & 




Framework 

2:15 Grand S Ruzek Measuring an Evidence‐Based Model of Implementation: Preliminary 


3:25 Grand S Saldana Common Elements for Implementing Evidence‐Based Practices in 

Children’s Mental Health 

2:35 Regent T Smith Implementation of the Family Check‐Up in Community Mental Health 

Agencies: Clinical Effectiveness, Fidelity, & Other Outcomes 

3:45 Grand Hurlburt Interagency Collaborative Teams for Capacity Building to Scale‐Up 

Evidence‐Based Practice 

4:05 Grand Aarons Collaboration, Negotiation, & Coalescence for Interagency‐ 

Collaborative Teams to Scale‐Up Evidence‐Based Practice 

4:25 Grand Gunderson Leadership & Practice in the Face of Policy: How Supervisors & 

Providers Exercise Discretion in Evidence‐Based Practice 


SUSTAINABILITY 


THURSDAY 

2:20 Chancellor G DuBose DBT Teams in Training 2008‐2012: Implementation Follow‐up in 2012 

2:40 Chancellor G Wiltsey Stirman Understanding Modifications to CBT in Community Settings: A 

Comparison of Providers in Adult & Child Mental Health Settings 


SUSTAINABILITY 


FRIDAY 

12:45 President L Zatzick Scaling Up & Sustaining Alcohol & PTSD Screening & Intervention in 

US Trauma Care Systems 


2:15 Grand Q Beidas Sustainability of CBT for Youth Anxiety in Community Settings 

Following Implementation 


Framework 

Schedule 

TECHNOLOGY 


THURSDAY 






2:20 Grand I Lyon User‐Centered Design & the Implementation of Evidence‐Based 

Interventions 



3:50 Grand Koerner* PracticeGround: An Online Platform to Help Therapists Learn, 

Implement, & Measure Impact of EBPs 




5:30 Grand Green Adapting a Research Tested Automated Electronic Health Record 

Intervention for Implementation in Safety Net Clinics 

FRIDAY 

1:05 President L Weingardt Lessons Learned from Implementing a Web‐Based Tool for Brief 

Alcohol Interventions in a Large Integrated Health Care System 



TRAINING 


THURSDAY 



Change 



12:50 Chancellor B Bryan Observed Barriers to Implementation of Empirically‐Supported 

Treatments by Clinicians Working with Military & Veterans 

2:20 Chancellor G DuBose DBT Teams in Training 2008‐2012: Implementation Follow‐up in 2012 

2:00 Regent H Herman* From Inception to Practice: Taking an Evidence‐Based Practice from 

Development to Implementation 

2:00 Grand I Best Matching Training to Setting: A New Implementation Model for DBT 


TRAINING 


3:50 Grand Koerner* PracticeGround: An Online Platform to Help Therapists Learn, 

Implement, & Measure Impact of EBPs 




5:30 Grand Aarons Implementation of an HIV Preventive Intervention in Mexico: The 

Roles of Context, Organizational Structure & Process, & Community 

Violence 

5:30 Grand Landy Improving Our Capacity for Evidence‐Based PTSD Treatment: 

Developing an Effective Model of Post‐Workshop Consultation 

FRIDAY 












2:15 Grand Q Beidas Sustainability of CBT for Youth Anxiety in Community Settings 

Following Implementation 

2:15 Regent T Mancebo Team‐Based Exposure & Ritual Prevention for Adults with Obsessive 

Compulsive Disorder: An Open Trial in a CMHC 


May 16. 8:15‐8:30 

SHOULD EBPS BE LOCALLY GROWN 

OR FACTORY FARMED? 

(MC: Kate Comtois, PhD, MPH) 


Group Health Research Institute 

Symposia – May 16 


SHOULD EBPS BE LOCALLY GROWN OR FACTORY FARMED? 


Group Health Research Institute 

Contact: simon.g@ghc.org 

Efforts to implement evidence‐based psychosocial treatments have typically assumed that those treatments would 

be delivered by local providers. Recent research regarding care management and structured psychotherapy 

programs for depression should cause us to question this assumption. Effectiveness trials strongly support the 

fidelity and clinical effectiveness of centrally produced or “factory‐farmed” depression treatment programs. The 

limited data available suggest that centrally produced treatments clearly out‐perform “locally grown” models of 

depression care. For any specific treatment, the likelihood that a centralized delivery model will prove superior 

depends on two questions: First, can this treatment be provided over distance (via telephone, video conference, or 

some other telehealth medium) without significant loss of clinical effectiveness? Second, does local variation in this 

treatment lead to better outcomes or simply lower quality? Ultimately, the choice between centrally produced and 

locally produced mental health treatments should depend not on the needs or preferences of providers or 

researchers, but on the clinical benefits and value they provide to patients or consumers. 

Track(s): Fidelity, Scale‐Up, Technology 

NOTES 


May 16. 8:30‐9:45 

INNOVATIVE APPROACHES FOR 

MAKING EPBS WORK 

(MC: Cameo Borntrager, PhD) 

Bridge Over Troubled Waters: The 

Interactive Systems Framework for 

Dissemination & Implementation 


University of South Carolina 

Implementation of School‐Wide 

Positive Behavior Interventions & 

Supports: The Influence of Emotion, 

Self‐Efficacy, & Organizational 

Commitment 

Zed Kramer, MA, Molly K. McDonald, 

MA, Brandon Rennie, EdS, Cameo 

Borntrager, PhD 

University of Montana 


Seeing is Believing: Behavioral Research 

Methodology 

Shannon Dorsey, PhD, 1 Rinad Beidas, 2 

PhD, & Wendi Cross, PhD 3 

1 University of Washington; 2 University of 

Pennsylvania, Perelman School of 

Medicine; 3 University of Rochester 

Medical Center 


BRIDGE OVER TROUBLED WATERS: THE INTERACTIVE SYSTEMS FRAMEWORK FOR DISSEMINATION & 

IMPLEMENTATION 


University of South Carolina 

Contact: wandersman@sc.edu 

Reducing the gap between research and practice requires a practical framework that brings funders, practitioners, 

researchers/evaluators, and consumers together for effective implementation and scaling up. Abe Wandersman and 

colleagues, in collaboration with CDC Division of Violence Prevention staff, developed the Interactive Systems 

Framework for Dissemination and Implementation (ISF) (2008 and 2012 special issues of the American Journal of 

Community Psychology). The ISF is now being widely cited and used in domains throughout public health and 

in education. The presentation will (a) introduce the ISF; (b) briefly describe one of the contributions of the ISF 

special issue: Practical Implementation Science; and (c) provide examples of bridge‐building including at CDC and 

elsewhere. 

Track(s): Scale‐Up 

NOTES 


TM 

Applying Science. 

Advancing Practice. A S A P 

Understanding the Interactive Systems Framework 

for Dissemination and Implementation 

State public health agencies and sexual assault coalitions 

have developed prevention plans with goals and 

objectives to prevent first time perpetration of sexual 

assault as part of the Rape Prevention Education (RPE) 

program. These plans were intended to prepare the 

way for successful and sustainable implementation of 

evidence-based prevention programs. However, very 

few of these evidence-based programs exist for the 

prevention of sexual violence. This lag in the development 

of evidence-based programs continues to challenge both 

sexual violence prevention researchers and practitioners. 

The public health approach to violence prevention 

(Figure 1) uses four steps to systematically define the 

problem, identify risk and protective factors, develop 

and test prevention strategies, and finally, ensure 

widespread adoption. This model assumes that the tested 

interventions will be used in the field, but it provides very 

little information on how this should be accomplished. 

The Interactive Systems Framework (ISF) for 

Dissemination and Implementation was developed to 

address the “how to” gap that exists between scientifically 

determining what works and moving that knowledge into 

the field for the benefit of the public. 

Those who work in the various fields of violence prevention 

are motivated to develop, evaluate, disseminate, and 

implement effective strategies for preventing violence 

with the goal of building a safer, healthier society. Ideally 

we would select programs, practices, or policies that have 

been proven to be effective—meaning there is strong, 

scientific evidence that they work. 

Applying Science. 

Advancing Practice. 

What is ASAP? Applying Science. Advancing 

Practice. (ASAP) is a series of informational 

briefs created by CDC’s Division of Violence 

Prevention to help apply scientific knowledge 

to the practice of primary prevention of 

violence. 

Who is it for? This series of ASAPs are written 

for state health departments and statewide 

sexual assault coalitions, the current support 

system for rape prevention education activities. 

Sharing with other violence prevention 

partners is encouraged. 

How will it help? ASAP offers specialized, topicspecific 

information necessary for successful, 

sustainable violence prevention efforts. This 

particular series on sexual violence prevention is 

intended to provide information and resources 

for RPE grantees who provide prevention 

support to community-based prevention 

education activities. 

National Center for Injury Prevention and Control 

Division of Violence Prevention

Applying Science. Advancing Practice. 

Figure 1: The Public Health Approach to Violence Prevention and the ISF 

Define the 

Problem 

Identify Risk 

and Protective 

Factors 

Develop and 

Test Prevention 

Strategies 

ISF 

Ensure 

Widespread 

Adoption 

The Interactive Systems Framework (ISF) for Dissemination and Implementation provides a framework 

for understanding how to address the gap between the third and fourth stages of the public 

health approach to violence prevention, often referred to as the research to practice gap. 

Figure 2: The Interactive Systems Framework for Dissemination and Implementation 

Funding 

Implementing Prevention—Prevention Delivery System 

General 

Capacity Use 

Innovation-Specific 

Capacity Use 

Macro 

Policy 

Supporting the Work—Prevention Support System 

General 

Capacity Building 

Innovation-Specific 

Capacity Building 

Climate 

Distilling Information—Prevention Synthesis and Translation System 

Synthesis 

Translation 

Existing Research and Theory

A S A P 

The ISF was developed specifically with the fields of 

youth violence and child maltreatment prevention in 

mind, where much evidence has been gathered over the 

past several decades about what works and does not 

work. Despite this growing evidence, wide-spread use of 

these effective strategies has been less than ideal. The ISF 

resolves this by addressing some of these questions: 

• How do we achieve the widespread use of effective 

practices, policies, and programs to prevent violence? 

• What infrastructures or systems are necessary to 

ensure that dissemination and implementation are 

carried out successfully? 

• How do organizations and practitioners build the 

capacity necessary to bring effective violence 

prevention strategies to scale community wide? 

One advantage the ISF offers to the sexual violence 

prevention field is a well thought out, underlying process 

for how to move science to practice. By spending the 

time understanding these underlying processes now, 

the field will be better prepared to more rapidly move 

effective programs, practices, or policies into the hands of 

communities as they become available later. 

A Closer Look at the Interactive 

Systems Framework 

Figure 2 shows the ISF and how it connects three systems 

to work together for successful dissemination and 

implementation of prevention innovations. The term 

“system” is used broadly to describe a set of activities that 

accomplish one of the three identified functions that 

make dissemination and implementation possible. These 

systems are: 

Prevention Synthesis and Translation System 

Here scientific knowledge is distilled into understandable 

and actionable information. Research institutions, 

universities, and the Division of Violence Prevention (DVP) 

at CDC are all institutional examples of this system. 

Prevention Support System 

This system supports the work of the other two systems 

through building capacity for carrying out prevention 

activities. Agencies like CDC, state health departments, 

or state sexual assault coalitions are often in the role of 

prevention support for grantees or local programs. 

Prevention Delivery System 

This is where innovations are actually implemented or 

where “the rubber meets the road.” Community-based 

organizations often function in the role of the prevention 

delivery system. 

As depicted in figure 2, these three systems work 

together and are embedded within an underlying 

context that influences decision-making and adoption 

of prevention strategies. These underlying conditions 

include: legislation that supports funding for sexual 

assault prevention, the best available theory and 

research related to the prevention of sexual assault, 

the community and/or organizational context in which 

sexual assault strategies are implemented and macrolevel 

policy factors such as state or federal level budget 

constraints or legislative changes. These underlying 

considerations are graphically displayed as the climate 

in which the three systems exist, and all of these have an 

impact on successful dissemination and implementation. 

Each system within the ISF also builds upon or 

influences the functions of the other two systems. These 

relationships and influences are represented by the 

arrows that connect the systems to each other. 

“If we keep doing what we are 

doing, we will keep getting 

what we are getting.” 

–Anonymous 

For sexual violence prevention, where the research 

evidence is scant and still being built, the ISF can be 

especially helpful. What the ISF can do is take what we do 

know about effective prevention principles and processes 

and distill that knowledge into understandable concepts 

through the Prevention Synthesis and Translation System. 

The Prevention Support System builds the capacity of 

local organizations to put these prevention principles and 

processes into practice. The Prevention Delivery System 

serves to strengthen and deliver prevention principles 

and processes on the ground. 

To illustrate how the ISF would function in the prevention 

of sexual violence, consider the following examples of 

activities that may occur within each system:

Applying Science. Advancing Practice. 

Distill (PSTS) 

• Review and condense scientific literature on risk and 

protective factors for sexual violence. 

• Translate research findings about risk and protective 

factors for sexual violence into user friendly 

language. 

Support (PSS) 

• Build the capacity of local organizations to develop 

strong leaders, understand how to use data, or form 

long-lasting partnerships. 

• Provide training and technical assistance about 

specific prevention strategies. 

Delivery (PDS) 

• Implement sexual violence prevention strategies 

across a community. 

• Support the spread and uptake of effective sexual 

violence prevention principles. 

• Monitor and evaluate programmatic activities to 

further improve the program. 

While the ISF includes activities or functions that are 

carried out by people in many different kinds of roles and 

within three distinct systems, these systems are working 

together to distill, support, and deliver prevention 

strategies. By understanding the functions of these three 

systems and how they work together, organizations, 

stakeholders, funders, and practitioners can 

communicate better and work together to disseminate 

and more effectively implement prevention strategies. 

You may have noticed that in the example above, much 

of the RPE grantee roles and/or functions showed up 

in the Prevention Support System. This makes sense 

because as an RPE grantee, the role of state public 

health agencies and state-level sexual assault coalitions 

is to provide support for local programs to ensure 

they can implement rape prevention education at the 

community level. These support activities can be seen as 

an important link between taking scientifically derived 

information and putting it into practice. 

Future editions of ASAP will focus on the PSS in more 

detail. Specifically, they will describe how to understand 

the capacities necessary for individuals and organizations 

(which are linked through systems) to prevent sexual 

violence and build healthier and safer communities. 

A S A P 

Key Terms 

The following key terms are found throughout this brief. 

Capacity: The ability, skills, and motivations to 

conduct and sustain prevention work at the individual, 

organizational, and systems level. The ISF views capacity 

as carrying out important functions in two distinct ways: 

• General Capacity – a capacity to implement or 

improve any programmatic strategy or activity. 

• Innovation Specific Capacity – a capacity needed 

to plan, implement, evaluate and sustain primary 

prevention strategies. 

Dissemination: The intentional, targeted spreading 

of an innovation from the originators to the intended 

users that result in a targeted and facilitated process of 

distributing information and materials to organizations 

and individuals who want and can use them to improve 

health. 

Implementation: A purposeful set of specific activities 

that result in individual or organizational use of an 

innovation. 

Innovation: New prevention knowledge or information 

- product, practice, program, policy, idea, research 

findings, or results. 

Strategy: An approach to address a problem such as 

the promotion of respectful relationships to reduce 

interpersonal violence. 

Synthesis: A process for obtaining and summarizing 

scientifically derived information, including evidence of 

effectiveness (risk and protective factors, core elements, 

and key features, etc.). 

Translation: The process of converting scientific and 

technically complex research into everyday language and 

applicable/actionable concepts in the practice setting. 

More Information 

More information about the ISF can be found in the 

following article at www.cdc.gov/ViolencePrevention/ 

sexualviolence/translation.html: 

Wandersman, A., Duffy, J., Flasphor, P., Noonan, R., Lubell, K., Stillman, L., 

et al. (2008). Bridging the gap between prevention research and practice: 

The Interactive Systems Framework for Dissemination and Implemetation. 

American Journal of Community Psychology. 41, 3-4. 

For more information, please contact: 

Centers for Disease Control and Prevention 

1-800-CDC-INFO • www.cdc.gov/violenceprevention • cdcinfo@cdc.gov

IMPLEMENTATION OF SCHOOL‐WIDE POSITIVE BEHAVIOR INTERVENTIONS & SUPPORTS: THE INFLUENCE 

OF EMOTION, SELF‐EFFICACY, & ORGANIZATIONAL COMMITMENT 

Zed Kramer, MA, Molly K. McDonald, MA, Brandon Rennie, EdS, & Cameo Borntrager, PhD 

University of Montana 

Contact: zed.kramer@umontana.edu 

Both organizational and individual factors are known to influence implementation. However, there is little 

research investigating the interaction of these influences. For example, Klimes‐Dougan et al. (2009) found 

counterintuitive results from a study examining factors related to fidelity: organizational culture was negatively 

related to fidelity; whereas, low job satisfaction was positively related to fidelity. Additionally, research on the 

role of emotion suggests complex dynamics are at work. Emotional valence has been found to function as a 

barrier and facilitator to implementation (Choi et al., 2011; Peters et al., 2011). Self‐efficacy has also been 

identified as a factor involved in the implementation of evidence‐based practices (Aarons et al., 2012). This 

study further assesses the relationship between organizational initiatives that appear facilitative and individual 

factors presupposed to affect implementation. Using self‐report data from elementary school teachers, the 

relationships between emotion, self‐efficacy, and organizational commitment are investigated within the 

context of an ongoing implementation project initially presented at SIRC 2011 involving School‐Wide Positive 

Behavior Interventions & Supports (SWPBIS). SWPBIS emphasizes prevention, data based decision‐making, and 

evidence‐based practices for addressing difficult behaviors in youth. Implications for the potential existence of 

recursive feedback between affect and self‐efficacy and its influence on organizational climate are discussed. 


Track(s): Fidelity 

NOTES 


SEEING IS BELIEVING: BEHAVIORAL REHEARSAL METHODOLOGY 

Shannon Dorsey, PhD, 1 Rinad Beidas, PhD, 2 & Wendi Cross, PhD 3 

1 University of Washington; 2 University of Pennsylvania, Perelman School of Medicine; 3 University of Rochester 

Medical Center 

Contact: dorsey2@uw.edu 

Evidence‐based interventions (EBI) are available for a variety of problems, populations, and settings. 

Nevertheless, intervention effectiveness hinges upon the skill level of the people implementing them, making 

cost‐effective training and supervision programs a critical focus for implementation science. Adult learning 

theory indicates that implementer skill development is most likely when active learning strategies, which include 

modeling and practice opportunities, are employed. In addition, behavioral rehearsal and standardized patient 

(SP) methods offers a more rigorous and objective means of assessing skill development and fidelity than 

commonly used strategies (e.g., self‐report), and a more cost‐effective strategy than audio or video coding. 

Understanding how to use BR and SP strategies for cost‐effective dissemination and implementation, and more 

rigorous evaluation, is an important emerging research focus. 

We present findings from three studies using similar BR methodology (Cross et al., 2007) as both a training and a 

fidelity assessment tool. Presentation will include an in‐depth focus on the use of BR in a CBT common elements 

training initiative in Washington State for child and adolescent depression, anxiety, trauma, and behavior 

problems. Presentation will include BR vignette examples, decision points in BR use (e.g., actor selection; coding 

strategy), actual audio/video of BR, and study outcomes. 

Track(s): Fidelity, Measurement, Training 

NOTES 


May 16. 10:00‐11:15 

LEADERSHIP & IMPLEMENTATION 

(MC: Adam Carmel, PhD) 

Leadership & Implementation 

Bruce J. Avolio, PhD 

Center for Leadership & Strategic 

Thinking, Michael G. Foster School of 

Business, University of Washington 

Taking a Lesson from Usual Care: 

Predictors of Use of Evidence‐Based 

Practices for Youth 

Charmaine K. Higa‐McMillan, PhD 

University of Hawaii at Hilo 



SOURCING & TRANSMITTING LEADERSHIP TO OPTIMIZE ORGANIZATIONAL CHANGE 

Bruce J. Avolio, PhD, Marion B. Ingersoll Professor, Executive Director, Center for Leadership & Strategic Thinking, 

Michael G. Foster School of Business, University of Washington 

Contact: bavolio@uw.edu 

The focus of my presentation will be on examining the various sources of leadership and how those sources are 

transmitted and contribute to or detract from individual, unit and organizational transformation. Today, the source 

of leadership is recognized as not being just associated with the designated or formal leader. Indeed, the source of 

leadership can be a group or even a crowd! Moreover, leadership is now being distributed throughout organizations, 

communities and nation states in ways that are creating opportunities for fundamental change in the way we 

configure our institutions and lead them in the 21 st century, including for profit, not for profit, government agencies. 

Participants can expect to learn the following from our discussion: 

 

 

 

 

Where does their leadership come from and how can it be effectively transmitted. 

How the source and transmission of leadership can drive more ownership at all organizational levels. 

What it means to consider the total leadership system in one’s organization and its development when 

engaging in organizational transformation. 

One insight that each participant can apply to his or her own leadership development. 


NOTES 


TAKING A LESSON FROM USUAL CARE: PREDICTORS OF USE OF EVIDENCE‐BASED PRACTICES FOR YOUTH 

Charmaine K. Higa‐McMillan, PhD, 1 Ashley Usita, MA, 1 & Brad J. Nakamura, PhD 2 

1 University of Hawaii at Hilo; 2 University of Hawaii at Manoa 

Contact: higac@hawaii.edu 

A growing body of research suggests that in addition to examining adoption of evidence‐based psychosocial 

interventions (EBPIs), studying practices in usual care might be a complimentary approach to solving a number 

of implementation research dilemmas. In a recent study of usual care for youth with disruptive behavior 

problems, Brookman‐Frazee, Garland, et al. (2010) found few youth, family, and therapist characteristics that 

predicted use of EBPIs. The current study expanded on these findings by examining therapists (N=74) providing 

services for youth with anxiety, trauma, depressive, attentive, and disruptive behavior problems (N=514) in a 

large public mental health system. Using multilevel modeling, this study examined therapist‐level characteristics 

(e.g., training background, theoretical orientation) that predict use of EBPIs after accounting for child‐level 

characteristics (e.g., age, gender, diagnosis, functional impairment, episode length). Results suggest that while 

no child‐level characteristics predict therapist use of EBPIs, child age, gender, functional impairment at episode 

start, and service type predict use of practices that do not have a strong evidence‐base. Further, while most 

therapist characteristics do not predict use of EBPIs, theoretical orientation accounts for a significant amount of 

variance in provider use of EBPIs with youth in usual care. Implications for implementation research will be 

discussed. 


Track(s): None specific to this conference 

NOTES 



May 16. 12:30‐1:45 

Breakout A: 

EBP Champion Symposium 

(MC: Sara J. Landes, PhD) 

Is My Patient Getting Better? 

Implementation of Mental Health 

Progress Monitoring/Outcomes System 

in an Integrated Delivery System 

Bradley Steinfeld, PhD 

Group Health Cooperative 



IS MY PATIENT GETTING BETTER? IMPLEMENTATION OF MENTAL HEALTH PROGRESS MONITORING/ 

OUTCOMES SYSTEM IN AN INTEGRATED DELIVERY SYSTEM 

Bradley Steinfeld, PhD, Allie Franklin, MSSW, Mariam Sarikhan, MA, & Brian Mercer 

Group Health Cooperative 

Contact: steinfeld.b@ghc.org 

Knowing whether patients are getting better is foundational to providing evidence‐based care. Implementing a 

progress monitoring system that can both be effectively used by clinicians in real time to track progress at the 

individual patient level while at the same time provide feedback on mental health outcomes at a clinic or 

program level is a complex endeavor. This symposium will describe the multiyear experience of a behavioral 

health department in an integrated delivery system in implementing a progress monitoring system. Strategies 

for engaging practicing clinicians in use of a progress monitoring tool as well as how to spread tool use across 

multiple locations and providers will be discussed. A particular focus will be on how to integrate a progress 

monitoring system into an electronic medical record. This symposium will involve multiple speakers providing 

perspectives on implementation of a progress monitoring system from the individual clinician, clinic manager, 

organizational leaders, and information systems analyst. 

Track(s): EBP Champions, Measurement 

NOTES 


May 16. 12:30‐1:45 

BREAKOUT B: LEARNING FROM 

IMPLEMENTATION OBSERVATION 

(MC: Suzanne Kerns, PhD) 

Training in Triple P (Positive Parenting 

Program): Exploring Implementation 

Outcomes Across Practitioner Groups in 

the United States, Australia, England, & 

Canada 

Suvena Sethi, PhD 

Parenting & Family Support Centre, 

University of Queensland, Australia 

Factors Associated with the Adoption of 

a Mental Health Intervention for 

Autism Spectrum Disorders 

Colby Chlebowski, PhD 

University of California, San Diego 


Observed Barriers to Implementation of 

Empirically‐Supported Treatments by 

Clinicians Working with Military & 

Veteran Patients 

Craig J. Bryan, PsyD, ABPP, 1 & David S. 

Riggs, PhD 2 

1 National Center for Veterans Studies; 

2 Center for Deployment Psychology 


TRAINING IN TRIPLE P (POSITIVE PARENTING PROGRAM): EXPLORING IMPLEMENTATION OUTCOMES ACROSS 

PRACTITIONER GROUPS IN THE UNITED STATES, AUSTRALIA, ENGLAND & CANADA 

Suvena Sethi, PhD, & Matthew Sanders, PhD 

Parenting & Family Support Centre, University of Queensland, Australia 

Contact: s.sethi@uq.edu.au 

The Triple P Positive Parenting Program is a multi‐level parenting and family support strategy that aims to reduce 

behavioral, emotional, and developmental problems in children by enhancing parental knowledge, skills, and 

confidence to promote positive and supportive relationships with their children. An integral element of the 

implementation of Triple P is practitioner training. This paper explores the results of an analysis of training outcomes 

of participants who received practitioner training in Level 4, Group Triple P (n=4080; for example, GP’s, psychologists, 

nurses, teachers, social and family support workers). A series of ANOVAs were conducted to analyze training 

outcomes across practitioner groups including perceived adequacy of skills and proficiency in the delivery of Triple P 

strategies. While we highlight that across each country, outcomes illustrate significant improvements in practitioner 

confidence in the delivery of positive parenting strategies, qualitative responses of participants also highlight the 

potential challenges faced by practitioners in accessing evidence‐based interventions. The successful 

implementation of a Triple P Online Training Program represents the next crucial step in program development. 

Discussion will draw on a public health model that engages a wider practitioner cohort and will include an emerging 

focus on translating Triple P provider training for an online implementation strategy. The opportunity and challenges 

of such a process will be discussed. 

Track(s): Global Perspectives, Training, Technology 

NOTES 


FACTORS ASSOCIATED WITH ADOPTION OF A MENTAL HEALTH INTERVENTION FOR AUTISM SPECTRUM 

DISORDERS 

Colby Chlebowski, PhD, & Lauren Brookman‐Frazee, PhD 

Department of Psychiatry, Child & Adolescent Services Research Center, University of California, San Diego 

Contact: cchlebowski@ucsd.edu 

Community and school‐based mental health programs play an important role in caring for school‐age children 

with Autism Spectrum Disorders (ASD). The AIM HI (“An Individualized Mental Health Intervention for ASD”) 

was developed to address concerns about the quality of publicly‐funded MH care for this population. It is a 

package of EB strategies developed specifically for delivery in MH services and designed in collaboration with 

community stakeholders based on a community needs assessment and data on EB strategies for this clinical 

population. This presentation will report findings related to intervention adoption from a large‐scale 

effectiveness and implementation study of AIM HI. Child MH programs from one large, geographically diverse 

county are being recruited to participate and randomized to immediate AIM HI implementation or a waitlist 

control condition. To date, 100% (n=18) of eligible programs approached to participate have enrolled in the 

study. Preliminary themes emerging from the program recruitment process highlight implementation 

facilitators at the system and organizational levels (e.g., system leader champions, program leader support 

based on perceived need for therapist ASD training, and strong implementation climate). Data from agency and 

program leader and therapist surveys reporting on their decisions to adopt AIM HI, implementation climate, 

leadership, and attitudes towards evidence‐based practices will be presented. 



NOTES 


OBSERVED BARRIERS TO IMPLEMENTATION OF EMPIRICALLY‐SUPPORTED TREATMENTS BY CLINICIANS 

WORKING WITH MILITARY & VETERAN PATIENTS 

Craig J. Bryan, PsyD, ABPP, 1 & David S. Riggs, PhD 2 

1 National Center for Veterans Studies; 2 Center for Deployment Psychology 

Contact: craig.bryan@utah.edu 

During the past two years, the National Center for Veterans Studies and Center for Deployment Psychology have 

provided training workshops to over 9000 military and civilian mental health professionals focused on several 

empirically‐supported treatments for psychiatric conditions of particular relevance to military and veteran patients: 

prolonged exposure (PE) and cognitive processing therapy (CPT) for PTSD, cognitive behavioral therapy for insomnia 

(CBTi), imagery rehearsal therapy (IRT) for nightmares, brief cognitive behavioral therapy (BCBT) for suicide risk, and 

cognitive behavioral therapy for chronic pain. Very few mental health professionals have been exposed to these 

treatments, and several common barriers to full adoption of these treatment models have been noted, including 

misconceptions about treatment safety and efficacy, strong commitment to unsupported theories and practices, and 

pseudoscientific thinking. Furthermore, adoption of these practices has been limited despite these efforts. In this 

presentation, individual and systemic barriers to the implementation of these protocols will be discussed, as well as a 

few examples of successful adoption that may provide guidance to improve successful dissemination of empiricallysupported 

treatments across large and dispersed health care systems. 

Track(s): EBP Champions, Training 

NOTES 


May 16. 12:30‐1:45 

BREAKOUT C: IMPLEMENTATION 

THROUGH COLLABORATIONS WITH 

POLICYMAKERS 

(MC: Aaron R. Lyon, PhD) 

Effective Implementation of EBP 

Legislation by Engaging Providers in a 

Coaching Process 

Eric Trupin, PhD, & Gabrielle D’Angelo, 

MSW 

University of Washington Division of 

Public Behavioral Health & Justice Policy 

Negotiating Implementation Science & 

Evaluation Research: Lessons Learned 

from a National Teen Pregnancy 

Prevention Implementation Study 

Jacqueline Berman, PhD 

Mathematica Policy Research 


Identifying the Needs of IEF/OIF 

Veterans with TBI & Co‐Occurring 

Behavioral Health Issues 

Lisa Brenner, PhD, Jennifer Olson‐ 

Madden, PhD, Bridget Matarazzo, PsyD, 

& Gina Signoracci, PhD 

Veterans Integrated Service Network 

(VISN) 19 Mental Illness Research, 

Education, & Clinical Center (MIRECC) 


EFFECTIVE IMPLEMENTATION OF EBP LEGISLATION BY ENGAGING PROVIDERS IN A COACHING PROCESS 

Eric Trupin, PhD, & Gabrielle D'Angelo, MSW 

Division of Public Behavioral Health & Justice Policy, University of Washington 

Contact: dgabriel@uw.edu 

Legislation to re‐define evidence‐based practice (EBP) and inventory all EBPs currently in practice in Washington was 

passed in Spring 2012 (HB 2536). Responding to HB 2536, The University of Washington Evidence‐Based Practice 

Institute and Washington State Institute for Public Policy (WSIPP) engaged with community providers to overcome 

barriers to these policy changes. First, UW and WSIPP engaged in education of providers about the law, and 

dialogued about implications for practice as usual on all levels of practice. Next, UW engaged in a coaching process 

with selected providers to help them move their existing interventions into the promising or research‐based 

categories, or prepare their organizations for implementation of current EBPs. Case studies of organizational 

evolution toward evidence‐based practice reflect the ongoing challenges on the national, state, and individual 

stakeholder levels. The strategies used by UW and WSIPP for engaging community providers provide extractable 

models for overcoming EBP implementation barriers across service systems. The case studies and strategies 

presented could suggest solutions to many of the “Implementation Research Dilemmas”. Common stakeholder 

needs for coaching include: maintaining fidelity, accessibility of training especially considering turnover rates and 

referral streams, and analysis of existing qualitative data to demonstrate effectiveness of existing interventions. 


NOTES 


NEGOTIATING IMPLEMENTATION SCIENCE & EVALUATION RESEARCH: LESSONS LEARNED FROM A 

NATIONAL TEEN PREGNANCY PREVENTION IMPLEMENTATION STUDY 

Jacqueline Berman, PhD, 1 & Ellen Kisker, PhD 2 

1 Mathematica Policy Research; 2 Twin Peaks Partners, LLC 

Contact: jberman@mathematica‐mpr.com 

Implementation research seeks to guide the adoption, initial implementation, and continuous improvement of 

evidence‐based programs over time. Implementation science, which focuses on how to translate, replicate, 

adapt, and scale up evidence‐based programs or practices in “real world” settings, can serve as a key support to 

implementation research. This paper explores emergent lessons about the application of tools from 

implementation science to the design and execution of systematic, high‐quality implementation evaluations. 

Because implementation science evolved primarily in clinical settings, however, its practices require some 

negotiation and translation when applied to the evaluation of public policies and programs. This paper examines 

lessons learned with regard to how to (1) use concepts and tools from implementation science to identify key 

drivers and elements of implementation necessary for replication and scalability of effective programs; (2) select 

valid quantitative and qualitative measures of these elements; and (3) determine multiple, appropriate data 

sources for these measures. These lessons are drawn from the development of a conceptual framework and 

measures of core program elements for an in‐depth implementation study, embedded in a large‐scale impact 

evaluation of evidence‐based teen pregnancy prevention programs funded by the Administration for Children 

and Families, DHHS. 


Track(s): Measurement 

NOTES 


IDENTIFYING THE NEEDS OF OEF/OIF VETERANS WITH TRAUMATIC BRAIN INJURY (TBI) & CO‐OCCURRING 

BEHAVIORAL HEALTH ISSUES AND THEIR FAMILIES 

Lisa Brenner, PhD, Jennifer Olson‐Madden, PhD, Bridget Matarazzo, PsyD, & Gina Signoracci, PhD 

Veterans Integrated Service Network (VISN) 19 Mental Illness Research, Education, & Clinical Center (MIRECC) 

Contact: lisa.brenner@va.gov 

Background: Estimates of TBI among Soldiers deployed to Iraq and Afghanistan vary, and the long‐term implications 

of such injuries are not understood. In light of concerns regarding the behavioral health needs of Veterans with TBI, 

the U.S. Department of Health and Human Services Administration awarded the State of Colorado, in collaboration 

with the Veterans Integrated Service Network (VISN) 19 Mental Illness Research Education and Clinical Center 

(MIRECC) a grant to improve the community mental health system’s ability to provide services to individuals with TBI. 

Methods: The MIRECC team has conducted focus groups with stakeholders throughout Colorado. In addition, a 

consensus conference with national experts specializing in the assessment and treatment of TBI and co‐occurring 

behavioral health issues was conducted. 

Results: Data obtained from focus groups will be shared along with findings from the consensus conference. 

Conclusions: Utilizing data obtained, researchers will work with Statewide Steering Committee (SSC) members to 

develop assessment and treatment guidelines. A training module and toolkit based on these guidelines will also be 

created. Additionally, the VISN 19 MIRECC and the SSC will develop a statewide Brain Injury Resource Team 

comprised of community mental health providers to act as internal facilitators for the above described best practices. 


NOTES 


May 16. 12:30‐1:45 

BREAKOUT D: ADVANCING 

FIDELITY MEASUREMENT 

(MC: Jennifer Villatte, PhD(c)) 

Fidelity Measurements in the Real 

World: Feasibility of BECCI & MITI for 

Motivational Interviewing in Child & 

Youth Mental Health 

Melissa Kimber, MSW, PhD(c), Raluca 

Barac, MA, PhD, & Melanie Barwick, 

PhD, CPsych 

Hospital for Sick Children, Toronto, 

Canada 

Comparisons Among Six Methods for 

Measuring Fidelity: Implications for 

Research & Practice 

Kristin Duppong Hurley, PhD 

University of Nebraska‐Lincoln 


An Update on Project BEST (Bringing 

Evidence‐Supported Treatments to 

South Carolina Children & Families): 

Challenges to Measuring Provider 

Fidelity 

Rochelle F. Hanson, PhD 

National Crime Victims Research & 

Treatment Center, Medical University of 

South Carolina 


FIDELITY MEASUREMENT IN THE REAL WORLD: FEASIBILITY OF BECCI & MITI FOR MOTIVATIONAL 

INTERVIEWING IN CHILD & YOUTH MENTAL HEALTH 

Melissa Kimber, MSW, PhD(c), Raluca Barac, MA, PhD, & Melanie Barwick, PhD, CPsych 

Hospital for Sick Children, Toronto, Canada 

Contact: melanie.barwick@sickkids.ca 

Treatment fidelity is an essential element of good scientific research and clinical practice (Waltz et al., 1993). 

Treatment fidelity helps researchers and practitioners draw solid conclusions about treatment effects, inform on 

implementation of evidence‐based practices (EBPs) so that cross‐site comparisons can be made, and monitor 

therapist training and need for additional training and supervision. Despite its undisputed importance, treatment 

fidelity is often neglected in practice. Schulte et al. (2009) noted that fidelity monitoring is least likely to occur 

precisely when there is high risk of compromising fidelity, namely during EBP implementation. Fidelity neglect stems 

from lack of theoretical knowledge and implementation guidelines and the increased costs associated with its use 

(Perepletchikova et al., 2009). METHOD: We addressed the issue of fidelity in the context of implementing 

Motivational Interviewing (MI) in four child and youth mental health organizations in Ontario, Canada. Across the 

four organizations, 20 clinicians audiotaped monthly therapy sessions with their clients to capture practice at several 

time points: 3 months leading to MI training, while receiving coaching support over 9 months, and 3 months 

following support. All sessions were coded using the Behavioral Change Counseling Index (BECCI; Lane et al., 2005) 

and 50% of the sessions were coded by an expert coder using the Motivational Interviewing Treatment Integrity scale 

(MITI; Moyers et al., 2005). Both instruments measure therapists’ competence in MI and have acceptable 

psychometric properties (Wallace et al., 2009). However, the two instruments differ in practical and economical 

aspects: BECCI is a brief tool, to be scored in one pass and requiring minimal amounts of training, whereas MITI is 

more elaborate, shown to be reliable when used by expert raters, and requires relatively longer training. The brevity 

of the BECCI makes it a very appealing instrument for practice settings, with great potential for use in mental health 

settings to ensure MI fidelity. This is important because fidelity assessment will only become a reality if it is simple 

and practicable. To date, the BECCI has only been tested with data from simulated therapy sessions and how it 

stands up when tested in real world clinical practice or how it relates to other MI fidelity instruments is less known. 

Thus, the present study examined (a) the concordance between the MITI and BECCI and (b) the extent to which the 

two fidelity measures detect change in therapists’ competence following training and coaching in MI. These findings 

have significant implications for implementing fidelity checks as standard practice in mental health. 

Track(s): Fidelity, Global Perspectives, Measurement 

NOTES 


COMPARISONS AMONG SIX METHODS FOR MEASURING FIDELITY: IMPLICATIONS FOR RESEARCH & 

PRACTICE 

Kristin Duppong Hurley, PhD, 1 & Mark Van Ryzin, PhD 2 

1 University of Nebraska‐Lincoln; 2 Oregon Social Learning Center 

Contact: kristin.hurley@unl.edu 

Developing valid, reliable, and cost‐effective fidelity assessment tools that can be used in practice‐settings is a 

challenge for many evidence‐based interventions. One of goals of this pilot study was to compare the 

psychometrics of six methods for measuring the fidelity of an adaptation of the Teaching Family Model. These 

six methods of assessing fidelity included external observations, internal‐agency observations, supervisor 

ratings, staff self‐ratings, youth ratings, and archival data. The study included 145 youth with disruptive 

behavior disorders, 120 direct‐care staff, and 16 supervisors. Fidelity process data and youth mental health 

outcome data were collected longitudinally. We will briefly discuss the findings of this pilot study comparing the 

fidelity measurement approaches. Individually, each fidelity approach had acceptable psychometric properties 

and their ratings were correlated over time. However, the different assessment approaches were not strongly 

correlated among each other. Interestingly, the supervisor ratings had some issues with bias in ratings. By 

comparing the multiple assessments, certain supervisors were identified as likely being stricter in their ratings, 

causing an unexpected inverse correlation with youth outcomes. Only youth ratings of fidelity and some of the 

archival data were predictive of positive youth mental health outcomes. Implications and directions for future 

research will be discussed. 


Track(s): Fidelity, Measurement 

NOTES 


AN UPDATE ON PROJECT BEST (BRINGING EVIDENCE‐SUPPORTED TREATMENTS TO SOUTH CAROLINA 

CHILDREN & FAMILIES): CHALLENGES TO MEASURING PROVIDER FIDELITY 

Rochelle F. Hanson, PhD, 1 Benjamin E. Saunders, PhD, 1 Libby Ralston, PhD, 2 Michael de Arellano, PhD, 1 & Angela 

Moreland, PhD 1 

1 National Crime Victims Research & Treatment Center, Medical University of South Carolina; 2 Dee Norton Low Country 

Children's Center 

Contact: hansonrf@musc.edu 

This presentation provides an update on Project BEST (Bringing Evidence‐Supported Treatments to South Carolina 

children and families; funded by Duke Endowment), an ongoing statewide initiative designed to support the 

dissemination and implementation of Trauma‐focused Cognitive Behavioral Therapy (TF‐CBT). Project BEST utilizes 

the Community‐Based Learning Collaborative (CBLC) dissemination/implementation model to build community 

capacity to deliver and sustain trauma‐informed services to abused children and their families. Since its onset, we 

have completed three CBLCs, and two are nearing completion. These have involved 477 clinicians, brokers, and 

senior leaders from 105 different agencies serving 38 of South Carolina’s 46 counties. One of our key goals is to train 

mental health providers in the delivery of TF‐CBT, and an ongoing challenge is to determine the most feasible, cost 

effective ways to measure therapist fidelity to the model. After providing an update on Project BEST activities to 

date (e.g., total number of participants who completed all training requirements; pre/post treatment outcome data 

for TF‐CBT training cases), the focus will be to discuss our measure of therapist fidelity to TF‐CBT, including factors 

associated with TF‐CBT fidelity; and preliminary findings on the relations among therapist fidelity and child/family 

treatment outcomes. The presentation will conclude with a discussion of challenges encountered in this statewide 

initiative, lessons learned, and future plans. 

Track(s): Fidelity, Measurement, Scale‐up 

NOTES 


May 16. 12:30‐1:45 

BREAKOUT E: IMPLEMENTING 

PRIMARY CARE INTERVENTIONS 

(MC: Maria Monroe‐DeVita, PhD) 

Transformation & Spread of Primary 

Care Clinics into Medical Homes: It’s 

Slow, Hard Work 

Leif I. Solberg, MD 

HealthPartners Institute for Education & 

Research 

A Qualitative Study of Fidelity: 

Understanding Variations in 

Implementation of the Patient‐ 

Centered Medical Home 


Mathematica Policy Research 


CADET: Clinical & Cost Effectiveness of 

Collaborative Care for Depression in UK 

Primary Care: A Cluster Randomized 

Controlled Trial 


University of Exeter, United Kingdom 


TRANSFORMATION & SPREAD OF PRIMARY CARE CLINICS INTO MEDICAL HOMES: IT’S SLOW, HARD WORK 

Leif I. Solberg, MD, Juliana Tillema, MPA, A. Lauren Crain, PhD, Robin Whitebird, PhD, & Patricia Fontaine, MD 

HealthPartners Institute for Education & Research 

Contact: leif.i.solberg@healthpartners.com 

Although many advocates of patient‐centered medical homes talk as though transformation occurs quickly and fairly 

abruptly, there is very little evidence to support that. We have studied the first 135 primary care clinics in Minnesota 

to be certified as health care homes (our local term for medical homes) through a fairly rigorous review and 

certification process. Our data shows that while they have implemented fairly large changes in the practice systems 

needed to function as medical homes over a three year period, there has been relatively little change in standardized 

performance rates for diabetes and cardiovascular disease care. Moreover, while average performance rates for 

certified health care homes are higher than for uncertified primary care clinics, there is wide variation in both groups 

and considerable overlap. Based on intensive interviews with leaders in nine certified clinics and a survey of them all, 

we have been able to identify the strategies and characteristics that most clearly differentiate the certified clinics 

with the greatest transformation from those with the least. This information has been translated into 

recommendations for what clinics should pay most attention to if they wish to truly transform and achieve triple aim 

results for their patients. 


NOTES 


A QUALITATIVE STUDY OF FIDELITY: UNDERSTANDING VARIATION IN IMPLEMENTATION OF THE PATIENT‐ 

CENTERED MEDICAL HOME 


Mathematica Policy Research, Inc. 

Contact: rkeith@mathematica‐mpr.com 

Evaluating fidelity is particularly salient for complex, multi‐faceted interventions, such as the patient‐centered 

medical home (PCMH) model of care delivery, where poor implementation of different model components can 

compromise the effectiveness of the intervention as a whole in improving patient outcomes. This presentation 

will focus on the qualitative assessment of fidelity to the PCMH model. A comparative case analysis was 

conducted to examine fidelity to the PCMH in six primary care clinics affiliated with a large, academic, integrated 

health system. The clinics in the sample had similar organizational structures and resources (e.g., health 

information systems, incentives, quality initiatives centralized within the system, PCMH tools and processes, 

collaborative learning opportunities focused on PCMH implementation). Interview and observational data were 

analyzed deductively to assess variation in fidelity across the clinics. Categorical measures were constructed to 

reflect relative ratings of clinic level fidelity for each PCMH component. The findings show that despite having 

similar organizational structures, considerable variation in fidelity to the various PCMH components existed 

across the six clinics. 



NOTES 


CADET: CLINICAL & COST EFFECTIVENESS OF COLLABORATIVE CARE FOR DEPRESSION IN UK PRIMARY CARE: 

A CLUSTER RANDOMIZED CONTROLLED TRIAL 


University of Exeter, UK 

Contact: d.a.richards@exeter.ac.uk 

Background: Collaborative care for depression is effective in the US but effects are uncertain internationally. 

Design: Multi‐centre, cluster randomised controlled trial with two parallel group arms. 

Results: 581 participants recruited from 49 primary care practices; 276 in collaborative care, 305 usual care. 

Participants in collaborative care had a mean reduction in depression score 1.33 PHQ‐9 points greater (95% CI 0.35 to 

2.31, p=0.009) compared to usual care (effect size = 0.26, 95% CI 0.07 to 0.46). Odds ratios for PHQ‐9 scores below 

depression threshold (PHQ

May 16. 2:00‐3:15 

BREAKOUT F: 

EBP CHAMPION SYMPOSIUM 

(MC: Cara C. Lewis, PhD, & Cameo 

Borntrager, PhD) 

Solving Research Dilemmas Related to 

Implementation Fidelity 

Rebecca Selove, PhD, MPH, Kathryn 

Mathes, BSN, RN, MS, PhD, & Heather 

Wallace, PhD 

Centerstone Research Institute 



SOLVING RESEARCH DILEMMAS RELATED TO IMPLEMENTATION FIDELITY 

Rebecca Selove, PhD, MPH, Kathryn Mathes, BSN, RN, MS, PhD, & Heather Wallace, PhD 

Centerstone Research Institute 

Contact: rebecca.selove@centerstone.org 

The Interactive Systems Framework (ISF) offers a broad context within which a variety of implementation projects 

can be described and improved. Centerstone Research Institute (CRI) is uniquely positioned to evaluate 

implementation processes and outcomes for a diverse range of community‐based health improvement projects. 

Three CRI‐evaluated programs were selected for systematic retrospective review using the ISF to identify barriers to 

and facilitators of implementation with fidelity. The services provided by these programs include (a) community 

development of a system of care for transition‐age youth, (b) intensive in‐home intervention with parents involved in 

the judicial system in connection with substance abuse histories, and (c) a school‐based educational program to 

reduce risk of teen pregnancy. Data came from staff observations, evaluation records, and interviews with program 

managers and service providers. The symposium will provide an overview of potentially critical elements of fidelity 

related to each of the three systems of the ISF, and description of the investigation methodology. Presenters will 

highlight challenges to researchers in the Translation and Synthesis System and their implications for implementation 

planning. The three panelists will discuss cross‐cutting approaches to research dilemmas related to planning for 

implementation fidelity in a variety of programs and services. 

Track(s): EBP Champions, Fidelity 

NOTES 


May 16. 2:00‐3:15 

BREAKOUT G: SUSTAINABILITY & 

ADAPTATION IN SOCIAL SERVICES 


Implementation Strategies in Social 

Service Settings: A Research Agenda 

Byron J. Powell, AM 

Washington University in St. Louis 

DBT Teams in Training 2008‐2011: 

Implementation Follow‐up in 2012 

Anthony DuBose, PsyD, 1 & André 

Ivanoff, PhD 1,2 

1 Behavioral Tech, LLC; 2 Columbia 

University 


Understanding Modifications to CBT in 

Community Settings: A Comparison of 

Providers in Adults & Child Mental 

Health Service Settings 


VA Boston Healthcare System, National 

Center for PTSD, & Boston University 


IMPLEMENTATION STRATEGIES IN SOCIAL SERVICE SETTINGS: A RESEARCH AGENDA 

Enola K. Proctor, PhD, 1 Byron J. Powell, AM, 1 & J. Curtis McMillen, PhD 2 

1 Washington University in St. Louis; 2 The University of Chicago 

Contact: bjpowell@wustl.edu 

The prioritization of implementation research has yielded conceptual, empirical, and methodological advances that 

contribute to our understanding of the structures, processes, and outcomes of implementation. This is perhaps an 

ideal time to generate a rich set of research questions pertaining to the use of implementation strategies. This paper 

draws upon both the published literature and ongoing implementation research to demonstrate the challenges and 

opportunities associated with a number of these key questions, including: 

• How can community stakeholders inform research on implementation strategies? 

• Who are the appropriate stakeholders to be deploying implementation strategies? 

• To what extent are implementation strategies generalizable? 

• Are tailored implementation strategies more effective than other approaches? 

• Can the methods for selecting and designing implementation strategies be strengthened? 

• Can implementation strategies be adequately specified? 

• How can we disentangle the mutative factors of multifaceted implementation strategies? 

• How can (inexpensive) technologies be harnessed in developing innovative implementation strategies? 

• What is the economic impact of implementation strategies? 

• Can we develop “learning organizations” and evidence‐based systems of care? 

In exploring these questions, we identify associated methodological challenges and innovative approaches to 

addressing them, and suggest promising directions for future studies aimed at increasing our knowledge of how to 

implement effective evidence‐based treatments. 


NOTES 


DBT TEAMS IN TRAINING 2008‐2011: IMPLEMENTATION FOLLOW‐UP IN 2012 

Anthony DuBose, PsyD, 1 André Ivanoff, PhD, 1,2 Erin Miga, PhD, 1,3 Linda Dimeff, PhD, 4 & Marsha Linehan, PhD 3 

1 Behavioral Tech, LLC; 2 Columbia University; 3 Behavioral Research & Therapy Clinics, University of Washington; 

4 Portland DBT Institute 

Contact: apdubose@behavioraltech.org 

To address the challenges of implementing evidence‐based therapies in large systems, Behavioral Tech, LLC 

(BTECH) has begun a significant evaluation of team and system‐based DBT implementations toward developing 

improved methods and outcomes. As part of this, a pilot study consisted of a random sample of 50% (n=77) of 

teams was drawn from those completing BTECH intensive training from 2008‐2011 (n=154). Mixed methods 

data collection was used; an online survey including the DBT Program Elements of Treatment Questionnaire 

(PETQ: Schmidt, Ivanoff & Linehan, 2009) was followed by a semi‐structured telephone interview to discuss 

team leader questions, programs goals, and next steps. Data reveals that most programs offer individual 

treatment, skills training, weekly DBT consultation team, and after‐hours coaching. Approximately half of 

programs provide regular DBT supervision. Thirty‐nine percent of programs conduct manual‐based, selfassessment 

of DBT adherence, although fewer (18%) provide adherence data to staff/or clients for purposes of 

quality improvement. Significant implementation barriers included time constraints and staff turnover. Team 

leads identified “careful selection of intensive team members” and “sending more team members to intensive 

training” as ways to improve program functioning. In sum, while the majority of programs deliver all DBT 

modes, onsite DBT supervision and fidelity/adherence assessment are insufficient. 


Track(s): Training, Sustainability 

NOTES 


UNDERSTANDING MODIFICATIONS TO CBT IN COMMUNITY SETTINGS: A COMPARISON OF PROVIDERS IN 

ADULT & CHILD MENTAL HEALTH SERVICE SETTINGS 

Shannon Wiltsey Stirman, PhD, 1,2 Rinad Beidas, PhD, 3 Christopher Miller, PhD, 4 Julie Edmunds, MA, 5 Mary Margaret 

Downey, BA, 3 Matthew Gallagher, BA, 3 Philip Kendall, PhD, ABPP, 5 Katherine Toder, 3 & Amber Calloway 6 

1 National Center for PTSD; 2 VA Boston Healthcare System, & Boston University; 3 University of Pennsylvania; 4 VA 

Center for Leadership, Organization, & Management Research; 5 Temple University; 6 University of Massachusetts at 

Boston 

Contact: sws@bu.edu 

Little is known about modifications to CBT that providers make following training and consultation. To optimize 

clinician implementation and sustained use of CBT, it is necessary to investigate provider perspectives of 

modifications that are necessary to make CBT feasible and sustainable in a community setting. However, a challenge 

to understanding the nature and implications of modifications to EBPs is a lack of consistency or comprehensiveness 

in the classification of adaptations. This study allows us to do so with both adult and child providers in the 

Philadelphia community, using a framework that was developed to characterize adaptations made to evidence‐based 

interventions. The present study examines follow‐up interviews conducted 2 years following the CBT training and 

consultation provided in Beidas et al. (2012) and Stirman et al. (2010, 2012) with adult (n=30) and child (n=50) mental 

health providers. The same coding system was used for both samples to examine modifications made to CBT to 

facilitate comparisons between groups. The findings from this study will shed much needed insight on whether and 

how providers modify evidence‐based treatments to make them more usable in their settings. Implications for CBT 

implementation and for further research will be discussed. 

Track(s): Sustainability 

NOTES 


May 16. 2:00‐3:15 

BREAKOUT H: IMPLEMENTATION 

OF CRITICAL TIME INTERVENTION 

(MC: Meghan Keough, PhD) 

From Inception to Practice: Taking an 

Evidence‐Based Practice from 

Development to Implementation 

Challenges & Successes in Assessing 

Fidelity to the CTI Model Over Time 

Assessing the Implementation of the 

Critical Time Intervention Model Across 

20 Homeless‐Service Agencies 


R. Neil Greene, PhD, & Melissa Martin, 

MSW 

Center for Social Innovation, Needham, 

MA 


FROM INCEPTION TO PRACTICE: TAKING AN EVIDENCE‐BASED PRACTICE FROM DEVELOPMENT TO 


Daniel Herman, PhD, 1 Sarah Conover, MS, 1 Jeff Olivet, MA, 2 & Melissa Martin, MSW 2 

1 Critical Time Intervention Global Network, Silberman School of Social Work at Hunter College, New York, NY; 2 Center 

for Social Innovation, Needham, MA 

Contact: mmartin@center4si.com 

Critical Time Intervention (CTI) is a time‐limited case management model designed to prevent homelessness and 

other adverse outcomes in people with severe mental illness (SMI) during periods of transition in their lives, such as 

following discharge from hospitals, shelters, prisons and other institutions. During transitions, people often have 

difficulty re‐establishing themselves in satisfactory living arrangements with access to needed supports. CTI provides 

focused, time‐limited case management assistance during this critical period and can have enduring positive impacts. 

In this session, presenters will provide an overview of the CTI model and how it was developed to fit an urgent need, 

as well as the subsequent experimental and quasi‐experimental research on the model in a variety of settings. 

Presenters will describe effective approaches used to provide consultation and training of individuals and agencies 

for adaptation and implementation of the CTI model, and current collaborations that are enabling the model to be 

brought to scale to broader national and international audiences. 

Track(s): Training 

NOTES 


CHALLENGES & SUCCESSES IN ASSESSING FIDELITY TO THE CTI MODEL OVER TIME 

Sarah Conover, MS, 1 Suzanne Zerger, PhD, 2 & R. Neil Greene, PhD 2 

1 Critical Time Intervention Global Network, Office of Scholarship & Research, Silberman School of Social Work at 

Hunter College, New York, NY; 2 Center for Social Innovation, Needham, MA 


In this session, presenters will describe the development of the original “fidelity scale” for the CTI model and 

share considerations of how to assess fidelity within existing case management paperwork practices and 

protocols. The fidelity scale was developed by Sarah Conover at the Silberman School of Social Work at Hunter 

College. The scale has been tested and used to assist numerous agencies to implement the model in a wide 

range of settings and increasingly diverse populations. In our recently completed Phase II SBIR study, Sarah 

Conover helped the research team modify the scale and accompanying case management tracking forms. These 

tools were adapted to “fit” the needs and practices of these 20 homeless‐service agencies enrolled in the study, 

and to enable tracking of both provider and client‐level outcomes. We describe challenges and successes in 

helping these agencies adapt the fidelity measures to their existing practices, and in collecting, analyzing, 

interpreting, and sharing results from the fidelity scale. With funding from this study, the research team was 

able to adapt the pen‐and‐paper fidelity assessment process into an electronic format; we describe our 

development of a user‐friendly electronic fidelity assessment product that agencies can employ when they 

choose to learn and implement the CTI model. 

Track(s): Fidelity, Measurement, Technology 


Notes 


ASSESSING THE IMPLEMENTATION OF THE CRITICAL TIME INTERVENTION MODEL ACROSS 20 HOMELESS‐ 

SERVICE AGENCIES 

Suzanne Zerger, PhD, R. Neil Greene, PhD, Melissa Martin, MSW, & Rachael R. Kenney, MA 

Center for Social Innovation, Needham, MA 


With funding from a Phase II Small Business Innovation Research (SBIR) grant from the National Institute of Mental 

Health, the Center for Social Innovation developed an online multi‐media training on CTI which incorporates a 

Community of Practice approach to encourage peer‐based learning. The primary aim of this longitudinal, 

randomized‐control study was to compare and contrast this online training modality with a face‐to‐face training on 

implementation of and fidelity to the CTI model over time. In this presentation, we describe research methodologies 

and lessons learned in our exploration and documentation of the implementation (and adaptation) experiences of 20 

diverse homeless‐service agencies across the U.S. and nearly two‐hundred direct service providers engaged in 

implementing the model over the course of one‐year. We describe challenges and promising approaches used to 

assess the “readiness” of the agencies to implement the model prior to training their staff, and with documenting 

experiences of implementation facilitators and barriers from the perspectives of agency administrators and direct 

service providers through surveys, interviews, fidelity tracking forms, and in‐depth case studies. We also talk about 

strategies and challenges associated with assessing impacts on clients enrolled in the model. 


NOTES 


May 16. 2:00‐3:15 

BREAKOUT I: MATCHING 

IMPLEMENTATION TO SETTING 

(MC: Sara J. Landes, PhD) 

Matching Training to Setting: A New 

Implementation Model for Dialectical 

Behavior Therapy 

Helen Best, MEd, 1 Kate Comtois, PhD, 

MPH, 2 Nancy A. McDonald, MS, CADC, 

LPC, 3 & Jamie F. Edwards, LCSW, 

CMFSW 4 

1 Treatment Implementation 

Collaborative, LLC; 2 University of 

Washington; 3 Chester County 

Department of Human Services; 

4 Community Care Behavioral Health 


User‐Centered Design & the 

Implementation of Evidence‐Based 

Interventions 



Designing an Implementation Strategy 

to Support the Multi‐Site Scale‐Up of an 

Evidence‐Based, Culturally Appropriate 

Practice Model for Intensive Family 

Support Services Across the Northern 

Territory, Australia 

Robyn Mildon, PhD 

Knowledge Exchange & Implementation, 

Parenting Research Centre, Australia 


MATCHING TRAINING TO SETTING: A NEW IMPLEMENTATION MODEL FOR DIALECTICAL BEHAVIOR THERAPY 

Helen Best, MEd, 1 Katherine Anne Comtois, PhD, MPH, 2 Nancy A. McDonald, MS, CADC, LPC, 3 & Jamie F. Edwards, 

LCSW, CMFSW 4 

1 Treatment Implementation Collaborative, LLC; 2 University of Washington; 3 Chester County Department of Human 

Services; 4 Community Care Behavioral Health 

Contact: hbest@ticllc.org 

While Dialectical Behavior Therapy has been widely disseminated, most large scale system initiatives have focused on 

training DBT to adherence and how to integrate the EBP into standard system structures. The Treatment 

Implementation Collaborative, LLC, is testing a new model of implementation that is organized in terms of how 

systems implement a new treatment rather than how to train clinicians in the treatment. It is no accident that DBT 

skills training is the component most often misconstrued as comprehensive DBT, by clinicians and consumers, as it is 

the component of DBT that is most accessible to a broad audience. With this in mind, the implementation model 

being tested by TIC focuses first on laying a solid foundation on administrative orientation for leadership in 

conjunction with a solid base in DBT skills training for clinicians and programs. Once this core component is in place, 

clinicians and teams are trained to implement all modes of comprehensive DBT. This presentation will highlight TIC’s 

model for implementation using a case example and data from the implementation of DBT across 11 teams in three 

counties in PA, including Chester County. 

Track(s): EBP Champions, Training 

NOTES 


USER‐CENTERED DESIGN & THE IMPLEMENTATION OF EVIDENCE‐BASED INTERVENTIONS 



Contact: lyona@uw.edu 

A well‐documented “research‐to‐practice gap” exists in which evidence‐based interventions (EBI) are unlikely to 

be adopted by mental health practitioners working in community settings, limiting their public health impact. 

This presentation discusses how the design of EBI is detracting from their ability to be effectively implemented 

on a large scale. Although EBI frequently produce robust effects for well‐specified problems, their design is 

characterized by excessive complexity, inflexibility (e.g., fidelity requirements), and steep learning curves. In this 

way, EBI can be said to be very well engineered (functional and able to produce their intended outcome), but 

badly designed. 

This presentation will draw from the literature on user experience and user‐centered design to address EBI 

design as a key implementation issue. To date, the mental health research community has done relatively little 

to ensure that existing EBI are appealing and accessible to their target audience. A variety of design heuristics 

and principles of good design will be applied to the construction of EBI with the goal of better meeting the needs 

of the end user (i.e., mental health practitioners). These include building EBI that are more readily learnable, 

demonstrate functional minimalism, decrease a user’s cognitive load, and exploit the natural constraints of the 

context of use. Examples drawn from ongoing projects initiated to create contextually‐appropriate and usable 

supports for quality improvement in school‐based mental health will also be presented. 


Track(s): Technology 

NOTES 


DESIGNING AN IMPLEMENTATION STRATEGY TO SUPPORT THE MULTI‐SITE SCALE‐UP OF AN EVIDENCE‐BASED, 

CULTURALLY APPROPRIATE PRACTICE MODEL FOR INTENSIVE FAMILY SUPPORT SERVICES ACROSS THE 

NORTHERN TERRITORY, AUSTRALIA 

Robyn Mildon, PhD, 1 & Fiona Arney, PhD 2 

1 Director of Knowledge Exchange & Implementation, Parenting Research Centre; 2 Director, Australian Centre for Child 

Protection, University of South Australia 

Contact: rmildon@parentingrc.org.au 

In recent years, we have seen a growth in Australia of funding and delivery of “Intensive Family Support Services” for 

vulnerable families in an effort to improve health, safety and wellbeing of children and prevent family involvement in 

our child protection system, including out of home care. Despite this trend, few services adopt a coherent, evidence‐‐ 

based program model, effective and full implementation is rarely reached or sustained, and little evaluation is done 

in child protection and family support on any large scale. 

Family support service providers and policy makers seeking an evidence‐based Practice Model will find that there is 

little written for a community service setting about how to go about making the critical decision of choosing and 

refining a comprehensive Model with the potential for significantly impacting child and family outcomes. 

Furthermore, there are relatively few comprehensive guidelines on quality implementation of practice specific to 

intensive family support. A framework based on the Quality Implementation Framework (Meyers, Durlak & 

Wandersman, in press) and the work of the National Implementation Research Network (NIRN), is being applied to 

support the scale‐up of a purpose built, evidence‐based, culturally appropriate Practice Model in multiple health and 

child welfare service delivery sites across the Northern Territory, Australia. 

This paper will describe in detail the implementation framework being applied, implementation support strategies 

being utilized to date to achieve early practice change and the effect these have had, and the conceptual model 

being used to guide the evaluation of this work. 

Track(s): Global Perspectives, Scale‐Up 

NOTES 


May 16. 2:00‐3:15 

BREAKOUT J: RESEARCH‐ 

COMMUNITY RELATIONSHIPS 


Evaluating the Success of a Statewide 

EBP Scale‐Up Project: The Children’s 

Administration‐University of 

Washington EBP Partnership 

Eric Bruns, PhD, 1 Andrea Negrete, MEd, 1 

& Tammy Cordova, MSW 2 

1 University of Washington; 2 Washington 

State Children’s Administration 

Reviewing the Use of Research‐ 

Community Partnerships to Facilitate 


Practices in Children’s Community 

Services 

Nicole Stadnick, MS, MPH 

SDSU/UCSD Joint Doctoral Program in 

Clinical Psychology & Child & Adolescent 

Services Research Center 


Developing the Autism Model of 

Implementation for ASD Community 

Providers: Use of Research‐Community 

Partnership 

Amy Drahota, PhD 

San Diego State University & Child & 

Adolescent Services Research Center 


EVALUATION OF THE SUCCESS OF A STATEWIDE EBP SCALE‐UP PROJECT: THE CHILDREN’S ADMINISTRATION‐ 

UNIVERSITY OF WASHINGTON EBP PARTNERSHIP 

Eric Bruns, PhD, 1 Eric Trupin, PhD, 1 Suzanne Kerns, PhD, 1 Sarah Walker, PhD, 1 Andrea Negrete, MEd, 1 Rima Ellard, 

MSW, 1 Tim Kelly, 2 & Tammy Cordova, MSW 2 

1 University of Washington; 2 Washington State Children's Administration 

Contact: ebruns@uw.edu 

In 2012, the Washington State Children’s Administration (CA) and the University of Washington (UW) Division of 

Public Behavioral Health and Justice Policy (PBHJP) launched an initiative to expand availability and utilization, and 

improve fidelity and outcomes of evidence‐based practices (EBP) relevant to core child welfare outcomes of child 

safety, permanency, and well‐being. This presentation will describe how key theoretical models for implementation 

science (e.g., Fixsen et al., 2005; Proctor et al., 2004; Shortell et al., 2009) were used to (1) create a logic model for 

the project, (2) develop core implementation strategies, (3) identify key indicators of success, and (4) keep the 

project and its collaborators organized as the Partnership evolved over time. The presentation will go on to present 

data on our identified indicators of success, such as number of providers trained, number of agencies implementing 

EBPs, rate of EBP referrals, rate of providers meeting criteria for fidelity, rate of billed units of service, and 

associations between EBPs and child welfare outcomes. We will conclude with a discussion of how these results point 

to the successes and challenges faced by this innovative state‐academic partnership. 


NOTES 


REVIEWING THE USE OF RESEARCH‐COMMUNITY PARTNERSHIPS TO FACILITATE IMPLEMENTATION OF 

EVIDENCE‐BASED PRACTICES IN CHILDREN’S COMMUNITY SERVICES 

Nicole Stadnick, MS, MPH, 1,2 Lauren Brookman‐Frazee, PhD, 2,3 Aubyn Stahmer, PhD, 2 Amy Herschell, PhD, 4 & 

Ann Garland, PhD 2,5 

1 SDSU/UCSD Joint Doctoral Program in Clinical Psychology; 2 Child & Adolescent Services Research Center; 

3 Department of Psychiatry, University of California, San Diego; 4 University of Pittsburgh School of Medicine; 

5 University of San Diego 

Contact: nstadnic@ucsd.edu 

There is growing conceptualization of research‐community partnerships (RCPs) in implementation models as 

critical in facilitating the uptake and sustainability of evidence‐based practices (EBPs) in community‐based 

services. There are a growing number of RCPs in the field of mental health services, particularly within pediatric 

service settings. The purpose of this study is to examine RCPs that have been used to adapt EBP interventions, 

training, and broader implementation models to address mental health and behavioral issues for children served 

in community‐based service systems. Through a comprehensive literature and grants search, independent 

review and consensus coding, 38 studies using RCPs for these purposes were identified. A web‐based survey 

completed by project principal investigators and community partners will be used to characterize the use of 

RCPs by examining: (1) characteristics of research studies using RCP models; (2) RCP functioning, processes, and 

products; (4) processes of tailoring EBPs for implementation in the community; and (3) investigator perceptions 

of the benefits and challenges of collaborating with community providers and consumers. Respondents 

included 28 PIs and community partners reporting on 18 studies involving RCPs. Survey data were analyzed 

using mixed quantitative and qualitative methods. Common themes may inform future collaborative projects 

and the development of RCP theory. 



NOTES 


DEVELOPING THE AUTISM MODEL OF IMPLEMENTATION FOR ASD COMMUNITY PROVIDERS: USE OF A 

RESEARCH‐COMMUNITY PARTNERSHIP 

Amy Drahota, PhD, 1,2 Gregory A. Aarons, PhD, 2,3 & Aubyn C. Stahmer, PhD 2,3 

1 San Diego State University; 2 Child & Adolescent Services Research Center; 3 University of California, San Diego 

Contact: adrahota@projects.sdsu.edu 

ASD community providers (ASD‐CPs) provide services to children with any severity of ASD symptoms using a 

combination of treatment paradigms, some with an evidence‐base and some without. When evidence‐based 

practices (EBPs) are successfully implemented by ASD‐CPs, they can result in positive outcomes. Despite this, EBPs 

are often implemented unsuccessfully and other treatments used by ASD‐CPs lack supportive evidence, especially for 

school‐age children with ASD. While it is not well understood why ASD‐CPs are not implementing EBPs, 

organizational and individual characteristics likely play a role. An academic‐community collaboration (ACC) 

partnering ASD‐CPs and researchers will develop the autism model of implementation (AMI), a systematic process 

specifically for use by ASD community‐based agencies to facilitate implementation of EBPs. Using social networking 

and focused recruitment strategies, 13 members have joined the ACC, and begun building the collaborative. Using 

mixed methods, the purpose of this study is to evaluate the (1) development, (2) collaborative process, (3) function, 

and (4) tangible products (i.e., the AMI and related materials) of the ACC. Qualitative and quantitative data will be 

integrated and analyzed using triangulation, expansion, and complementarity. This study is designed to address the 

real‐world implications of EBP implementation in ASD community‐based agencies. 


NOTES 


May 16. 3:30‐4:45 

LEVERAGING TECHNOLOGY 

(MC: Rinad Beidas, PhD) 

Scaling Up Assessment of Therapist 

Fidelity in Motivational Interviewing: 

Preliminary Development of the 

AutoMITI 

David C. Atkins, PhD 


PracticeGround: An Online Platform to 

Help Therapists Learn, Implement, & 

Measure Impact of EBPs 

Gareth Holman, PhD 

Evidence‐Based Practice Institute, 



Dialectical Behavior Therapy 

Implementation Process & Outcomes in 


Sara J. Landes, PhD, 1 & Matthew Ditty, 

MSW 2 

1 National Center for PTSD, VA Palo Alto 

Health Care System; 2 University of 

Pennsylvania School of Social Policy & 

Practice 


SCALING UP ASSESSMENT OF THERAPIST FIDELITY IN MOTIVATIONAL INTERVIEWING: PRELIMINARY 

DEVELOPMENT OF THE AUTOMITI 

David C. Atkins, PhD, 1 Zac E. Imel, PhD, 2 Doğan Can, MSc, 3 Bo Xiao, 3 Panayiotis Georgiou, MEng, 3 & Shrikanth 

Narayanan, PhD 3 

1 University of Washington; 2 University of Utah; 3 University of Southern California 

Contact: datkins@uw.edu 

Implementation and dissemination are by nature large‐scale endeavors: How do we take evidence‐based practices 

and move them to general clinical use? As such, common tools in the clinical research setting do not easily translate 

to general use. One example is assessing therapist fidelity, in which the typical technology for assessing fidelity is to 

use behavioral coding systems and human raters. This “low tech” route to assessing fidelity does not scale up to 

larger applications and is a non‐starter for wide‐spread use. The current authors are part of a larger, interdisciplinary 

team developing automated methods for assessing therapist fidelity in Motivational Interviewing (MI). The current 

talk will provide an overview and current status of this work, discussing initial examination of automated detection of 

therapist reflections and empathy. Thus far, detecting reflections using linguistic tools has been quite successful, 

whereas assessing therapist empathy has proved more challenging. We will review both of these tasks and some 

reasons why there is differential effectiveness across these two domains of therapist fidelity to MI. In addition, we 

will briefly comment on the underlying methodology for this work, arising out of the new field of “behavioral 

informatics” with tools from engineering, computer science, and related disciplines. 

Track(s): Fidelity, Measurement, Scale‐Up, Technology 

NOTES 


PRACTICEGROUND: AN ONLINE PLATFORM TO HELP THERAPISTS LEARN, IMPLEMENT, & MEASURE 

IMPACT OF EBPS 

Kelly Koerner, PhD, & Gareth Holman, PhD 

Evidence‐Based Practice Institute, University of Washington 

Contact: kellykpracticeground@gmail.com 

Effective post‐graduate training and consultation are essential to successfully disseminate and implement 

evidence‐based practices (EBPs). However, commonly used continuing education methods produce little change 

in practitioner behavior. Instead intensive training models, those combining training and ongoing practice with 

supervision, appear most effective (Rakovshik & McManus, 2010). Yet such intensive models are expensive and 

difficult to take to scale. Even in the best case, in‐person expert‐led training and consultation can only reach a 

limited number of practitioners. 

PracticeGround is a scalable online alternative to traditional continuing education methods. PracticeGround is a 

training and performance support platform through which practitioners, trainers, and researchers work together 

to achieve the best possible therapy outcomes for clients. PracticeGround integrates learning, implementation 

support and measurement into practitioners’ routine workflow. In this talk I will layout our long‐range strategy 

to develop and test training and implementation methods using PracticeGround. I will report findings from our 

first three studies (training to do behavioral activation (Puspitasari et al (in press), enhance therapeutic 

relationship skills (Kanter et al, in press); and implement progress monitoring (Persons et al, 2012)). 


Track(s): Technology, Training 

NOTES 


DIALECTICAL BEHAVIOR THERAPY IMPLEMENTATION PROCESS & OUTCOMES IN VA & COMMUNITY SETTINGS 

Sara J. Landes, PhD, 1 & Matthew Ditty, MSW 2 

1 National Center for PTSD, VA Palo Alto Health Care System; 2 University of Pennsylvania School of Social Policy & 

Practice 

Contact: sjlandes@uw.edu 

Dialectical Behavior Therapy (Linehan, 1993) is an evidence‐based cognitive behavioral psychotherapy for suicidal 

individuals with Borderline Personality Disorder (BPD); it is considered the gold standard treatment for BPD, suicidal 

behavior, and severe behavioral dyscontrol. DBT is a comprehensive treatment and consists of four modes: group 

skills training, individual therapy, skills coaching outside of session, and therapist consultation team. Limited data is 

available about how teams in real‐world settings implement full DBT programs following intensive training. Two 

different ongoing projects are evaluating the process of implementing DBT programs, in community settings and in 

Department of Veterans Affairs (VA) settings. We will present data from these studies, including what components 

are implemented at different time points following intensive training, barriers encountered, and qualitative 

descriptions of the process. We will discuss whether these data support implementation strategies that encourage 

implementation of all components at once or a modular approach (e.g., implementing one component at a time) and 

future research directions. 

Track(s): Scale‐Up, Technology, Training 

NOTES 


May 17. 8:15‐8:45 

SIRC INSTRUMENT REVIEW 

TASKFORCE: AN OVERVIEW OF 

PROGRESS MADE & PLANS FOR 

THE FUTURE 





UPDATE FROM THE SIRC INSTRUMENT REVIEW TASK FORCE 

Cara C. Lewis, PhD, 1 Ruben Martinez, BA, 1 Cameo Borntrager, PhD, 2 & Bryan Weiner, PhD 3 

1 Indiana University; 2 University of Montana; 3 University of North Carolina at Chapel Hill 

It is critical that researchers utilize psychometrically validated instruments when studying their implementation 

efforts to build a strong knowledge base and to avoid drawing incorrect or inappropriate conclusions. Navigating the 

seemingly dense landscape of instruments used in implementation science is an arduous task even for the most 

experienced reviewer. In an effort to move the field forward, the Seattle Implementation Research Collaborative 

(SIRC) has coordinated a multi‐site effort that attempts to systematically review, compile, and empirically rate 

instruments relevant to the study of implementation. SIRC first identified 33 distinct constructs integral to the 

implementation process delineated by the Consolidated Framework for Implementation Research (Damschroder et 

al., 2009) and the Implementation Outcomes (Proctor et al., 2010). Through SIRC’s systematic review of these 

constructs, we have identified and categorized over 450 instruments to be empirically rated and made available to 

members of SIRC. SIRC’s Instrument Review Task Force (containing over 50 members) will use the modified 

evidence‐based assessment criteria to rate each instrument for its psychometric strength (e.g., reliability and 

validity). This talk will present the results of the rating process for approximately 115 instruments tapping 

implementation outcomes (i.e., acceptability, adoption, appropriateness, feasibility, penetration, sustainability). 

Additionally, we will unveil the interactive SIRC Instrument Review Project website page. 


NOTES 


May 17. 8:45‐10:00 

KEY FINDINGS & FUTURE PATHS IN 

IMPLEMENTATION RESEARCH 


Implementation Science in an Era of 

Health Reform & Patient‐Centered 

Comparative Effectiveness Research: 

New Threats, New Expectations, New 

Opportunities 

Brian S. Mittman, PhD 

VA Greater Los Angeles Healthcare 

System & Kaiser Permanente Southern 

California 

Synthesis of Findings from 3 Lifestyle 

Behavior Change Program 

Implementation in the VA 

Laura J. Damschroder, MS, MPH 

VA Ann Arbor Healthcare System 

Racial/Ethnic Disparities & the 


Practices in Public Youth‐Serving 

Systems 

Antonio R. Garcia, PhD 

School of Social Policy & Practice, University 

of Pennsylvania 



IMPLEMENTATION SCIENCE IN AN ERA OF HEALTH REFORM & PATIENT‐CENTERED COMPARATIVE 

EFFECTIVENESS RESEARCH: NEW THREATS, NEW EXPECTATIONS, NEW OPPORTUNITIES 

Brian S. Mittman, PhD 

VA Greater Los Angeles Healthcare System & Kaiser Permanente Southern California 

Contact: brian.mittman@va.gov 

The field of implementation science is enjoying a surge of attention and interest throughout the world, driven by 

increasing recognition of gaps in the quality and outcomes of health services (and other social services) – and gaps in 

the appropriate use of effective practices and innovations – as well as by economic pressures and additional policy 

initiatives (e.g., health reform in the US). This new attention offers the promise of increased funding and other 

support, but accompanied by heightened expectations and scrutiny. This presentation begins with a brief overview 

of these and other trends in the field of health research (e.g., the rise of patient‐centeredness and engagement as a 

dominant value in clinical, health services and health behavior research), and then identifies several specific threats 

and opportunities they pose for implementation science. Possible responses are suggested, including suggested 

actions by individual implementation scientists, research program leaders and funding agencies, and other key 

stakeholders. 


NOTES 


SYNTHESIS OF FINDINGS FROM 3 LIFESTYLE BEHAVIOR CHANGE PROGRAM IMPLEMENTATIONS IN THE VA 

Laura J. Damschroder, MS, MPH, & Julie C. Lowery, PhD 

VA Ann Arbor Healthcare System 

Contact: laura.damschroder@va.gov 

There is urgent need to implement evidence‐based lifestyle change interventions more widely in healthcare 

settings. However, evidence supporting multi‐dimensional, complex implementation strategies and techniques 

consistently shows mixed results with repeated calls for needed research into “context.” Research on context 

has been dominated by single or small sample case studies without using a theoretical structure to promote 

comparison across studies. We used the Consolidated Framework for Implementation Research (CFIR), a 

theory‐based taxonomy of contextual constructs, in a series of 3 implementation studies of lifestyle behavior 

change programs for Veterans. Some contextual factors were influential regardless of the program, such as 

nature and quality of networks and communications, though specific behaviors varied by program. Other 

contextual factors varied; e.g., the group‐based program implementation was heavily influenced by stakeholder 

perceptions of the relative advantage of the new program compared to other options but this construct did not 

influence success for the other programs. Insights into why and how these constructs manifest differently based 

on differences in intervention characteristics, settings, and processes will be presented. Use of a common 

theoretical framework allowed this synthesis to take place, demonstrating how a knowledge‐base can be more 

readily built to help accelerate implementation of evidence‐based lifestyle behavior change interventions. 


NOTES 



RACIAL/ETHNIC DISPARITIES & THE IMPLEMENTATION OF EVIDENCE‐BASED PRACTICES IN PUBLIC YOUTH‐ 

SERVING SYSTEMS 

Antonio R. Garcia, PhD, 1 Lawrence A. Palinkas, PhD, 2 Lonnie Snowden, PhD, 3 Lisa Saldana, PhD, 4 & Patricia 

Chamberlain, PhD 4 

1 School of Social Policy & Practice, University of Pennsylvania; 2 School of Social Work, University of Southern 

California; 3 School of Public Health, University of California‐Berkeley; 4 Center for Research to Practice, Eugene, Oregon 

Contact: antgar@sp2.upenn.edu; 

Much of the disparities in the delivery and outcomes of child welfare services can be attributed to differences in the 

implementation of high‐quality evidence‐based practices (EBPs). However, no research to date has identified socioenvironmental 

and organizational barriers to and facilitators of successful implementation of EBPs among high and 

low minority concentrated areas (MCAs). To address this gap and understand the role of research evidence use in 

bringing EBPs to scale, the current study utilized data from a larger randomized trial comparing the use of 

Community Development Teams (CDTs) versus standard implementation strategies to implement Multidimensional 

Treatment Foster Care to scale in California and Ohio. While findings point to no differences in the number of 

implementation activities completed, multi‐group path analyses revealed that there are significant differences in 

which socio‐ecological and organizational factors significantly predict implementation outcomes between high and 

low MCAs. Study findings highlight that more attention should be devoted to improving the organizational social 

context in preparing for and actively implementing EBPs to scale to disrupt structural disparities. Further implications 

for practice innovation and research to improve implementation outcomes will be discussed. 


NOTES 


May 17. 10:15‐11:30 

IMPLEMENTATION IN ZAMBIA 

(MC: Shannon Dorsey, PhD) 

Implementation of TF‐CBT in Zambia: 

Perspectives from Local Supervisors & 

Counselors 


Serenity Harm Reduction Programme 

Zambia (SHARPZ), Lusaka, Zambia 

Organizational Implementation Barriers 

& Facilitators for Mental Health 

Programs in Zambia: A Mixed Methods 

Study 


Johns Hopkins University Bloomberg 

School of Public Health 

Mixed Methods Assessment of 

Implementation Barriers & Facilitators 

for Mental Health Programs in Zambia: 

Provider Level Themes 


University of Pennsylvania 



IMPLEMENTATION OF TF‐CBT IN ZAMBIA: PERSPECTIVES FROM LOCAL SUPERVISORS & COUNSELORS 


Serenity Harm Reduction Programme Zambia (SHARPZ), Lusaka, Zambia 

Contact: chimfwembe1957@yahoo.co.uk 

Global mental health is receiving increasing attention given the high burden of mental health disease and in low and 

middle income countries (LMIC) and the fewer than 10% of individuals with need who receive treatment. One 

implementation strategy for scaling up care that has received a great deal of attention is task‐shifting mental health 

care to lay counselors, who have little or no mental health experience. However, training lay counselors requires 

additional time, supervision and supports. In addition, most evidence‐based practices (EBP) are developed in highincome 

countries, and transported to LMIC, which have different cultures and contexts. 

This unique presentation focuses on the perspectives and experiences of a local counselor and now supervisor, Ms. 

Kasoma, on learning, implementing, and supervising a child mental health EBP (trauma‐focused CBT) in Lusaka, 

Zambia over seven years. Ms. Kasoma first served as a counselor on a NIH‐funded feasibility study and currently is a 

supervisor on a nearly completed USAID randomized trial. 

Ms. Kasoma will speak to issues related to learning an EBP, counselor and consumer acceptability, monitoring 

counselor fidelity, and cultural adaptation and acceptability. 

Track(s): EBP Champions, Fidelity, Global Perspectives, Scale‐Up, Training 

NOTES 


ORGANIZATIONAL IMPLEMENTATION BARRIERS & FACILITATORS FOR MENTAL HEALTH PROGRAMS IN 

ZAMBIA: A MIXED‐METHODS STUDY 

Laura Murray, PhD, 1 Rinad Beidas, PhD, 2 Stephanie Skavenski, MPH, MSW, 1 Shannon Dorsey, PhD, 3 & John 

Mayeya 4 

1 Johns Hopkins University Bloomberg School of Public Health; 2 University of Pennsylvania; 3 University of 

Washington; 4 Ministry of Health, Zambia 

Contact: lamurray@jhsph.edu 

The Dissemination and Implementation (D&I) literature in low‐resource countries (LRC) is often described as 

being in its infancy to that of the West (Thorncraft et al., 2009). Although the field of global mental health has 

now shown that various evidence‐based practices (EBPs) are feasible, adaptable, and effective (e.g., Bolton et 

al., 2007), the uptake of these interventions by Ministry of Health (MoH), non‐governmental (NGOs) or 

community‐based organizations (CBOs) has been sluggish at best (despite no evidence that “psychosocial 

programming” is effective, Bryant et al., 2012). It is likely that major barriers to uptake of EBPs include 

implementation factors. 

Participants include 65 individuals who were part of a Trauma Focused‐Cognitive Behavioral Therapy (TF‐CBT) 

feasibility and/or effectiveness pilot in Zambia. A mixed‐methods design will include a sequential collection of 

qualitative, followed by quantitative data (Palinkas et al., 2011). 

This study will examine how organizational structure impacts implementation factors, including adoption, 

appropriateness, feasibility, penetration, and sustainability (Proctor et al., 2011). A semi‐structured interview 

will be followed by administration of the DOOR and ORC. Results will be discussed across different 

organizational levels including JHU, CBOs, NGOs, and the MoH. Policy implications and future research ideas will 

be discussed related to implementation dilemmas. 

Track(s): Global Perspectives 

NOTES 



MIXED METHODS ASSESSMENT OF IMPLEMENTATION BARRIERS & FACILITATORS FOR MENTAL HEALTH 

PROGRAMS IN ZAMBIA: PROVIDER LEVEL THEMES 

Rinad Beidas, PhD, 1 Laura Murray, PhD, 2 Shannon Dorsey, PhD, 3 Stephanie Skavenski, MSW, MPH, 2 Margaret 

Kasoma, 4 & John Mayeya 5 

1 University of Pennsylvania; 2 Johns Hopkins University Bloomberg School of Public Health; 3 University of Washington; 

4 Serenity Harm Reduction Programme Zambia (SHARPZ); 5 Ministry of Health 

Contact: rbeidas@upenn.edu 

A better understanding of the implementation factors involved in the uptake of mental health programming in lowresource 

countries (LRC) is needed. Although the field of global mental health has now shown that various evidencebased 

treatments (EBTs) are feasible, adaptable and effective (Bolton et al., 2003; 2007; Rahman et al., 2008), the 

uptake of these interventions by locally‐based organizations has been sluggish at best. It is likely that major barriers 

to uptake of EBPs include implementation factors. To our knowledge, no study has been conducted globally around 

barriers and facilitators of implementation of evidence‐based mental health practices. Participants will include 65 

individuals who were part of a TF‐CBT feasibility and effectiveness pilot in Zambia. The goal of this study is to 

qualitatively and quantitatively examine implementation factors, including acceptability, adoption, appropriateness, 

feasibility, fidelity, penetration and sustainability (Proctor et al., 2011). This study will employ mixed‐methods, 

specifically the semi‐structured interviews and the EBPAS‐50 to examine stakeholders perspectives and attitudes on 

implementation of evidence‐based mental health programming in Zambia. This presentation will focus on themes 

related to the ecological level of the individual‐provider level. 


NOTES 


May 17. 12:45‐2:00 

BREAKOUT K: GLOBAL MODELS OF 


(MC: Rinad Beidas, PhD) 

Scaling Up Care for Orphans in 

Tanzania: A Task‐Sharing Approach to 


Shannon Dorsey, PhD 


A Transdiagnostic Mental Health 

Intervention in Low Resource Countries: 

An Alternative Solutions to Mental 

Health Implementation Challenges 


Johns Hopkins University Bloomberg 

School of Public Health 

Implementation of Cognitive Processing 

Therapy Provided by Community‐Based 

Paraprofessionals in the Democratic 

Republic of Congo: Influence of 

Therapist Factors Randomized Clinical 

Trial 

Debra Kaysen, PhD 




SCALING UP CARE FOR ORPHANS IN TANZANIA: A TASK‐SHARING APPROACH TO MENTAL HEALTH 

TREATMENT 

Shannon Dorsey, PhD, 1 Karen O'Donnell, PhD, 2 Kate Whetten, PhD, 3 Wenfeng Gong, MA, 4 Dafrosa Itemba, 5 & Rachel 

Manongi 6 

1 University of Washington; 2 Duke University School of Medicine; 3 Duke University; 4 Johns Hopkins University; 

5 Tanzania Women Research Foundation; 6 Kilimanjaro Christian Medical Centre 

Contact: dorsey2@uw.edu 

Global mental health is increasingly receiving research attention. Nearly ten randomized clinical trials in low and 

middle income countries (LMIC) have demonstrated the effectiveness of evidence‐based practices (EBP), however, 

only one trial focused on adolescents, none on children, despite the high mental health gap for this population 

(Saxena et al., 2007). Furthermore, very few D&I questions have been included in trials, despite the focus on tasksharing 

(training non‐mental health professionals to deliver mental health interventions) and its relevance to 

implementation science. We examined feasibility and clinical outcomes for children and adolescents receiving 

Trauma‐focused Cognitive Behavioral Therapy (TF‐CBT) in Moshi, Tanzania, an area of high HIV prevalence and 

orphaned children. TF‐CBT was provided to single sex groups (ages 7‐10; 11‐13), using a task‐sharing approach. The 

study employed the Apprenticeship Training Model (Murray, Dorsey et al., in press), developed specifically for 

training, supervision, and iterative, collaborative adaption with local lay counselors. Post‐treatment, children had 

significantly reduced PTSD and traumatic grief or shame at post‐treatment (PTSD Child‐report; β=15.38; p

A TRANSDIAGNOSTIC MENTAL HEALTH INTERVENTION IN LOW RESOURCE COUNTRIES: AN ALTERNATIVE 

SOLUTION TO MENTAL HEALTH IMPLEMENTATION CHALLENGES 

Laura Murray, PhD, 1 Shannon Dorsey, PhD, 2 Maythem Alyasiry, 3 Amir Haydary, 4 & Paul Bolton, MBBS, MPH 1 

1 Johns Hopkins University Bloomberg School of Public Health; 2 University of Washington; 3 Department of 

Psychiatry, Iraq; 4 Ministry of Health, Iraq 

Contact: lamurray@jhsph.edu 

A growing research base demonstrates that evidence‐based treatments (EBT) are transportable, adaptable, 

acceptable, and effective in low and middle income countries (LMIC) (e.g., Bolton et al., 2007; Patel et al., 2011) 

using a task‐shifting approach (i.e., counselors with limited or no prior mental health training). However, the 

singular focus of most EBT on one clinical problem (e.g., depression) is a barrier to scale up, reducing ability to 

address the mental substantial health treatment gap in LMIC. Transdiagnostic interventions teach a set of 

common practice elements delivered in varying combinations to address a range of problems. Components‐ 

Based Intervention (CBI) is a transdiagnostic approach developed for LMIC and is currently being tested in two 

RCTs for adult survivors of torture in Southern Iraq and at the Thailand‐Burma border. Presentation will focus on 

the novel intervention development, training, supervision, and outcomes in both sites. Pilot cases showed a 

75% decrease in clinical symptoms (Iraq) and 54.5%/55.2% decrease in depression and trauma symptoms 

respectively (Thailand). Functioning impairment also decreased at both sites. These findings, combined with 

growing US‐based evidence, suggest that a transdiagnostic approach may be an alternative for implementation 

and scale‐up challenges of addressing mental health problems in LMIC. 


NOTES 



IMPLEMENTATION OF COGNITIVE PROCESSING THERAPY PROVIDED BY COMMUNITY‐BASED 

PARAPROFESSIONALS IN THE REPUBLIC OF CONGO: INFLUENCE OF THERAPIST FACTORS RANDOMIZED CLINICAL 

TRIAL 

Debra Kaysen, PhD, 1 Shelly Griffiths, MSW, LICSW, 1 Cindy Stappenbeck, PhD, 1 Janny Jinor, MSW, LCSW, 2 Paul Bolton, 

MBBS, MPH, 3 Jeannie Annan, PhD, 4 Katie Robinette, MPH, 4 & Judith Bass, PhD, MPH 3 

1 University of Washington; 2 Morgan State, Baltimore, MD; 3 Johns Hopkins University Bloomberg School of Public 

Health; 4 International Rescue Committee 

Contact: dkaysen@uw.edu 

Need for mental health care services for sexual violence victims in eastern Democratic Republic of Congo (DRC) has 

been documented but few services exist. Therapies developed in the West with established efficacy with female 

rape victims have not been tested in low resource settings like DRC. Growing literature addresses the adaptability of 

evidence‐based psychotherapies cross‐culturally and in resource‐poor contexts. One dilemma for implementation in 

low resource settings is to what extent complex treatments can be delivered successfully by paraprofessionals. In 

this study, we will discuss results of adapted Cognitive Processing Therapy for use in DRC. Congolese communitybased 

paraprofessionals were trained and supervised in delivering group CPT. Hierarchical linear modeling was used 

to examine change over time. Based on preliminary analyses of weekly self‐report measures, there was a significant 

reduction in mental health symptoms over time (b= ‐2.04, p

May 17. 12:45‐2:00 

BREAKOUT L: INNOVATIVE 

SUBSTANCE ABUSE TREATMENT 


(MC: Doyanne Darnell, PhD) 

Scaling Up & Sustaining Alcohol & PTSD 

Screening & Intervention in US Trauma 

Care Systems 


University of Washington School of 

Medicine, Harborview Injury Prevention 

& Research Center 

Lessons Learned from Implementing a 

Web‐Based Tool for Brief Alcohol 

Interventions in a Large Integrated 


Kenneth R. Weingardt, PhD 

Veterans Health Administration & 

Stanford University 

Disseminating Contingency 

Management: A Training & 



Alcohol & Drug Abuse Institute, 




SCALING UP & SUSTAINING ALCOHOL & PTSD SCREENING & INTERVENTION IN US TRAUMA CARE SYSTEMS 

Douglas Zatzick, MD, 1,2 Dennis Donovan, PhD, 1,3 Chris Dunn, PhD, 1,2 Frederick Rivara, MD, MPH, 1,2 Larry Gentilello, 

MD, 1,2 Joan Russo, PhD, 1 Jin Wang, PhD, 2 Jeff Love,BA, 1 Collin McFadden, BA, 1 & Gregory Jurkovich, MD 4 

1 University of Washington School of Medicine; 2 Harborview Injury Prevention & Research Center; 3 Alcohol & Drug 

Abuse Institute; 4 Denver Health Care System, Denver, CO 

Contact: dzatzick@uw.edu 

The American College of Surgeons Committee on Trauma tightly regulates United States (US) trauma center care 

through policy mandates and clinical guideline best practice recommendations. College mandates are reinforced 

through verification site visit implementation criteria. The American College of Surgeons has successfully linked 

trauma center funding to verification site visits and other quality indicators. This presentation will describe a unique 

investigative‐policy collaboration whereby federally funded empiric research on posttraumatic stress disorder (PTSD), 

and alcohol/drug screening and intervention has been directly translated to policy mandates and clinical guidelines 

for acute care medical trauma centers nationwide. The presentation will first describe the randomized clinical trial 

evidence base supporting alcohol and PTSD screening and intervention in acute care medical settings. Next, the 

presentation will focus on recent policy summits with the American College of Surgeons that have allowed for the 

scaling up of results of single site randomized clinical trials across trauma care systems nationwide. Findings suggest 

that regulatory requirements developed in concert with multidisciplinary implementation team oversight may 

optimally enhance the scaling up and sustainability of behavioral health treatment integration in acute care trauma 

center medical settings. 

Track(s): Scale‐Up, Sustainability 

NOTES 


LESSONS LEARNED FROM IMPLEMENTING A WEB‐BASED TOOL FOR BRIEF ALCOHOL INTERVENTIONS IN A 

LARGE INTEGRATED HEALTH CARE SYSTEM 

Kenneth R. Weingardt, PhD, 1,2 Michael A. Cucciare, PhD, 2,3 Paula L. Wilbourne, PhD, 1 & John S. Baer, PhD 4 

1 Veterans Health Administration; 2 Stanford University; 3 Center for Health Care Evaluation, VA Palo Alto; 

4 University of Washington & VA Puget Sound Health Care System 

Contact: kenweingardt@va.gov 

Research has demonstrated that computer‐based brief motivational interventions can be an efficacious means 

of reducing alcohol use and alcohol‐related problems. The authors designed and built a web‐based brief alcohol 

intervention for use in the Department of Veterans Affairs (VA) in 2005. This presentation summarizes the 

projects and initiatives that have been undertaken in the ensuing years to support the implementation of this 

tool into clinical practice. Projects have focused on a variety of clinical settings, including outpatient 

Readjustment Counseling (Vet Centers), specialty care for PTSD and Substance Use Disorders, and Integrated 

Primary Care. The authors will provide a narrative description of each initiative, and use the Consolidated 

Framework for Implementation Research (CFIR) to communicate how various contextual factors acted as 

barriers and facilitators within each setting. The presentation concludes with a summary of lessons learned, and 

a description of how these lessons are informing current efforts to implement the web‐based brief alcohol 

interventions in HCV clinics, and as part of a national initiative to roll out Motivational Enhancement Therapy 

(MET). 

Track(s): Scale‐Up, Technology 

NOTES 



DISSEMINATING CONTINGENCY MANAGEMENT: A TRAINING & IMPLEMENTATION TRIAL 


Alcohol & Drug Abuse Institute, University of Washington 

Contact: hartzb@uw.edu 

Contingency Management (CM) is an empirically‐validated behavioral treatment, for which community dissemination 

has been surprisingly slow. Given the considerable evidence accumulated for its efficacy, examination of 

implementation outcomes is paramount. A recent study involved the collaborative design of a CM intervention by a 

university investigator and partnering community addiction treatment clinic. The study then formally evaluated 

immediate impacts of a 16‐hour training workshop with clinic personnel, as well as well as eventual impacts following 

a 90‐day trial implementation period. A mixed method design allowed repeated measurement of clinician‐focused 

implementation (acceptability, appropriateness, adoption, fidelity) among 17 staff clinicians and qualitative 

retrospective measurement of management‐focused implementation outcomes (cost, feasibility, penetration, 

sustainability) among 5 executive staff. Broad clinical outcomes of the intervention were also examined via review 

of patient medical records, and compared to a historical control. The presentation will outline: 1) the processes of 

intervention design and staff training, 2) immediate and eventual impacts of training on clinician‐focused 

implementation domains, 3) qualitative report of clinic executives concerning management‐focused implementation 

domains following the trial implementation period, and 4) patient‐based intervention outcomes. The study offers a 

formal, comprehensive evaluation of staff training and implementation of CM at a community addiction treatment 

clinic, and collective results may provide useful insights for those working in or collaborating with community 

treatment programs. 


NOTES 


May 17. 12:45‐2:00 

BREAKOUT M: STATISTICAL 

METHODS WORKSHOP PART I 


Design & Analysis Challenges with 

Multilevel Implementation Data 

David C. Atkins, PhD, 1 & Scott A. 

Baldwin, PhD 2 

1 University of Washington; 2 Brigham 

Young University 



DESIGN & ANALYSIS CHALLENGES WITH MULTILEVEL IMPLEMENTATION DATA 

David C. Atkins, PhD, 1 & Scott A. Baldwin, PhD 2 

1 University of Washington; 2 Brigham Young University 

Contact: datkins@uw.edu; scott_baldwin@byu.edu 

Implementation research often involves multilevel data (sometimes called hierarchical, clustered, or nested data). 

Examples of such data include patients clustered within providers, providers clustered within sites, and therapist 

fidelity items clustered within therapists. Such multilevel data present a number of design and analysis challenges. 

Our presentation will provide a brief, general overview of multilevel models and then focus on specific challenges 

related to implementation research. Topics will include: 

 

 

 

 

How sample sizes at provider and site levels affect the use of multilevel models 

Advantages and disadvantages of randomizing between or within clusters (e.g., randomizing therapists 

within a site or randomizing sites to treatment condition) 

Power and sample size calculations, including costs (e.g., costs of adding providers vs. sites) and attrition 

How multilevel designs influence the assessment of therapist fidelity, including reliability and psychometric 

considerations 

Our goal is to provide a non‐technical introduction to these topics, emphasizing concepts as opposed to statistics. 

Moreover, we hope to have a highly interactive session with input and questions from the audience. Finally, there 

will be time for general questions on implementation designs at the end of the session, not necessarily specific to 

multilevel data. 


NOTES 


May 17. 12:45‐2:00 

BREAKOUT N: LEARNING FROM 

SCALE‐UP 


Overcoming Implementation Research 

Challenges While Studying CPT Training 

& Implementation Across Canada 


VA Boston Healthcare System, National 

Center for PTSD, & Boston University 

Financing & Scaling Up Early 

Intervention Services 

Howard H. Goldman, MD, PhD 

Department of Psychiatry, University of 

Maryland School of Medicine 

System Improvement Through Service 

Collaboratives: Closing Gaps & 


Brian Rush, PhD 

Centre for Addiction & Mental Health 

Health Systems & Health Equity 

Research, Department of Psychiatry, 

University of Toronto, Canada 



OVERCOMING IMPLEMENTATION RESEARCH CHALLENGES WHILE STUDYING CPT TRAINING & 

IMPLEMENTATION ACROSS CANADA 

Shannon Wiltsey Stirman, PhD, 1 Candice Monson, PhD, 2 Josh Deloriea, BA, 2 Jennifer Belus, BA, 3 Marta Maslej, BA, 2 & 

Norman Shields, PhD 4 

1 VA Boston Healthcare System, National Center for PTSD, & Boston University; 2 Ryerson University; 3 University of 

North Carolina; 4 Operational Stress Injuries National Network, Veteran Affairs Canada 

Contact: sws@bu.edu 

The overall objectives of this study are 1) to compare three different post‐workshop consultation strategies 

(technology‐enhanced, standard, and independent use of resources) for the implementation of Cognitive Processing 

Therapy with clinicians who serve Veterans in Canada, and 2) to identify system‐, site‐ and provider‐level barriers and 

facilitators to implementation. Consistent with the theme of this year’s conference, we will present data relevant to 

our efforts to solve implementation research dilemmas that arose as we executed the study. Two grant submissions 

and three cycles of recruitment have allowed us to iteratively refine strategies to address a number of challenges 

inherent in implementation research, including considerations regarding study design, recruiting and engaging 

clinicians, managing over 16 separate IRB submissions, addressing problems with study technology, collecting data 

from clinicians based at clinics throughout the country, consenting of patient participants, ethical dilemmas, budget 

constraints, and fidelity monitoring (for both consultation condition fidelity at the consultant level CPT fidelity at the 

clinician level). We will present data gathered and analyzed from meeting minutes, data tracking logs, monthly 

clinician activity reports, weekly consultation reports, and interviews with clinicians and consultants to demonstrate 

both the decision making processes around addressing these challenges, as well as the outcomes of our 

implementation of these strategies. 

Track(s): Global Perspectives, Scale‐Up, Technology, Training 

NOTES 


FINANCING & SCALING UP EARLY INTERVENTION SERVICES 

Howard Goldman, MD, PhD, 1 Mustafa C. Karakus, PhD, 2 & Kirsten Beronio, JD 3 

1 Department of Psychiatry, University of Maryland School of Medicine; 2 Westat; 3 US Department of Health & 

Human Services, ASPE 

Contact: hh.goldman@verizon.net 

This presentation will focus on the scalability of supported employment and early intervention services in 

mental health care. It is based on a series of studies performed by Westat for the Office of the Assistant 

Secretary for Planning and Evaluation (ASPE) in the Department of Health and Human Services. ASPE has been 

interested in examining policies that would promote the implementation of supported employment services and 

early intervention services in mental disorders over the past several years. The passage of the Affordable Care 

Act (ACA) has improved the prospects for financing some of these services, but barriers still remain for paying 

for supported employment and supported education, as well as scaling up other aspects of team‐based early 

intervention services. We will report on our discussions with policy makers, service providers and our 

observations from site visits across the U.S. 


NOTES 



SYSTEM IMPROVEMENT THROUGH SERVICE COLLABORATIVES: CLOSING GAPS & IMPROVING ACCESS & 

COORDINATION 

Brian Rush, PhD, 1 Fiona C. Thomas, 2 & Heather McKee 2 

1 Centre for Addiction & Mental Health, Department of Psychiatry, University of Toronto; 2 Centre for Addiction & 

Mental Health, University of Toronto 

Contact: brian.rush@camh.ca 

The Systems Improvement through Service Collaboratives (SISC) initiative, sponsored by the Centre for Addiction and 

Mental Health, aims to improve access and coordination for those with mental health and addictions problems, with 

priority on children and youth in regions across Ontario, Canada. This provincial initiative spans across multiple 

sectors and six different Ministries, including those in the health, justice and education sectors. SISC is grounded in 

Implementation Science (IS) frameworks, including Quality Improvement (QI), and has a strong emphasis on both 

developmental evaluation and more traditional performance measurement. 

The presentation will focus on how this initiative embedded the National Implementation Research Network (NIRN) 

framework for implementing evidence‐based interventions (EBI) across the province. Evaluation data will be 

presented, which includes performance indicators related to key transitions (e.g. hospital – community; health – 

justice; youth‐adult) and measures of implementation progress, including those of maintaining fidelity to EBI. The 

evaluation is further comprised of assessments of partnership and collaboration, measures of health equity and case 

studies. The challenges of balancing between both provincial and local indicators will be discussed. Finally, 

contributions to the theoretical framework of IS, based on learnings from this initiative will be shared. 


NOTES 


May 17. 12:45‐2:00 

BREAKOUT O: FIDELITY OF 

INTERVENTIONS ACROSS THE AGE 

SPECTRUM 


Implementation of the Program to 

Encourage Active & Rewarding Lives for 

Seniors (PEARLS) 

Leslie Steinman, MSW, MPH 

UW Health Promotion Research Center 

Common Issues with Assessing Fidelity 

to Complex Multi‐Modal Service 

Programs: Lessons Learned from 

Assessing Fidelity to the ACT Model 


Department of Psychiatry & Behavioral 

Sciences, University of Washington 

Assessing Implementation Fidelity of 

the Family Check‐Up: Development & 

Validation of the COACH Rating System 


Child & Family Center, University of 

Oregon 



IMPLEMENTATION OF THE PROGRAM TO ENCOURAGE ACTIVE & REWARDING LIVES FOR SENIORS (PEARLS) 

Lesley Steinman, MSW, MPH, 1 & Mark Snowden, MD, MPH 2 

1 Health Promotion Research Center, Department of Health Services, University of Washington; 2 Department of 

Psychiatry & Behavioral Sciences, University of Washington 

Contact: lesles@uw.edu 

Background: Depression is often undertreated in older adults. PEARLS is an evidence‐based depression care 

management program for homebound elders. Working with multiple community partners, we studied several 

approaches for improving implementation and adapting PEARLS to address commonly identified barriers. 

Methods: Ten focus groups with 40 staff were analyzed using thematic analysis to identify barriers to 

implementation. A formal agency plan was developed to improve implementation and the plan was evaluated using 

process and outcome measures. We developed a 20‐item fidelity instrument through key informant interviews and 

validated the instrument using known‐groups method with 12 agencies. 

Results: Focus groups revealed strict eligibility criteria interfered with agency’s mission to serve all clients. PEARLS 

modifications were piloted with interpreters for limited English‐speaking clients and for clients with major 

depression. Depression response and remission rates were similar to the original model (80%). Implementation 

coaching resulted in modest improvements in referral (9% to 15%) and enrollment rates (4% to 8%). Fidelity 

instrument testing showed PEARLS programs had higher fidelity scores compared to other types of depression 

programs (p

COMMON ISSUES WITH ASSESSING FIDELITY TO COMPLEX MULTI‐MODAL SERVICE PROGRAMS: LESSONS 

LEARNED FROM ASSESSING FIDELITY TO THE ACT MODEL 


Department of Psychiatry & Behavioral Sciences, University of Washington School of Medicine 

Contact: mmdv@uw.edu 

Fidelity assessment is a key element to ensuring that treatment programs are adherent to the intended model 

and can therefore anticipate achieving desired clinical outcomes; however, comprehensive evaluation of fidelity 

to more complex multi‐modal service programs can be difficult to achieve. For some programs, the broad range 

of biopsychosocial service needs of the population served require clinicians to employ more than one evidencebased 

practice (EBP) within the context of the larger program; in some cases, service recipients may receive 

treatments delivered by more than one clinician and/or in a variety of community‐based settings outside of the 

office. This presentation will focus on how these complex program elements may be assessed by using the new 

Assertive Community Treatment (ACT) fidelity scale – the TMACT – as a case study, focusing on key issues in 

fidelity assessment such as balancing evaluation of: (1) process and structure; (2) team and individual clinical 

skills; and (3) other EBPs integrated or blended within the larger service program. While this presentation will 

use ACT, an EBP for adults with serious mental illness, as an illustration of how these core dilemmas in fidelity 

assessment can be handled, implications for other service programs for different populations will be discussed. 


NOTES 



ASSESSING IMPLEMENTATION FIDELITY OF THE FAMILY CHECK‐UP: DEVELOPMENT & VALIDATION OF THE 

COACH RATING SYSTEM 

Justin D. Smith, PhD 1 Elizabeth A. Stormshak, PhD, 1 & Thomas J. Dishion, PhD 1,2 

1 Child & Family Center, University of Oregon; 2 Prevention Research Center, Arizona State University 

Contact: jsmith6@uoregon.edu 

Objective: We present a series of studies concerning the development and validation of an observation fidelity of 

implementation rating system for the Family Check‐Up (FCU). The FCU is a family‐based intervention shown to 

improve family management practices and reduce problem behaviors in youth ages 2‐18 (e.g., Dishion et al, 2008; 

Stormshak & Dishion, 2009). Method: Therapists treating families of children ages 2–17 from two randomized trials 

(one efficacy and one effectiveness) were rated for fidelity to the FCU in three separate studies, the final study being 

an experimental manipulation of the rating procedures in an attempt to improve reliability of the ratings. Results: 

Study 1: Variations in fidelity were associated with observed positive parenting of toddler‐age children one year after 

receipt of the FCU, which in turn predicted reductions in child problem behaviors the following year. Study 2: 

Therapists employed at community mental health agencies achieved adequate levels of fidelity, which was 

associated with family and child level outcomes. Study 3: Two factors of previous fidelity rating studies were 

identified that likely contributed to less than optimal reliability: Coder training in the FCU and access to family’s 

assessment data. Observed caregiver engagement, a single item in the rating system that has been found to be an 

important intervening variable in the relationship between fidelity and family outcomes, was also examined for 

validity. Conclusions: The FCU is an effective family‐based intervention that is feasible for scale‐up in multiple 

community service settings. The accurate and reliable assessment of fidelity of implementation is a crucial factor in 

training providers to deliver the intervention as intended. These studies demonstrate the validity and reliability of 

our fidelity rating system. 


NOTES 


May 17. 2:15‐3:30 

BREAKOUT P: 

EBP CHAMPION SYMPOSIUM 

(MC: Shannon Dorsey, PhD) 

Implementation of TF‐CBT Across 

Washington State 

Joe Leroy, MSW 

HopeSparks Family Services 

Dan Fox, MSW 

Lutheran Community Services Northwest 

Ron Gengler, MS 

Central WA Comprehensive Mental 

Health 

Lori Vanderburg, MS 

Compass Health 



IMPLEMENTATION OF TF‐CBT ACROSS WASHINGTON STATE 

Joe Leroy, MSW, 1 Dan Fox, MSW, 2 Ron Gengler, MS, 3 & Lori Vanderburg, MS 4 

1 HopeSparks Family Services; 2 Lutheran Community Services Northwest; 3 Central WA Comprehensive Mental Health; 

4 Compass Health 

Contact: jleroy@hopesparks.org 

The dissemination and implementation literature is replete with challenges, barriers, and some successes to 

implementing and scaling up evidence‐based treatments (EBT) in community‐based settings. With a few exceptions, 

generally missing from the research dialogue; however, are the perspectives and voices of clinicians, supervisors, and 

administrators in public mental health agencies who are implementing, or being asked or mandated to implement 

EBT. 

In Washington State, the Division of Behavioral Health and Recovery funded a yearly Trauma‐focused CBT training 

initiative for public mental health agencies. Now in its seventh year, with a broadened, common elements focus on 

CBT for anxiety, depression and behavioral problems, a number of agencies have adopted CBT. This presentation 

brings together champions from a wide range of EBT‐adopting agencies in Washington State to present their 

experiences and perspectives on facilitators, barriers, success stories, and ongoing challenges at the various stages of 

implementation. Of particular focus for the presentation will be the balance between effective and feasible strategies 

for obtaining: 1) agency‐wide reach of EBT and 2) assessing clinician competence and fidelity given challenges of 

turnover, tight budget climate, and in Washington State, pending EBT legislation. 

Track(s): EBP Champions, Fidelity, Scale‐Up, Sustainability, Training 

NOTES 


May 17. 2:15‐3:30 

BREAKOUT Q: SUSTAINABILITY 

(MC: Adam Carmel, PhD) 

Sustainability of CBT for Youth Anxiety 

in Community Settings Following 




Supporting Implementation of the 

Triple P System: A Standard Framework 

Jacquie Brown, MES, RSW, 1 & Sara van 

Driel, PhD 2 

1 Triple P International; 2 Triple P America 

Research Implementation within a 

Clinical Practice: Resolving the 

Science/Practice Dialectic 

Sally A. Moore, PhD 

Evidence‐Based Treatment Centers of 

Seattle & University of Washington 

Department of Psychiatry & Behavioral 

Sciences 



SUSTAINABILITY OF CBT FOR YOUTH ANXIETY IN COMMUNITY SETTINGS FOLLOWING IMPLEMENTATION 

Rinad Beidas, PhD, 1 Julie Edmunds, MA, 2 Margaret Mary Downey, BA, 1 Mark Gallagher, BA, 1 Jessica Watkins, 3 & Philip 

C. Kendall, PhD, ABPP 2 

1 University of Pennsylvania; 2 Temple University; 3 Bryn Mawr College; 

Contact: rbeidas@upenn.edu 

A recent experimental study examined whether various training methods and ongoing support resulted in differential 

outcomes in clinicians’ skill and adherence in implementing CBT for youth anxiety (Beidas, Edmunds, & Kendall, 

2012). Results indicated that any of the three training methods resulted in somewhat improved skill and adherence 

but that consultation was the most robust predictor of therapist outcomes. However, little is known about the 

sustainability of therapist implementation of CBT following training and consultation. To optimize therapist 

implementation and sustained use of CBT, it is necessary to investigate therapist perspectives on their sustained use 

of CBT, as well as barriers and facilitators to use of CBT. The present study used semi‐structured interviews 

conducted 2 years following the training and consultation provided in Beidas et al. (2012) with 50 therapists who 

participated in the initial study. Provider interviews were coded for the following themes: attitudes, practice change, 

facilitators, barriers, adaptation, organizational factors, self‐efficacy, eclecticism, client factors, treatment factors, 

EBP language, and consultation. The findings from this study will shed much needed insight on how CBT for youth 

anxiety is sustained in community settings following training and consultation. 

Track(s): Sustainability, Training 

NOTES 


SUPPORTING IMPLEMENTATION OF THE TRIPLE P SYSTEM: A STANDARDIZED FRAMEWORK 

Jacquie Brown, MES, RSW, 1 Jenna McWilliam, PhD, 1 Sara van Driel, PhD, 1 Randy Ahn, PhD, MLIS, 1 Debbie 

Easton, 2 Thomas Dirscherl, 3 Ronja Born, 3 Brad Thomas, 4 Sarah Munro, 1 & Rita Bostick, MA, LPC 4 

1 Triple P International; 2 Triple P Parenting Canada Inc.; 3 Triple P Deutschland; 4 Triple P America 

Contact: jacquie.brown@triplep.net; sara@triplep.net 

As a result of 15 years of dissemination and expansion in more than 20 countries, Triple P recognized the need 

to develop a flexible but standardized framework to support implementation of the Triple P System. The 

standardized framework is based in the RE‐AIM (Reach, Effectiveness, Adoption, Implementation, and 

Maintenance) and National Implementation Research Network (NIRN) Active Implementation Frameworks and 

includes 5 main steps: 1) Engagement, 2) Commitment and Contracting, 3) Implementation Planning, 4) Training 

and Accreditation, and 5) Implementation Maintenance. In using this framework, Triple P hopes to increase 

utilization rates of trained practitioners, improve sustainability of the program in communities, and expand the 

use of Triple P as a public health approach. 

In this presentation, we will provide an overview the framework, including each of the steps and the key 

activities within those steps, and discuss how the integration of implementation experience and implementation 

science contributed to the development of the framework. 

Track(s): Global Perspectives, Scale‐Up, Sustainability 

NOTES 



RESEARCH IMPLEMENTATION WITHIN A CLINICAL PRACTICE: RESOLVING THE SCIENCE/PRACTICE DIALECTIC 

Sally A. Moore, PhD, Stacy Shaw Welch, PhD, Travis Osborne, PhD, Jennifer Sayrs, PhD, & Jennifer Tininenko, PhD 

Evidence‐Based Treatment Centers of Seattle & University of Washington Department of Psychiatry & Behavioral 

Sciences 

Contact: smoore@ebtseattle.com 

Conducting research within a clinical setting is a complex endeavor, particularly given that research and clinical work 

are often viewed as conflicting interests. Clinicians may fear that investment in research will detract from providing 

optimal treatment. We will discuss the Evidence‐Based Treatment Centers of Seattle (EBTCS) as a case study 

illustrating the process of research implementation in a clinical setting. EBTCS is a treatment center with two primary 

missions: to provide evidence‐based specialty care for individuals with anxiety disorders, borderline personality 

disorder, and other difficulties, and to conduct research relevant to the individuals we treat. We will discuss the 

tension that can naturally arise between research and clinical domains and our efforts to move research from the 

back burner into the spotlight without sacrificing clinical care. When implementing a research program, clinical 

settings face multiple dilemmas, including resource allocation, creating a research infrastructure, and data collection 

methods. We will discuss our evolution in meeting these challenges and potential future directions for research at 

EBTCS, such as outcome benchmarking, mechanisms of change, and modular treatment of anxiety disorders. We 

have come to view research as complementary to our clinical mission, and this perspective promotes successful 

research implementation in clinical practice. 

Track(s): EBP Champion 

NOTES 


May 17. 2:15‐3:30 

BREAKOUT R: STATISTICAL 

METHODS WORKSHOP PART II 


Design & Analysis Challenges with 

Multilevel Implementation Data 

David C. Atkins, PhD, 1 & Scott A. 

Baldwin, PhD 2 

1 University of Washington; 2 Brigham 

Young University 



DESIGN & ANALYSIS CHALLENGES WITH MULTILEVEL IMPLEMENTATION DATA 

David C. Atkins, PhD, 1 & Scott A. Baldwin, PhD 2 

1 University of Washington; 2 Brigham Young University 

Contact: datkins@uw.edu; scott_baldwin@byu.edu 

Implementation research often involves multilevel data (sometimes called hierarchical, clustered, or nested data). 

Examples of such data include patients clustered within providers, providers clustered within sites, and therapist 

fidelity items clustered within therapists. Such multilevel data present a number of design and analysis challenges. 

Our presentation will provide a brief, general overview of multilevel models and then focus on specific challenges 

related to implementation research. Topics will include: 

 

 

 

 

How sample sizes at provider and site levels affect the use of multilevel models 

Advantages and disadvantages of randomizing between or within clusters (e.g., randomizing therapists 

within a site or randomizing sites to treatment condition) 

Power and sample size calculations, including costs (e.g., costs of adding providers vs. sites) and attrition 

How multilevel designs influence the assessment of therapist fidelity, including reliability and psychometric 

considerations 

Our goal is to provide a non‐technical introduction to these topics, emphasizing concepts as opposed to statistics. 

Moreover, we hope to have a highly interactive session with input and questions from the audience. Finally, there 

will be time for general questions on implementation designs at the end of the session, not necessarily specific to 

multilevel data. 


NOTES 


May 17. 2:15‐3:30 

BREAKOUT S: 

NEW IMPLEMENTATION MEASURES 

(MC: Doyanne Darnell, PhD) 

Measuring an Evidence‐Based Model of 

Implementation: Preliminary 


Josef I. Ruzek, PhD 

National Center for PTSD, VA Palo Alto 


Solving Measurement Issues in 

Implementation Science 

Ruben Martinez, BA, & Cara C. Lewis, 

PhD 


Common Elements for Implementing 

Evidence‐Based Practices in Children’s 

Mental Health 

Lisa Saldana, PhD 

Oregon Social Learning Center 



MEASURING AN EVIDENCE‐BASED MODEL OF IMPLEMENTATION: PRELIMINARY DEVELOPMENT OF A SURVEY 

INSTRUMENT 

Josef I. Ruzek, PhD, 1,2 Joan M. Cook, PhD, 1,3 Richard Thompson, PhD, 4 Stephanie Dinnen, MS, 3 James C. Coyne, PhD, 5 

Paula P. Schnurr, PhD, 1,6 

1 National Center for PTSD; 2 Stanford University; 3 Yale School of Medicine; 4 University of Illinois at Chicago; 5 University 

of Pennsylvania & University of Groningen, the Netherlands; 6 Geisel School of Medicine at Dartmouth 

Contact: josef.ruzek@va.gov 

To facilitate testing of comprehensive models of implementation, measures are needed that assess a broad range of 

theory‐based constructs within a single measurement instrument. We developed a measure assessing the six broad 

elements of the Greenhalgh et al. (2005) model: perceived innovation characteristics; individual characteristics of 

potential adopters; communication and influence; system antecedents and readiness; outer context; and 

implementation process. Survey and interview measures were developed, via systematic literature review of 

measures for associated constructs. Keywords representing 53 separate model constructs were searched to identify 

existing measures for each construct. These were assessed for adequate reliability, validity, and applicability to 

healthcare settings. Items meeting these criteria were used to guide survey/interview design; where no measure 

was deemed appropriate, items were created through a consensus‐based process. Approximately two items were 

used to assess each construct, to ensure low respondent burden. The measure was used to assess factors affecting 

implementation of two PTSD treatments (Prolonged Exposure, Cognitive Processing Therapy). All 229 PTSD 

treatment providers in 38 VA residential PTSD treatment settings were asked to complete the measure. 216 (94.3%) 

completed the survey. Internal consistency was generally good and results suggested that the measure may be 

helpful in researching and planning implementation. CPT scored significantly higher than PE on a number of factors 

(including relative advantage, compatibility, trialability, potential for reinvention, task issues, and augmentationtechnical 

support) and lower on perceived risk. The measure has several possible applications for research and 

implementation and can be adapted for other organizations, settings, and innovations. 

Themes: Measurement, Scale‐up 

NOTES 


MEASUREMENT ISSUES IN IMPLEMENTATION SCIENCE 

Ruben Martinez, BA, & Cara C. Lewis, PhD 


Contact: rgm@indiana.edu 

According to Siegel (1964), “Science is measurement”, which begs the question, is measurement necessarily 

scientific? Unfortunately, the answer is “no”; particularly for the field of implementation science, which in its 

nascent state, has become vulnerable to measurement issues that threaten the strength of the developing 

knowledge base. It is as though the demand for the implementation of evidence‐based practices is outpacing 

the science (Chamberlain, Brown, & Saldana, 2011), resulting in measurement that is not always scientific (Cook 

& Beckman, 2006; Proctor et al., 2011; Weiner, 2009). This situation presents an alarming paradox whereby 

investigators are drawing conclusions based off of instruments that have not been psychometrically validated, 

leading to an unstable methodological ground. In order to make interpretations or draw conclusions from data 

and confidently generalize findings to different settings it is imperative that the measures we utilize assess what 

we think they are measuring‐‐a test only repeated analysis and reporting of psychometrics can affirm (Hunsley & 

Mash, 2011). If our measures are not empirically validated, we run the risk of constructing “a magnificent house 

with no foundation” (Achenbach, 2005). Perhaps a bold question is worthy of our consideration: what is the 

value of performing evaluative research if it is not possible to place confidence in your interpretations of the 

data? We will present data from a survey completed by 80 implementation stakeholders who shared their 

perspectives of the most pressing measurement issues in the field. Specifically, we will report on their use of 

measures, theoretical frameworks, mixed‐methods, and recommendations for advancing the science of 

implementation. 


NOTES 



COMMON ELEMENTS FOR IMPLEMENTING EVIDENCE‐BASED PRACTICES IN CHILDREN’S MENTAL HEALTH 

Lisa Saldana, PhD, & Patricia Chamberlain, PhD 

Oregon Social Learning Center 

Contact: lisas@cr2p.org 

With the increased focus and effort to scale‐up evidence‐based practices (EBPs) in real‐world settings comes 

recognition of the complexity of the process, which involves planning, training, quality assurance, and interactions 

among developers, system leaders, practitioners, and consumers. Little is known about which aspects of these 

methods are essential for successful implementation, or how to measure if they have occurred well. The Stages of 

Implementation Completion (SIC), was developed to assess sites’ implementation process and obtainment of 

implementation milestones for Multidimensional Treatment Foster Care (MTFC). The SIC is an 8‐stage tool that maps 

onto three phases of implementation (pre‐implementation, implementation, and sustainability). Items are defined 

by the activities that are identified by the developer/purveyor as necessary to implement MTFC. The SIC is being 

adapted for other EBPs in children’s services that address highly prevalent (e.g., anxiety) and costly (e.g., parenting 

for child welfare populations) mental health and behavioral problems. This presentation will examine the 

common/universal elements of the implementation process that have been identified through the SIC adaptation 

process with multiple EBP developers/purveyors. Preliminary data will be presented to demonstrate the frequency 

with which these elements are completed by adopting sites and the influence that this completion has on successful 

scale‐up. 

Track(s): Measurement, Scale‐Up 

NOTES 


May 17. 2:15‐3:30 

BREAKOUT T: OUTCOMES FROM 

NEW INTERVENTIONS 


Team‐Based Exposure & Ritual 

Prevention for Adults with Obsessive 

Compulsive Disorder: An Open Trial 

Implemented in a Community Mental 

Health Center 

Maria Mancebo, PhD 

Butler Hospital, Brown University 

Implementation of the Family Check‐Up 

in Community Mental Health Agencies 


Child & Family Center, University of 

Oregon 

Cognitive Retraining (CR) for Attention 

& Working Memory for Older Adults: 

What to Train, to Whom, & How Long 

Lee Hyer, PhD, ABPP 

Psychiatry & Family Health, Mercer 

Georgia Neurosurgical Institute & 

Mercer School of Medicine 



TEAM‐BASED EXPOSURE & RITUAL PREVENTION FOR ADULTS WITH OBSESSIVE COMPULSIVE DISORDER: AN 

OPEN TRIAL IMPLEMENTED IN A COMMUNITY MENTAL HEALTH CENTER 

Maria Mancebo, PhD, Gail Steketee, PhD, Jordana Muroff, PhD, Steven Rasmussen, MD, & Caron Zlotnick, PhD 

Butler Hospital, Alpert Medical School at Brown University 

Contact: maria_mancebo@brown.edu 

Objectives: This study assessed the acceptability and feasibility of a portable training program and team‐based 

approach to delivering Exposure and Ritual Prevention (Ex/RP) for OCD in a community mental health center (CMHC). 

Method: Group Ex/RP for OCD was adapted to meet the needs of low‐income adults receiving treatment in a CMHC. 

A training program, treatment manual, and fidelity ratings were developed to train community therapists and case 

managers to deliver Ex/RP. Low‐income individuals with OCD were included if they: identified OCD as their primary 

DSM‐IV disorder, had at least moderate OCD symptoms, were on a stable psychotropic medication regiment, and had 

never received a trial of Ex/RP. Participants received 14 group therapy sessions with a therapist and 10 individual 

coaching sessions with a case manager. Results: Nine participants entered active treatment. Feasibility and 

acceptability of the intervention were high, with two‐thirds (n=6) of patients completing treatment, 89% of 

scheduled sessions attended, and high treatment satisfaction ratings. Two therapists and six case managers 

completed the training program. Fidelity ratings indicated staff were able to effectively deliver manualized Ex/RP. 

Treatment completers experienced significant reductions in symptoms on the Yale‐Brown Obsessive‐Compulsive 

Scale at Posttreatment. Conclusions: Team‐based Ex/RP appears to be a feasible and acceptable intervention for 

individuals receiving treatment for OCD at CMHCs. 

Track(s): Fidelity, Training 

NOTES 


IMPLEMENTATION OF THE FAMILY CHECK‐UP IN COMMUNITY MENTAL HEALTH AGENCIES: CLINICAL 

EFFECTIVENESS, FIDELITY, & OTHER OUTCOMES 

Justin D. Smith, PhD, Elizabeth A. Stormshak, PhD, & Katherine Kavanagh 

Child & Family Center, University of Oregon 

Contact: jsmith6@uoregon.edu 

Objective: We examine the process and outcomes of implementation of the Family Check‐Up (FCU; Dishion & 

Stormshak, 2007) in low resource community mental health agencies serving high‐risk families. The FCU is a 

family‐based intervention shown to improve family management practices and reduce problem behaviors in 

youth ages 2‐18 (e.g., Dishion et al, 2008; Stormshak & Dishion, 2009). The model was also designed for 

implementation in community settings serving families, such as schools, primary care, and community mental 

health. Method: Of the 36 therapists engaged in the study, 17 were randomly assigned to the FCU condition and 

19 to TAU. Therapists in the FCU condition received intensive training followed by ongoing implementation 

support to promote uptake and counter drift. Families were pseudo‐randomly assigned to a therapist in one of 

the conditions: 42 families received the FCU while 32 received community treatment as usual (TAU). Results: ITT 

analysis revealed intervention effects on youths’ conduct problems and increased parental monitoring, which 

differed for male and female youth. Interventionists were found to have positive attitudes toward evidencebased 

practices, a desire to learn evidence‐based approaches, and acceptable implementation fidelity, assessed 

using the COACH observational rating system. Furthermore, variations in fidelity were associated with change in 

specific parenting behaviors. Conclusions: The FCU is a viable and effective family‐based community intervention 

as a precursor to typical community mental health services. The implications of these findings for future scale‐up 

efforts of the FCU will be discussed. 


NOTES 



COGNITIVE RETRAINING (CR) FOR ATTENTION & WORKING MEMORY FOR OLDER ADULTS: WHAT TO TRAIN, 

TO WHOM, & HOW LONG? 

Lee Hyer, PhD, ABPP, 1 Christine Mullin, 2 & Laura McKensie 2 

1 Georgia Neurosurgical Institute & Mercer School of Medicine; 2 Mercer School of Medicine 

Contact: lhyer@ganeuroandspine.com 

It is now accepted that cognitive retraining (CR) improves attention and working memory for older adults. What to 

train, to whom, and how long are now important questions. We conducted three independent clinical studies in a 

medical school addressing different memory programs; cogmed, memory/attention training, and brainwaveR. These 

studies were distinctive, one being computer based (N=64), one attention/memory focused and holistic (N=112), and 

one a structured memory class (N= 24). All subjects had memory complaints ranging from AAMI to mild dementia. 

Pre and post testing were applied in all studies. We had a sham or control group in all three studies; cogmed used 

sham and control; memory/attention used control, and brainwaveR had a control group. We also separated the 

Memory Group by Risk Status, Low, Medium and High. Targets addressed new learning, memory and executive 

functioning, and memory habits, function, and affect. Results showed that the interventions improved on most 

outcomes, especially cognitive and adjustment markers, emphasizing far transfer. We discuss the research and 

practical aspects of the three studies, implementation issues, and lessons learned. 


NOTES 


May 17. 3:45‐5:00 

INTERAGENCY COLLABORATIVE TEAMS 

TO SCALE‐UP EVIDENCE‐BASED 

PRACTICES: PRELIMINARY RESULTS 

FROM A LARGE SCALE IMPLEMENTATION 


Symposium Chair: Gregory A. Aarons, PhD, 1,2 

Interagency Collaborative Teams for Capacity 

Building to Scale‐Up Evidence‐Based Practice 

Michael Hurlburt, PhD, 2,3 Gregory A. Aarons, 

PhD, 1,2 Danielle Fettes, 1,2 Cathleen Willging, 4 

Lawrence A. Palinkas, PhD, 2,3 & Mark J. 

Chaffin 5 

Collaboration, Negotiation, & Coalescence for 

Interagency‐Collaborative Teams to Scale‐Up 

Evidence‐Based Practice 

Gregory A. Aarons, PhD, 1,2 Michael Hurlburt, 

PhD, 2,3 Danielle Fettes, 1,2 Cathleen Willging, 4 

Lara Gunderson, MA, 4 Mark Chaffin, 5 & 

Lawrence A. Palinkas, PhD 2,3 

Leadership & Practice in the Face of Policy: 

How Supervisors & Providers Exercise 

Discretion in Evidence‐Based Practice 


Lara Gunderson, MA 4 & Cathleen Willging 4 


1 University of California, San Diego; 2 Child & 

Adolescent Services Research Center; 

3 University of Southern California; 4 Pacific 

Institute for Research & Evaluation; 5 University 

of Oklahoma Health Science Center 


INTERAGENCY COLLABORATIVE TEAMS FOR CAPACITY BUILDING TO SCALE‐UP EVIDENCE‐BASED PRACTICE 

Michael Hurlburt, PhD, 1,2 Gregory A. Aarons, PhD, 1,3 Danielle Fettes, 1,3 Cathleen Willging, 4 Lawrence A. Palinkas, 

PhD, 1,2 & Mark J. Chaffin 5 

1 Child & Adolescent Services Research Center; 2 University of Southern California; 3 University of California, San Diego; 

4 Pacific Institute for Research & Evaluation; 5 University of Oklahoma Health Science Center 

Contact: gaarons@ucsd.edu 

Background: System‐wide scale up of evidence‐based practice (EBP) in child welfare is a complex process that 

requires consideration of multiple system and organizational levels. Yet few strategic approaches to address such 

concerns exist to support EBP implementation and sustainment across a service system. Building on the Exploration, 

Preparation, Implementation, and Sustainment (EPIS) implementation framework, we developed and are testing the 

Interagency Collaborative Team (ICT) process model to implement an evidence‐based home visitation model within a 

large children’s service system. The ICT model emphasizes the role of local agency collaborations in creating 

structured supports for successful implementation. Methods: In this paper, we describe utilization of the ICT model 

to scale‐up an EBP and present preliminary qualitative results from use of the implementation model. Qualitative 

interviews were conducted to assess challenges in building system, organization, and home visitor capacity to 

implement the EBP. Data collection and analysis centered on EBP implementation issues, as well as the experiences 

of home visitors under the ICT model. Results: Six notable issues relating to implementation process emerged from 

participant interviews, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) 

communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early 

successes. These issues highlight strengths and areas for development in the ICT model. Conclusions: Use of the ICT 

model led to sustained and widespread use of SafeCare in San Diego County. Although some aspects of the 

implementation model may benefit from enhancement, rich qualitative data from implementation of SafeCare in San 

Diego suggest that the process model generates strong structural supports for implementation and creates 

conditions in which tensions between EBP structure and local contextual needs can be resolved in ways that support 

the expansion and maintenance of an EBP while preserving potential for public health benefit. 


NOTES 


COLLABORATION, NEGOTIATION, & COALESCENCE FOR INTERAGENCY‐COLLABORATIVE TEAMS TO SCALE‐ 

UP EVIDENCE‐BASED PRACTICE 

Gregory A. Aarons, PhD, 1,2 Michael Hurlburt, PhD, 2,3 Danielle Fettes, 1,2 Cathleen Willging, 4 Lara Gunderson, MA, 4 

Mark Chaffin, 5 & Lawrence A. Palinkas, PhD 2,3 

1 University of California, San Diego; 2 Child & Adolescent Services Research Center; 3 University of Southern 

California; 4 Pacific Institute for Research & Evaluation; 5 University of Oklahoma Health Science Center 


Objective: Implementation and scale‐up of evidence‐based practices (EBPs) involves multiple stakeholders and a 

process that requires collaboration, negotiation, compromise and a shared vision of improving care. The present 

study examined the complex process involved in EBP scale‐up across an entire service system using the 

Interagency Collaborative Team (ICT) approach and utilizing the EPIS implementation framework. Methods: 

Participants were key stakeholders in a large‐scale county‐wide implementation of an EBP to reduce child 

neglect, SafeCare®. Semi‐structured interviews and/or focus groups were conducted with 27 individuals 

representing various constituents in the service system. A grounded theory approach to qualitative data 

collection and analysis was utilized. Results: Several challenges affected the eventual coalescence of community 

stakeholders in their implementation of SafeCare including, differing organizational cultures, varied 

organizational strategic approaches, differing definitions of collaboration, variations and competing priorities 

across leadership and leaders, resolution of power struggles, role ambiguity, and communication effectiveness. 

While the process resulted in eventual stakeholder coalescence and collaboration, each of the factors identified 

above impacted how stakeholders approached the EBP implementation process. Conclusions: System wide 

scale‐up of EBPs involves multiple stakeholders in a complex process that provides a nexus for differing agendas, 

priorities, leadership styles, and negotiation strategies. The term “collaboration” oversimplifies the multifaceted 

nature of the scale‐up process. Implementation efforts should openly acknowledge and consider the complex 

agendas, priorities, and interaction styles of organizations and individual stakeholders during the Exploration, 

Preparation, and Implementation phases across outer (system) and inner (organizational) contexts. This will 

allow for facilitative resolution of the concerns of each participant in the process. 



NOTES 


LEADERSHIP & PRACTICE IN THE FACE OF POLICY: HOW SUPERVISORS & PROVIDERS EXERCISE DISCRETION IN 

EVIDENCE‐BASED PRACTICE IMPLEMENTATION 

Lara Gunderson, MA, & Cathleen Willging, PhD 

Pacific Institute for Research & Evaluation, University of New Mexico 

Contact: lgunderson@pire.org 

Background: As more community‐based organizations (CBOs) contract with government to deliver public services, we 

analyze how a unique partnership forged among a child welfare agency, a private foundation, and CBOs facilitated 

use of a Collaborative Training and Supervision Model (CTSM) to roll‐out an evidence‐based practice (EBP) to 

decrease child neglect throughout a large county in California. Partners were connected via complex, frequently 

evolving contracts, the contents of which were filtered through the lenses of diverse stakeholders, ranging from 

upper‐level county administrators to those at the frontlines of service delivery. Methods: Qualitative data collected 

via focus groups and semi‐structured interviews were collected and grounded theory analysis methods were utilized. 

Results: Analyses provided insight into how different stakeholders experienced and understood the implementation 

process, allowing for mid‐course corrections and informing replication of both the CTSM and EBP within other service 

systems. Drawing on Michael Lipsky’s theoretical approach to street‐level bureaucracy, we describe how two sets of 

stakeholders directly involved in EBP provision—home visitors and their supervisors—negotiated the constraints of a 

seemingly rigid contract to implement the EBP. We illustrated how both parties also exert discretion in their 

respective efforts to (a) realize their ethical obligation to serve clients, (b) adhere to CBO policies, and (c) function 

under limited financing. Discussion: The sometimes conflicting roles of supervisor and supervisee are complex and 

impacted by factors in both the outer system context and inner organizational context. Such complexity, if not 

resolved, may compromise fidelity to CTSM and EBP implementation requirements. 


NOTES 


May 16. 5:30 pm 

POSTER SESSION 4 

1. Barriers to & Facilitators of Implementation of 

Evidence‐Based Mental Health Treatments 

Adam Chuong, BA, Marlanea E. Peabody, BA, Shannon 

Wiltsey Stirman, PhD, & Jennifer E. Johnson, PhD 

3. Adapting a Research Tested Automated Electronic 

Health Record Intervention (Systems of Support to 

Increase Colorectal Cancer Screening) for 

Implementation in Safety Net Clinics (Strategies & 

Opportunities to Stop Colorectal Cancer in Priority 

Populations) 

Beverly Green MD, MPH, Jennifer DeVoe, MD, DPhil, 

& Gloria D. Coronado, PhD 

5. Development of An Assessment of Organizational 

Readiness for EBP Implementation in Public Child 

Welfare 

Catherine Roller White, MA, Adam Darnell, PhD, Lien 

Bragg, Kirk O'Brien, PhD, & Erin Maher, PhD 

7. Using Qualitative Research to Understand VA 

Provider Perspectives 

Gina M. Signoracci, PhD, Nazanin H. Bahraini, PhD, 

Bridget B. Matarazzo, PsyD, Jennifer H. Olson‐ 

Madden, PhD, & Lisa A. Brenner, PhD 

2. Fidelity Assessment of Widely‐Disseminated but Understudied 

Prevention Programs: A Framework & Illustration from the 

Common Sense Parenting Trial 

W. Alex Mason, PhD, Robert G. Oats, MA, Wendi F. Cross, PhD, 

Mary Casey‐Goldstein, MSEd, Kevin G. Haggerty, PhD, & Koren G. 

Hansen 

4. Adapting a Multidimensional Knowledge Translation Strategy to 

Improve Pediatric Pain Practices in Canada: Evidence‐Based 

Practice for Improving Quality 

Bonnie Stevens, RN, PhD, Janet Yamada, RN, MSc, Carole A. 

Estabrooks, PhD, Jennifer Stinson, PhD, RN‐EC, CPNP, Fiona 

Campbell, MD, FRCA, & Shannon D. Scott, RN, PhD 

6. Development & Use of a Fidelity Checklist for Permanency 

Roundtables: A New Child Welfare Intervention 

Catherine Roller White, MA, Kirk O'Brien, PhD, Tyler Corwin, MA, 

& Anne Buher 

8. Implementation of an HIV Preventive Intervention in Mexico: 

The Roles of Context, Organizational Structure & Process, & 

Community Violence 

Gregory A. Aarons, PhD, Thomas L. Patterson, PhD, Claudia V. 

Chavarin, MD, & Lawrence A. Palinkas, PhD 

9. Utilization of the Hybrid Model to Evaluate an 

Adolescent Treatment Engagement Intervention 

Heather Spielvogle, PhD, & Faye Mishna, PhD 

10. Doulas to Fill the Gap: A Proposed Model of Doula Delivery of 

Cognitive‐Behavioral Therapy for Maternal Anxiety & 

Depression 

Margaret Mary Downey, BA, & Rinad Beidas, PhD 

11. A Mixed‐Methods Approach to the Intersection of 

Attitudes & Organizational Factors by Provider Type 

in Dissemination & Implementation of Evidence‐ 

Based Practice for Child Anxiety 

Margaret Mary Downey, BA, Mark Gallagher, BA, 

Jessica Watkins, BA, Prianna Pathak, Julie Edmunds, 

MA & Rinad Beidas, PhD 

13. Elucidating the Practical Challenges & 

Opportunities for Implementing & Sustaining 

Alcohol Screening & Brief Intervention Services in 

Level I Trauma Centers 

Mijung Park, PhD, MPH, RN, Ashley Jones, Jeff Love, 

BA, Jin Wang, PhD Joan Russo, PhD, Dennis Donovan, 

PhD, Chris Dunn, PhD, Gregory Jurkovich, MD, 

Frederick Rivara, MD, Larry Gentilello, MD, & 


12. Conceptualizing the Dilemmas of Implementation Through a 

Socio‐Cultural Lens: A Qualitative Study Examining Tools, 

People, & Organizations 

Meaghan McCollow, MST, BCBA, Grace Blum, MA, Jacob Hackett, 

MEd, Yelena Patish, & Jennifer Pierce 

14. Identifying the Predictors of Early Versus Late Engagement in a 

Foster Parent Training Intervention 

Natalia Escobar Walsh, MS, & Joseph M. Price, PhD 

(Continued on next page) 

Posters 

4 The poster list includes all authors, but all may not necessarily be present for the poster session. 


May 16. 5:30 pm 

POSTER SESSION CONTINUED 

15. Determining Factors Important in Influencing ASD 

Community Stakeholders Participation in an 

Academic‐Community Collaboration 

Rosemary Meza, BA 

17. Exploring Training Therapist Use of Cognitive 

Behavioral Therapy to the Prediction of Early 

Depression Symptom Reduction 

Taylor Marshall, Brigid Marriott, Sofia Braga, MA, 

Meredith Boyd, Mark Crossen, BA, & Cara C. Lewis, 

PhD 

19. Who Receives Evidence‐Based Psychotherapies? 

Characteristics of Female Veterans from a Practice 

Setting 

Andrea DeVito, Cassidy Gutner, Alexandra Dick, MA, 

Sam Meisel, & Shannon Wiltsey Stirman, PhD 

16. A Multi‐Level Framework for Implementation Science 

Ruben G. Martinez, BA, & Cara C. Lewis, PhD 

18. Community Assessment & EBP Implementation: Illness 

Management & Recovery (IMR) & Integrated Dual Disorder 

Treatment (IDDT) 

Shannon Blajeski, MSW 

20. Comparing Self, Clinician, & Observer Reports of Cognitive 

Processing Therapy (CPT) Adherence 

Samuel Meisel, Amber Calloway, Alexandra Dick, MA, Andrea 

DeVito, Ann Rasmusson, & Shannon Wiltsey Stirman, PhD 

21. Improving our Capacity for Evidence‐Based PTSD 

Treatment: Developing an Effective Model of Post‐ 

Workshop Consultation 

Meredith S. H. Landy, Kelly McShane, Sheena Bance, 

Shannon Wiltsey Stirman, PhD, Josh Deloriea, BA, 

Marta Maslej, & Candice M. Monson, PhD 


1. BARRIERS TO & FACILITATORS OF IMPLEMENTATION OF EVIDENCE‐BASED MENTAL HEALTH 

TREATMENTS 

Adam Chuong, BA, 1 Marlanea E. Peabody, 1 Shannon Wiltsey Stirman, PhD, 2 & Jennifer E. Johnson, PhD 1 

1 Brown University; 2 VA Boston Healthcare System, National Center for PTSD, & Boston University 

Contact: adam_chuong@brown.edu 

Individuals with psychiatric disorders have substantially increased risk of incarceration. There is a critical need to 

incorporate evidence‐based treatment (EBT) into prison mental health treatment. Number of Stakeholders from 

two different prison systems was surveyed to identify barriers and facilitators of EBT implementation. 

Descriptive analyses suggest that stakeholders view rehabilitation as an important aspect of incarceration. 

Furthermore, the majority of stakeholders express a high interest in, awareness of, and concern about 

treatments of depression. Stakeholders rated workload burden and potential inconvenience as the greatest 

obstacles to implementation. Supervision and clinical fit were rated as the greatest facilitators of 

implementation. 


NOTES 

Posters 


2. FIDELITY ASSESSMENT OF WIDELY‐DISSEMINATED BUT UNDERSTUDIED PREVENTION PROGRAMS: A 

FRAMEWORK & ILLUSTRATION FROM THE COMMON SENSE PARENTING TRIAL 

W. Alex Mason, PhD, 1 Robert G. Oats, MA, 1 Wendi F. Cross, PhD, 2 Mary Casey‐Goldstein, MSEd, 3 Kevin G. Haggerty, 

PhD, 3 & Koren G. Hansen 3 

1 Boys Town National Research Institute; 2 University of Rochester Medical Center; 3 University of Washington 

Contact: walter.mason@boystown.org 

Common Sense Parenting (CSP) is a promising and widely disseminated but understudied parenting program that is 

being tested in a recently initiated randomized trial. To facilitate evaluation, programs must be implemented as 

manualized, which requires monitoring of fidelity and feedback for performance improvement. This is challenging for 

programs, like CSP, developed in community contexts, which often lack well‐established fidelity assessment tools and 

protocols that meet rigorous research standards. This presentation presents a framework for fidelity assessment 

relevant for widely used but understudied preventive interventions, and reports fidelity data from the CSP trial. 

Overall, 321 families were recruited into the study and randomly assigned to either the standard CSP program (n = 

118), a modified CSP+ program (n = 95), or a control condition (n = 108). Parent workshops were videotaped, and a 

random sample of videotapes were coded for adherence to the key components of this structured program, and for 

the quality of delivery. There was a high degree of correspondence in the ratings (96% agreement) and adherence 

was high (95%). Quality ratings met overall standards and generally showed improvement from the beginning to the 

end of the intervention phase, as fidelity data were used for supervision and feedback. 


NOTES 


3. ADAPTING A RESEARCH TESTED AUTOMATED ELECTRONIC HEALTH RECORD INTERVENTION (SYSTEMS 

OF SUPPORT TO INCREASE COLORECTAL CANCER SCREENING) FOR IMPLEMENTATION IN SAFETY NET 

CLINICS (STRATEGIES & OPPORTUNITIES TO STOP COLORECTAL CANCER IN PRIORITY POPULATIONS) 

Beverly Green MD, MPH, 1 Jennifer DeVoe, MD, DPhil, 2 & Gloria D. Coronado, PhD 3 

1 Group Health Research Institute; 2 OCHIN; 3 The Center for Health Research, Kaiser Permanente Northwest 

Contact: green.b@ghc.org 

Background: Systems of Support to Increase Colorectal Cancer (CRC) Screening (SOS) was a randomized trial that 

leveraged electronic health record (EHR) data to implement a stepwise intervention to increase CRC screening. 

Compared to usual care (UC), intervention patients were more likely to be current for CRCS in both study years, 

with incremental increases by intervention intensity: UC 26.5%, Automated 50.7%, Assisted 57.7%, and 

Navigated 64.4% (P

4. ADAPTING A MULTIDIMENSIONAL KNOWLEDGE TRANSLATION STRATEGY TO IMPROVE PEDIATRIC PAIN 

PRACTICES IN CANADA: EVIDENCE‐BASED PRACTICE FOR IMPROVING QUALITY 

Bonnie Stevens, RN, PhD, 1 Janet Yamada, RN, MSc, 1,2 Carole A. Estabrooks, PhD, 2 Jennifer Stinson, PhD, RN‐EC, 

CPNP, 1,2 Fiona Campbell, MD, FRCA, 1,2 & Shannon D. Scott, RN, PhD 2 

1 Hospital for Sick Children & University of Toronto, 2 University of Alberta 

Contact: b.stevens@utoronto.ca 

An innovative, multidimensional knowledge translation (KT) approach, the Evidence‐Based Practice for Improving 

Quality (EPIQ), was originally developed and effectively used to reduce nosocomial infection and bronchopulmonary 

dysplasia in Neonatal Intensive Care Units (Lee, et al. 2009). EPIQ was adapted by the CIHR Team in Children’s Pain 

(Stevens et al., 2006‐2011) and evaluated in a broader hospitalized pediatric population in terms of pediatric pain 

practices (assessment and management). EPIQ is guided by the Promoting Action on Research Implementation in 

Health Sciences (PARiHS) framework and rooted in continuous quality improvement (CQI) methods. EPIQ consists of 

two phases: 1.) Preparation Phase: establishing a group of healthcare professional facilitators to promote practice 

change at the unit level, training them to implement the EPIQ strategy, and identifying a pain practice aim as the 

focus of improvement. 2.) Implementation and Change Phase: implementing change processes in rapid Plan‐Do‐ 

Study‐Act (PDSA) cycles; choosing, developing and implementing KT strategies (e.g., audit and feedback) to address 

the unique needs and culture of the unit; and monitoring improvement. Results indicate that EPIQ was successful in 

promoting improved pediatric pain practices in 8 pediatric hospitals across Canada. 


NOTES 


5. DEVELOPMENT OF AN ASSESSMENT OF ORGANIZATIONAL READINESS FOR EBP IMPLEMENTATION IN 

PUBLIC CHILD WELFARE 

Catherine Roller White, MA, Adam Darnell, PhD, Lien Bragg, Kirk O'Brien, PhD, & Erin Maher, PhD 

Casey Family Programs 

Contact: crwhite@casey.org 

Although recent Children’s Bureau waiver and grant applications encourage the use of implementation science 

(e.g., Title IV‐E waiver applications), the child welfare sector has been particularly slow to embrace this 

methodology to examine successful adoption of evidence‐based practice. Implementation science has identified 

a number of factors at the organization and system level that can hinder or promote adoption of evidence‐based 

practice. Further, implementation science is developing a body of literature to examine these factors specifically 

in the child welfare arena. Based on a review of this literature and first‐hand knowledge of challenges faced in 

promoting organizational and system change in public child welfare jurisdictions, we will describe a conceptual 

model of organizational readiness for adoption of evidence‐based practice. This conceptual model serves as the 

foundation for development of a brief instrument to assess organizational readiness for change. The instrument 

is intended for use by internal or external organizational change agents and is expected to serve a dual role to 

support organization development and to provide data for implementation research. A poster session is an ideal 

forum for dialogue as it will assist us in refining the content of the instrument, ultimately maximizing its utility 

for both practitioners and researchers. 


NOTES 

Posters 


6. DEVELOPMENT & USE OF A FIDELITY CHECKLIST FOR PERMANENCY ROUNDTABLES: A NEW CHILD WELFARE 

INTERVENTION 

Catherine Roller White, MA, Kirk O'Brien, PhD, Tyler Corwin, MA, & Anne Buher 

Casey Family Programs 

Contact: crwhite@casey.org 

Permanency Roundtables (PRTs) are intensive case consultations in which action plans are developed to assist 

children and youth in foster care achieve legal permanency (defined as adoption, guardianship, or reunification). To 

date, PRTs have occurred in over 130 jurisdictions in 30 states across the country, and while specific elements are 

considered to be central to the PRT model, there has been no standard instrument with which to measure fidelity. 

The PRT Fidelity Checklist was developed in response to this need. Surveys were administered to 248 case managers 

who participated in PRTs in Alabama, Colorado, Florida, and Ohio in 2010. Using confirmatory factor analysis, four 

subscales were created from 16 items: Engagement, Resources, Identifying Relatives, and Focus. Items that did not 

load onto any factors were discarded or used as single‐item predictors. Predictive analyses examined the degree to 

which fidelity to the PRT model predicts achievement of legal permanency in a sample of 726 youth, controlling for 

youth demographics, risk factors, placement history, and caseworker background and attitudes. We will discuss 

lessons learned about the creation of a fidelity checklist with conference attendees and seek their input on further 

enhancement of the instrument. 


NOTES 


7. USING QUALITATIVE RESEARCH TO UNDERSTAND VA PROVIDER PERSPECTIVES 

Gina M. Signoracci, PhD, Nazanin H. Bahraini, PhD, Bridget B. Matarazzo, PsyD, Jennifer H. Olson‐Madden, PhD, 

& Lisa A. Brenner, PhD 

Veterans Integrated Service Network (VISN) 19 Mental Illness Research, Education, & Clinical Center (MIRECC); 

University of Colorado Denver, School of Medicine, Department of Psychiatry 

Contact: Gina.Signoracci@va.gov 

Background: Operation Enduring Freedom (OEF) and Operation Iraqi Freedom (OIF) military personnel are 

presenting to the Veterans Affairs (VA) healthcare system with complex needs that may require utilization of 

various service lines for comprehensive treatment. This study aimed to gather information from VA 

professionals regarding necessary resources to provide services to OEF/OIF Veterans. Thirteen semi‐structured 

interviews were conducted to describe provider perspectives regarding: OEF/OIF Veteran clinical needs; 

collaboration and referral processes; barriers to providing treatment; needs and resources regarding service 

delivery; psychiatric outcomes; and professional satisfaction. 

Methods: De‐identified interviews were transcribed in their entirety and cross‐checked for accuracy. Members 

of the research team first identified themes in the above‐listed domains independently, and then met as a group 

to achieve thematic consensus. Saturation was considered to have been reached at the point at which no new 

information or themes were identified. 

Results: Themes included acuity and intensity of OEF/OIF distress and symptoms, need for professional trainings, 

and a sense of purpose with regard to providing services to this cohort. In‐depth information regarding themes 

and select quotes will be presented. 

Conclusions: Study results may help inform VA healthcare systems about provider experiences and service needs 

of OEF/OIF Veterans. 


NOTES 

Posters 


8. IMPLEMENTATION OF AN HIV PREVENTIVE INTERVENTION IN MEXICO: THE ROLES OF CONTEXT, 

ORGANIZATIONAL STRUCTURE & PROCESS, & COMMUNITY VIOLENCE 

Gregory A. Aarons, PhD, 1 Thomas L. Patterson, PhD, 1 Claudia V. Chavarin, MD, 1 & Lawrence A. Palinkas, PhD 2 

1 University of California, San Diego, Department of Psychiatry; 2 University of Southern California 


This study describes the initial implementation of Mujer Segura (Healthy Woman), an evidence‐based HIV/STI 

preventive intervention for female sex workers in Mexico. Our conceptual framework addresses outer (system) and 

inner (organizational) contextual factors that can enhance or limit effective implementation. Mixed 

quantitative/qualitative methods are being used to assess the train‐the‐trainer implementation model being utilized 

in twelve cities and eight states in Mexico. Mujer Segura is currently being implemented at thirteen sites throughout 

Mexico. We conducted focus groups and interviews with participants at multiple organizational levels (i.e., site 

directors, physicians, nurses, outreach workers) and completed observations of meetings and interactions of study 

stakeholders. For the present study we analyzed focus group data using grounded theory techniques to identify 

predominant themes and issues impact implementation progress. Preliminary results indicate that early 

implementation was influenced by variability in leadership, team member commitment to effective outreach, 

workforce stability across organization levels, creative problem solving, and a facilitative and supportive work 

environment. Variability in state level policies and community violence also impacted implementation progress and 

effectiveness. Our examination of contextual, organizational, and individual implementation factors will inform the 

further development of strategies to support effective evidence‐based practice implementation and sustainment in 

low‐resourced settings. 

Track(s): Global Perspectives, Training 

NOTES 


9. UTILIZATION OF THE HYBRID MODEL TO EVALUATE AN ADOLESCENT TREATMENT ENGAGEMENT 

INTERVENTION 

Heather Spielvogle, PhD, 1 & Faye Mishna, PhD 2 

1 University of Washington; 2 University of Toronto 

Corresponding author contact: spielvog@uw.edu 

Premature service dropout is a common problem in child and adolescent community mental health settings. 

Blending elements of efficacy and effectiveness research, this research used a hybrid model (Carroll & 

Rounsaville, 2003) to explore the impact of an engagement intervention (i.e., motivational interviewing) on 

adolescent treatment attendance in partnership with four Toronto‐based community mental health centers. 

The results of this randomized pilot study (n=51) demonstrated that the engagement intervention had a 

medium effect (d=.51) on initial treatment attendance. This poster will focus on the outcomes of the 

intervention on mediating variables (i.e., self‐efficacy, autonomy, and alliance) and initial treatment attendance. 

This poster will also demonstrate the process of collaborating with agency leadership and clinical staff from 

stakeholder engagement to follow‐up. 


NOTES 

Posters 


10. DOULAS TO FILL THE GAP: A PROPOSED MODEL OF DOULA DELIVERY OF COGNITIVE‐BEHAVIORAL 

THERAPY FOR MATERNAL ANXIETY & DEPRESSION 

Margaret Mary Downey, BA, & Rinad Beidas, PhD 


Contact: mdowney@upenn.edu 

Evidence‐based interventions (EBIs) have grown exponentially in recent decades. However, calls continue for broader 

adoption in the United States in order for their potential to be affordably and effectively realized. Proposals to close 

the development‐implementation gap include brief interventions (O’Connor and Whaley 2003) and utilizing 

paraprofessionals (Rotheram‐Borus et al 2012). One sector of consideration for such proposals is maternal mental 

health. It remains a pressing issue in EBIs given the prevalence of maternal anxiety and depression and their 

association with other health indicators for mother, partner, and newborn (Patel 2004). Using task‐shifting/sharing 

approaches, this concept poster outlines a hypothetical model of Cognitive Behavioral Therapy (CBT) delivery, an EST 

for anxiety and depression, by doulas. Doulas are trained individuals who provide emotional, physical, and 

informational support to clients across pregnancy. Continuous labor support as offered by doulas is a demonstrated 

effective practice for improving several maternal health outcomes (e.g., shortening labors, lowering instrumental 

intervention rates, improving newborn health factors and maternal self‐esteem and satisfaction) and continues to 

grow in popularity (Hodnett 2007). Doulas operate as “paraprofessionals”, e.g., agents of knowledge transfer 

between mother and clinical staff and sharing support duties. Doula provision of CBT effectively and efficiently 

complements doula care models to potentially fill a critical EBI gap. 


NOTES 


11. A MIXED‐METHODS APPROACH TO THE INTERSECTION OF ATTITUDES & ORGANIZATIONAL FACTORS 

BY PROVIDER TYPE IN DISSEMINATION & IMPLEMENTATION OF EVIDENCE‐BASED PRACTICE FOR CHILD 

ANXIETY 

Margaret Mary Downey, BA, 1 Mark Gallagher, BA, 2 Jessica Watkins, BA, 3 Prianna Pathak, 3 Julie Edmunds, MA 2 & 

Rinad Beidas, PhD 1 

1 University of Pennsylvania; 2 Temple University; 3 Bryn Mawr College 

Contact: mdowney@upenn.edu 

As research on dissemination and implementation of evidence‐based practice (EBP) proliferates, understanding 

contextual characteristics that influence implementation is critical. Two levels of the ecological model, therapist 

and organizational, are worthy of consideration, as quantitative evidence suggests attitudes and organizational 

characteristics remain significant factors in implementation (Aarons 2006). However, little research explores the 

relationship between therapist attitudes toward EBPs, therapist type (e.g., doctoral vs. non‐doctoral), 

organizational factors, and their impact on dissemination and implementation (Aarons 2004). Previous work 

indicates doctoral providers may endorse differences in implementation factors including adequacy of resources 

and motivation for change compared to non‐doctoral counterparts (Downey et al 2012). The current study 

approaches this question using a mixed‐methods framework. Participants include 50 therapists trained in an EBP 

for child anxiety two years prior (Beidas, Edmunds, Marcus & Kendall, 2012). We conducted semi‐structured 

qualitative interviews to examine themes and relationships between attitudes and organizational characteristics 

stratified by therapist type (doctoral (6) vs. non‐doctoral (44)). We developed and applied a comprehensive 

coding scheme to produce fine‐grained descriptive analyses of the role of organizational and therapist‐level 

characteristics in implementation. To assess reliability and robustness, a sample of transcripts was separately 

coded by researchers to compare applications of the scheme. 

This research was supported by NIMH grant: F31 MH083333 


NOTES 

Posters 


12. CONCEPTUALIZING THE DILEMMAS OF IMPLEMENTATION THROUGH A SOCIO‐CULTURAL LENS: A 

QUALITATIVE STUDY EXAMINING TOOLS, PEOPLE, & ORGANIZATIONS 

Meaghan McCollow, MST, BCBA, Grace Blum, MA, Jacob Hackett, MEd, Yelena Patish, & Jennifer Pierce 


Contact: meaghm@uw.edu 

With the emphasis placed on evidence‐based practices (EBPs) in special education (IDEiA, 2004) and the gap that 

exists between research and practice (Fixsen et al., 2005; Odom, 2008), it is important to know how evidence‐based 

practices are understood by implementers. 

A framework was developed that drew upon socio‐cultural theory of practice (Rogoff, Baker‐Sennett, Lacasa, & 

Goldsmith, 1995). The framework provides a means of analyzing factors of implementation related to implementers 

(people), classrooms/schools/districts (organizations), and methods of assessment/curriculum/practice (tools). A 

qualitative study was then conducted to examine how special education teachers and administrators in five school 

districts interpret and make decisions about the implementation of EBPs. The findings indicate the complexities and 

factors that shape practitioners’ interpretations and actions related to the implementation of evidence‐based 

practices. The analysis of the data using the framework made evident potential variables that might influence the 

implementation of EBPs in schools. We will propose and discuss the implications for how this conceptual framework 

can deepen our understanding of the complexities of implementation. 


NOTES 


13. ELUCIDATING THE PRACTICAL CHALLENGES & OPPORTUNITIES FOR IMPLEMENTING & SUSTAINING 

ALCOHOL SCREENING & BRIEF INTERVENTION SERVICES IN LEVEL I TRAUMA CENTERS 

Mijung Park, PhD, MPH, RN, 1 Ashley Jones, 2 Jeff Love, BA, 2 Jin Wang, PhD 2 Joan Russo, PhD, 2 Dennis Donovan, 

PhD, 2 Chris Dunn, PhD, 2 Gregory Jurkovich, MD, 2 Frederick Rivara, MD, 2 Larry Gentilello, MD, 2 & Douglas Zatzick, 

MD 2 

1 University of Pittsburgh School of Nursing, 2 University of Washington School of Medicine 

Contact: parkm@pitt.edu 

Alcohol‐use is common among patients who are admitted to a Level I trauma centers. The American College of 

Surgeons’ mandated level I trauma centers screen injured patients for an alcohol‐use problems provide an 

intervention to those who screen positive. However, marked variability has been observed in prior nationwide 

surveys of trauma center readiness to implement alcohol screening and brief intervention (SBI). 

The purpose of this paper is to elucidate the challenges and opportunities for implementing and sustaining 

alcohol SBI at trauma centers. We conducted semi‐structured telephone interviews with clinicians from 

intervention (n=10) and control (n=8) trauma centers that were part of a multi‐site randomized clinical trial 

designed to test the implementation of high quality alcohol SBI. Transcribed interviews were organized and 

managed using the Atlas‐Ti computer program. Thematic analysis was used as an analytic method. 

The ability to implement alcohol SBI across sites varied markedly, yet occurred in all sites in response to the 

American College of Surgeons mandate. Overall, intervention sites reported that receiving implementation 

guidance was helpful for sustaining SBI services after study completion. Practical challenges occurred at all sites 

and included lack of implementation resources and/or existing resources that were not well aligned with SBI 

implementation needs. 


NOTES 

Posters 


14. IDENTIFYING THE PREDICTORS OF EARLY VERSUS LATE ENGAGEMENT IN A FOSTER PARENT TRAINING 

INTERVENTION 

Natalia Escobar Walsh, MS, 1 & Joseph M. Price, PhD 2 

1 SDSU/UCSD Joint Doctoral Program in Clinical Psychology; 2 Department of Psychology, San Diego State University 

Contact: nataliawalsh@gmail.com 

While several studies have identified predictors of participant engagement, most have measured engagement 

through attendance and none have examined engagement among foster parents. During an effectiveness trial, the 

KEEP intervention reduced foster child behavior problems by providing parent training to foster parents in a group 

setting. In the current study, pre‐intervention foster parent and child characteristics in the KEEP intervention group 

(n = 359) were used to predict process‐oriented engagement (e.g., homework completion, level of participation) in 

early and later stages of the intervention. Regression analyses showed that the number of baseline foster child 

behavior problems (b = 0.01, SE, = 0.01, p = .029) was associated with early engagement (R2 = .03, F(2, 317) = 5.40, p 

= .005). Baseline foster parent distress (b = 0.16, SE = 0.07, p = .03) and number of children in the home (b = ‐0.03, SE 

= 0.02, p = .029) were associated with late engagement (R2 = .04, F(3, 302) = 4.78, p = .003). These findings provide 

an opportunity to give targeted recommendations to child welfare agencies that will facilitate the effective 

dissemination and implementation of the KEEP intervention. 


NOTES 


15. DETERMINING FACTORS IMPORTANT IN INFLUENCING ASD COMMUNITY STAKEHOLDERS 

PARTICIPATION IN AN ACADEMIC‐COMMUNITY COLLABORATION 

Rosemary Meza, BA 

San Diego State University & Child & Adolescent Services Research Center 

Contact: Rosemarydmeza@gmail.com 

Successful academic‐community collaborations (ACCs) increase communication, cooperation and trust between 

researchers and community stakeholders, generate feasible and useful innovations, and help to close the gap 

between research and community practice. However, factors influencing an individuals’ decision to participate 

in an ACC are not well understood. ASD community stakeholders, previously contacted to participate in an ACC, 

completed the Decision to Participate questionnaire (DPQ). Ten ACC participants and 8 non‐participants 

completed the DPQ, which asks individuals to rate the importance of items selected a priori in their decision to 

participate in an ACC or not. Using independent sample T‐tests, four items were found to be statistically and 

meaningfully different between the groups. ACC participants rated networking with other providers (p=.007; 

effect size (ES)=1.74), the fit of collaboration with agency philosophy (p=.011; ES=1.31), and the opportunity for 

future training/consultations (p=.034; ES=1.16) as factors more important in their decision to participate in the 

ACC than non‐participants. Non‐participants reported the number of requests to participate in research more 

important in their decision to participate (p=0.15; ES=1.48) than participants. Considering the networking 

opportunities, collaboration philosophy, opportunities for training, and amount of research requests being made 

of community stakeholders may aid in building and sustaining successful ACCs. 


NOTES 

Posters 


16. A MULTI‐LEVEL FRAMEWORK FOR IMPLEMENTATION SCIENCE 

Ruben G. Martinez, BA, & Cara C. Lewis, PhD 


Contact: rgm@indiana.edu 

Over 60 theoretical implementation frameworks exist (Tabak et al, 2012). These frameworks range from heuristic 

models, or models that detail what constructs are important to measure, to process models, which detail a temporal 

sequence of implementation. Even so, a gap in the framework literature has been identified. No testable theoretical 

framework exists that simultaneously delineates the temporal process of an implementation, acknowledges the 

multiple stakeholder levels ripe for investigation, and situates the constructs in the stage and level within which they 

are implicated. The author extracted, utilized, and combined aspects from three well‐established theoretical 

frameworks: the Consolidated Framework for Implementation Research (Damschroder, et al., 2009), Implementation 

Outcomes (Proctor et al., 2010), and the Stages of Implementation (Fixsen, et al., 2006). The proposed framework 

will detail not only what constructs to measure at what time, but at what stakeholder level (client, provider, 

administration, organization) and the relation between constructs at specific points in the implementation. The 

proposed framework will be populated with instruments identified in the SIRC Instrument Review resulting in a 

practical guide for stakeholders. The proposed framework will be presented using two contrasting examples of 

implementation in mental health. Implications for the field will be discussed. 


NOTES 


17. EXPLORING TRAINING THERAPIST USE OF COGNITIVE BEHAVIORAL THERAPY TO THE PREDICTION OF 

EARLY DEPRESSION SYMPTOM REDUCTION 

Taylor Marshall, Brigid Marriott, Sofia Braga, MA, Meredith Boyd, Mark Crossen, BA, & Cara C. Lewis, PhD 


Contact: taymarsh@indiana.edu 

In an effort to address the well‐documented science‐practice gap, implementation researchers have focused on 

training doctoral student therapists to deliver empirically supported treatments, such as cognitive behavioral 

therapy (CBT), that deliver rapid early symptom changes for clients. This current study sought to identify 

components of CBT responsible for early changes in depressive symptoms in order to isolate which core features 

are ripe for dissemination. Participants were advanced doctoral student therapists (n=6), and clients (n=14) who 

presented with an Axis I diagnosis at a psychotherapy training clinic. The training model included a weekly 

didactic session and 1.5 hours of weekly group supervision over one school semester (approximately 16 weeks). 

Each therapy session was video recorded and coded using the Collaborative Study Psychotherapy Rating Scale to 

identify use of Cognitive therapy elements. The Beck Depression Inventory or Patient Health Questionaire‐9 

were completed by clients at each session. We hypothesize the Behavioral Methods/Homework subscale will be 

most closely related to early depressive symptom reduction. Results will be presented with respect to 

implications for the dissemination of CBT in a community mental health setting. 


NOTES 

Posters 


18. COMMUNITY ASSESSMENT & EBP IMPLEMENTATION: ILLNESS MANAGEMENT & RECOVERY (IMR) & 

INTEGRATED DUAL DISORDER TREATMENT (IDDT) 

Shannon Blajeski, MSW 

University of Washington School of Social Work 

Contact: blajes@uw.edu 

Beginning in 2010, The Washington Institute for Mental Health Research & Training, part of the University of 

Washington Department of Psychiatry & Behavioral Sciences, Division of Public Behavioral Health & Justice Policy, 

worked to implement pilots of the Illness Management & Recovery (IMR) and Integrated Dual Disorder Treatment 

(IDDT) models in Washington State. IMR is a manualized module‐based psychoeducational wellness program while 

IDDT is a team‐based community mental health treatment program for adults with severe mental illness and cooccurring 

substance use disorders. Both are evidence‐based practices linked to positive outcomes for adults with 

both Schizophrenia‐spectrum and Bipolar disorders who have significant functional impairments in the community. 

With annual fiscal support from the State of Washington Division of Behavioral Health & Recovery, the Institute 

worked to develop these EBP’s at the agency level including initial conversations, assessment of desire/fit, training, 

consultation and fidelity measures. 

The poster session will explore these two EBP pilots with a focus on community assessment at the agency and 

Regional Support Network (RSN) level as a necessary first step to the implementation process. Initial program fidelity 

data and first‐hand experience will be presented and linked to the level of community assessment that was granted 

within the scope of the project’s funding. Lessons learned and future ideas will be presented. 


NOTES 


19. WHO RECEIVES EVIDENCE‐BASED PSYCHOTHERAPIES? CHARACTERISTICS OF FEMALE VETERANS FROM 

A PRACTICE SETTING 

Andrea DeVito, 1 Cassidy Gutner, 1 Alexandra Dick, MA, 2 Sam Meisel, 1 & Shannon Wiltsey Stirman,PhD 1,2,3 

1 Boston University; 2 VA Boston Healthcare System; 3 National Center for PTSD 

Contact: adevito@bu.edu 

The Department of Veterans Affairs is working to implement the use of Evidence‐Based Psychotherapies (EBPs) 

in their clinics to returning veterans that seek psychological treatment. A key implementation challenge is 

estimating the penetration of EBPs into mental health systems. The goal of our study was to determine the 

proportion of women in a VA clinic who a) were offered an EBP, b) engaged in EBPs, and c) to understand why 

others were not receiving these treatments. Using a program evaluation tool designed to be feasible for delivery 

in a clinical practice setting, and including validated symptom, satisfaction, and functioning inventories, we 

surveyed 165 women Veterans and their providers over 13 months during intake and follow up appointments. 

Of those women, 58 presented for individual psychotherapy and 55 were offered an EBP through the clinic. 

Clinicians identified 14 individuals who refused treatment, were not judged stable enough for trauma‐focused 

treatment, or were noncompliant. We found that the most common form of EBPs offered were Cognitive 

Processing Therapy (CPT) and cognitive behavioral therapy (CBT), with 37.9% of women reporting engaging in 

one of these treatments after intake. We also found that a significant proportion of women received a different 

type of treatment not listed on the patient and clinician forms. We compared characteristics of women 

Veterans to eligibility criteria for EBPs offered in the clinic. We also compared a subset of these instruments to 

clinical documentation to better understand what occurred in session when they did and did not receive EBPs. 

The data gathered from our study hopefully will shed light on EBP penetration in a VA setting as well as why 

some women veterans do not receive EBPs. 


NOTES 

Posters 


20. COMPARING SELF, CLINICIAN, & OBSERVER REPORTS OF COGNITIVE PROCESSING THERAPY (CPT) 

ADHERENCE 

Samuel Meisel, 1 Amber Calloway, 2 Alexandra Dick, MA, 3 Andrea DeVito, 1 Ann Rasmusson, 1,3 & Shannon Wiltsey 

Stirman, PhD 1,3,4 

1 Boston University; 2 UMass Boston; 3 National Center for PTSD, 4 VA Boston Healthcare System 

Contact: meisels@bu.edu 

Most previous research has assessed treatment fidelity using time‐and cost‐intensive observer ratings. In line with 

the theme of this year’s conference, the current study looks to offer a new, more efficient way of analyzing fidelity of 

treatment for CPT for PTSD. The objectives of the current study are to (1) assess the degree in which clinicians, 

clients, and an objective rater agree on a newly developed CPT adherence measure and (2) explore its psychometric 

properties. The current study will assess agreement between clients (n> 50), clinicians (n>10), and observers (n=3). 

Reliability of the new measure will be determined through an intraclass correlation coefficient and criterion‐related 

validity will be assessed a comparison of the measure to the CPT Adherence and Competence Observer Rating Scale. 

We will conduct a preliminary evaluation of the predictive validity of this measure by examining the association 

between adherence and PTSD symptom change. Validation of the self‐report measure could greatly improve the 

efficiency of monitoring adherence, a key implementation outcome. 


NOTES 


21. IMPROVING OUR CAPACITY FOR EVIDENCE‐BASED PTSD TREATMENT: DEVELOPING AN EFFECTIVE 

MODEL OF POST‐WORKSHOP CONSULTATION 

Meredith S. H. Landy, 1 Kelly McShane, 1 Sheena Bance, 2 Shannon Wiltsey Stirman, PhD, 3 Josh Deloriea, BA, 1 

Marta Maslej, 1 & Candice M. Monson, PhD 1 

1 Ryerson University; 2 Ontario Institute of Studies in Education (OISE), University of Toronto; 3 VA Boston 

Healthcare System, National Center for PTSD, & Boston University 

Contact: meredith.landy@psych.ryerson.ca 

Current best practice in training clinicians in the administration of evidence‐based psychotherapies (EBP) 

includes workshop attendance followed by post‐workshop consultation. In spite of data suggesting that 

consultation is important, little is known about what makes for successful consultation. The aim of this study is 

to understand how consultation changes the behaviors of clinicians learning to administer Cognitive Processing 

Therapy, an EBP for Posttraumatic Stress Disorder. We will undertake a qualitative analysis of consultant and 

clinician behaviors and interactions, and develop and test a theory detailing the context and mechanisms that 

make for successful consultation. Consistent with the theme of this year’s conference, in this presentation we 

will describe the challenges we faced in our efforts to develop a model of clinical consultation. Specifically, we 

will describe how we selected which consultation calls to code, the process of developing the plan for analyzing 

data, and used how we used this early data to further refine the plan. We will also discuss how we used a realist 

approach, a paradigm frequently adopted by social scientists, to develop and test a theory describing the 

context and mechanisms that make for successful consultation. The strengths and limitations of this approach 

will be discussed. 

Track(s): Global Perspectives, Training 

NOTES 

Posters 


ADDITIONAL NOTES 










RESTAURANTS NEAR HOTEL DECA 

Agua Verde Cafe 

(206) 545‐8570 

1303 NE Boat Street, Seattle, WA 98105 

Espresso ‐ Monday‐Friday 7:30 am to 2:00 pm 

Monday‐Saturday 11:00 am to 9:00 pm 

http://www.aguaverde.com/ 

Bulldog News & Fast Expresso 

(206) 632‐6397 

4208 University Way NE 

Newstand open daily 8am to 8pm 

Weekdays 6:30am to 7pm; Weekends 8am to 7pm 

www.bulldognews.com 

Chaco Canyon Organic Café 

(206) 522‐6966 

4757 12ave NE 

Seattle, WA 98105 

http://chacocanyoncafe.com/ 

Costas Restaurant 

(206) 633‐2751 


Open Everyday 7am to 10pm 

www.costasontheave.com 

Earl's on the Ave 

(206) 525 4493 


Open Everyday 11am to 2am 

www.earlsuw.com 

Pagliacci Pizzeria 

(206) 726‐1717 


Monday‐Thursday 11am to 11pm Friday‐ Saturday 11am to 

12am 

www.pagliacci.com 

Shultzy's 

(206) 548‐9461 


Monday‐Saturday 10am to 2am 

Sunday 10am to 12am 

www.shultzys.com 

Which Wich? 

(206) 588‐0471 

4730 University Way NE #102 

Monday‐Thursday 11am to 8pm Friday‐Saturday 11am to 9pm 

Sunday 11am to 8pm 

www.whichwich.com 

Big Time Brewery 

(206) 545‐4509 


Monday‐Thursday 11:30am to 12:30am (kitchen closes at 

11pm) 

Friday‐Saturday 11:30am to 1:30am (kitchen closes at 12am) 

www.bigtimebrewery.com 

Cafe Allegro 

(206) 633‐3030 


Monday‐Friday 6am to 10pm 

Saturday 7:30am to 10pm Sunday 8am to 10pm 

www.cafeallegro.com 

Continental Restaurant & Pastry Shop 

(206) 632‐4700 


Open Everyday 7pm to 11pm 

Dick's Drive In 

(206) 633‐2751 

111 NE 45th St 

Open Everyday 10am to 2am 

www.ddir.com 

Ivar's Salmon House 

(206) 632‐0767 

401 NE Northlake Way 

Monday‐Thursday 11am to 9:30pm 

Friday‐Saturday 11am to 10pm Sunday 9:30am to 9:30pm 

www.ivars.com 

Qdoba Mexican Grill 

(206) 547‐0803 1200 NE 45th St 

Monday‐Saturday 10am to 11pm 

Sunday 11am to 10pm 

www.qdoba.com 

Village Sushi 

(206) 985‐6870 4741 12th Ave NE 

Tuesday‐Friday 11:30am to 2pm, 5pm to 9:30pm Saturday 

12pm to 9:30pm Sunday 5pm to 9pm 

www.villagesushiseattle.com 


9th Ave N 

8th Ave N 

D exter Ave N 

A urora Ave N 

6th Ave N 

Taylor Ave N 

5th Ave N 

Roy St 

To Lake Union 

W Republican St 

W Harrison St 

W Thomas St 

John St 

Elliott Avenue 

Queen Ann e 

Ave N 

1st Ave N 

SEATTLE 

CENTER 

2nd Ave N 

Chihuly 

Garden 

& Glass 

Pacific 

Science 

Center 

EMP 

Museum 

Space 

Needle 

Mercer St 

Republican St 

Harrison St 

Denny Way 

Thomas St 

John St 

Westlake Ave N 

Ter y r Ave N 

B oren Ave N 

F a irvie w Ave N 

7. 

M inor Ave N 

Terminal 91 

Olympic 

Sculpture Park 

Pier 70 

Victoria Clipper 

NORTH 

Major Attractions 

Pier 69 

Pier 67 

Broad St 

Clay St 

Alaskan Way 

Cedar St 

Vine St 

Bell Harbor International 

Conference Center 

Pier 66/Bell St. 

Cruise Terminal 

Ferry to Tillicum Village 

Tilikum Place 

Wall St 

Elliott Bay 

Battery St 

Pier 62 & 63 

Bell St 

Western Ave 

Ferry to Bainbridge Island 

Bus/Light Rail Tunnel Stops 

South Lake Union Streetcar 

Parks 

5th Ave 

Regrade Park 

Blanchard St 

Pier 59 

4th Ave 

Lenora St 

1st Ave 

Seattle Aquarium 

Waterfront Park 

Seattle Great Wheel 

Ferry to Bremerton 

Steinbrueck 

Park 

Pier 57 

Pier 56 

Pier 55 

Ferry to Vashon Island 

7th Ave 

6th Ave 

2nd Ave 

3rd Ave 

PIKE PLACE MARKET 

Pier 54 

Pier 52 

8th Ave 

WA State 

Ferries 

Pier 48 

Pine St 

Pike St 

9th Ave 

Union St 

Post Alley 

Virginia St 

Westlake 

Park 

Terry Ave 

Stewart St 

Olive Way 

University St 

Bus 

Terminal 

Seneca St 

Spring St 

Marion St 

Madison St 

Columbia St 

Pioneer 

Square Park 

Occidental 

Square 

1st Ave S 

Minor Ave 

Occidental Ave S 

Howell St 

4th Ave 

PIONEER 

SQUARE 

Yale Ave 

WASHINGTON STATE 

CONVENTION CENTER 

5th Ave 

City Hall 

Park 

2nd Ave S 

6th Ave 

Safeco Field 

(baseball) 

Freeway Park 

7th Ave 

8th Ave 

Jefferson St 

3rd Ave S 

King St. 

Station 

AMTRAK 

CenturyLink 

Field 

(soccer 

& 

football) 

CenturyLink 

Event Center 

Boren-Pike-Pine Park 

Terrace 

9th Ave 

Terry Ave 

Cherry St 

James St 

Kobe 

Terrace 

Park 

S Dearborn St 

Summit Ave 

Boren Ave 

Hing Hay Park 

CHINATOWN– 

INTERNAT’L DISTRICT 

International 

Children’s Park 

M aynard Ave S 

Royal Brougham Wy 

Boylston Ave 

Minor Ave 

Alder St 

Yesler Way 

B road way Ave 

S Washington 

7th Ave S 

S Main 

S Jackson 

S King 

S Weller 

S Lane

SEATTLE IMPLEMENTATION RESEARCH CONFERENCE (SIRC) 

MAY 16‐17, 2013 

PROGRAM EVALUATION 

Please complete this evaluation form and return it to the SIRC Registration Desk prior to your departure. 

The planning committee appreciates your reaction and suggestions to develop and improve future conferences. 

Thank you! 

Answer each by circling the appropriate number, per the following scale. Please provide additional information in 

the space provided. 

Scale 1 = Strongly Disagree 2 = Disagree 3 = Neutral 4 = Agree 5 = Strongly Agree 

1. Overall, the Seattle Implementation Research Conference met my expectations. 

1 2 3 4 5 

Why or Why Not? 

2. The Panel Discussions, Breakout Sessions, and Materials were useful. 

1 2 3 4 5 


3. I will recommend this conference to my colleagues. 

1 2 3 4 5 


4. The registration process and conference administration were effective. 

1 2 3 4 5 


5. Compared to other implementation‐focused conferences recently, SIRC offered a unique 

experience that made me glad I attended. (Please highlight below what contributed to your 

response) 

1 2 3 4 5 


6. What talks or topics were of particular use for you? 

7. What talks or topics were not particularly helpful or did not belong in this conference? 

8. What topics would you like to have covered at the next SIRC conference? 

9. What speakers would you like to hear from at the next SIRC conference? 

10. Apart from the conferences themselves, what ventures would you recommend SIRC take on in the next 1‐2 

years? (If you are interested in participating in such a venture should we take it on, please include your email so we 

can include you.) 

11. Was there anything you want to complement or raise a concern about regarding the facilities?

Conference Program PDF - SIRC : Seattle Implementation Research ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?