Evaluation Findings - SAMHSA Store - Substance Abuse and Mental ...

National Evaluation Study Components
Measures
Domain Instrument Phases Used
Services Received Multi-Sector Service Contacts Form II–V
Service Experience &
Satisfaction
Family Satisfaction Questionnaire (FSQ); Youth
Satisfaction Questionnaire (YSQ) (Brunk, Santiago,
Ewell, & Watts, 1997)
Youth Services Survey (YSS), Youth Services Survey–
Family (YSS–F) (Brunk, Koch, & McCall, 2000)
I–III
IV–V
Cultural Competence Culturally Competent Service Provision (CCSP) IV–V
Services and Costs Study
Using existing cost data in agency management information systems [MIS] and budgets, this study
describes the types of services used by children and families, their utilization patterns, and the
associated costs. Changes were made to this study after 2005 to provide communities with standard
templates for data about flexible fund expenditures, and to further standardize the delivery of crossagency
service and cost data. Because access to service and cost data varies among communities,
this study has used a tiered approach which accommodates availability of information from a single
child-serving system and multiple child-serving systems is employed.
Sustainability Study
Added in 2000 to assess communities 5 years post-funding, this study obtains information about (a)
availability of specific services in the system of care, (b) implementation of system of care principles,
(c) achievement of objectives related to system of care implementation, (d) role and impact of various
factors on the development or maintenance of the system of care, and (e) effectiveness of various
general and financing strategies for sustaining systems of care. Phase I communities were assessed 5
years post-funding and communities funded in 1997 were assessed in their final year of funding with a
Web survey and telephone interviews with key individuals in the communities and at the State level.
Communities funded in 1998–2000 were assessed with the Web survey in their final year of funding;
communities funded from 2002 to 2006 are assessed during the fiscal years in which the Federal–local
funding match requirements change (Years 3 and 4), and in their final year.
Comparison Studies
Comparison studies using quasi-experimental designs were conducted in three funded communities in
Phase I (Stephens et al., 2005) and two in Phase II (CMHS, 2001, 2003), and matched communities.
Non-funded communities were chosen by similar geographic and population characteristics, and their
willingness to participate in the project. The design called for enrollment of the same number of children
in the funded community and in the corresponding community, matched on age, gender, severity of
behavioral and emotional problems and functional impairment. Measures used in the descriptive and
outcome studies were used in the comparison studies, with some additions. The Phase II comparison
study included substudies of service experiences and provider characteristics.
The Comprehensive Community Mental Health Services for Children and Their Families Program Evaluation Findings
2006–2008 Annual Report to Congress ● Appendix D ● Page 3