4. Evaluating CHA Services

4. Evaluating CHA Services
There are few rigorous evaluation studies of CHA programs. We did an extensive review of the
published literature related to CHA evaluations. Additionally, we reviewed evaluation instruments
currently in use, obtaining many of these through personal contacts at national meetings as well as through
outreach to more than 40 published researchers evaluating CHA programs. The outreach was undertaken
in collaboration with the University of New Mexico School of Medicine CHA evaluation project funded
by the Robert Wood Johnson and Henry J. Kaiser Family Foundations’ Opening Doors program.
In the fall of 1996, a meeting of CHAs, program administrators and evaluators from the field was hosted
in conjunction with the University of New Mexico to review our research and data and to develop our
evaluation framework. The framework was based on a conceptual model created by Eng and Young in
1992. Its components also draw from our review of literature and existing evaluation instruments.
Evaluating CHA Services: Findings
Many CHA programs do not assess their practices and outcomes. These programs often lack
feedback to shape later decisions about which activities should be continued, refined, expanded,
downsized, or even discontinued. Without such information, program managers have difficulty
informing their partners and stakeholders whether their investment has been effective in achieving
their organizational or programmatic goals.
Barriers to Evaluation of CHA programs
Evaluation of CHA programs is complex and poorly understood. We found that not all programs collect
outcome or process data. Modestly funded community-based programs often do not know what data to
collect or how to manage it; nor do they always understand the benefits of collecting data. One strength of
CHA programs is their ability to adapt to community needs and to incorporate changes as they receive
feedback. However, program changes complicate evaluation by altering the processes that are being
measured. Conducting outcome evaluation is challenging for much the same reason: programs frequently
take on issues that were not part of their original plans.
CHA programs lack resources for evaluation training and implementation. Quality evaluation takes
training and time. CHA programs, like other community-based health promotion and disease
prevention efforts, often lack the staff expertise and resources to design and conduct evaluations.
Often programs lack specific, measurable goals and objectives or program planners fail to define
appropriate outcome measures. Community need for services may consume staff efforts and leave no
time for evaluation.
The nature of CHA interactions with clients does not always lend itself to easy documentation.
Much of the daily work that CHAs do and the positive changes their clients achieve are difficult to
track, and are not tracked by current program evaluations. This may be because the essential living
skills CHAs teach and the support they provide are not seen as important subjects for data collection.
There are few methods and little opportunity to measure long-term effects. CHAs acknowledge that
measurement of long-term effects of community-based interventions has been difficult. Short-term
funding of CHA programs may prevent the assessment of even intermediate program effects.
19