Lessons from the Field - Seer Consulting

Evaluation of Capacity Building: Lessons from the Field
Part IV: Case Studies
44
to build an evaluation that could report on the relationship
between technical assistance in various
areas—planning, fiscal management, governance,
fundraising, human resources, computer technology,
public relations/marketing, evaluation and HIV/
AIDS prevention—and the capacity of local CBOs
to then deliver HIV/AIDS prevention services.
In addition to the size and scope of the project,
other challenges presented themselves:
• None of the TA agencies had engaged in an evaluation
project that demanded as much close
monitoring and intensive data design as this large
federal project did.
• The four partners had not worked collaboratively
before and, therefore, had no foundation for
working collaboratively.
• The partners did not share a common approach
to technical assistance delivery.
• Not all the staff members assigned to the project
were experienced technical assistance providers.
Before rushing to develop outcomes, measures and
tools, the evaluation team had to take some steps
back and work with four partners on more formative
issues building an effective collaborative and
shared practice for the delivery of technical assistance.
4 The evaluators coached some TA providers
and gave them support in a number of areas, such
as how to write a work plan, how to conduct a
CBO needs assessment, and how to clarify training
objectives when conducting workshops.
Recent evaluation projects across the
country have stressed the need to coach
ground-level workers in how to evaluate
and/or to self-evaluate.
Recent evaluation projects across the country have
stressed the need to coach ground-level workers in
how to evaluate and/or to self-evaluate. Taking the
time at the beginning and throughout the project to
build the foundation for evaluation was a key
element of The Capacity Project evaluation. As a
result, the evaluation had an impact on the entire
project itself. The evaluation assisted the TA
providers to: work together to agree on quality standards
for technical assistance delivery; value
evaluation as an ongoing practice; and more closely
define a theory of technical assistance practice.
As a direct result of participating in this evaluation,
the two larger partners, CRE and FPWA, have
moved to institutionalize evaluation as an organizational
practice and/or service. FPWA has begun to
evaluate its TA practice using the model and instruments
developed through The Capacity Project.
John Glover, Senior Program Associate for
Management and Technical Assistance at FPWA,
credits The Capacity Project with enabling FPWA to
strengthen its already substantial commitment to
internal and external evaluation. CRE is in the planning
stages of institutionalizing the evaluation of its
TA work, described later in this report.
Approach: Despite some overwhelming odds—
working with as many as 30 TA providers employed
by four distinct capacity-building agencies and
240 diverse CBOs—The Capacity Project’s
evaluation team:
• Established critical relationships with the four
agencies to raise the level of trust so that the
evaluators could both evaluate and “coach”
as needed.
• Designed a uniform way to diagnose and document
overall organizational needs and capacity.
This ensured that the diversity and range of CBO
characteristics and services in this project would
be equally well described. Peter Block stresses the
importance of the entry phase in capacity
building, which is captured in the evaluation
intake and diagnosis instruments and process
developed by the evaluators in collaboration with
the TA providers. 5
• Developed an extensive, structured protocol to
ensure that there is a direct connection (even if
long-term) between the kinds of TA provided
and the CBO’s effectiveness at improving or
expanding the specific service for which
funding was awarded (in this case, HIV/AIDS
prevention services).