Lessons from the Field - Seer Consulting

18
Part III
Important Considerations and Lessons Learned
PROMISING PRACTICES IN
EVALUATION OF CAPACITY BUILDING
This study identified the following characteristics
based upon promising practices and/or lessons
learned from evaluation reports, interviews
with evaluators and capacity builders, and the
current literature on capacity building and
organizational effectiveness.
Timely and Planned
Evaluation works best when it is incorporated
from the beginning of the design of a capacitybuilding
effort. Before startup is a time to
question the project’s readiness for evaluation
and plan accordingly.
Timeliness leads to a sound design phase that
considers all aspects of the evaluation process from
who should be involved to what is being measured
and why, how to measure, for whom the evaluation
is intended and who will receive the findings and
how. The planning process should clarify why an
evaluation is gathering what information and for
whom and for what purpose from the beginning.
Accountability and evaluation are closely aligned
and this is the opportunity to review ethical and
practical questions underlying an evaluation
approach: time, costs, impact on those providing
data and so forth.
The need to plan the evaluation during the planning
and startup of an initiative was echoed by
nearly every informant interviewed for this study.
Attempting to connect measures and critical questions
at the beginning of the design may change the
actual design of the project (formative evaluation) and
improve it. If a capacity-building intervention is
looked at through the lens of how it will be measured
(either by an outside evaluator or through
self-evaluation), certain flaws in the logic of the
design may be caught and corrected.
The need to plan the evaluation during the planning
and startup of an initiative was echoed by nearly
every informant interviewed for this study.
Unfortunately, many evaluators interviewed for this
report experienced being called in after a project
was designed and operating. In some cases, all that
was left to do was to describe the process and its
impact after the fact. Some evaluators experienced
difficulty going back to the project’s beginning and
capturing baseline information Therefore it became
difficult to quantify change.
Even for a simple capacity-building activity, such as
providing a training, planning for the evaluation at
the same time the training is being designed allows
for synchronicity between the trainers’ guide,
learning points, and tools to measure whether or
not the participants attained the learning goals.
Stakeholder-based
If the question is the effectiveness of a capacitybuilding
intervention, then those for whom the
intervention is intended should be included in
shaping what defines effectiveness (outcomes), how
effectiveness might be shown (indicators), and
methods for measuring it (tools).
The various stakeholders—the capacity-building
agent, the nonprofit(s) or end users; consumers of
nonprofit services, funders/social investors—will
have different perspectives and needs regarding
evaluation. The richest evaluative experiences—and
the ones that appear to be leading to genuine institutionalization
of evaluation for the purpose of
ongoing learning—were those that included stakeholders.
These were, without exception, the
participatory evaluations that by their nature are
inclusive of multiple stakeholders (see Case Studies,
Part IV for more information).