Developing an Evaluation Capacity Building Network in the Field of Early Childhood Development
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
59<br />
<strong>Develop<strong>in</strong>g</strong> <strong>an</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> <strong>Network</strong> <strong>in</strong> <strong>the</strong><br />
<strong>Field</strong> <strong>of</strong> <strong>Early</strong> <strong>Childhood</strong> <strong>Development</strong><br />
Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar,<br />
Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Abstract This reflective essay traces <strong>the</strong> development <strong>of</strong> <strong>an</strong> evaluation capacity<br />
build<strong>in</strong>g network with<strong>in</strong> <strong>the</strong> early childhood development field. First, we describe <strong>the</strong><br />
context for build<strong>in</strong>g <strong>the</strong> network us<strong>in</strong>g a community-based participatory approach <strong>an</strong>d<br />
provide rationale for our specific focus on early childhood development. Second, we<br />
provide <strong>an</strong> expl<strong>an</strong>ation <strong>of</strong> <strong>the</strong> purpose <strong>an</strong>d processes <strong>in</strong>volved <strong>in</strong> three areas <strong>of</strong> signific<strong>an</strong>t<br />
engagement: partner, stakeholder, <strong>an</strong>d student. We reflect on <strong>the</strong> methods <strong>of</strong> engagement<br />
used across <strong>the</strong>se three areas <strong>an</strong>d <strong>the</strong>ir impact on <strong>the</strong> outcomes that we achieved. F<strong>in</strong>ally,<br />
we conclude <strong>the</strong> paper with some f<strong>in</strong>al considerations for guid<strong>in</strong>g engaged scholars <strong>an</strong>d<br />
with <strong>the</strong> next steps <strong>in</strong> our own work.<br />
KeyWords community-university partnerships, early childhood development,<br />
evaluation capacity build<strong>in</strong>g, partnership development, engagement methods <strong>an</strong>d practices<br />
The health <strong>an</strong>d well-be<strong>in</strong>g <strong>of</strong> children <strong>an</strong>d families has long been a focus <strong>of</strong> research <strong>an</strong>d<br />
social policy, as it impacts <strong>the</strong> economic <strong>an</strong>d social fabric <strong>of</strong> our communities (Akbari &<br />
McCuaig, 2017; Hertzm<strong>an</strong> & Boyce, 2010; Shonk<strong>of</strong>f & Levitt, 2010). Solutions to <strong>the</strong> issues<br />
that are faced by communities, <strong>in</strong>clud<strong>in</strong>g poverty, health <strong>in</strong>equalities, <strong>an</strong>d access to quality<br />
early learn<strong>in</strong>g <strong>an</strong>d care opportunities are not limited to a s<strong>in</strong>gle discipl<strong>in</strong>e or sector <strong>an</strong>d require<br />
<strong>the</strong> expertise <strong>an</strong>d collaborative efforts <strong>of</strong> community leaders, funders, <strong>the</strong> academy, <strong>an</strong>d all<br />
levels <strong>of</strong> government. There is grow<strong>in</strong>g recognition that collaboration through communityuniversity<br />
partnerships is <strong>an</strong> effective way to br<strong>in</strong>g community members, practitioners, <strong>an</strong>d<br />
researchers toge<strong>the</strong>r to discuss import<strong>an</strong>t issues <strong>in</strong> <strong>an</strong> environment where multiple worldviews<br />
are respected, solutions c<strong>an</strong> be generated, <strong>an</strong>d knowledge c<strong>an</strong> be co-constructed (Cargo &<br />
Mercer, 2008; Israel, Schultz, Parker, & Becker, 1998; Kajner, Fletcher, & Makokis, 2011).<br />
M<strong>an</strong>y <strong>of</strong> <strong>the</strong>se partnerships are guided by pr<strong>in</strong>ciples <strong>of</strong> community-based participatory<br />
research (CBPR), which encourages au<strong>the</strong>ntic collaboration by br<strong>in</strong>g<strong>in</strong>g toge<strong>the</strong>r knowledge<br />
<strong>an</strong>d expertise from multiple sectors <strong>an</strong>d discipl<strong>in</strong>es. CBPR pr<strong>in</strong>ciples are <strong>in</strong>tended to re<strong>in</strong>force<br />
<strong>the</strong> relev<strong>an</strong>ce <strong>of</strong> <strong>the</strong> partnership through shared leadership <strong>an</strong>d decision-mak<strong>in</strong>g, to foster<br />
ownership <strong>an</strong>d susta<strong>in</strong>ability (Israel et al., 1998; M<strong>in</strong>kler & Wallerste<strong>in</strong>, 2008).<br />
Community-university partnerships have been developed to tackle press<strong>in</strong>g social issues <strong>an</strong>d<br />
are well documented <strong>in</strong> <strong>the</strong> community engaged scholarship (CES) literature (e.g., Jagosh et al.,<br />
Volume 3/Issue 2/Fall 2017
60 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
2015; Pei, Feltham, Ford, & Schwartz, 2015; Visw<strong>an</strong>ath<strong>an</strong> et al., 2004). Fewer examples exist<br />
<strong>of</strong> susta<strong>in</strong>able community-university partnerships <strong>an</strong>d networks that encompass all aspects<br />
<strong>of</strong> CES (e.g., research, teach<strong>in</strong>g <strong>an</strong>d learn<strong>in</strong>g, student engagement, knowledge mobilization)<br />
with core resources (hum<strong>an</strong> <strong>an</strong>d f<strong>in</strong><strong>an</strong>cial) from both university <strong>an</strong>d community partners.<br />
Although less common, examples <strong>of</strong> such partnerships <strong>in</strong>clude <strong>the</strong> Community-University<br />
Partnership Program at <strong>the</strong> University <strong>of</strong> Brighton, Community-based Research C<strong>an</strong>ada,<br />
<strong>an</strong>d Participatory Research <strong>in</strong> Asia. In <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta, <strong>the</strong> Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families (CUP) <strong>in</strong> <strong>the</strong> Faculty <strong>of</strong> Extension<br />
at <strong>the</strong> University <strong>of</strong> Alberta represents <strong>an</strong>o<strong>the</strong>r example. CUP was launched <strong>in</strong> 2000 through<br />
shared community <strong>an</strong>d university leadership to improve <strong>the</strong> development <strong>of</strong> children, youth,<br />
<strong>an</strong>d families by promot<strong>in</strong>g <strong>in</strong>teractions among researchers <strong>an</strong>d community members (e.g.,<br />
practitioners, policymakers, families) <strong>in</strong> <strong>the</strong> areas <strong>of</strong> research, knowledge shar<strong>in</strong>g, <strong>an</strong>d lifelong<br />
learn<strong>in</strong>g (Chapm<strong>an</strong>, 2015; McCaffrey, 2007). CUP has <strong>the</strong> m<strong>an</strong>date to nurture environments<br />
where evidence is used effectively to develop practices, programs, <strong>an</strong>d policies that support <strong>the</strong><br />
healthy development <strong>of</strong> children, youth, families <strong>an</strong>d communities across four priority areas:<br />
policy, poverty, early childhood development, <strong>an</strong>d evaluation.<br />
Over its 17 years <strong>of</strong> operation, university <strong>an</strong>d community partners have shared jo<strong>in</strong>t<br />
responsibility for guid<strong>in</strong>g <strong>an</strong>d susta<strong>in</strong><strong>in</strong>g CUP, <strong>an</strong>d are represented by CUP’s Steer<strong>in</strong>g<br />
Committee. Reflect<strong>in</strong>g <strong>the</strong> partnership, <strong>an</strong> academic <strong>an</strong>d community member co-chair <strong>the</strong><br />
CUP steer<strong>in</strong>g committee, <strong>an</strong>d its membership is currently comprised <strong>of</strong> 27 members that<br />
represent foundations, municipal <strong>an</strong>d prov<strong>in</strong>cial government, academics, research <strong>an</strong>d policy<br />
centres, <strong>an</strong>d community-based agencies. This govern<strong>an</strong>ce structure has long provided <strong>the</strong><br />
impetus for collaborative opportunities <strong>in</strong> CUP’s key focus on evaluation <strong>an</strong>d early childhood<br />
development from its m<strong>an</strong>y community requests for research, measurement, <strong>an</strong>d evaluation<br />
support (Bis<strong>an</strong>z, Edwards, & Shaw, 2013). These requests have resulted <strong>in</strong> shar<strong>in</strong>g resources,<br />
broker<strong>in</strong>g relationships with o<strong>the</strong>r faculty <strong>an</strong>d graduate students on campus, <strong>an</strong>d develop<strong>in</strong>g<br />
participatory research <strong>an</strong>d/or evaluation projects. However, with ever-<strong>in</strong>creas<strong>in</strong>g requests from<br />
community-based agencies that are not always accomp<strong>an</strong>ied by resources, it became necessary<br />
for CUP to determ<strong>in</strong>e a more systematic <strong>an</strong>d effective way to respond to <strong>the</strong>se needs.<br />
This reflective essay provides <strong>an</strong> <strong>in</strong>-depth account <strong>of</strong> how community <strong>an</strong>d university<br />
members <strong>of</strong> CUP spearheaded <strong>the</strong> development <strong>of</strong> a network to adv<strong>an</strong>ce evaluation capacity<br />
<strong>in</strong> <strong>the</strong> early childhood development field, us<strong>in</strong>g a CBPR approach. First, we describe <strong>the</strong><br />
context for <strong>the</strong> development <strong>of</strong> <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN) 1 <strong>an</strong>d <strong>the</strong> reason for<br />
focus<strong>in</strong>g on early childhood development. This is followed by <strong>an</strong> expl<strong>an</strong>ation <strong>of</strong> <strong>the</strong> purpose<br />
<strong>an</strong>d processes <strong>in</strong>volved <strong>in</strong> three areas <strong>of</strong> signific<strong>an</strong>t engagement: partners, stakeholders, <strong>an</strong>d<br />
students. We reflect on <strong>the</strong> methods <strong>of</strong> engagement used across <strong>the</strong>se three areas <strong>an</strong>d <strong>the</strong><br />
impact <strong>of</strong> <strong>the</strong>se methods on <strong>the</strong> outcomes that we achieved. F<strong>in</strong>ally, we conclude <strong>the</strong> paper<br />
with some f<strong>in</strong>al considerations for guid<strong>in</strong>g engaged scholars <strong>an</strong>d with <strong>the</strong> next steps <strong>in</strong> our<br />
own work.<br />
1 For <strong>the</strong> sake <strong>of</strong> consistency, we refer to <strong>the</strong> ECN <strong>an</strong>d its development <strong>in</strong> <strong>the</strong> past tense throughout this paper. However,<br />
<strong>the</strong> ECN is a long-term project <strong>an</strong>d is <strong>the</strong>refore ongo<strong>in</strong>g.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
61<br />
Address<strong>in</strong>g Community <strong>Evaluation</strong> Needs Through Partnership<br />
CUP has received a steadily <strong>in</strong>creas<strong>in</strong>g number <strong>of</strong> requests from community agencies for<br />
assist<strong>an</strong>ce with research <strong>an</strong>d evaluation (Bis<strong>an</strong>z et al., 2013). This reflects <strong>an</strong> <strong>in</strong>creas<strong>in</strong>g dem<strong>an</strong>d<br />
across C<strong>an</strong>ada for evaluation. Funders, program pl<strong>an</strong>ners, <strong>an</strong>d policymakers are seek<strong>in</strong>g<br />
rigorous <strong>an</strong>d reliable evidence to <strong>in</strong>form resource allocation <strong>an</strong>d improve essential services<br />
(McSh<strong>an</strong>e, Usher, T<strong>an</strong>don, & Steel, 2015). However, <strong>the</strong> dem<strong>an</strong>d for evaluation currently<br />
surpasses <strong>the</strong> required resources <strong>an</strong>d supply <strong>of</strong> evaluation knowledge <strong>an</strong>d expertise available<br />
to m<strong>an</strong>y community agencies (Gauthier et al., 2010). This has placed community agencies <strong>in</strong> <strong>an</strong><br />
untenable situation. With limited fund<strong>in</strong>g, hum<strong>an</strong> resources, <strong>an</strong>d evaluation expertise to collect<br />
<strong>an</strong>d use evaluation evidence, community agencies <strong>of</strong>ten struggle to justify cont<strong>in</strong>ued support<br />
through evaluation (Bakken, Núñez, & Couture, 2014; Cous<strong>in</strong>s, Goh, Elliott, & Bourgeois,<br />
2014; J<strong>an</strong>zen et al., 2017).<br />
To ga<strong>in</strong> a deeper underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> widespread evaluation capacity issues with <strong>the</strong> <strong>in</strong>tent <strong>of</strong><br />
develop<strong>in</strong>g <strong>an</strong> effective response, CUP hosted two focus groups <strong>in</strong> 2012. The 14 particip<strong>an</strong>ts<br />
<strong>in</strong>cluded leaders (e.g., CEOs, executive directors, <strong>an</strong>d m<strong>an</strong>agers) from nonpr<strong>of</strong>its, foundations,<br />
<strong>an</strong>d prov<strong>in</strong>cial government, as well as pr<strong>of</strong>essional evaluators <strong>an</strong>d university academics familiar<br />
with evaluation related issues. What emerged from <strong>the</strong> focus groups was a locally-relev<strong>an</strong>t<br />
underst<strong>an</strong>d<strong>in</strong>g that community agencies f<strong>in</strong>d <strong>the</strong> process <strong>of</strong> evaluation challeng<strong>in</strong>g <strong>in</strong> common<br />
ways. These f<strong>in</strong>d<strong>in</strong>gs resonated with what o<strong>the</strong>rs had already reported <strong>in</strong> <strong>the</strong> literature: funders<br />
<strong>of</strong>ten request specific evaluation methods <strong>an</strong>d outcomes to meet <strong>the</strong>ir needs for accountability<br />
that do not realistically reflect org<strong>an</strong>izational strategic learn<strong>in</strong>g goals, time, <strong>an</strong>d resources<br />
(Carm<strong>an</strong> & Milleson, 2005; Leviton, 2014); agencies f<strong>in</strong>d <strong>the</strong> process <strong>of</strong> evaluation challeng<strong>in</strong>g<br />
due to <strong>in</strong>sufficient fund<strong>in</strong>g, <strong>an</strong>d lack <strong>of</strong> hum<strong>an</strong> resource capacity (Bakken et al., 2014;<br />
Cous<strong>in</strong>s, et al., 2014; J<strong>an</strong>zen et al., 2017); experts f<strong>in</strong>d it difficult to provide all <strong>the</strong> required<br />
resources, knowledge, <strong>an</strong>d capacity to community agencies; <strong>an</strong>d evaluation outcomes <strong>of</strong>ten<br />
prove un<strong>in</strong>formative for program development <strong>an</strong>d practice. Common across <strong>the</strong> focus group<br />
particip<strong>an</strong>ts was <strong>the</strong> urgent need for fur<strong>the</strong>r dialogue among <strong>in</strong>tersectoral stakeholders who<br />
support evaluation <strong>of</strong> programs. They also validated <strong>the</strong> need to create a central po<strong>in</strong>t where<br />
stakeholders could access coord<strong>in</strong>ated evaluation capacity build<strong>in</strong>g resources, <strong>an</strong>d ensure high<br />
quality tra<strong>in</strong><strong>in</strong>g, practice, <strong>an</strong>d research <strong>in</strong> evaluation.<br />
Form<strong>in</strong>g <strong>the</strong> Partnership<br />
Focus group f<strong>in</strong>d<strong>in</strong>gs were presented to <strong>the</strong> CUP steer<strong>in</strong>g committee, <strong>an</strong>d a work<strong>in</strong>g group<br />
was established to realize <strong>the</strong> evaluation <strong>in</strong>itiative. The work<strong>in</strong>g group consisted <strong>of</strong> a<br />
foundation CEO, two executive directors <strong>of</strong> large nonpr<strong>of</strong>it agencies, <strong>an</strong>d <strong>an</strong> academic <strong>an</strong>d<br />
research associate affiliated with CUP. The key task <strong>of</strong> <strong>the</strong> work<strong>in</strong>g group was to identify<br />
<strong>an</strong>d br<strong>in</strong>g toge<strong>the</strong>r leaders (funders, nonpr<strong>of</strong>its, academics, <strong>an</strong>d government) from <strong>the</strong> social<br />
sector to form a partnership. The <strong>in</strong>tent <strong>of</strong> <strong>the</strong> partnership was to develop <strong>an</strong>d operationalize<br />
a robust, coord<strong>in</strong>ated pl<strong>an</strong> for <strong>in</strong>creas<strong>in</strong>g <strong>the</strong> availability <strong>of</strong> quality evaluation knowledge,<br />
resources, expertise, <strong>an</strong>d tailored capacity build<strong>in</strong>g opportunities. In September 2013, a group<br />
<strong>of</strong> 18 leaders was brought toge<strong>the</strong>r for a full-day meet<strong>in</strong>g to determ<strong>in</strong>e (1) what <strong>the</strong> focus<br />
Volume 3/Issue 2/Fall 2017
62 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
<strong>of</strong> <strong>the</strong> evaluation <strong>in</strong>itative should be, (2) how to fund <strong>the</strong> <strong>in</strong>itative, (3) what <strong>in</strong>dividuals <strong>an</strong>d<br />
org<strong>an</strong>izations needed to be <strong>in</strong>volved, <strong>an</strong>d (4) what steps <strong>the</strong> partnership should take mov<strong>in</strong>g<br />
forward. The group decided that <strong>the</strong> focus <strong>of</strong> <strong>the</strong> <strong>in</strong>itative would be on evaluation <strong>in</strong> <strong>the</strong> early<br />
childhood development (ECD) field. Much <strong>of</strong> <strong>the</strong> group’s discussion focused on <strong>the</strong> need<br />
to foster <strong>an</strong>d support “evaluative th<strong>in</strong>k<strong>in</strong>g” <strong>in</strong> <strong>the</strong> ECD sector. Evaluative th<strong>in</strong>k<strong>in</strong>g has been<br />
def<strong>in</strong>ed as<br />
Critical th<strong>in</strong>k<strong>in</strong>g applied <strong>in</strong> <strong>the</strong> context <strong>of</strong> evaluation, motivated by <strong>an</strong> attitude<br />
<strong>of</strong> <strong>in</strong>quisitiveness <strong>an</strong>d a belief <strong>in</strong> <strong>the</strong> value <strong>of</strong> evidence, that <strong>in</strong>volves identify<strong>in</strong>g<br />
assumptions, pos<strong>in</strong>g thoughtful questions, pursu<strong>in</strong>g deeper underst<strong>an</strong>d<strong>in</strong>g through<br />
reflection <strong>an</strong>d perspective tak<strong>in</strong>g, <strong>an</strong>d <strong>in</strong>form<strong>in</strong>g decisions <strong>in</strong> preparation for action.<br />
(Buckley, Archibald, Hargraves, & Trochim, 2015, p. 378).<br />
To stimulate <strong>an</strong>d support <strong>the</strong> <strong>in</strong>tiative, several partners <strong>of</strong>fered <strong>in</strong>-k<strong>in</strong>d <strong>an</strong>d cash<br />
contributions to support <strong>an</strong> application for a Social Sciences <strong>an</strong>d Hum<strong>an</strong>ities Research Council<br />
Partnership <strong>Development</strong> Gr<strong>an</strong>t (SSHRC PDG), which we were successful <strong>in</strong> obta<strong>in</strong><strong>in</strong>g<br />
<strong>in</strong> 2014. As determ<strong>in</strong>ed dur<strong>in</strong>g <strong>the</strong> September pl<strong>an</strong>n<strong>in</strong>g meet<strong>in</strong>g, <strong>the</strong> primary objectives <strong>of</strong><br />
<strong>the</strong> SSHRC PDG were to (1) conduct <strong>an</strong> <strong>in</strong>tersectoral needs assessment us<strong>in</strong>g community<br />
forums with <strong>the</strong> aim <strong>of</strong> identify<strong>in</strong>g common evaluation knowledge (i.e., l<strong>an</strong>guage, metrics,<br />
methods, <strong>the</strong>ories <strong>an</strong>d issues) <strong>an</strong>d capacity gaps; (2) develop <strong>an</strong>d deliver educational resources<br />
<strong>an</strong>d tra<strong>in</strong><strong>in</strong>g opportunities that address <strong>the</strong>se gaps <strong>an</strong>d to subsequently evaluate <strong>an</strong>d ref<strong>in</strong>e<br />
<strong>the</strong> resources <strong>an</strong>d tra<strong>in</strong><strong>in</strong>g; <strong>an</strong>d (3) nurture <strong>an</strong>d susta<strong>in</strong> <strong>an</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> that<br />
supports ongo<strong>in</strong>g dialogue <strong>of</strong> evaluation experts, government, funders, <strong>an</strong>d community<br />
agencies at a national level, <strong>an</strong>d knowledge mobilization <strong>of</strong> community-engaged evaluative<br />
practices across <strong>the</strong> r<strong>an</strong>ge <strong>of</strong> sectors that impact ECD. The partnership now had a clear focus<br />
on build<strong>in</strong>g a network to adv<strong>an</strong>ce evaluation practice <strong>in</strong> <strong>the</strong> ECD field, <strong>an</strong>d <strong>the</strong> necessary<br />
fund<strong>in</strong>g <strong>an</strong>d committed leaders to move th<strong>in</strong>gs forward.<br />
Rationale for Focus<strong>in</strong>g on <strong>Early</strong> <strong>Childhood</strong> <strong>Development</strong><br />
The rationale for <strong>the</strong> partnership’s focus on evaluation <strong>in</strong> ECD was based on several<br />
considerations. First, robust scientific evidence <strong>in</strong> <strong>the</strong> areas <strong>of</strong> child <strong>an</strong>d family health <strong>an</strong>d wellbe<strong>in</strong>g<br />
demonstrates that experiences <strong>an</strong>d environments <strong>in</strong> <strong>the</strong> early years pr<strong>of</strong>oundly impact<br />
children’s development (Akbari & McCuaig, 2017; Hertzm<strong>an</strong> & Boyce, 2010; Shonk<strong>of</strong>f &<br />
Levitt, 2010; Shonk<strong>of</strong>f, 2017). Incorporat<strong>in</strong>g multiple sectors <strong>an</strong>d systems, <strong>the</strong> field <strong>of</strong> ECD<br />
is <strong>in</strong>terdiscipl<strong>in</strong>ary <strong>an</strong>d complex yet provides critical opportunities for <strong>in</strong>novations <strong>in</strong> social<br />
policy <strong>an</strong>d practices. When policies <strong>an</strong>d programs that target <strong>the</strong> early years are responsive to<br />
<strong>the</strong> complexity <strong>of</strong> <strong>the</strong> field, <strong>the</strong>y c<strong>an</strong> reduce expensive <strong>in</strong>terventions <strong>in</strong> later years (Akbari &<br />
McCuaig, 2017; Dunc<strong>an</strong> et al., 2007; Heckm<strong>an</strong>, 2008; OECD, 2012; Shonk<strong>of</strong>f, 2017). Despite<br />
signific<strong>an</strong>t <strong>in</strong>vestments <strong>in</strong> early years programm<strong>in</strong>g, approximately one <strong>in</strong> four C<strong>an</strong>adi<strong>an</strong><br />
children lack <strong>the</strong> social, emotional, <strong>an</strong>d cognitive capacities to benefit from <strong>the</strong> public education<br />
system (CIHI, 2014). This rate doubles for Indigenous children <strong>an</strong>d English-l<strong>an</strong>guage learners<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
63<br />
<strong>of</strong> immigr<strong>an</strong>t <strong>an</strong>d refugee backgrounds, who experience social vulnerabilities as a result <strong>of</strong><br />
economic, cultural, <strong>an</strong>d l<strong>an</strong>guage differences (Cabrera, 2013; Georgis, Gokiert, & Kirova, 2018;<br />
Gokiert et al., 2014; Government <strong>of</strong> C<strong>an</strong>ada, 2011). The prevalence <strong>of</strong> mental health disorders<br />
among children <strong>in</strong> C<strong>an</strong>ada is about 13% (Waddell, Shepherd, Schwartz & Baric<strong>an</strong>, 2014), <strong>an</strong>d<br />
roughly a quarter <strong>of</strong> C<strong>an</strong>adi<strong>an</strong> children are liv<strong>in</strong>g <strong>in</strong> low-<strong>in</strong>come households (Statistics C<strong>an</strong>ada,<br />
2011). Fur<strong>the</strong>rmore, <strong>in</strong> 2016 UNICEF r<strong>an</strong>ked C<strong>an</strong>ada 26th <strong>of</strong> 35 high-<strong>in</strong>come countries for<br />
child well-be<strong>in</strong>g. Such statistics <strong>in</strong>dicate <strong>the</strong>re is a press<strong>in</strong>g need for improvement <strong>in</strong> exist<strong>in</strong>g<br />
C<strong>an</strong>adi<strong>an</strong> child- <strong>an</strong>d family-focused policies <strong>an</strong>d practices.<br />
Second, policy frameworks have been emerg<strong>in</strong>g for over a decade that encourage a<br />
common underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> development <strong>in</strong> <strong>the</strong> early years, promote shared l<strong>an</strong>guage <strong>an</strong>d<br />
outcomes, create cont<strong>in</strong>uity across jurisdictions <strong>an</strong>d sett<strong>in</strong>gs, <strong>an</strong>d serve as a resource to support<br />
program <strong>an</strong>d policy development (CMEC, 2014; Government <strong>of</strong> Alberta, 2013; Government<br />
<strong>of</strong> M<strong>an</strong>itoba, 2013; Government <strong>of</strong> NWT, 2013; Munro, 2006). Unfortunately, <strong>the</strong>se policy<br />
frameworks tend to create un<strong>in</strong>tended complications for <strong>the</strong> early childhood system. Funders<br />
adopt <strong>the</strong>se policy frameworks, <strong>an</strong>d do not always provide clear expectations on how to<br />
use <strong>the</strong>m or contribute to <strong>the</strong>m <strong>in</strong> me<strong>an</strong><strong>in</strong>gful ways. This leaves <strong>the</strong> public sector collect<strong>in</strong>g<br />
considerable amounts <strong>of</strong> data that result <strong>in</strong> a signific<strong>an</strong>t “data burden,” as nonpr<strong>of</strong>its <strong>in</strong>vest<br />
resources beyond <strong>the</strong>ir me<strong>an</strong>s to produce perform<strong>an</strong>ce data that is <strong>of</strong> little use to both <strong>the</strong><br />
org<strong>an</strong>izations produc<strong>in</strong>g <strong>the</strong> data <strong>an</strong>d <strong>the</strong> funders request<strong>in</strong>g it (e.g. Snibbe, 2006; Carm<strong>an</strong>,<br />
2010; Leviton, 2014; Liket, Rey-Garcia, & Maas, 2014).<br />
F<strong>in</strong>ally, <strong>the</strong>re is no common approach to quality evaluation knowledge, resources,<br />
expertise, <strong>an</strong>d capacity-build<strong>in</strong>g opportunities tailored to <strong>the</strong> needs <strong>of</strong> <strong>the</strong> <strong>in</strong>tersectoral <strong>an</strong>d<br />
<strong>in</strong>terdiscipl<strong>in</strong>ary nature <strong>of</strong> <strong>the</strong> ECD field. This void compromises high-quality research,<br />
tra<strong>in</strong><strong>in</strong>g, <strong>an</strong>d practice <strong>in</strong> evaluation, <strong>an</strong>d ultimately impacts <strong>the</strong> programs <strong>an</strong>d practices <strong>in</strong><br />
ECD. A coord<strong>in</strong>ated approach was <strong>the</strong> solution, <strong>an</strong>d so we extensively engaged partners,<br />
stakeholders, <strong>an</strong>d students to underst<strong>an</strong>d <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> ECD field. In <strong>the</strong><br />
next sections, we will explore our process <strong>of</strong> develop<strong>in</strong>g <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong><br />
(ECN) through three dist<strong>in</strong>ct but related <strong>the</strong>mes <strong>of</strong> engagement: (1) partner engagement, (2)<br />
stakeholder engagement, <strong>an</strong>d (3) student engagement.<br />
Partner Engagement: <strong>Develop<strong>in</strong>g</strong> a CBPR Partnership for <strong>Evaluation</strong><br />
As <strong>the</strong> ECN is grounded <strong>in</strong> a CBPR approach (Israel et al., 1998), it <strong>in</strong>tegrates research, action,<br />
reflection, <strong>an</strong>d communication. As such, a partnership was <strong>the</strong> first step towards build<strong>in</strong>g<br />
<strong>the</strong> ECN because it provides pr<strong>in</strong>ciples <strong>an</strong>d methods to guide <strong>the</strong> work <strong>of</strong> <strong>the</strong> network, c<strong>an</strong><br />
stimulate <strong>in</strong>tersectoral <strong>an</strong>d <strong>in</strong>terdiscipl<strong>in</strong>ary dialogue, <strong>an</strong>d c<strong>an</strong> ensure that community needs <strong>an</strong>d<br />
values are at <strong>the</strong> foundation <strong>of</strong> <strong>the</strong> network. Partners were carefully chosen for <strong>the</strong> import<strong>an</strong>t<br />
expertise <strong>the</strong>y had, <strong>an</strong>d <strong>the</strong> role <strong>the</strong>y could play <strong>in</strong> work<strong>in</strong>g toge<strong>the</strong>r to recognize <strong>an</strong>d address<br />
<strong>the</strong> complexity <strong>of</strong> <strong>in</strong>tersectoral evaluative th<strong>in</strong>k<strong>in</strong>g. Partners represented stakeholder groups<br />
whose def<strong>in</strong>ition(s) <strong>of</strong> evaluation effectiveness, practices, <strong>an</strong>d outcomes are <strong>in</strong>fluenced by<br />
<strong>the</strong> sector with<strong>in</strong> which <strong>the</strong>y worked <strong>an</strong>d <strong>the</strong> role <strong>the</strong>y played <strong>in</strong> support<strong>in</strong>g ECD programs,<br />
practices, or policies.<br />
Volume 3/Issue 2/Fall 2017
64 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
ECN Partnership Govern<strong>an</strong>ce<br />
The partnership that was developed <strong>an</strong>d now susta<strong>in</strong>s <strong>the</strong> ECN comprised three ma<strong>in</strong><br />
govern<strong>in</strong>g bodies <strong>an</strong>d <strong>in</strong>volves <strong>the</strong> authors <strong>in</strong> various roles: a steer<strong>in</strong>g committee, core research<br />
team, <strong>an</strong>d project m<strong>an</strong>agement team (see Figure 1). The 19-member Steer<strong>in</strong>g Committee had<br />
representation from government, community agencies, funders, evaluation consult<strong>an</strong>ts <strong>an</strong>d<br />
academia. The Steer<strong>in</strong>g Committee was <strong>the</strong> decision-mak<strong>in</strong>g body for <strong>the</strong> ECN <strong>an</strong>d provided<br />
high-level guid<strong>an</strong>ce, support, <strong>an</strong>d direction for partner engagement, research, data <strong>in</strong>terpretation,<br />
knowledge mobilization, <strong>an</strong>d fund<strong>in</strong>g <strong>in</strong>itiatives. The Steer<strong>in</strong>g Committee comprised 19<br />
members, <strong>in</strong>clud<strong>in</strong>g academics (Gokiert, K<strong>in</strong>gsley), funders, government representatives,<br />
evaluation consult<strong>an</strong>ts, <strong>an</strong>d nonpr<strong>of</strong>it representatives. The Core Research Team met monthly<br />
<strong>an</strong>d was responsible for <strong>the</strong> design, development, <strong>an</strong>d implementation <strong>of</strong> <strong>the</strong> research<br />
component <strong>of</strong> <strong>the</strong> ECN. The Core Research Team comprised n<strong>in</strong>e members, <strong>in</strong>clusive <strong>of</strong><br />
<strong>an</strong> executive director from a national nonpr<strong>of</strong>it (Hopk<strong>in</strong>s), a government representative, five<br />
academics (Cor, Gokiert, Poth, Spr<strong>in</strong>gett), <strong>an</strong>d one postdoctoral fellow (K<strong>in</strong>gsley). The Project<br />
M<strong>an</strong>agement Team met weekly <strong>an</strong>d was responsible for <strong>the</strong> development <strong>an</strong>d monitor<strong>in</strong>g <strong>of</strong><br />
<strong>the</strong> pl<strong>an</strong>s, schedules, budgets, <strong>an</strong>d deliverables <strong>of</strong> <strong>the</strong> ECN with<strong>in</strong> <strong>the</strong> established time frames<br />
<strong>an</strong>d quality guidel<strong>in</strong>es approved by <strong>the</strong> Steer<strong>in</strong>g Committee. The Project M<strong>an</strong>agement Team<br />
comprised <strong>the</strong> pr<strong>in</strong>cipal <strong>in</strong>vestigator (Gokiert), a postdoctoral fellow (K<strong>in</strong>gsley), graduate<br />
research assist<strong>an</strong>ts (El Hassar, T<strong>in</strong>k, Tremblay), practicum students, <strong>an</strong>d student volunteers.<br />
Figure 1: <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> Govern<strong>an</strong>ce<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
65<br />
Engag<strong>in</strong>g Partners through Focus Groups<br />
To develop guid<strong>in</strong>g goals, common pr<strong>in</strong>ciples, <strong>in</strong>dicators <strong>of</strong> success for <strong>the</strong> partnership, <strong>an</strong>d<br />
a framework for build<strong>in</strong>g a prov<strong>in</strong>cial evaluation agenda, we hosted two focus groups with<br />
16 <strong>in</strong>dividuals across <strong>the</strong> levels <strong>of</strong> govern<strong>an</strong>ce. The questions posed were: What was <strong>the</strong><br />
orig<strong>in</strong>al reason you agreed to participate <strong>in</strong> <strong>the</strong> ECN <strong>an</strong>d what do you hope to ga<strong>in</strong> from<br />
your <strong>in</strong>volvement? What are <strong>the</strong> essential elements <strong>of</strong> <strong>the</strong> community-university partnership<br />
that will contribute to develop<strong>in</strong>g a successful ECN? What would success look like for <strong>the</strong><br />
ECN? The focus groups were audio-recorded, <strong>an</strong>d three graduate research assist<strong>an</strong>ts took<br />
extensive notes. Members <strong>of</strong> <strong>the</strong> Project M<strong>an</strong>agement Team completed a <strong>the</strong>matic <strong>an</strong>alysis <strong>of</strong><br />
this data <strong>an</strong>d org<strong>an</strong>ized <strong>the</strong> <strong>in</strong>formation <strong>in</strong>to core partnership pr<strong>in</strong>ciples, project outcomes <strong>an</strong>d<br />
particular actions that partners described w<strong>an</strong>t<strong>in</strong>g from <strong>the</strong> ECN.<br />
The pr<strong>in</strong>ciples that arose from <strong>the</strong> focus groups were highly reflective <strong>of</strong> <strong>the</strong> guid<strong>in</strong>g<br />
pr<strong>in</strong>ciples <strong>of</strong> a CBPR approach (Israel et al., 1998). They <strong>in</strong>cluded <strong>the</strong> need for trust, mutual<br />
benefit, equity, co-creation, accessibility, collaboration, strong communication, a commitment<br />
to action, <strong>an</strong>d engagement at all levels. The focus groups provided <strong>an</strong> import<strong>an</strong>t opportunity<br />
not only to identify key pr<strong>in</strong>ciples to guide <strong>the</strong> partnership, but to also ascribe me<strong>an</strong><strong>in</strong>g to<br />
<strong>the</strong>se pr<strong>in</strong>ciples <strong>in</strong> <strong>the</strong> context <strong>of</strong> <strong>the</strong> ECN.<br />
The partners identified several immediate <strong>an</strong>d <strong>in</strong>termediate outcomes that were import<strong>an</strong>t<br />
to guide <strong>the</strong> ECN towards <strong>the</strong> ultimate outcome articulated through focus groups: children are<br />
provided <strong>the</strong> best possible start through evaluation-<strong>in</strong>formed policies <strong>an</strong>d practices. Immediate<br />
outcomes <strong>in</strong>cluded <strong>the</strong> need to <strong>in</strong>crease awareness <strong>an</strong>d underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> <strong>the</strong> purpose <strong>of</strong><br />
<strong>the</strong> ECN, <strong>in</strong>crease <strong>the</strong> perceived import<strong>an</strong>ce <strong>of</strong> evaluation, <strong>an</strong>d <strong>in</strong>crease evaluation capacity<br />
through elevated evaluative knowledge <strong>an</strong>d practice. Intermediate outcomes <strong>in</strong>cluded <strong>the</strong> need<br />
for a culture that values evaluation, risk-tak<strong>in</strong>g <strong>an</strong>d tr<strong>an</strong>sparency, <strong>an</strong> <strong>in</strong>crease time <strong>an</strong>d f<strong>in</strong><strong>an</strong>cial<br />
<strong>in</strong>vestments <strong>in</strong> evaluation, evaluations <strong>in</strong> ECD that are relev<strong>an</strong>t <strong>an</strong>d me<strong>an</strong><strong>in</strong>gful, <strong>an</strong>d <strong>the</strong><br />
<strong>in</strong>tegration <strong>of</strong> evaluation <strong>in</strong>to org<strong>an</strong>izational processes through a utilization focused approach.<br />
F<strong>in</strong>ally, a number <strong>of</strong> actions were identified that partners felt would contribute to <strong>the</strong><br />
success <strong>of</strong> <strong>the</strong> ECN. Actions <strong>in</strong>cluded <strong>the</strong> need to engage <strong>an</strong>d connect current <strong>an</strong>d future<br />
evaluators, users <strong>of</strong> evaluation, <strong>an</strong>d evaluation capacity builders; provide evaluation mentorship<br />
<strong>an</strong>d expertise; develop education tools <strong>an</strong>d resources; develop common evaluation l<strong>an</strong>guage<br />
through effective communication mech<strong>an</strong>isms; provide pr<strong>of</strong>essional development <strong>an</strong>d<br />
learn<strong>in</strong>g opportunities; <strong>an</strong>d share best practices <strong>in</strong> evaluation. For <strong>the</strong>se actions to occur, it<br />
was determ<strong>in</strong>ed that <strong>the</strong>re would be a need to mobilize <strong>an</strong>d create synergy between <strong>the</strong> levels<br />
<strong>of</strong> govern<strong>an</strong>ce <strong>an</strong>d <strong>the</strong> broader membership <strong>of</strong> <strong>the</strong> <strong>Network</strong>.<br />
Challenges <strong>an</strong>d Opportunities <strong>in</strong> Partner Engagement<br />
The focus group process was beneficial as a me<strong>an</strong>s <strong>of</strong> engag<strong>in</strong>g partners <strong>in</strong> a dialogue about<br />
shared pr<strong>in</strong>ciples, outcomes, <strong>an</strong>d actions that <strong>the</strong>y w<strong>an</strong>ted for <strong>the</strong> ECN. The focus groups also<br />
generated ideas, enthusiasm, <strong>an</strong>d a shared commitment for <strong>the</strong> ECN that o<strong>the</strong>rwise might not<br />
have happened. Despite <strong>the</strong> positive impacts <strong>of</strong> <strong>the</strong> focus groups, ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g a concerted<br />
level <strong>of</strong> engagement over time <strong>an</strong>d adher<strong>in</strong>g to <strong>the</strong> pr<strong>in</strong>ciples that were identified by <strong>the</strong><br />
Volume 3/Issue 2/Fall 2017
66 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
partners has been challeng<strong>in</strong>g for a number <strong>of</strong> reasons. For one, <strong>the</strong> breadth <strong>of</strong> <strong>the</strong> project<br />
– engag<strong>in</strong>g partners from across so m<strong>an</strong>y contexts (universities, fund<strong>in</strong>g agencies, nonpr<strong>of</strong>its,<br />
government, policy <strong>an</strong>d research centres <strong>an</strong>d consult<strong>in</strong>g firms) – has, to some extent, affected<br />
<strong>the</strong> depth <strong>of</strong> engagement that we were able to achieve. Hav<strong>in</strong>g 19 people on <strong>the</strong> steer<strong>in</strong>g<br />
committee alone limited <strong>the</strong> extent to which face-to-face meet<strong>in</strong>gs could be held <strong>an</strong>d rich<br />
discussion could be facilitated. In addition, m<strong>an</strong>y <strong>of</strong> <strong>the</strong> steer<strong>in</strong>g committee members were<br />
high-level decision makers <strong>an</strong>d leaders <strong>in</strong> <strong>the</strong> field. As a result, f<strong>in</strong>d<strong>in</strong>g meet<strong>in</strong>g times when<br />
everyone was available <strong>an</strong>d expect<strong>in</strong>g signific<strong>an</strong>t engagement <strong>in</strong> <strong>the</strong> ECN was difficult beyond<br />
meet<strong>in</strong>g once per year <strong>an</strong>d communicat<strong>in</strong>g project milestones through email.<br />
Although we <strong>in</strong>itially <strong>in</strong>tended <strong>the</strong> Steer<strong>in</strong>g Committee to be <strong>the</strong> primary decision mak<strong>in</strong>g<br />
body for <strong>the</strong> ECN, <strong>the</strong> Core Research Team took on m<strong>an</strong>y <strong>of</strong> <strong>the</strong> responsibilities <strong>of</strong> <strong>the</strong><br />
Steer<strong>in</strong>g Committee <strong>an</strong>d provided signific<strong>an</strong>t expertise <strong>in</strong> measurement, evaluation, <strong>an</strong>d<br />
engagement. Although not <strong>in</strong>itially <strong>in</strong>tended, <strong>the</strong> Core Research Team members suggested<br />
meet<strong>in</strong>g monthly to ga<strong>in</strong> momentum on <strong>the</strong> project. The Steer<strong>in</strong>g Committee still functioned,<br />
however, with a more distal role th<strong>an</strong> we orig<strong>in</strong>ally <strong>in</strong>tended, with strategic th<strong>in</strong>k<strong>in</strong>g, fund<strong>in</strong>g<br />
opportunities, <strong>an</strong>d prov<strong>in</strong>cial connection be<strong>in</strong>g <strong>the</strong> ma<strong>in</strong> areas <strong>of</strong> focus. To compensate for<br />
this limited engagement, we have <strong>in</strong>stead engaged with members <strong>of</strong> <strong>the</strong> Steer<strong>in</strong>g Committee<br />
on <strong>an</strong> <strong>in</strong>dividual basis depend<strong>in</strong>g on <strong>the</strong> task at h<strong>an</strong>d or <strong>the</strong> stage <strong>of</strong> <strong>the</strong> project, reflect<strong>in</strong>g a<br />
need to connect on a more personal level.<br />
Stakeholder Engagement: Underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong> <strong>Evaluation</strong> Needs <strong>an</strong>d Capacities <strong>of</strong> <strong>the</strong><br />
<strong>Field</strong><br />
<strong>Build<strong>in</strong>g</strong> <strong>an</strong> agenda for adv<strong>an</strong>c<strong>in</strong>g evaluation practices <strong>in</strong> <strong>the</strong> field <strong>of</strong> ECD required determ<strong>in</strong><strong>in</strong>g<br />
evaluation needs <strong>an</strong>d assets, which <strong>in</strong> turn required extensive stakeholder engagement.<br />
Stakeholder, <strong>in</strong> this sense, refers to <strong>in</strong>dividuals who might be affected by decisions about <strong>an</strong><br />
evaluation agenda (Freem<strong>an</strong>, 2010) – namely funders, evaluators, nonpr<strong>of</strong>it <strong>an</strong>d government<br />
employees, <strong>an</strong>d academics. Consistent with a CBPR approach, we w<strong>an</strong>ted to develop <strong>an</strong> agenda<br />
that had mutual benefit across <strong>the</strong> ECD field (Israel et al., 1998). It was <strong>the</strong>refore essential to<br />
underst<strong>an</strong>d <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> field from <strong>the</strong> perspectives <strong>of</strong> stakeholders work<strong>in</strong>g<br />
across a r<strong>an</strong>ge <strong>of</strong> contexts. A subsequent goal <strong>of</strong> stakeholder engagement was to ga<strong>in</strong> a wider<br />
sense <strong>of</strong> ownership <strong>an</strong>d momentum across <strong>the</strong> ECD field towards work<strong>in</strong>g collectively to<br />
address <strong>the</strong> evaluation needs that exist.<br />
To accomplish <strong>the</strong> needs <strong>an</strong>d assets assessment, we used three primary methods <strong>of</strong><br />
stakeholder engagement across <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta: survey<strong>in</strong>g stakeholders, prim<strong>in</strong>g<br />
stakeholders <strong>an</strong>d consult<strong>in</strong>g stakeholders. The methods were complementary, <strong>in</strong> that<br />
<strong>an</strong> evaluation capacity build<strong>in</strong>g survey <strong>an</strong>d stimulus paper were sent to <strong>the</strong> particip<strong>an</strong>ts <strong>in</strong><br />
adv<strong>an</strong>ce <strong>of</strong> attend<strong>in</strong>g <strong>the</strong> forums, <strong>an</strong>d <strong>the</strong> survey results <strong>an</strong>d stimulus paper were used to spark<br />
discussion throughout <strong>the</strong> forums. We will describe each method <strong>of</strong> engagement <strong>in</strong> more<br />
detail with its contribution to underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> ECD field.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
67<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> Survey<br />
It is import<strong>an</strong>t to recognize that evaluation is not simply about develop<strong>in</strong>g <strong>in</strong> people <strong>an</strong>d<br />
org<strong>an</strong>izations <strong>the</strong> capacity to do evaluation but also about develop<strong>in</strong>g <strong>the</strong> capacity to use<br />
evaluation (Cous<strong>in</strong>s et al., 2014). The construct <strong>of</strong> evaluation capacity has been conceptualized<br />
<strong>an</strong>d def<strong>in</strong>ed <strong>in</strong> <strong>the</strong> literature <strong>an</strong>d operationalized <strong>in</strong> <strong>the</strong> form <strong>of</strong> assessment tools (e.g., Lab<strong>in</strong>,<br />
2014; Preskill & Boyle, 2008; Taylor-Ritzler, Suarez-Balcazar, Garcia-Iriarte, Henry, & Balcazar,<br />
2013). However, m<strong>an</strong>y <strong>of</strong> <strong>the</strong>se frameworks <strong>an</strong>d tools are not context specific, mak<strong>in</strong>g <strong>the</strong>m less<br />
reflective <strong>of</strong> <strong>the</strong> complex field <strong>of</strong> ECD <strong>an</strong>d thus were not appropriate for underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong><br />
<strong>in</strong>dividual <strong>an</strong>d org<strong>an</strong>izational capacity <strong>of</strong> our stakeholders. As a result, it was necessary to draw<br />
on <strong>the</strong>se frameworks <strong>an</strong>d tools, through <strong>an</strong> extensive review <strong>of</strong> <strong>the</strong> literature <strong>an</strong>d to develop a<br />
survey that better reflected our context. A doctoral student (El Hassar) took a leadership role<br />
<strong>in</strong> develop<strong>in</strong>g <strong>the</strong> survey as part <strong>of</strong> her doctoral <strong>the</strong>sis. She engaged <strong>the</strong> Core Research Team at<br />
several po<strong>in</strong>ts <strong>in</strong> <strong>the</strong> development process such as determ<strong>in</strong><strong>in</strong>g <strong>the</strong> most contextually relev<strong>an</strong>t<br />
survey components <strong>an</strong>d items, measurement scales, <strong>an</strong>d survey format (onl<strong>in</strong>e). The f<strong>in</strong>al<br />
survey consisted <strong>of</strong> items across three ma<strong>in</strong> areas <strong>of</strong> evaluation capacity: <strong>in</strong>dividual capacity<br />
(e.g., attitudes, motivation, knowledge, <strong>an</strong>d skills), org<strong>an</strong>izational capacity (e.g., leadership,<br />
org<strong>an</strong>izational processes, <strong>an</strong>d available resources), <strong>an</strong>d tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d pr<strong>of</strong>essional development<br />
(e.g., tra<strong>in</strong><strong>in</strong>g experiences, desires for capacity build<strong>in</strong>g). Stakeholders that were <strong>in</strong>vited to <strong>the</strong><br />
evaluation capacity build<strong>in</strong>g forums (described below) were purposefully selected based on<br />
experience, expertise, or leadership <strong>in</strong> evaluation <strong>an</strong>d/or early childhood development <strong>in</strong> <strong>the</strong><br />
prov<strong>in</strong>ce <strong>of</strong> Alberta. The survey was sent out to <strong>the</strong>se stakeholders prior to <strong>the</strong>ir attend<strong>an</strong>ce<br />
at <strong>the</strong> prov<strong>in</strong>cial forums. Approximately 164 surveys were sent to <strong>in</strong>vitees, <strong>an</strong>d a total <strong>of</strong> 101<br />
surveys were returned. As mentioned, <strong>the</strong> result<strong>in</strong>g data were presented back to particip<strong>an</strong>ts<br />
at <strong>the</strong> forums.<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> Forums<br />
In total, <strong>the</strong> ECN hosted four forums across <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta <strong>in</strong> W<strong>in</strong>ter/Spr<strong>in</strong>g 2016<br />
as a way to fur<strong>the</strong>r underst<strong>an</strong>d <strong>the</strong> collective evaluation needs <strong>an</strong>d capacities <strong>of</strong> <strong>the</strong> ECD field.<br />
The specific purpose <strong>of</strong> <strong>the</strong> forums was to engage <strong>in</strong>fluential stakeholders <strong>in</strong> conversation<br />
about <strong>the</strong> evaluation barriers, facilitators, <strong>an</strong>d needs experienced <strong>in</strong> <strong>the</strong>ir work <strong>an</strong>d to generate<br />
<strong>in</strong>novative <strong>an</strong>d collective solutions for address<strong>in</strong>g <strong>the</strong>m. A total <strong>of</strong> 164 leaders from 78 different<br />
org<strong>an</strong>izations (fund<strong>in</strong>g, government, nonpr<strong>of</strong>it, university, <strong>an</strong>d consult<strong>in</strong>g) were <strong>in</strong>vited, <strong>an</strong>d a<br />
total <strong>of</strong> 122 attended. A large facilitation team guided <strong>an</strong>d stimulated discussion dur<strong>in</strong>g each<br />
forum <strong>an</strong>d comprised a mixed group that mirrored <strong>the</strong> diversity <strong>of</strong> <strong>the</strong> stakeholders. The team<br />
<strong>in</strong>cluded current <strong>an</strong>d retired leaders <strong>in</strong> <strong>the</strong> ECD field from school boards, fund<strong>in</strong>g agencies,<br />
<strong>an</strong>d nonpr<strong>of</strong>its; academics (Gokiert & K<strong>in</strong>gsley) <strong>an</strong>d graduate students from CUP (El Hassar,<br />
T<strong>in</strong>k, & Tremblay); <strong>an</strong>d a highly-respected consult<strong>an</strong>t <strong>in</strong> <strong>the</strong> social sector. For each forum,<br />
<strong>the</strong>re was a primary moderator who led <strong>the</strong> agenda <strong>an</strong>d several co-facilitators sat at each<br />
discussion table to guide conversation, take notes, <strong>an</strong>d report back to <strong>the</strong> larger group. The<br />
agenda for <strong>the</strong> forums was co-created by <strong>the</strong> Core Research Team, <strong>the</strong> Project M<strong>an</strong>agement<br />
Team, <strong>an</strong>d <strong>the</strong> same external consult<strong>an</strong>t who aided <strong>the</strong> facilitation.<br />
Volume 3/Issue 2/Fall 2017
68 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
To provide contextual ground<strong>in</strong>g, shared l<strong>an</strong>guage <strong>an</strong>d def<strong>in</strong>itions for <strong>the</strong> forum, <strong>an</strong>d to<br />
spark some <strong>in</strong>itial ideas, we developed <strong>an</strong>d sent a stimulus paper to all <strong>the</strong> <strong>in</strong>vitees (<strong>Evaluation</strong><br />
<strong>Capacity</strong> <strong>Network</strong>, 2015). The stimulus paper comprised five ma<strong>in</strong> sections that situated<br />
evaluation capacity build<strong>in</strong>g <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood <strong>an</strong>d asked particip<strong>an</strong>ts to th<strong>in</strong>k<br />
about <strong>the</strong> possibilities for adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation <strong>in</strong> <strong>the</strong> field. Discussion questions<br />
were <strong>in</strong>tegrated throughout <strong>an</strong>d provided <strong>in</strong> <strong>an</strong> accomp<strong>an</strong>y<strong>in</strong>g worksheet to generate ideas <strong>in</strong><br />
preparation for <strong>the</strong> forum (e.g., What ECD evaluation framework, if <strong>an</strong>y, have you found most<br />
useful <strong>an</strong>d why? What characteristics does your org<strong>an</strong>ization have that support <strong>an</strong> evaluation<br />
learn<strong>in</strong>g culture?). The stimulus paper was one <strong>of</strong> several engagement techniques implemented<br />
with <strong>the</strong> aim <strong>of</strong> stimulat<strong>in</strong>g <strong>in</strong>novative, ch<strong>an</strong>ge-focused ideas. We also used a comb<strong>in</strong>ation<br />
<strong>of</strong> small <strong>an</strong>d larger table discussions, group plenaries, a world café (www.<strong>the</strong>worldcafe.com),<br />
<strong>an</strong>d a design th<strong>in</strong>k<strong>in</strong>g challenge (St<strong>an</strong>ford University, 2017). The facilitation team made<br />
modifications to <strong>the</strong> agenda through a reflective debrief after each forum.<br />
The <strong>in</strong>formation shared dur<strong>in</strong>g each forum was captured on particip<strong>an</strong>t worksheets that<br />
were given to stakeholders at <strong>the</strong> start <strong>of</strong> <strong>the</strong> day, <strong>in</strong> addition to sticky notes used dur<strong>in</strong>g<br />
some <strong>of</strong> <strong>the</strong> group activities. This <strong>in</strong>formation was electronically <strong>in</strong>putted <strong>an</strong>d a basic <strong>the</strong>matic<br />
<strong>an</strong>alysis was conducted to org<strong>an</strong>ize <strong>the</strong> ideas. These ideas were presented as three action areas<br />
for <strong>the</strong> ECN to focus on to adv<strong>an</strong>ce evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood <strong>an</strong>d were<br />
reported back to stakeholders <strong>in</strong> a “What We Heard” summary document (T<strong>in</strong>k, K<strong>in</strong>gsley, &<br />
Gokiert, 2016). In addition, feedback from stakeholders dur<strong>in</strong>g <strong>the</strong> forums <strong>in</strong>dicated that <strong>the</strong>y<br />
w<strong>an</strong>ted us to send <strong>the</strong> survey out more broadly <strong>in</strong> order to reach all levels <strong>of</strong> <strong>an</strong> org<strong>an</strong>ization,<br />
from frontl<strong>in</strong>e staff through to <strong>the</strong> leadership. With this advice, we modified <strong>the</strong> survey to be<br />
reflective <strong>of</strong> a more diverse audience <strong>an</strong>d redistributed it. We sent <strong>the</strong> survey to all particip<strong>an</strong>ts<br />
<strong>an</strong>d asked that <strong>the</strong>y forward it through <strong>the</strong>ir org<strong>an</strong>ization to staff us<strong>in</strong>g a snowball technique<br />
<strong>of</strong> sampl<strong>in</strong>g. The second adm<strong>in</strong>istration generated over 329 responses. The survey data was<br />
reported back to stakeholders <strong>in</strong> a report that is posted on <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong><br />
website (T<strong>in</strong>k, Gokiert, K<strong>in</strong>gsley, & El Hassar, 2017).<br />
Au<strong>the</strong>ntic stakeholder engagement is not <strong>an</strong> easy or straightforward process, even with<br />
subst<strong>an</strong>tial experience. For this reason, we relied heavily on pre-exist<strong>in</strong>g, well-established,<br />
<strong>an</strong>d trust<strong>in</strong>g relationships with pivotal stakeholders <strong>an</strong>d org<strong>an</strong>izations across <strong>the</strong> prov<strong>in</strong>ce<br />
(developed over <strong>the</strong> past decade <strong>of</strong> CUP’s history). One <strong>of</strong> <strong>the</strong> ECN partners, <strong>the</strong> Muttart<br />
Foundation, had carried out a series <strong>of</strong> forums across Alberta <strong>an</strong>d Saskatchew<strong>an</strong> with <strong>the</strong><br />
ECD sector to determ<strong>in</strong>e a system <strong>of</strong> early learn<strong>in</strong>g <strong>an</strong>d care (Muttart Foundation, 2013). We<br />
reached out to <strong>the</strong> Muttart Foundation for advice about <strong>the</strong>ir engagement process, costs, ideal<br />
locations <strong>an</strong>d venues, skilled facilitators, <strong>an</strong>d <strong>the</strong> leaders <strong>the</strong>y had previously <strong>in</strong>vited. It was<br />
under <strong>the</strong>ir guid<strong>an</strong>ce that we recruited facilitators <strong>in</strong> each region as <strong>the</strong>y had <strong>the</strong> relationships<br />
necessary to <strong>in</strong>crease <strong>the</strong> likelihood that stakeholders would participate. The Foundation used<br />
stimulus papers for each <strong>of</strong> <strong>the</strong>ir forums, <strong>an</strong>d so we adopted this idea <strong>an</strong>d created a stimulus<br />
paper to contextually ground our forums. They also provided <strong>in</strong>sight <strong>in</strong>to <strong>the</strong> pr<strong>of</strong>essional<br />
cultures <strong>of</strong> each location so we could adapt our style <strong>of</strong> engagement. For example, one <strong>of</strong> <strong>the</strong><br />
larger cities <strong>in</strong> which we hosted a forum had more <strong>of</strong> a ‘corporate’ culture, which was quite<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
69<br />
different from <strong>the</strong> ‘relaxed’ culture <strong>of</strong> one <strong>of</strong> <strong>the</strong> smaller rural locations. We made subsequent<br />
decisions about <strong>the</strong> locations based on <strong>the</strong>se differences, with <strong>the</strong> larger urb<strong>an</strong> forums held<br />
at a conference centre <strong>an</strong>d hotel <strong>in</strong> <strong>the</strong> downtown core, compared to <strong>the</strong> rural forums which<br />
were held at a public library <strong>an</strong>d a school.<br />
We w<strong>an</strong>ted <strong>the</strong> forums to be ch<strong>an</strong>ge-oriented ra<strong>the</strong>r th<strong>an</strong> deficit-focused. With this <strong>in</strong><br />
m<strong>in</strong>d, we purposefully <strong>in</strong>vited leaders who we believed were driven to f<strong>in</strong>d unique solutions to<br />
complex problems, <strong>an</strong>d possessed <strong>the</strong> <strong>in</strong>fluence to effect such ch<strong>an</strong>ge. Despite <strong>the</strong>se <strong>in</strong>tentions,<br />
<strong>the</strong> extent to which this made a difference to <strong>the</strong> quality <strong>of</strong> <strong>the</strong> discussion is difficult to gauge.<br />
In focus<strong>in</strong>g on <strong>in</strong>vit<strong>in</strong>g ch<strong>an</strong>ge-oriented leaders, we also restricted <strong>the</strong> number <strong>an</strong>d breadth<br />
<strong>of</strong> perspectives shap<strong>in</strong>g <strong>the</strong> direction <strong>of</strong> <strong>the</strong> ECN, particularly those <strong>of</strong> frontl<strong>in</strong>e staff <strong>an</strong>d<br />
service providers.<br />
Position<strong>in</strong>g <strong>the</strong> forums around ch<strong>an</strong>ge created expectations from our stakeholders for<br />
action to result from <strong>the</strong> discussions. This was exacerbated by <strong>the</strong> fact that a number <strong>of</strong> o<strong>the</strong>r<br />
consultations had been recently conducted <strong>in</strong> <strong>the</strong> field. For this reason, it was import<strong>an</strong>t to<br />
provide time-sensitive reports to summarize learn<strong>in</strong>g, <strong>an</strong>d to f<strong>in</strong>d a bal<strong>an</strong>ce between community<br />
need <strong>an</strong>d academic def<strong>in</strong>itions <strong>of</strong> rigour (K<strong>in</strong>gsley & Chapm<strong>an</strong>, 2013). In this process, our<br />
ability to respond to dem<strong>an</strong>ds for action has been impacted by our own limited resource, time<br />
<strong>an</strong>d expertise, which must grow signific<strong>an</strong>tly if we are to adequately respond to <strong>the</strong> needs<br />
presented dur<strong>in</strong>g <strong>the</strong> forums. This is fur<strong>the</strong>r tested by <strong>the</strong> <strong>in</strong>terest <strong>an</strong>d awareness generated<br />
through <strong>the</strong> engagement process. Rais<strong>in</strong>g <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> ECN through public dialogue has<br />
also resulted <strong>in</strong> <strong>an</strong> <strong>in</strong>creased number <strong>of</strong> requests for evaluation support <strong>an</strong>d resources. In <strong>an</strong><br />
attempt to meet this need, we have drawn upon graduate students as a source <strong>of</strong> capacity for<br />
<strong>the</strong> ECN, <strong>the</strong> details <strong>of</strong> which are described <strong>in</strong> <strong>the</strong> next section.<br />
Student Engagement: Tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d Mentorship<br />
Graduate student engagement has been fundamental to <strong>the</strong> success <strong>of</strong> <strong>the</strong> ECN, as <strong>the</strong>y<br />
have provided signific<strong>an</strong>t capacity. Some students have completed <strong>the</strong>ir masters’ <strong>an</strong>d doctoral<br />
research with <strong>the</strong> ECN, while o<strong>the</strong>rs contributed through <strong>in</strong>dependent studies, research<br />
assist<strong>an</strong>tships, practicums <strong>an</strong>d course placements, <strong>an</strong>d volunteer<strong>in</strong>g. For example, two students<br />
jo<strong>in</strong>ed <strong>the</strong> project m<strong>an</strong>agement team to fulfill a 150-hour practicum as part <strong>of</strong> <strong>an</strong> embedded<br />
graduate certificate <strong>in</strong> community-based research <strong>an</strong>d evaluation. Two graduate students from<br />
<strong>the</strong> Master <strong>of</strong> Arts <strong>in</strong> Communication <strong>an</strong>d Technology program <strong>in</strong> <strong>the</strong> Faculty <strong>of</strong> Extension<br />
developed a communications pl<strong>an</strong> <strong>an</strong>d website for <strong>the</strong> ECN as part <strong>of</strong> <strong>an</strong> <strong>in</strong>dependent study<br />
course. One <strong>of</strong> <strong>the</strong>se students became fur<strong>the</strong>r <strong>in</strong>volved <strong>in</strong> <strong>the</strong> ECN <strong>an</strong>d completed her f<strong>in</strong>al<br />
capstone project with <strong>the</strong> ECN. Two doctoral c<strong>an</strong>didates are pursu<strong>in</strong>g <strong>the</strong>ir dissertation<br />
research focused on different research questions relat<strong>in</strong>g to <strong>the</strong> ECN (El Hassar, Poth, Gokiert,<br />
K<strong>in</strong>gsley, & Krishn<strong>an</strong>, 2016; Gokiert et al., 2017; Tremblay et al., 2016). In <strong>the</strong> three-year sp<strong>an</strong><br />
<strong>of</strong> <strong>the</strong> ECN thus far, eleven graduate students <strong>an</strong>d one postdoctoral fellow have been mentored<br />
through, <strong>an</strong>d provided support to, <strong>the</strong> ECN. Mentorship is generally provided by academics<br />
that are community-engaged scholars <strong>the</strong>mselves <strong>an</strong>d/or who specialize <strong>in</strong> evaluation.<br />
Volume 3/Issue 2/Fall 2017
70 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Emerg<strong>in</strong>g Community-Engaged Scholars<br />
The <strong>in</strong>tent <strong>of</strong> engag<strong>in</strong>g students <strong>in</strong> <strong>the</strong> ECN is to support <strong>the</strong>ir development as communityengaged<br />
scholars <strong>an</strong>d evaluators. The focus on such enh<strong>an</strong>ced learn<strong>in</strong>g opportunities is based<br />
on a grow<strong>in</strong>g need for pr<strong>of</strong>essionals who are equipped to navigate a social sector with complex<br />
challenges (Armitage & Levac, 2015; C<strong>an</strong>tor, DeLauer, Mart<strong>in</strong>, & Rog<strong>an</strong>, 2015). Students seem<br />
to be recogniz<strong>in</strong>g this need to exp<strong>an</strong>d <strong>the</strong>ir learn<strong>in</strong>g <strong>an</strong>d build new skills, which has resulted <strong>in</strong><br />
<strong>an</strong> <strong>in</strong>creas<strong>in</strong>g number seek<strong>in</strong>g community-based research <strong>an</strong>d evaluation opportunities. These<br />
emerg<strong>in</strong>g scholars are acquir<strong>in</strong>g skills, knowledge <strong>an</strong>d experiences that would not be available<br />
through conventional academic tra<strong>in</strong><strong>in</strong>g. For example, students are exposed to several research<br />
methods <strong>in</strong> <strong>an</strong> <strong>in</strong>terdiscipl<strong>in</strong>ary context, <strong>an</strong>d developed skills <strong>in</strong> knowledge mobilization <strong>an</strong>d<br />
engagement with academic <strong>an</strong>d non-academic audiences. More specifically, <strong>the</strong>y have facilitated<br />
meet<strong>in</strong>gs <strong>an</strong>d forums, produced grey literature <strong>an</strong>d scholarly m<strong>an</strong>uscripts, <strong>an</strong>d developed <strong>an</strong>d<br />
ma<strong>in</strong>ta<strong>in</strong>ed <strong>the</strong> ECN’s onl<strong>in</strong>e presence through email campaigns <strong>an</strong>d blog posts.<br />
Community Service Learn<strong>in</strong>g<br />
In addition to prepar<strong>in</strong>g students as community-engaged scholars, <strong>the</strong> ECN also supports<br />
students to develop <strong>the</strong>ir skills as evaluators. As mentioned, <strong>the</strong>re is a subst<strong>an</strong>tial need for<br />
evaluation support <strong>in</strong> <strong>the</strong> social sector, which was reiterated by <strong>the</strong> <strong>in</strong>formation ga<strong>the</strong>red<br />
dur<strong>in</strong>g <strong>the</strong> forums. To meet this need, while provid<strong>in</strong>g experiential learn<strong>in</strong>g <strong>in</strong> evaluation for<br />
students, service learn<strong>in</strong>g placements were established. Service learn<strong>in</strong>g is a method to enh<strong>an</strong>ce<br />
students’ learn<strong>in</strong>g <strong>an</strong>d development through org<strong>an</strong>ized service experiences <strong>in</strong> <strong>the</strong> community,<br />
<strong>an</strong>d is <strong>in</strong>tegrated <strong>in</strong>to academic curriculum (Taylor et al., 2015). Service learn<strong>in</strong>g, particularly <strong>in</strong><br />
community-based participatory projects, is widely considered to exp<strong>an</strong>d <strong>the</strong> student learn<strong>in</strong>g,<br />
provid<strong>in</strong>g opportunities to develop critical th<strong>in</strong>k<strong>in</strong>g <strong>an</strong>d a sense <strong>of</strong> civic responsibility (Taylor<br />
et al., 2015). Service learn<strong>in</strong>g also provides additional capacity to <strong>the</strong> org<strong>an</strong>izations <strong>in</strong> which<br />
<strong>the</strong> placements occur. Learn<strong>in</strong>g opportunities are <strong>the</strong>refore <strong>in</strong>tended to be <strong>of</strong> equal benefit to<br />
both <strong>the</strong> student <strong>an</strong>d <strong>the</strong> recipient <strong>of</strong> <strong>the</strong> service (Furco, 1996).<br />
The service learn<strong>in</strong>g placements <strong>in</strong> this project are embedded with<strong>in</strong> three graduate-level<br />
evaluation courses at <strong>the</strong> University <strong>of</strong> Alberta: Health Promotion Pl<strong>an</strong>n<strong>in</strong>g <strong>an</strong>d <strong>Evaluation</strong><br />
<strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health, Program <strong>Evaluation</strong> <strong>in</strong> <strong>the</strong> Department <strong>of</strong> Educational<br />
Psychology, <strong>an</strong>d Program Pl<strong>an</strong>n<strong>in</strong>g <strong>an</strong>d <strong>Evaluation</strong> <strong>in</strong> <strong>the</strong> Department <strong>of</strong> Hum<strong>an</strong> Ecology.<br />
To fulfill <strong>the</strong>ir course requirements, graduate students are asked to work with community<br />
org<strong>an</strong>izations to build contextually relev<strong>an</strong>t evaluation pl<strong>an</strong>s. To build a comprehensive<br />
pl<strong>an</strong>, <strong>the</strong>y meet with org<strong>an</strong>izations several times to ga<strong>in</strong> a contextual underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> <strong>the</strong>ir<br />
evaluation needs, develop a logic model <strong>an</strong>d ga<strong>in</strong> feedback from staff. The course <strong>in</strong>structors<br />
mentor <strong>the</strong> students to enh<strong>an</strong>ce learn<strong>in</strong>g <strong>an</strong>d serve as a form <strong>of</strong> quality assur<strong>an</strong>ce for <strong>the</strong><br />
community <strong>an</strong>d government agencies. This approach has been generally positive; however,<br />
<strong>the</strong>re is a need to fur<strong>the</strong>r extend this service learn<strong>in</strong>g to assist org<strong>an</strong>izations with evaluation<br />
beyond <strong>the</strong> creation <strong>of</strong> <strong>an</strong> evaluation pl<strong>an</strong>. To this end, adv<strong>an</strong>ced courses <strong>in</strong> evaluation are<br />
required that <strong>in</strong>volve more extensive <strong>an</strong>d concerted student engagement over time. This need<br />
to build <strong>the</strong> evaluation capacity <strong>of</strong> university students <strong>in</strong> C<strong>an</strong>ada is well supported <strong>in</strong> <strong>the</strong><br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
71<br />
literature (e.g. McDavid & Dev<strong>in</strong>e, 2009; McSh<strong>an</strong>e et al., 2015), <strong>an</strong>d will become a focus for<br />
<strong>the</strong> ECN mov<strong>in</strong>g forward.<br />
Fund<strong>in</strong>g Graduate Students<br />
Due to limited funds to engage graduate students <strong>in</strong> <strong>the</strong> ECN, we have sought o<strong>the</strong>r fund<strong>in</strong>g<br />
opportunities to supplement student <strong>in</strong>volvement. Several funders have supported <strong>the</strong> ECN’s<br />
student engagement <strong>an</strong>d created various learn<strong>in</strong>g opportunities. The Women <strong>an</strong>d Children’s<br />
Health Research Institute (WCHRI) based at <strong>the</strong> University <strong>of</strong> Alberta has several student<br />
fund<strong>in</strong>g streams <strong>in</strong>clud<strong>in</strong>g <strong>the</strong> Patient <strong>an</strong>d Community Engagement Tra<strong>in</strong><strong>in</strong>g program<br />
<strong>an</strong>d conference travel. Through this program, graduate students <strong>an</strong>d postdoctoral fellows<br />
conduct<strong>in</strong>g engaged research are funded for 8 months to support <strong>the</strong>ir work. In each fund<strong>in</strong>g<br />
year, a community <strong>of</strong> practice is formed to discuss issues relat<strong>in</strong>g to engagement. One doctoral<br />
student (El Hassar) <strong>an</strong>d one postdoctoral fellow (K<strong>in</strong>gsley) have participated <strong>in</strong> <strong>the</strong> tra<strong>in</strong><strong>in</strong>g<br />
program <strong>an</strong>d received travel gr<strong>an</strong>ts to present <strong>the</strong>ir research at conferences. Mitacs, a C<strong>an</strong>adi<strong>an</strong><br />
fund<strong>in</strong>g agency that builds partnerships between academia <strong>an</strong>d nonpr<strong>of</strong>it or <strong>in</strong>dustry partners,<br />
has also provided subst<strong>an</strong>tial graduate student fund<strong>in</strong>g support. Mitacs funded a postdoctoral<br />
fellow (K<strong>in</strong>gsley) for three years, two doctoral (Tremblay) <strong>an</strong>d one masters’ student up to one<br />
year, to assist community agencies with evaluation <strong>an</strong>d evaluation capacity build<strong>in</strong>g.<br />
Graduate students have added signific<strong>an</strong>t capacity to <strong>the</strong> ECN; however, we have also run<br />
<strong>in</strong>to some challenges with a project reli<strong>an</strong>t on such extensive student <strong>in</strong>volvement. F<strong>in</strong>d<strong>in</strong>g<br />
fund<strong>in</strong>g for students has been difficult at times <strong>an</strong>d as mentioned, it was necessary to seek<br />
alternative forms <strong>of</strong> fund<strong>in</strong>g. Apply<strong>in</strong>g for this fund<strong>in</strong>g consumes available resources such as<br />
time. In addition, students – no matter <strong>the</strong>ir level <strong>of</strong> experience – require mentorship, or at<br />
<strong>the</strong> very least, m<strong>an</strong>agement <strong>an</strong>d supervision. Allocat<strong>in</strong>g university mentors or supervisors who<br />
have sufficient time to work with students, above <strong>the</strong>ir regular academic responsibilities, is also<br />
challeng<strong>in</strong>g. For this reason, much <strong>of</strong> this workload falls on <strong>the</strong> project m<strong>an</strong>agement team, <strong>an</strong>d<br />
is <strong>the</strong> reason that hav<strong>in</strong>g a postdoctoral fellow <strong>in</strong>volved <strong>in</strong> <strong>the</strong> project has been essential for<br />
shar<strong>in</strong>g <strong>the</strong> supervision load <strong>of</strong> students.<br />
An additional issue relates to student retention, as most students are only available on a<br />
short-term basis, ei<strong>the</strong>r because <strong>the</strong>y f<strong>in</strong>ish <strong>the</strong>ir degrees, accept alternative opportunities, or<br />
because our ability to fund <strong>the</strong>m dim<strong>in</strong>ishes. Although we have had some students volunteer<br />
for us <strong>in</strong> <strong>the</strong> past, we would prefer to be able to pay <strong>in</strong>dividuals for <strong>the</strong>ir time. F<strong>in</strong>ally, as<br />
with all CBPR projects, <strong>the</strong>re c<strong>an</strong> <strong>of</strong>ten be a disson<strong>an</strong>ce between community <strong>an</strong>d academic<br />
timel<strong>in</strong>es. Graduate students have milestones <strong>the</strong>y need to achieve before <strong>the</strong>y c<strong>an</strong> progress<br />
with <strong>the</strong>ir research (e.g., a c<strong>an</strong>didacy exam), yet <strong>the</strong> project needs to progress regardless. This<br />
c<strong>an</strong> put additional pressure on <strong>the</strong> student dur<strong>in</strong>g what is already a stressful process <strong>an</strong>d has <strong>the</strong><br />
potential to prevent <strong>the</strong> project from mov<strong>in</strong>g ahead as pl<strong>an</strong>ned. Similarly, academic st<strong>an</strong>dards<br />
<strong>an</strong>d expectations that <strong>in</strong>form student processes <strong>an</strong>d products are sometimes at odds with<br />
<strong>the</strong> flexible, responsive approaches needed when partner<strong>in</strong>g with community. For example,<br />
one <strong>of</strong> <strong>the</strong> students (El Hassar) took <strong>the</strong> lead on <strong>the</strong> development <strong>of</strong> <strong>the</strong> evaluation capacity<br />
survey as part <strong>of</strong> her dissertation <strong>an</strong>d engaged partners dur<strong>in</strong>g this process. In addition<br />
Volume 3/Issue 2/Fall 2017
72 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
to reflect<strong>in</strong>g conventional st<strong>an</strong>dards <strong>of</strong> validity, this engagement (<strong>an</strong>d <strong>the</strong> tool itself) also<br />
reflects <strong>an</strong> alternative form <strong>of</strong> rigour that more closely reflected a community-based research<br />
approach (K<strong>in</strong>gsley & Chapm<strong>an</strong>, 2013). To allow for a broader conceptualization <strong>of</strong> rigour,<br />
it is import<strong>an</strong>t to form graduate committees with faculty who have some underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong><br />
community-based <strong>an</strong>d engaged research to avoid potential friction between academic <strong>an</strong>d<br />
community needs <strong>an</strong>d expectations.<br />
Conclud<strong>in</strong>g Remarks<br />
The value <strong>of</strong> co-created underst<strong>an</strong>d<strong>in</strong>gs is highlighted by <strong>the</strong> evaluation-focused efforts <strong>of</strong><br />
McSh<strong>an</strong>e <strong>an</strong>d colleagues (2015) who state, “community engagement is <strong>of</strong>ten touted as a goal<br />
for universities <strong>an</strong>d community collaboration is <strong>in</strong>creas<strong>in</strong>gly viewed as favourable <strong>in</strong> research”<br />
(p.149). <strong>Evaluation</strong> experts have long recognized <strong>the</strong> import<strong>an</strong>ce <strong>of</strong> <strong>the</strong> <strong>in</strong>terests, views,<br />
<strong>in</strong>volvement, needs, <strong>an</strong>d roles <strong>of</strong> all stakeholders <strong>in</strong> evaluation practice <strong>an</strong>d <strong>the</strong>ory (e.g., Alk<strong>in</strong>,<br />
2004; Cockerill, Myers, & Allm<strong>an</strong>, 2000; Cous<strong>in</strong>s & Earl, 1992). Us<strong>in</strong>g a CBPR approach <strong>an</strong>d<br />
through multiple forms <strong>of</strong> engagement, <strong>the</strong> ECN was developed to br<strong>in</strong>g toge<strong>the</strong>r m<strong>an</strong>y<br />
voices from across academic, government, nonpr<strong>of</strong>it <strong>an</strong>d consult<strong>in</strong>g contexts around <strong>the</strong> issue<br />
<strong>of</strong> evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> ECD.<br />
The purpose <strong>of</strong> this reflective essay was to provide <strong>an</strong> <strong>in</strong>-depth account <strong>of</strong> <strong>the</strong> development<br />
<strong>of</strong> <strong>the</strong> network <strong>an</strong>d reflect on our engagement processes. Illustrat<strong>in</strong>g <strong>the</strong> context for <strong>the</strong><br />
development <strong>of</strong> <strong>the</strong> ECN, we provided a detailed description <strong>of</strong> <strong>the</strong> purpose <strong>an</strong>d process<br />
<strong>of</strong> engagement with partners, stakeholders, <strong>an</strong>d students <strong>in</strong> <strong>the</strong> development <strong>of</strong> a prov<strong>in</strong>cial<br />
agenda for <strong>the</strong> ECN, reflect<strong>in</strong>g on each method <strong>an</strong>d its impact on <strong>the</strong> outcomes we were able<br />
to achieve. In <strong>the</strong>se conclud<strong>in</strong>g remarks, we <strong>of</strong>fer some f<strong>in</strong>al considerations for communityengaged<br />
scholars relat<strong>in</strong>g to <strong>the</strong>se three forms <strong>of</strong> engagement, provide details <strong>of</strong> our <strong>in</strong>tended<br />
next steps for <strong>the</strong> ECN, <strong>an</strong>d <strong>in</strong>vite <strong>in</strong>dividuals <strong>an</strong>d org<strong>an</strong>izations to contact us for more detailed<br />
<strong>in</strong>formation.<br />
In a summary <strong>of</strong> learn<strong>in</strong>g from a conference co-hosted by Community-Campus<br />
Partnerships for Health (CCPH) <strong>an</strong>d <strong>the</strong> University <strong>of</strong> Guelph, Wenger, Hawk<strong>in</strong>s, <strong>an</strong>d Seifer<br />
(2011, 2012) articulated <strong>the</strong> need for flexibility <strong>in</strong> community-engaged scholarship. Through<br />
our engagement with ECN partners, we also learned early on that we needed to be flexible<br />
<strong>in</strong> our approach. Despite a comprehensive pl<strong>an</strong> for govern<strong>an</strong>ce – three committees with<br />
specific roles, responsibilities, <strong>an</strong>d expectations – it did not play out <strong>in</strong> practice as we orig<strong>in</strong>ally<br />
<strong>in</strong>tended. This did not appear to negatively impact <strong>the</strong> quality <strong>of</strong> <strong>the</strong> process, but did require<br />
us to adjust our engagement expectations <strong>an</strong>d modify our collaboration processes to align<br />
with <strong>the</strong>se. To be respectful <strong>of</strong> partners’ time, we only convened <strong>the</strong> entire group for specific<br />
<strong>an</strong>d necessary purposes <strong>an</strong>d <strong>in</strong>stead met one-on-one to access specific expertise on <strong>an</strong> ad hoc<br />
basis. It is difficult to know <strong>the</strong> effects <strong>of</strong> this shift, <strong>an</strong>d it may have led to limited ownership<br />
<strong>an</strong>d learn<strong>in</strong>g amongst those less engaged. To access this <strong>in</strong>formation, it would be beneficial<br />
to ask our partners if this reflected <strong>the</strong>ir preferences <strong>an</strong>d how it may have impacted <strong>the</strong>ir<br />
<strong>in</strong>volvement.<br />
Relat<strong>in</strong>g to stakeholder engagement, <strong>the</strong> role <strong>of</strong> CUP was pivotal <strong>in</strong> provid<strong>in</strong>g a stable<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
73<br />
foundation for reach<strong>in</strong>g out to community <strong>an</strong>d government agencies <strong>an</strong>d receiv<strong>in</strong>g such a<br />
positive <strong>an</strong>d enthusiastic response. With CUP’s 17-year history as <strong>an</strong> effective relationshipbuilder,<br />
<strong>in</strong> develop<strong>in</strong>g <strong>the</strong> ECN we drew heavily on CUP’s exist<strong>in</strong>g relationships as a trusted<br />
org<strong>an</strong>ization with<strong>in</strong> <strong>the</strong> prov<strong>in</strong>ce’s social sector to determ<strong>in</strong>e scope <strong>an</strong>d pool resources. The<br />
benefits <strong>of</strong> this are unsurpris<strong>in</strong>g s<strong>in</strong>ce trust is identified as a fundamental pillar <strong>of</strong> au<strong>the</strong>ntic<br />
community-based research partnerships (e.g., Cargo & Mercer, 2008). Key partners, such as <strong>the</strong><br />
Muttart Foundation, which we relied on for guid<strong>an</strong>ce <strong>in</strong> our prov<strong>in</strong>cial forums, were critical <strong>in</strong><br />
avoid<strong>in</strong>g missteps, ga<strong>in</strong><strong>in</strong>g momentum quickly, <strong>an</strong>d <strong>in</strong>creas<strong>in</strong>g our reach subst<strong>an</strong>tially. Through<br />
our stakeholder engagement process, we also identified a number <strong>of</strong> assets that have led to<br />
new evaluation-related opportunities <strong>in</strong> <strong>the</strong> community.<br />
F<strong>in</strong>ally, engag<strong>in</strong>g students was a mutually beneficial process for <strong>the</strong> ECN <strong>an</strong>d for <strong>the</strong><br />
students <strong>the</strong>mselves. Students, through research assist<strong>an</strong>tships, practicums, course-based<br />
learn<strong>in</strong>g, <strong>an</strong>d volunteer<strong>in</strong>g, provided much needed capacity. In return, students <strong>the</strong>mselves<br />
were mentored as community-engaged, <strong>in</strong>terdiscipl<strong>in</strong>ary scholars who were better equipped<br />
to respond to “wicked problems” (C<strong>an</strong>tor et al., 2015, p. 407). Engag<strong>in</strong>g students to <strong>the</strong> extent<br />
that we did helped to highlight <strong>the</strong> press<strong>in</strong>g need to build better <strong>in</strong>frastructure to support<br />
student engagement. Although <strong>the</strong>re are a number <strong>of</strong> courses through which students are<br />
connected to <strong>the</strong> ECN, <strong>the</strong>re is currently a signific<strong>an</strong>t lack <strong>of</strong> evaluation courses <strong>in</strong> C<strong>an</strong>adi<strong>an</strong><br />
universities generally (McDavid & Dev<strong>in</strong>e, 2009), <strong>an</strong>d <strong>the</strong> University <strong>of</strong> Alberta specifically<br />
(Bis<strong>an</strong>z et al., 2013). This makes both <strong>the</strong> tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d recruitment <strong>of</strong> students who have<br />
evaluation experience difficult <strong>an</strong>d limits <strong>the</strong> supply <strong>of</strong> students available to <strong>the</strong> ECN. In<br />
addition, hir<strong>in</strong>g students or facilitat<strong>in</strong>g practicum placements requires mentorship, supervision,<br />
<strong>an</strong>d m<strong>an</strong>agement. This is currently <strong>an</strong> area <strong>of</strong> lack for <strong>the</strong> ECN, with most <strong>of</strong> <strong>the</strong> supervision<br />
fall<strong>in</strong>g on <strong>the</strong> pr<strong>in</strong>ciple <strong>in</strong>vestigator <strong>an</strong>d a postdoctoral fellow on <strong>the</strong> project. This will not be<br />
susta<strong>in</strong>able over time <strong>an</strong>d is <strong>an</strong> area that requires attention.<br />
To our knowledge, <strong>the</strong> ECN is <strong>the</strong> first <strong>in</strong>itiative <strong>of</strong> its k<strong>in</strong>d to use a systems approach to<br />
build evaluation capacity that extends beyond <strong>the</strong> scope <strong>of</strong> a particular org<strong>an</strong>ization. To effect<br />
ch<strong>an</strong>ge, we believe a broad systems approach is essential to mobilize <strong>in</strong>fluential players from<br />
across <strong>the</strong> entire early childhood field (S<strong>an</strong>derson, 2000; Suárez-Herrera, Spr<strong>in</strong>gett, & Kag<strong>an</strong>,<br />
2009; Waldrop, 1992). Do<strong>in</strong>g so will not only foster a collective <strong>an</strong>d coord<strong>in</strong>ated effort to build<br />
capacity, it will <strong>in</strong>crease <strong>the</strong> likelihood that <strong>the</strong> questions asked <strong>of</strong> evaluation are valuable to <strong>the</strong><br />
field as a whole <strong>an</strong>d will be more effectively used to support improvements to early childhood<br />
policies <strong>an</strong>d practices. In addition, while this broad reach is necessary, we must also develop<br />
resources <strong>an</strong>d opportunities that are tailored to different users <strong>an</strong>d <strong>the</strong>refore contextually<br />
relev<strong>an</strong>t. As such, <strong>the</strong> ECN aims to be, simult<strong>an</strong>eously, a broad yet localized approach. This<br />
has required us to draw on <strong>the</strong> various forms <strong>of</strong> engagement described <strong>in</strong> this paper. However,<br />
foster<strong>in</strong>g deep relationships with partners rema<strong>in</strong>s a challenge with a project <strong>of</strong> this scale <strong>an</strong>d<br />
<strong>an</strong> area requir<strong>in</strong>g fur<strong>the</strong>r attention.<br />
Volume 3/Issue 2/Fall 2017
74 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Next Steps<br />
The <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN) was developed to support dialogue among<br />
<strong>in</strong>tersectoral stakeholders <strong>an</strong>d create a central po<strong>in</strong>t through which stakeholders could<br />
access coord<strong>in</strong>ated evaluation capacity build<strong>in</strong>g resources tailored to <strong>the</strong> field <strong>of</strong> ECD, <strong>an</strong>d<br />
ensure high quality tra<strong>in</strong><strong>in</strong>g, practice, <strong>an</strong>d research <strong>in</strong> evaluation. The ECN has provided,<br />
<strong>an</strong>d cont<strong>in</strong>ues to provide a mech<strong>an</strong>ism for foster<strong>in</strong>g dialogue among academics, funders,<br />
government representatives, evaluation consult<strong>an</strong>ts, <strong>an</strong>d nonpr<strong>of</strong>it representatives. It also<br />
provides a way to more easily share <strong>an</strong>d develop capacity build<strong>in</strong>g resources, expertise <strong>an</strong>d<br />
opportunities. Through our engagement with partners, stakeholders, <strong>an</strong>d students, we have<br />
developed a prov<strong>in</strong>cial agenda for <strong>the</strong> ECN that we <strong>in</strong>tend to implement over <strong>the</strong> next several<br />
years, while cont<strong>in</strong>u<strong>in</strong>g to exp<strong>an</strong>d <strong>the</strong> network <strong>an</strong>d establish new partnerships <strong>in</strong> C<strong>an</strong>ada <strong>an</strong>d<br />
<strong>in</strong>ternationally. It is also through engagement that our learn<strong>in</strong>g <strong>an</strong>d those we engaged with<br />
rema<strong>in</strong> <strong>in</strong> motion <strong>an</strong>d challenge us to adapt <strong>in</strong> new ways. In light <strong>of</strong> this, it is import<strong>an</strong>t to<br />
acknowledge that engagement is not <strong>an</strong> <strong>in</strong>herently necessary or equally adv<strong>an</strong>tageous process.<br />
Although <strong>the</strong> various forms <strong>of</strong> engagement through <strong>the</strong> ECN have helped to generate <strong>in</strong>terest<br />
<strong>an</strong>d develop a mutual agenda, it will be essential as we move forward to cont<strong>in</strong>ually assess<br />
whe<strong>the</strong>r <strong>the</strong> ways we are engag<strong>in</strong>g are me<strong>an</strong><strong>in</strong>gful <strong>an</strong>d appropriate. To evaluate our partnership<br />
<strong>an</strong>d our engagement processes systematically, we will use multiple methods <strong>an</strong>d tools (e.g.<br />
<strong>the</strong> PARTNERtool; Varda, Ch<strong>an</strong>dra, Stern, & Lurie, 2008), which will allow us to respond to<br />
un<strong>an</strong>ticipated challenges throughout <strong>the</strong> life <strong>of</strong> <strong>the</strong> partnership.<br />
With <strong>an</strong> <strong>in</strong>creas<strong>in</strong>g number <strong>of</strong> <strong>in</strong>stitutions attempt<strong>in</strong>g to dist<strong>an</strong>ce <strong>the</strong>mselves from <strong>an</strong> ivory<br />
tower status (Furco, 2001), <strong>the</strong> concept <strong>of</strong> engagement appears to have ga<strong>in</strong>ed signific<strong>an</strong>t<br />
momentum across <strong>the</strong> academy. There is a subsequent need for caution when mak<strong>in</strong>g decisions<br />
about <strong>the</strong> forms <strong>an</strong>d extent <strong>of</strong> engagement appropriate <strong>in</strong> each research project to avoid<br />
tokenism (at best) <strong>an</strong>d tyr<strong>an</strong>ny (at worst) (Cooke & Kothari, 2001). We commit ourselves to<br />
this ongo<strong>in</strong>g deliberation <strong>an</strong>d <strong>in</strong>vite o<strong>the</strong>rs to jo<strong>in</strong> us <strong>in</strong> a collective effort to engage carefully.<br />
About <strong>the</strong> Authors<br />
Rebecca Gokiert, PhD, R.Psych, (correspond<strong>in</strong>g author) is <strong>an</strong> Associate Pr<strong>of</strong>essor <strong>an</strong>d Associate<br />
Director, Community-University Partnership (CUP), University <strong>of</strong> Alberta. Her teach<strong>in</strong>g <strong>an</strong>d<br />
research is focused <strong>in</strong> community engagement <strong>an</strong>d participatory research <strong>in</strong> cross-cultural<br />
early childhood contexts. She is a registered psychologist <strong>in</strong> <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta <strong>an</strong>d works<br />
<strong>in</strong> partnership with m<strong>an</strong>y school divisions. Email: rgokiert@ualberta.ca<br />
Beth<strong>an</strong> K<strong>in</strong>gsley, PhD, is a postdoctoral fellow with <strong>the</strong> Community-University Partnership<br />
(CUP) at <strong>the</strong> University <strong>of</strong> Alberta. Beth<strong>an</strong>’s current focus is to build <strong>an</strong>d support <strong>the</strong> <strong>Evaluation</strong><br />
<strong>Capacity</strong> <strong>Network</strong>, with a keen <strong>in</strong>terest <strong>in</strong> community-based research <strong>an</strong>d evaluation. Her<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
75<br />
doctoral dissertation explored marg<strong>in</strong>aliz<strong>in</strong>g practices <strong>in</strong> recreation for young people liv<strong>in</strong>g<br />
with lower <strong>in</strong>comes.<br />
Cheryl Poth, PhD, CE, is <strong>an</strong> Associate Pr<strong>of</strong>essor <strong>in</strong> <strong>the</strong> Centre for Research <strong>an</strong>d Applied<br />
Measurement <strong>an</strong>d <strong>Evaluation</strong> (CRAME), University <strong>of</strong> Alberta. Cheryl has expertise<br />
conduct<strong>in</strong>g research <strong>an</strong>d teach<strong>in</strong>g <strong>in</strong> <strong>the</strong> areas <strong>of</strong> program evaluation, mixed methods <strong>an</strong>d<br />
qualitative research, classroom assessment <strong>an</strong>d health sciences education.<br />
Karen Edwards, Med, is <strong>the</strong> Director <strong>of</strong> <strong>the</strong> Community-University Partnership (CUP) at <strong>the</strong><br />
University <strong>of</strong> Alberta. In her role, she fosters ongo<strong>in</strong>g network <strong>an</strong>d partnership development,<br />
facilitates <strong>the</strong> implementation <strong>of</strong> <strong>the</strong> org<strong>an</strong>izational strategic pl<strong>an</strong>, brokers requests from<br />
network partners, <strong>an</strong>d pursues fund<strong>in</strong>g <strong>an</strong>d resource development.<br />
Btissam El Hassar, MPP, is a doctoral c<strong>an</strong>didate <strong>in</strong> Measurement <strong>an</strong>d <strong>Evaluation</strong> through<br />
<strong>the</strong> Centre for Research <strong>an</strong>d Applied Measurement <strong>an</strong>d <strong>Evaluation</strong> (CRAME), University<br />
<strong>of</strong> Alberta. Her dissertation is on <strong>the</strong> development <strong>an</strong>d validation <strong>of</strong> <strong>an</strong> evaluation capacity<br />
<strong>in</strong>strument that is contextually relev<strong>an</strong>t to <strong>the</strong> early childhood development field <strong>in</strong> Alberta.<br />
Lisa T<strong>in</strong>k, MA, is a doctoral student <strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health at <strong>the</strong> University <strong>of</strong><br />
Alberta. Prior to purs<strong>in</strong>g a doctorate degree, Lisa held numerous positions <strong>in</strong> <strong>the</strong> non-pr<strong>of</strong>it<br />
sector, prov<strong>in</strong>cial government, <strong>an</strong>d <strong>the</strong> academy. Lisa’s work focuses on how historical social<br />
<strong>an</strong>d political arr<strong>an</strong>gements create <strong>the</strong> conditions for policy development.<br />
Melissa Tremblay, MSc., is a doctoral c<strong>an</strong>didate <strong>in</strong> <strong>the</strong> School <strong>an</strong>d Cl<strong>in</strong>ical Child Psychology<br />
Program at <strong>the</strong> University <strong>of</strong> Alberta. Melissa has been a graduate research assist<strong>an</strong>t with <strong>the</strong><br />
ECN project m<strong>an</strong>agement team. Through her doctoral research, Melissa is exam<strong>in</strong><strong>in</strong>g <strong>the</strong><br />
impact <strong>of</strong> a supportive hous<strong>in</strong>g program for teen parents.<br />
J<strong>an</strong>e Spr<strong>in</strong>gett, PhD, is a Pr<strong>of</strong>essor <strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health at <strong>the</strong> University <strong>of</strong><br />
Alberta. She has expertise <strong>in</strong> participatory practice, network <strong>an</strong>d system development, <strong>an</strong>d<br />
realist approaches to syn<strong>the</strong>sis <strong>an</strong>d evaluation. She br<strong>in</strong>gs over 25 years <strong>of</strong> <strong>in</strong>ternational<br />
evaluation experience <strong>an</strong>d graduate supervision to <strong>the</strong> ECN.<br />
References<br />
Akbari, E., & McCuaig, K. (2017). <strong>Early</strong> <strong>Childhood</strong> Education Report. Retrieved from http://ecereport.<br />
ca/media/uploads/2017-report-pdfs/ece-report2017-en-feb6.pdf<br />
Volume 3/Issue 2/Fall 2017
76 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Alk<strong>in</strong>, M. C. (2004). <strong>Evaluation</strong> roots: Trac<strong>in</strong>g <strong>the</strong>orists’ views <strong>an</strong>d <strong>in</strong>fluences. Thous<strong>an</strong>d Oaks, CA: Sage<br />
Publications.<br />
Armitage, T., & Levac, L. (2015). The development <strong>of</strong> community-engaged scholars through coursebased<br />
learn<strong>in</strong>g: A student perspective. Engaged Scholar Journal: Community-Engaged Research,<br />
Teach<strong>in</strong>g <strong>an</strong>d Learn<strong>in</strong>g, 1(1), 148-163.<br />
Bakken, L. L., Núñez, J., & Couture, C. (2014). A course model for build<strong>in</strong>g evaluation capacity<br />
through a university-community partnership. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 35(4), 579-593.<br />
Bis<strong>an</strong>z, J., Edwards, K., & Shaw, K. (2013). <strong>Develop<strong>in</strong>g</strong> support for evaluation at CUP. Edmonton,<br />
AB: Community-University Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families,<br />
University <strong>of</strong> Alberta.<br />
Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Def<strong>in</strong><strong>in</strong>g <strong>an</strong>d teach<strong>in</strong>g evaluative<br />
th<strong>in</strong>k<strong>in</strong>g: Insights from research on critical th<strong>in</strong>k<strong>in</strong>g. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 36(3), 375-<br />
388.<br />
Cabrera, N. J. (2013). Positive development <strong>of</strong> m<strong>in</strong>ority children. Social Policy Report, 27(2), 1-30.<br />
C<strong>an</strong>adi<strong>an</strong> Institute for Health Information (CIHI). (2014). Children vulnerable <strong>in</strong> areas <strong>of</strong> early<br />
development: A determ<strong>in</strong><strong>an</strong>t <strong>of</strong> child health. Retrieved from https://secure.cihi.ca/free_products/<br />
Children_Vulnerable_<strong>in</strong>_Areas_<strong>of</strong>_<strong>Early</strong>_<strong>Development</strong>_EN.pdf<br />
C<strong>an</strong>tor, A., DeLauer, V., Mart<strong>in</strong>, D., & Rog<strong>an</strong>, J. (2015). Interdiscipl<strong>in</strong>ary “wicked problem” solvers:<br />
Apply<strong>in</strong>g lessons from HERO <strong>in</strong> community-based research experiences for undergraduates.<br />
Geography <strong>in</strong> Higher Education, 39(3), 407-419.<br />
Cargo, M. & Mercer, S. L. (2008). The value <strong>an</strong>d challenges <strong>of</strong> participatory research: Streng<strong>the</strong>n<strong>in</strong>g<br />
its practice. Annual Review <strong>of</strong> Public Health, 29, 325-350.<br />
Carm<strong>an</strong>, J. G. (2010). The accountability movement: What’s wrong with this <strong>the</strong>ory <strong>of</strong> ch<strong>an</strong>ge?<br />
Nonpr<strong>of</strong>it <strong>an</strong>d Voluntary Sector Quarterly, 39(2), 256-274.<br />
Carm<strong>an</strong>, J. G., & Millesen, J. L. (2005). Nonpr<strong>of</strong>it program evaluation: Org<strong>an</strong>izational challenges <strong>an</strong>d<br />
resource needs. Journal <strong>of</strong> Volunteer Adm<strong>in</strong>istration, 23(3), 36-43.<br />
Chapm<strong>an</strong>, S. (2015). Community-University Partnerships: A Case Study. In G. Higg<strong>in</strong>bottom & P.<br />
Liamputtong (Eds.), Participatory Qualitative Research Methodologies <strong>in</strong> Health (pp. 200-221). Los<br />
Angeles, US: SAGE.<br />
Cockerill, R., Myers, T., & Allm<strong>an</strong>, D. (2000). Pl<strong>an</strong>n<strong>in</strong>g for Community-based <strong>Evaluation</strong>. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 21(3), 351-357.<br />
Cooke, B., & Kothari, U. (2001). The case for participation as tyr<strong>an</strong>ny. In B. Cooke & U. Kothari<br />
(Eds.), Participation: The new tyr<strong>an</strong>ny? (pp. 1-15). New York, NY: Zed Books.<br />
Council <strong>of</strong> M<strong>in</strong>isters <strong>of</strong> Education, C<strong>an</strong>ada (CMEC). (2014). CMEC early learn<strong>in</strong>g <strong>an</strong>d development<br />
framework. Retrieved from http://www.cmec.ca/Publications/Lists/ Publications/<br />
Attachments/ 327/2014-07-<strong>Early</strong>-Learn<strong>in</strong>g-Framework-EN.pdf<br />
Cous<strong>in</strong>s, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational <strong>Evaluation</strong> <strong>an</strong>d<br />
Policy Analysis, 14(4), 397-418.<br />
Cous<strong>in</strong>s, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Fram<strong>in</strong>g <strong>the</strong> capacity to do <strong>an</strong>d use<br />
evaluation. New Directions for <strong>Evaluation</strong>, 141, 7-23.<br />
Doherty, G., Friendly, M., & Beach, J. (2003). OECD <strong>the</strong>matic review <strong>of</strong> early childhood education <strong>an</strong>d<br />
care: C<strong>an</strong>adi<strong>an</strong> background report. Retrieved from http://www.oecd.org/education/school/<br />
33852192.pdf<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
77<br />
Dunc<strong>an</strong>, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Kleb<strong>an</strong>ov, P., … Japel, C.<br />
(2007). School read<strong>in</strong>ess <strong>an</strong>d later achievement. <strong>Development</strong>al Psychology, 43(6), 1428-1446.<br />
<strong>Early</strong> <strong>Childhood</strong> Mapp<strong>in</strong>g Project (ECMap). (2014). How are our young children do<strong>in</strong>g? F<strong>in</strong>al report <strong>of</strong> <strong>the</strong><br />
early child development mapp<strong>in</strong>g project (ECMap). Edmonton, AB: ECMap, Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth <strong>an</strong>d Families, University <strong>of</strong> Alberta. Retrieved<br />
from http://www.ecmap.ca/images/results/ECMap_F<strong>in</strong>al_Report_20141118.pdf<br />
El Hassar, B., Poth, C., Gokiert, R. K<strong>in</strong>gsley, B., & Krishn<strong>an</strong>, V. (2016, October). Design <strong>an</strong>d validation<br />
<strong>of</strong> <strong>an</strong> evaluation capacity build<strong>in</strong>g <strong>in</strong>strument for <strong>the</strong> early childhood sector. Presentation delivered at <strong>the</strong><br />
Americ<strong>an</strong> <strong>Evaluation</strong> Association Conference. Atl<strong>an</strong>ta, GA, U.S.<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN). (2015). Adv<strong>an</strong>c<strong>in</strong>g evaluation practices <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood<br />
development. Edmonton, AB: ECN, Community-University Partnership for <strong>the</strong> Study <strong>of</strong><br />
Children, Youth <strong>an</strong>d Families, University <strong>of</strong> Alberta. Retrieved from http://www.cup.<br />
ualberta.ca/wp-content/uploads/2015/12/ECN-Community-Primer-December-17.pdf<br />
Family <strong>an</strong>d Community Support Services, City <strong>of</strong> Edmonton. (2015). 2015 common outcomes report<br />
summary. Retrieved from https://www.edmonton.ca/programs_services/documents/PDF/<br />
CommonOutcomesDataSummary2015.pdf<br />
Freem<strong>an</strong>, R. E. (2010). Strategic m<strong>an</strong>agement: A stakeholder approach. Cambridge, UK: Cambridge<br />
University Press.<br />
Furco, A. (1996). Service learn<strong>in</strong>g: A bal<strong>an</strong>ced approach to experiential education. In B.Taylor, (Ed.),<br />
Exp<strong>an</strong>d<strong>in</strong>g boundaries: Serv<strong>in</strong>g <strong>an</strong>d learn<strong>in</strong>g (pp. 2-6). Wash<strong>in</strong>gton, DC: Corporation for National<br />
Service.<br />
Furco, A. (2001). Adv<strong>an</strong>c<strong>in</strong>g service-learn<strong>in</strong>g at research universities. New Directions for Higher<br />
Education, 114, 67-78.<br />
Gauthier, B., Barr<strong>in</strong>gton, G. V., Bozzo, S. L., Chaytor, K., Dignard, A., Lahey, R., … Roy, S. (2010).<br />
The lay <strong>of</strong> <strong>the</strong> l<strong>an</strong>d: <strong>Evaluation</strong> practice <strong>in</strong> C<strong>an</strong>ada <strong>in</strong> 2009. The C<strong>an</strong>adi<strong>an</strong> Journal <strong>of</strong> Program<br />
<strong>Evaluation</strong>, 24(1), 1-49.<br />
Georgis, R., Gokiert, R.J., & Kirova, A. (2018). Research considerations <strong>in</strong> early childhood<br />
partnerships with immigr<strong>an</strong>t <strong>an</strong>d refugee communities: Researcher reflections. In S. Madrid,<br />
M.J. Mor<strong>an</strong>, R. Brookshire, & M. Buch<strong>an</strong><strong>an</strong> (Eds.), Collaborative research methodologies <strong>in</strong> diverse<br />
early care <strong>an</strong>d education contexts. New York, NY: Routledge<br />
Gokiert, R.J., Tremblay, M., K<strong>in</strong>gsley, B., Mottershead, K., Appleyard, R., & Spr<strong>in</strong>gett, J. (2017). The<br />
life we deserve: A model <strong>of</strong> supportive hous<strong>in</strong>g for teen families. [technical report]. Homeward Trust<br />
Edmonton, Edmonton, Alberta, C<strong>an</strong>ada.<br />
Gokiert, R. J., Georgis, R., Rob<strong>in</strong>son, T., Alex<strong>an</strong>der First Nation, Alexis Nakota Sioux Nation,<br />
O’Chiese First Nation, Sunchild First Nation, & Yellowhead Tribal College. (2014). First<br />
Nation child development: Collective community report. Edmonton, AB: Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families, University <strong>of</strong> Alberta.<br />
Government <strong>of</strong> Alberta. (2013). Toge<strong>the</strong>r we raise tomorrow: An Alberta approach to early childhood<br />
development. Retrieved from https://open.alberta.ca/publications/6881618<br />
Government <strong>of</strong> C<strong>an</strong>ada (2011). The Wellbe<strong>in</strong>g <strong>of</strong> C<strong>an</strong>ada’s Young Children. Government <strong>of</strong> C<strong>an</strong>ada.<br />
Government <strong>of</strong> M<strong>an</strong>itoba. (2013). Start<strong>in</strong>g early, start<strong>in</strong>g strong: M<strong>an</strong>itoba’s early childhood development<br />
framework. Retrieved from http://www.gov.mb.ca/healthychild/ecd/ ecd_sessframework.pdf<br />
Volume 3/Issue 2/Fall 2017
78 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Government <strong>of</strong> Northwest Territories. (2013). A framework for early childhood development <strong>in</strong> <strong>the</strong> Northwest<br />
Territories: Right from <strong>the</strong> start. Retrieved from https://www.ece.gov.nt.ca/sites/ www.ece.gov.<br />
nt.ca/files/resources/ecd_framework_<strong>an</strong>d_action_pl<strong>an</strong>.pdf<br />
Heckm<strong>an</strong>, J. J. (2008). The case for <strong>in</strong>vest<strong>in</strong>g <strong>in</strong> disadv<strong>an</strong>taged young children. In First Focus (Ed.),<br />
Big ideas for children: Invest<strong>in</strong>g <strong>in</strong> our nation’s future (pp. 49-58). Wash<strong>in</strong>gton, DC: First Focus.<br />
Hertzm<strong>an</strong>, C., & Boyce, T. (2010). How experience gets under <strong>the</strong> sk<strong>in</strong> to create gradients <strong>in</strong><br />
developmental health. Annual Review <strong>of</strong> Public Health, 31(1), 329-347.<br />
Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (1998). Review <strong>of</strong> community-based<br />
research: Assess<strong>in</strong>g partnership approaches to improve public health. Annual Review <strong>of</strong> Public<br />
Health, 19(1), 173-202.<br />
Jagosh, J., Bush, P., Salsberg, J., Macaulay, A.C., Greenhalgh, T., Wong, G., Cargo, M., Green, L.W.,<br />
Herbert, C.P., & Pluye, P. (2015). A realist evaluation <strong>of</strong> community-based participatory<br />
research: partnership synergy, trust build<strong>in</strong>g <strong>an</strong>d related ripple effects. BMC Public Health, 15,<br />
725.<br />
J<strong>an</strong>zen, R., Ochocka, J., Turner, L., Cook, T., Fr<strong>an</strong>kl<strong>in</strong>, M., & Deichert, D. (2017). <strong>Build<strong>in</strong>g</strong> a<br />
community-based culture <strong>of</strong> evaluation. <strong>Evaluation</strong> <strong>an</strong>d Program Pl<strong>an</strong>n<strong>in</strong>g, 65, 163-170.<br />
Kajner, T., Fletcher, F., & Makokis, P. (2011). Bal<strong>an</strong>c<strong>in</strong>g head <strong>an</strong>d heart: The import<strong>an</strong>ce <strong>of</strong> relational<br />
accountability <strong>in</strong> community-university partnerships. Innovative Higher Education, 37, 257-270.<br />
K<strong>in</strong>gsley, B. C., & Chapm<strong>an</strong>, S-A. (2013). Question<strong>in</strong>g <strong>the</strong> me<strong>an</strong><strong>in</strong>gfulness <strong>of</strong> rigour <strong>in</strong> communitybased<br />
research: Navigat<strong>in</strong>g a dilemma. International Journal <strong>of</strong> Qualitative Methods, 12, 551-569.<br />
Lab<strong>in</strong>, S. N. (2014). <strong>Develop<strong>in</strong>g</strong> Common Measures <strong>in</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> An Iterative<br />
Science <strong>an</strong>d Practice Process. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 35(1), 107-115.<br />
Leviton, L., (2014). Some underexam<strong>in</strong>ed aspects <strong>of</strong> evaluation capacity build<strong>in</strong>g. Americ<strong>an</strong> Journal <strong>of</strong><br />
<strong>Evaluation</strong>, 35(1), 90-94.<br />
Liket, K. C., Rey-Garcia, M., & Maas, K. E. H. (2014). Why aren’t evaluations work<strong>in</strong>g <strong>an</strong>d what to<br />
do about it: A framework for negotiat<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluations <strong>in</strong> nonpr<strong>of</strong>its. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 35(2), 171-188.<br />
McCaffrey, L. (2007). The History <strong>of</strong> <strong>the</strong> Community-University Partnership for <strong>the</strong> Study <strong>of</strong><br />
Children, Youth, <strong>an</strong>d Families (CUP). Edmonton, AB: McCaffery Consult<strong>in</strong>g. Retrieved<br />
from https://d1pbog36rugm0t.cloudfront.net/-/media/ualberta/faculties-<strong>an</strong>d-programs/<br />
centres-<strong>in</strong>stitutes/community-university-partnership/about-us/cup-history.pdf<br />
McDavid, J., & Dev<strong>in</strong>e, H. (2009). Consortium <strong>of</strong> Universities for <strong>Evaluation</strong> Education (CUEE) project:<br />
Research on evaluation education at <strong>the</strong> graduate level <strong>in</strong> C<strong>an</strong>adi<strong>an</strong> universities. F<strong>in</strong>al report.<br />
McSh<strong>an</strong>e, K., Usher, A. M., T<strong>an</strong>don, R., & Steel, J. (2015). Chart<strong>in</strong>g <strong>the</strong> trajectory <strong>of</strong> a flexible<br />
community-university collaboration <strong>in</strong> <strong>an</strong> applied learn<strong>in</strong>g ecosystem. Engaged Scholar Journal:<br />
Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g, 1(2), 149-165.<br />
M<strong>in</strong>kler, M., & Wallerste<strong>in</strong>, N. (Eds.). (2008). Community-based participatory research for health: From process<br />
to outcomes (2nd ed.). S<strong>an</strong> Fr<strong>an</strong>cisco, CA: Jossey-Bass.<br />
Munro, C. (2006). BC evaluation project report: C<strong>an</strong> we agree on common child outcomes for BC? Retrieved<br />
from http://www.successby6bc.ca/sites/default/files/BC%20<strong>Evaluation</strong>%20Project%20<br />
Report%202006.pdf<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
79<br />
Muttart Foundation. (2013). What We Heard: A Report on Fall 2012 Consultations with <strong>Early</strong><br />
Learn<strong>in</strong>g <strong>an</strong>d Care Stakeholders. Retrieved from http://www.muttart.org/sites/default/<br />
files/report/Toward%20A%20Prov<strong>in</strong>cial%20ELC%20Framework%20-%20What%20<br />
We%20Heard%20Report%20-%20f<strong>in</strong>al.pdf<br />
Org<strong>an</strong>isation for Economic Cooperation <strong>an</strong>d <strong>Development</strong>. (2012). Education at a gl<strong>an</strong>ce 2012: OECD<br />
<strong>in</strong>dicators. Retrieved from https://www.oecd.org/edu/EAG%202012_e-book_EN_200912.<br />
pdf<br />
Pei, N., Felth<strong>an</strong>, J., Ford, I., Schwartz, K. (2015). Best practices for implement<strong>in</strong>g a liv<strong>in</strong>g wage policy<br />
<strong>in</strong> C<strong>an</strong>ada: Us<strong>in</strong>g Community-Campus Partnerships to Fur<strong>the</strong>r <strong>the</strong> Community’s Goal.<br />
Engaged Scholar Journal, 1(2), 149-153.<br />
Preskill, H., & Boyle, S. (2008). A Multidiscipl<strong>in</strong>ary Model <strong>of</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong>. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 29(4), 443-459.<br />
Reed, E., & Morariu, J. (2010). State <strong>of</strong> evaluation 2010: <strong>Evaluation</strong> practice <strong>an</strong>d capacity <strong>in</strong> <strong>the</strong> nonpr<strong>of</strong>it sector.<br />
Retrieved from Innovation <strong>Network</strong> Website: http://www.po<strong>in</strong>tk.org/client_docs/<strong>in</strong>nonetstate-<strong>of</strong>-evaluation-2010.pdf<br />
S<strong>an</strong>derson, I. (2000). <strong>Evaluation</strong> <strong>in</strong> complex policy systems. <strong>Evaluation</strong>, 6(4), 433-454.<br />
Shonk<strong>of</strong>f, J. P., & Levitt, P. (2010). Neuroscience <strong>an</strong>d <strong>the</strong> future <strong>of</strong> early childhood policy: Mov<strong>in</strong>g<br />
from why to what <strong>an</strong>d how. Neuron, 67(5), 689-691.<br />
Shonk<strong>of</strong>f, J. P. (2017). Reth<strong>in</strong>k<strong>in</strong>g <strong>the</strong> def<strong>in</strong>ition <strong>of</strong> evidence-based <strong>in</strong>terventions to promote early<br />
childhood development. Pediatrics, 140(6), 1-2.<br />
Snibbe, A. C. (2006). Drown<strong>in</strong>g <strong>in</strong> data. St<strong>an</strong>ford Social Innovation Review, 4(3), 39-45.<br />
St<strong>an</strong>ford University (2017). A virtual crash course <strong>in</strong> design th<strong>in</strong>k<strong>in</strong>g. Retrieved from: https://<br />
dschool.st<strong>an</strong>ford.edu/resources-collections/a-virtual-crash-course-<strong>in</strong>-design-th<strong>in</strong>k<strong>in</strong>g<br />
Statistics C<strong>an</strong>ada, Government <strong>of</strong> C<strong>an</strong>ada. (2011). 2011 National Household Survey: Data Tables<br />
[C<strong>an</strong>ada]. Retrieved from http://www12.statc<strong>an</strong>.gc.ca/nhs-enm/2011/dp-pd/dt-td/Rp-eng.<br />
cfm?LANG=E&APATH=3&DETAIL=0&DIM=0&FL=A&FREE=0&GC=0&GID=0<br />
&GK=0&GRP=0&PID=106715&PRID=0&PTYPE=105277&S=0&SHOWALL=0&SUB<br />
=0&Temporal=2013&THEME=98&VID=0&VNAMEE=&VNAMEF=<br />
Suárez-Herrera, J. C., Spr<strong>in</strong>gett, J., & Kag<strong>an</strong>, C. (2009). Critical connections between<br />
participatory evaluation, org<strong>an</strong>izational learn<strong>in</strong>g <strong>an</strong>d <strong>in</strong>tentional ch<strong>an</strong>ge <strong>in</strong> pluralistic<br />
org<strong>an</strong>izations. <strong>Evaluation</strong>, 15(3), 321-342.<br />
Taylor, A., Butterwick, S., Raykov, M., Glick, S., Peikazadi, N., & Mehrabi, S. (2015). Community<br />
Service-Learn<strong>in</strong>g <strong>in</strong> C<strong>an</strong>adi<strong>an</strong> Higher Education. University <strong>of</strong> British Columbia. Retrieved from<br />
https://d1pbog36rugm0t.cloudfront.net/-/media/arts/departments-<strong>in</strong>stitutes-<strong>an</strong>d-centres/<br />
community-service-learn<strong>in</strong>g/documents/reports/ks-report-31-oct-2015-f<strong>in</strong>al.pdf<br />
Taylor-Ritzler, T., Suarez-Balcazar, Y., Garcia-Iriarte, E., Henry, D. B., & Balcazar, F. E. (2013).<br />
Underst<strong>an</strong>d<strong>in</strong>g <strong>an</strong>d Measur<strong>in</strong>g <strong>Evaluation</strong> <strong>Capacity</strong>: A Model <strong>an</strong>d Instrument Validation<br />
Study. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 34(2), 190-206.<br />
T<strong>in</strong>k, L.N., Gokiert, R.J., K<strong>in</strong>gsley, B.C., & El Hassar, B. (2017). <strong>Evaluation</strong> capacity build<strong>in</strong>g survey<br />
results: Adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood. Edmonton, AB, Community-<br />
University Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families, University <strong>of</strong> Alberta.<br />
Retrieved from www.evaluationcapacitynetwork.com.<br />
Volume 3/Issue 2/Fall 2017
80 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
T<strong>in</strong>k, L. N., K<strong>in</strong>gsley, B. C., & Gokiert, R. J. (2016). What we heard: Adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation<br />
<strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood. Retrieved from http://www.evaluationcapacitynetwork.com/<br />
wp-content/uploads/2017/03/ECN-Dialogues_What-we-heard_Sept-2016.pdf<br />
Tremblay, M., Gokiert, R. J., K<strong>in</strong>gsley, B. C., Mottershead, K., Ca<strong>in</strong>e, K., Appleyard, R., & Appleyard,<br />
R. (2016, March). Creat<strong>in</strong>g a research agenda to measure <strong>the</strong> impact <strong>of</strong> a supportive wraparound hous<strong>in</strong>g<br />
program for teen families. B<strong>an</strong>ff International Conference for Behavioural Sciences, B<strong>an</strong>ff, AB.<br />
UNICEF, Office <strong>of</strong> Research. (2016). Fairness for children (Innocenti Report Card 13). Retrieved from<br />
https://www.unicef.ca/en/unicef-report-card-13-fairness-for-children<br />
Varda, D. M., Ch<strong>an</strong>dra, A., Stern, S., & Lurie, L. (2008). Core dimensions <strong>of</strong> connectivity <strong>in</strong> public<br />
health collaboratives. Journal <strong>of</strong> Public Health M<strong>an</strong>agement Practice, 14, 1-7.<br />
Visw<strong>an</strong>ath<strong>an</strong>, M., Ammerm<strong>an</strong>, A., Eng, E., Garlehner, G., Lohr, K.N., Griffith, D., Rhodes, S.,<br />
Samuel-Hodge, C., Maty, S., Lux, L., Webb, L., Sutton, S.F., Sw<strong>in</strong>son, T., Jackm<strong>an</strong>, A., &<br />
Whitener, L. (2004). Community-Based Participatory Research: Assess<strong>in</strong>g <strong>the</strong> Evidence.<br />
Evidence Report/Technology Assessment No. 99 (Prepared by RTI-University <strong>of</strong> North<br />
Carol<strong>in</strong>a Evidence-based Practice Center under Contract No. 290-02-0016). AHRQ<br />
Publication 04-E022- 2. Rockville, MD: Agency for Healthcare Research <strong>an</strong>d Quality. July<br />
2004.<br />
Waddell, C., Shepherd, C. A., Shwartz, C., & Baric<strong>an</strong>, J. (2014). Child <strong>an</strong>d youth mental disorders: Prevalence<br />
<strong>an</strong>d evidence-based <strong>in</strong>terventions. V<strong>an</strong>couver, BC: Children’s Health Policy Centre, Simon Fraser<br />
University. Retrieved from http://childhealthpolicy.ca/wp-content/uploads/2015/11/2015-<br />
10-05-Waddell-et-al-Report-2014.06.16-w-errata.pdf<br />
Waldrop, M. M. (1992). Complexity: The emerg<strong>in</strong>g science at <strong>the</strong> edge <strong>of</strong> order <strong>an</strong>d chaos. New York, NY:<br />
Simon & Schuster.<br />
Wenger, L., Hawk<strong>in</strong>s, L., & Seifer, S. D. (Eds.). (2011). Community-engaged scholarships: Critical<br />
junctures <strong>in</strong> research, practice, <strong>an</strong>d policy: Conference report. Community-Campus<br />
Partnerships for Health & <strong>the</strong> University <strong>of</strong> Guelph. Retrieved from http://criticaljunctures.<br />
ca<br />
Wenger, L., Hawk<strong>in</strong>s, L., & Seifer, S. D. (2012). Community-engaged scholarships: Critical junctures<br />
<strong>in</strong> research, practice, <strong>an</strong>d policy. Journal <strong>of</strong> Higher Education Outreach <strong>an</strong>d Engagement, 16, 171-<br />
181.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g