Developing an Evaluation Capacity Building Network in the Field of Early Childhood Development
Transform your PDFs into Flipbooks and boost your revenue!
Leverage SEO-optimized Flipbooks, powerful backlinks, and multimedia content to professionally showcase your products and significantly increase your reach.
59<br />
<strong>Develop<strong>in</strong>g</strong> <strong>an</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> <strong>Network</strong> <strong>in</strong> <strong>the</strong><br />
<strong>Field</strong> <strong>of</strong> <strong>Early</strong> <strong>Childhood</strong> <strong>Development</strong><br />
Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar,<br />
Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Abstract This reflective essay traces <strong>the</strong> development <strong>of</strong> <strong>an</strong> evaluation capacity<br />
build<strong>in</strong>g network with<strong>in</strong> <strong>the</strong> early childhood development field. First, we describe <strong>the</strong><br />
context for build<strong>in</strong>g <strong>the</strong> network us<strong>in</strong>g a community-based participatory approach <strong>an</strong>d<br />
provide rationale for our specific focus on early childhood development. Second, we<br />
provide <strong>an</strong> expl<strong>an</strong>ation <strong>of</strong> <strong>the</strong> purpose <strong>an</strong>d processes <strong>in</strong>volved <strong>in</strong> three areas <strong>of</strong> signific<strong>an</strong>t<br />
engagement: partner, stakeholder, <strong>an</strong>d student. We reflect on <strong>the</strong> methods <strong>of</strong> engagement<br />
used across <strong>the</strong>se three areas <strong>an</strong>d <strong>the</strong>ir impact on <strong>the</strong> outcomes that we achieved. F<strong>in</strong>ally,<br />
we conclude <strong>the</strong> paper with some f<strong>in</strong>al considerations for guid<strong>in</strong>g engaged scholars <strong>an</strong>d<br />
with <strong>the</strong> next steps <strong>in</strong> our own work.<br />
KeyWords community-university partnerships, early childhood development,<br />
evaluation capacity build<strong>in</strong>g, partnership development, engagement methods <strong>an</strong>d practices<br />
The health <strong>an</strong>d well-be<strong>in</strong>g <strong>of</strong> children <strong>an</strong>d families has long been a focus <strong>of</strong> research <strong>an</strong>d<br />
social policy, as it impacts <strong>the</strong> economic <strong>an</strong>d social fabric <strong>of</strong> our communities (Akbari &<br />
McCuaig, 2017; Hertzm<strong>an</strong> & Boyce, 2010; Shonk<strong>of</strong>f & Levitt, 2010). Solutions to <strong>the</strong> issues<br />
that are faced by communities, <strong>in</strong>clud<strong>in</strong>g poverty, health <strong>in</strong>equalities, <strong>an</strong>d access to quality<br />
early learn<strong>in</strong>g <strong>an</strong>d care opportunities are not limited to a s<strong>in</strong>gle discipl<strong>in</strong>e or sector <strong>an</strong>d require<br />
<strong>the</strong> expertise <strong>an</strong>d collaborative efforts <strong>of</strong> community leaders, funders, <strong>the</strong> academy, <strong>an</strong>d all<br />
levels <strong>of</strong> government. There is grow<strong>in</strong>g recognition that collaboration through communityuniversity<br />
partnerships is <strong>an</strong> effective way to br<strong>in</strong>g community members, practitioners, <strong>an</strong>d<br />
researchers toge<strong>the</strong>r to discuss import<strong>an</strong>t issues <strong>in</strong> <strong>an</strong> environment where multiple worldviews<br />
are respected, solutions c<strong>an</strong> be generated, <strong>an</strong>d knowledge c<strong>an</strong> be co-constructed (Cargo &<br />
Mercer, 2008; Israel, Schultz, Parker, & Becker, 1998; Kajner, Fletcher, & Makokis, 2011).<br />
M<strong>an</strong>y <strong>of</strong> <strong>the</strong>se partnerships are guided by pr<strong>in</strong>ciples <strong>of</strong> community-based participatory<br />
research (CBPR), which encourages au<strong>the</strong>ntic collaboration by br<strong>in</strong>g<strong>in</strong>g toge<strong>the</strong>r knowledge<br />
<strong>an</strong>d expertise from multiple sectors <strong>an</strong>d discipl<strong>in</strong>es. CBPR pr<strong>in</strong>ciples are <strong>in</strong>tended to re<strong>in</strong>force<br />
<strong>the</strong> relev<strong>an</strong>ce <strong>of</strong> <strong>the</strong> partnership through shared leadership <strong>an</strong>d decision-mak<strong>in</strong>g, to foster<br />
ownership <strong>an</strong>d susta<strong>in</strong>ability (Israel et al., 1998; M<strong>in</strong>kler & Wallerste<strong>in</strong>, 2008).<br />
Community-university partnerships have been developed to tackle press<strong>in</strong>g social issues <strong>an</strong>d<br />
are well documented <strong>in</strong> <strong>the</strong> community engaged scholarship (CES) literature (e.g., Jagosh et al.,<br />
Volume 3/Issue 2/Fall 2017
60 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
2015; Pei, Feltham, Ford, & Schwartz, 2015; Visw<strong>an</strong>ath<strong>an</strong> et al., 2004). Fewer examples exist<br />
<strong>of</strong> susta<strong>in</strong>able community-university partnerships <strong>an</strong>d networks that encompass all aspects<br />
<strong>of</strong> CES (e.g., research, teach<strong>in</strong>g <strong>an</strong>d learn<strong>in</strong>g, student engagement, knowledge mobilization)<br />
with core resources (hum<strong>an</strong> <strong>an</strong>d f<strong>in</strong><strong>an</strong>cial) from both university <strong>an</strong>d community partners.<br />
Although less common, examples <strong>of</strong> such partnerships <strong>in</strong>clude <strong>the</strong> Community-University<br />
Partnership Program at <strong>the</strong> University <strong>of</strong> Brighton, Community-based Research C<strong>an</strong>ada,<br />
<strong>an</strong>d Participatory Research <strong>in</strong> Asia. In <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta, <strong>the</strong> Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families (CUP) <strong>in</strong> <strong>the</strong> Faculty <strong>of</strong> Extension<br />
at <strong>the</strong> University <strong>of</strong> Alberta represents <strong>an</strong>o<strong>the</strong>r example. CUP was launched <strong>in</strong> 2000 through<br />
shared community <strong>an</strong>d university leadership to improve <strong>the</strong> development <strong>of</strong> children, youth,<br />
<strong>an</strong>d families by promot<strong>in</strong>g <strong>in</strong>teractions among researchers <strong>an</strong>d community members (e.g.,<br />
practitioners, policymakers, families) <strong>in</strong> <strong>the</strong> areas <strong>of</strong> research, knowledge shar<strong>in</strong>g, <strong>an</strong>d lifelong<br />
learn<strong>in</strong>g (Chapm<strong>an</strong>, 2015; McCaffrey, 2007). CUP has <strong>the</strong> m<strong>an</strong>date to nurture environments<br />
where evidence is used effectively to develop practices, programs, <strong>an</strong>d policies that support <strong>the</strong><br />
healthy development <strong>of</strong> children, youth, families <strong>an</strong>d communities across four priority areas:<br />
policy, poverty, early childhood development, <strong>an</strong>d evaluation.<br />
Over its 17 years <strong>of</strong> operation, university <strong>an</strong>d community partners have shared jo<strong>in</strong>t<br />
responsibility for guid<strong>in</strong>g <strong>an</strong>d susta<strong>in</strong><strong>in</strong>g CUP, <strong>an</strong>d are represented by CUP’s Steer<strong>in</strong>g<br />
Committee. Reflect<strong>in</strong>g <strong>the</strong> partnership, <strong>an</strong> academic <strong>an</strong>d community member co-chair <strong>the</strong><br />
CUP steer<strong>in</strong>g committee, <strong>an</strong>d its membership is currently comprised <strong>of</strong> 27 members that<br />
represent foundations, municipal <strong>an</strong>d prov<strong>in</strong>cial government, academics, research <strong>an</strong>d policy<br />
centres, <strong>an</strong>d community-based agencies. This govern<strong>an</strong>ce structure has long provided <strong>the</strong><br />
impetus for collaborative opportunities <strong>in</strong> CUP’s key focus on evaluation <strong>an</strong>d early childhood<br />
development from its m<strong>an</strong>y community requests for research, measurement, <strong>an</strong>d evaluation<br />
support (Bis<strong>an</strong>z, Edwards, & Shaw, 2013). These requests have resulted <strong>in</strong> shar<strong>in</strong>g resources,<br />
broker<strong>in</strong>g relationships with o<strong>the</strong>r faculty <strong>an</strong>d graduate students on campus, <strong>an</strong>d develop<strong>in</strong>g<br />
participatory research <strong>an</strong>d/or evaluation projects. However, with ever-<strong>in</strong>creas<strong>in</strong>g requests from<br />
community-based agencies that are not always accomp<strong>an</strong>ied by resources, it became necessary<br />
for CUP to determ<strong>in</strong>e a more systematic <strong>an</strong>d effective way to respond to <strong>the</strong>se needs.<br />
This reflective essay provides <strong>an</strong> <strong>in</strong>-depth account <strong>of</strong> how community <strong>an</strong>d university<br />
members <strong>of</strong> CUP spearheaded <strong>the</strong> development <strong>of</strong> a network to adv<strong>an</strong>ce evaluation capacity<br />
<strong>in</strong> <strong>the</strong> early childhood development field, us<strong>in</strong>g a CBPR approach. First, we describe <strong>the</strong><br />
context for <strong>the</strong> development <strong>of</strong> <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN) 1 <strong>an</strong>d <strong>the</strong> reason for<br />
focus<strong>in</strong>g on early childhood development. This is followed by <strong>an</strong> expl<strong>an</strong>ation <strong>of</strong> <strong>the</strong> purpose<br />
<strong>an</strong>d processes <strong>in</strong>volved <strong>in</strong> three areas <strong>of</strong> signific<strong>an</strong>t engagement: partners, stakeholders, <strong>an</strong>d<br />
students. We reflect on <strong>the</strong> methods <strong>of</strong> engagement used across <strong>the</strong>se three areas <strong>an</strong>d <strong>the</strong><br />
impact <strong>of</strong> <strong>the</strong>se methods on <strong>the</strong> outcomes that we achieved. F<strong>in</strong>ally, we conclude <strong>the</strong> paper<br />
with some f<strong>in</strong>al considerations for guid<strong>in</strong>g engaged scholars <strong>an</strong>d with <strong>the</strong> next steps <strong>in</strong> our<br />
own work.<br />
1 For <strong>the</strong> sake <strong>of</strong> consistency, we refer to <strong>the</strong> ECN <strong>an</strong>d its development <strong>in</strong> <strong>the</strong> past tense throughout this paper. However,<br />
<strong>the</strong> ECN is a long-term project <strong>an</strong>d is <strong>the</strong>refore ongo<strong>in</strong>g.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
61<br />
Address<strong>in</strong>g Community <strong>Evaluation</strong> Needs Through Partnership<br />
CUP has received a steadily <strong>in</strong>creas<strong>in</strong>g number <strong>of</strong> requests from community agencies for<br />
assist<strong>an</strong>ce with research <strong>an</strong>d evaluation (Bis<strong>an</strong>z et al., 2013). This reflects <strong>an</strong> <strong>in</strong>creas<strong>in</strong>g dem<strong>an</strong>d<br />
across C<strong>an</strong>ada for evaluation. Funders, program pl<strong>an</strong>ners, <strong>an</strong>d policymakers are seek<strong>in</strong>g<br />
rigorous <strong>an</strong>d reliable evidence to <strong>in</strong>form resource allocation <strong>an</strong>d improve essential services<br />
(McSh<strong>an</strong>e, Usher, T<strong>an</strong>don, & Steel, 2015). However, <strong>the</strong> dem<strong>an</strong>d for evaluation currently<br />
surpasses <strong>the</strong> required resources <strong>an</strong>d supply <strong>of</strong> evaluation knowledge <strong>an</strong>d expertise available<br />
to m<strong>an</strong>y community agencies (Gauthier et al., 2010). This has placed community agencies <strong>in</strong> <strong>an</strong><br />
untenable situation. With limited fund<strong>in</strong>g, hum<strong>an</strong> resources, <strong>an</strong>d evaluation expertise to collect<br />
<strong>an</strong>d use evaluation evidence, community agencies <strong>of</strong>ten struggle to justify cont<strong>in</strong>ued support<br />
through evaluation (Bakken, Núñez, & Couture, 2014; Cous<strong>in</strong>s, Goh, Elliott, & Bourgeois,<br />
2014; J<strong>an</strong>zen et al., 2017).<br />
To ga<strong>in</strong> a deeper underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> widespread evaluation capacity issues with <strong>the</strong> <strong>in</strong>tent <strong>of</strong><br />
develop<strong>in</strong>g <strong>an</strong> effective response, CUP hosted two focus groups <strong>in</strong> 2012. The 14 particip<strong>an</strong>ts<br />
<strong>in</strong>cluded leaders (e.g., CEOs, executive directors, <strong>an</strong>d m<strong>an</strong>agers) from nonpr<strong>of</strong>its, foundations,<br />
<strong>an</strong>d prov<strong>in</strong>cial government, as well as pr<strong>of</strong>essional evaluators <strong>an</strong>d university academics familiar<br />
with evaluation related issues. What emerged from <strong>the</strong> focus groups was a locally-relev<strong>an</strong>t<br />
underst<strong>an</strong>d<strong>in</strong>g that community agencies f<strong>in</strong>d <strong>the</strong> process <strong>of</strong> evaluation challeng<strong>in</strong>g <strong>in</strong> common<br />
ways. These f<strong>in</strong>d<strong>in</strong>gs resonated with what o<strong>the</strong>rs had already reported <strong>in</strong> <strong>the</strong> literature: funders<br />
<strong>of</strong>ten request specific evaluation methods <strong>an</strong>d outcomes to meet <strong>the</strong>ir needs for accountability<br />
that do not realistically reflect org<strong>an</strong>izational strategic learn<strong>in</strong>g goals, time, <strong>an</strong>d resources<br />
(Carm<strong>an</strong> & Milleson, 2005; Leviton, 2014); agencies f<strong>in</strong>d <strong>the</strong> process <strong>of</strong> evaluation challeng<strong>in</strong>g<br />
due to <strong>in</strong>sufficient fund<strong>in</strong>g, <strong>an</strong>d lack <strong>of</strong> hum<strong>an</strong> resource capacity (Bakken et al., 2014;<br />
Cous<strong>in</strong>s, et al., 2014; J<strong>an</strong>zen et al., 2017); experts f<strong>in</strong>d it difficult to provide all <strong>the</strong> required<br />
resources, knowledge, <strong>an</strong>d capacity to community agencies; <strong>an</strong>d evaluation outcomes <strong>of</strong>ten<br />
prove un<strong>in</strong>formative for program development <strong>an</strong>d practice. Common across <strong>the</strong> focus group<br />
particip<strong>an</strong>ts was <strong>the</strong> urgent need for fur<strong>the</strong>r dialogue among <strong>in</strong>tersectoral stakeholders who<br />
support evaluation <strong>of</strong> programs. They also validated <strong>the</strong> need to create a central po<strong>in</strong>t where<br />
stakeholders could access coord<strong>in</strong>ated evaluation capacity build<strong>in</strong>g resources, <strong>an</strong>d ensure high<br />
quality tra<strong>in</strong><strong>in</strong>g, practice, <strong>an</strong>d research <strong>in</strong> evaluation.<br />
Form<strong>in</strong>g <strong>the</strong> Partnership<br />
Focus group f<strong>in</strong>d<strong>in</strong>gs were presented to <strong>the</strong> CUP steer<strong>in</strong>g committee, <strong>an</strong>d a work<strong>in</strong>g group<br />
was established to realize <strong>the</strong> evaluation <strong>in</strong>itiative. The work<strong>in</strong>g group consisted <strong>of</strong> a<br />
foundation CEO, two executive directors <strong>of</strong> large nonpr<strong>of</strong>it agencies, <strong>an</strong>d <strong>an</strong> academic <strong>an</strong>d<br />
research associate affiliated with CUP. The key task <strong>of</strong> <strong>the</strong> work<strong>in</strong>g group was to identify<br />
<strong>an</strong>d br<strong>in</strong>g toge<strong>the</strong>r leaders (funders, nonpr<strong>of</strong>its, academics, <strong>an</strong>d government) from <strong>the</strong> social<br />
sector to form a partnership. The <strong>in</strong>tent <strong>of</strong> <strong>the</strong> partnership was to develop <strong>an</strong>d operationalize<br />
a robust, coord<strong>in</strong>ated pl<strong>an</strong> for <strong>in</strong>creas<strong>in</strong>g <strong>the</strong> availability <strong>of</strong> quality evaluation knowledge,<br />
resources, expertise, <strong>an</strong>d tailored capacity build<strong>in</strong>g opportunities. In September 2013, a group<br />
<strong>of</strong> 18 leaders was brought toge<strong>the</strong>r for a full-day meet<strong>in</strong>g to determ<strong>in</strong>e (1) what <strong>the</strong> focus<br />
Volume 3/Issue 2/Fall 2017
62 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
<strong>of</strong> <strong>the</strong> evaluation <strong>in</strong>itative should be, (2) how to fund <strong>the</strong> <strong>in</strong>itative, (3) what <strong>in</strong>dividuals <strong>an</strong>d<br />
org<strong>an</strong>izations needed to be <strong>in</strong>volved, <strong>an</strong>d (4) what steps <strong>the</strong> partnership should take mov<strong>in</strong>g<br />
forward. The group decided that <strong>the</strong> focus <strong>of</strong> <strong>the</strong> <strong>in</strong>itative would be on evaluation <strong>in</strong> <strong>the</strong> early<br />
childhood development (ECD) field. Much <strong>of</strong> <strong>the</strong> group’s discussion focused on <strong>the</strong> need<br />
to foster <strong>an</strong>d support “evaluative th<strong>in</strong>k<strong>in</strong>g” <strong>in</strong> <strong>the</strong> ECD sector. Evaluative th<strong>in</strong>k<strong>in</strong>g has been<br />
def<strong>in</strong>ed as<br />
Critical th<strong>in</strong>k<strong>in</strong>g applied <strong>in</strong> <strong>the</strong> context <strong>of</strong> evaluation, motivated by <strong>an</strong> attitude<br />
<strong>of</strong> <strong>in</strong>quisitiveness <strong>an</strong>d a belief <strong>in</strong> <strong>the</strong> value <strong>of</strong> evidence, that <strong>in</strong>volves identify<strong>in</strong>g<br />
assumptions, pos<strong>in</strong>g thoughtful questions, pursu<strong>in</strong>g deeper underst<strong>an</strong>d<strong>in</strong>g through<br />
reflection <strong>an</strong>d perspective tak<strong>in</strong>g, <strong>an</strong>d <strong>in</strong>form<strong>in</strong>g decisions <strong>in</strong> preparation for action.<br />
(Buckley, Archibald, Hargraves, & Trochim, 2015, p. 378).<br />
To stimulate <strong>an</strong>d support <strong>the</strong> <strong>in</strong>tiative, several partners <strong>of</strong>fered <strong>in</strong>-k<strong>in</strong>d <strong>an</strong>d cash<br />
contributions to support <strong>an</strong> application for a Social Sciences <strong>an</strong>d Hum<strong>an</strong>ities Research Council<br />
Partnership <strong>Development</strong> Gr<strong>an</strong>t (SSHRC PDG), which we were successful <strong>in</strong> obta<strong>in</strong><strong>in</strong>g<br />
<strong>in</strong> 2014. As determ<strong>in</strong>ed dur<strong>in</strong>g <strong>the</strong> September pl<strong>an</strong>n<strong>in</strong>g meet<strong>in</strong>g, <strong>the</strong> primary objectives <strong>of</strong><br />
<strong>the</strong> SSHRC PDG were to (1) conduct <strong>an</strong> <strong>in</strong>tersectoral needs assessment us<strong>in</strong>g community<br />
forums with <strong>the</strong> aim <strong>of</strong> identify<strong>in</strong>g common evaluation knowledge (i.e., l<strong>an</strong>guage, metrics,<br />
methods, <strong>the</strong>ories <strong>an</strong>d issues) <strong>an</strong>d capacity gaps; (2) develop <strong>an</strong>d deliver educational resources<br />
<strong>an</strong>d tra<strong>in</strong><strong>in</strong>g opportunities that address <strong>the</strong>se gaps <strong>an</strong>d to subsequently evaluate <strong>an</strong>d ref<strong>in</strong>e<br />
<strong>the</strong> resources <strong>an</strong>d tra<strong>in</strong><strong>in</strong>g; <strong>an</strong>d (3) nurture <strong>an</strong>d susta<strong>in</strong> <strong>an</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> that<br />
supports ongo<strong>in</strong>g dialogue <strong>of</strong> evaluation experts, government, funders, <strong>an</strong>d community<br />
agencies at a national level, <strong>an</strong>d knowledge mobilization <strong>of</strong> community-engaged evaluative<br />
practices across <strong>the</strong> r<strong>an</strong>ge <strong>of</strong> sectors that impact ECD. The partnership now had a clear focus<br />
on build<strong>in</strong>g a network to adv<strong>an</strong>ce evaluation practice <strong>in</strong> <strong>the</strong> ECD field, <strong>an</strong>d <strong>the</strong> necessary<br />
fund<strong>in</strong>g <strong>an</strong>d committed leaders to move th<strong>in</strong>gs forward.<br />
Rationale for Focus<strong>in</strong>g on <strong>Early</strong> <strong>Childhood</strong> <strong>Development</strong><br />
The rationale for <strong>the</strong> partnership’s focus on evaluation <strong>in</strong> ECD was based on several<br />
considerations. First, robust scientific evidence <strong>in</strong> <strong>the</strong> areas <strong>of</strong> child <strong>an</strong>d family health <strong>an</strong>d wellbe<strong>in</strong>g<br />
demonstrates that experiences <strong>an</strong>d environments <strong>in</strong> <strong>the</strong> early years pr<strong>of</strong>oundly impact<br />
children’s development (Akbari & McCuaig, 2017; Hertzm<strong>an</strong> & Boyce, 2010; Shonk<strong>of</strong>f &<br />
Levitt, 2010; Shonk<strong>of</strong>f, 2017). Incorporat<strong>in</strong>g multiple sectors <strong>an</strong>d systems, <strong>the</strong> field <strong>of</strong> ECD<br />
is <strong>in</strong>terdiscipl<strong>in</strong>ary <strong>an</strong>d complex yet provides critical opportunities for <strong>in</strong>novations <strong>in</strong> social<br />
policy <strong>an</strong>d practices. When policies <strong>an</strong>d programs that target <strong>the</strong> early years are responsive to<br />
<strong>the</strong> complexity <strong>of</strong> <strong>the</strong> field, <strong>the</strong>y c<strong>an</strong> reduce expensive <strong>in</strong>terventions <strong>in</strong> later years (Akbari &<br />
McCuaig, 2017; Dunc<strong>an</strong> et al., 2007; Heckm<strong>an</strong>, 2008; OECD, 2012; Shonk<strong>of</strong>f, 2017). Despite<br />
signific<strong>an</strong>t <strong>in</strong>vestments <strong>in</strong> early years programm<strong>in</strong>g, approximately one <strong>in</strong> four C<strong>an</strong>adi<strong>an</strong><br />
children lack <strong>the</strong> social, emotional, <strong>an</strong>d cognitive capacities to benefit from <strong>the</strong> public education<br />
system (CIHI, 2014). This rate doubles for Indigenous children <strong>an</strong>d English-l<strong>an</strong>guage learners<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
63<br />
<strong>of</strong> immigr<strong>an</strong>t <strong>an</strong>d refugee backgrounds, who experience social vulnerabilities as a result <strong>of</strong><br />
economic, cultural, <strong>an</strong>d l<strong>an</strong>guage differences (Cabrera, 2013; Georgis, Gokiert, & Kirova, 2018;<br />
Gokiert et al., 2014; Government <strong>of</strong> C<strong>an</strong>ada, 2011). The prevalence <strong>of</strong> mental health disorders<br />
among children <strong>in</strong> C<strong>an</strong>ada is about 13% (Waddell, Shepherd, Schwartz & Baric<strong>an</strong>, 2014), <strong>an</strong>d<br />
roughly a quarter <strong>of</strong> C<strong>an</strong>adi<strong>an</strong> children are liv<strong>in</strong>g <strong>in</strong> low-<strong>in</strong>come households (Statistics C<strong>an</strong>ada,<br />
2011). Fur<strong>the</strong>rmore, <strong>in</strong> 2016 UNICEF r<strong>an</strong>ked C<strong>an</strong>ada 26th <strong>of</strong> 35 high-<strong>in</strong>come countries for<br />
child well-be<strong>in</strong>g. Such statistics <strong>in</strong>dicate <strong>the</strong>re is a press<strong>in</strong>g need for improvement <strong>in</strong> exist<strong>in</strong>g<br />
C<strong>an</strong>adi<strong>an</strong> child- <strong>an</strong>d family-focused policies <strong>an</strong>d practices.<br />
Second, policy frameworks have been emerg<strong>in</strong>g for over a decade that encourage a<br />
common underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> development <strong>in</strong> <strong>the</strong> early years, promote shared l<strong>an</strong>guage <strong>an</strong>d<br />
outcomes, create cont<strong>in</strong>uity across jurisdictions <strong>an</strong>d sett<strong>in</strong>gs, <strong>an</strong>d serve as a resource to support<br />
program <strong>an</strong>d policy development (CMEC, 2014; Government <strong>of</strong> Alberta, 2013; Government<br />
<strong>of</strong> M<strong>an</strong>itoba, 2013; Government <strong>of</strong> NWT, 2013; Munro, 2006). Unfortunately, <strong>the</strong>se policy<br />
frameworks tend to create un<strong>in</strong>tended complications for <strong>the</strong> early childhood system. Funders<br />
adopt <strong>the</strong>se policy frameworks, <strong>an</strong>d do not always provide clear expectations on how to<br />
use <strong>the</strong>m or contribute to <strong>the</strong>m <strong>in</strong> me<strong>an</strong><strong>in</strong>gful ways. This leaves <strong>the</strong> public sector collect<strong>in</strong>g<br />
considerable amounts <strong>of</strong> data that result <strong>in</strong> a signific<strong>an</strong>t “data burden,” as nonpr<strong>of</strong>its <strong>in</strong>vest<br />
resources beyond <strong>the</strong>ir me<strong>an</strong>s to produce perform<strong>an</strong>ce data that is <strong>of</strong> little use to both <strong>the</strong><br />
org<strong>an</strong>izations produc<strong>in</strong>g <strong>the</strong> data <strong>an</strong>d <strong>the</strong> funders request<strong>in</strong>g it (e.g. Snibbe, 2006; Carm<strong>an</strong>,<br />
2010; Leviton, 2014; Liket, Rey-Garcia, & Maas, 2014).<br />
F<strong>in</strong>ally, <strong>the</strong>re is no common approach to quality evaluation knowledge, resources,<br />
expertise, <strong>an</strong>d capacity-build<strong>in</strong>g opportunities tailored to <strong>the</strong> needs <strong>of</strong> <strong>the</strong> <strong>in</strong>tersectoral <strong>an</strong>d<br />
<strong>in</strong>terdiscipl<strong>in</strong>ary nature <strong>of</strong> <strong>the</strong> ECD field. This void compromises high-quality research,<br />
tra<strong>in</strong><strong>in</strong>g, <strong>an</strong>d practice <strong>in</strong> evaluation, <strong>an</strong>d ultimately impacts <strong>the</strong> programs <strong>an</strong>d practices <strong>in</strong><br />
ECD. A coord<strong>in</strong>ated approach was <strong>the</strong> solution, <strong>an</strong>d so we extensively engaged partners,<br />
stakeholders, <strong>an</strong>d students to underst<strong>an</strong>d <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> ECD field. In <strong>the</strong><br />
next sections, we will explore our process <strong>of</strong> develop<strong>in</strong>g <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong><br />
(ECN) through three dist<strong>in</strong>ct but related <strong>the</strong>mes <strong>of</strong> engagement: (1) partner engagement, (2)<br />
stakeholder engagement, <strong>an</strong>d (3) student engagement.<br />
Partner Engagement: <strong>Develop<strong>in</strong>g</strong> a CBPR Partnership for <strong>Evaluation</strong><br />
As <strong>the</strong> ECN is grounded <strong>in</strong> a CBPR approach (Israel et al., 1998), it <strong>in</strong>tegrates research, action,<br />
reflection, <strong>an</strong>d communication. As such, a partnership was <strong>the</strong> first step towards build<strong>in</strong>g<br />
<strong>the</strong> ECN because it provides pr<strong>in</strong>ciples <strong>an</strong>d methods to guide <strong>the</strong> work <strong>of</strong> <strong>the</strong> network, c<strong>an</strong><br />
stimulate <strong>in</strong>tersectoral <strong>an</strong>d <strong>in</strong>terdiscipl<strong>in</strong>ary dialogue, <strong>an</strong>d c<strong>an</strong> ensure that community needs <strong>an</strong>d<br />
values are at <strong>the</strong> foundation <strong>of</strong> <strong>the</strong> network. Partners were carefully chosen for <strong>the</strong> import<strong>an</strong>t<br />
expertise <strong>the</strong>y had, <strong>an</strong>d <strong>the</strong> role <strong>the</strong>y could play <strong>in</strong> work<strong>in</strong>g toge<strong>the</strong>r to recognize <strong>an</strong>d address<br />
<strong>the</strong> complexity <strong>of</strong> <strong>in</strong>tersectoral evaluative th<strong>in</strong>k<strong>in</strong>g. Partners represented stakeholder groups<br />
whose def<strong>in</strong>ition(s) <strong>of</strong> evaluation effectiveness, practices, <strong>an</strong>d outcomes are <strong>in</strong>fluenced by<br />
<strong>the</strong> sector with<strong>in</strong> which <strong>the</strong>y worked <strong>an</strong>d <strong>the</strong> role <strong>the</strong>y played <strong>in</strong> support<strong>in</strong>g ECD programs,<br />
practices, or policies.<br />
Volume 3/Issue 2/Fall 2017
64 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
ECN Partnership Govern<strong>an</strong>ce<br />
The partnership that was developed <strong>an</strong>d now susta<strong>in</strong>s <strong>the</strong> ECN comprised three ma<strong>in</strong><br />
govern<strong>in</strong>g bodies <strong>an</strong>d <strong>in</strong>volves <strong>the</strong> authors <strong>in</strong> various roles: a steer<strong>in</strong>g committee, core research<br />
team, <strong>an</strong>d project m<strong>an</strong>agement team (see Figure 1). The 19-member Steer<strong>in</strong>g Committee had<br />
representation from government, community agencies, funders, evaluation consult<strong>an</strong>ts <strong>an</strong>d<br />
academia. The Steer<strong>in</strong>g Committee was <strong>the</strong> decision-mak<strong>in</strong>g body for <strong>the</strong> ECN <strong>an</strong>d provided<br />
high-level guid<strong>an</strong>ce, support, <strong>an</strong>d direction for partner engagement, research, data <strong>in</strong>terpretation,<br />
knowledge mobilization, <strong>an</strong>d fund<strong>in</strong>g <strong>in</strong>itiatives. The Steer<strong>in</strong>g Committee comprised 19<br />
members, <strong>in</strong>clud<strong>in</strong>g academics (Gokiert, K<strong>in</strong>gsley), funders, government representatives,<br />
evaluation consult<strong>an</strong>ts, <strong>an</strong>d nonpr<strong>of</strong>it representatives. The Core Research Team met monthly<br />
<strong>an</strong>d was responsible for <strong>the</strong> design, development, <strong>an</strong>d implementation <strong>of</strong> <strong>the</strong> research<br />
component <strong>of</strong> <strong>the</strong> ECN. The Core Research Team comprised n<strong>in</strong>e members, <strong>in</strong>clusive <strong>of</strong><br />
<strong>an</strong> executive director from a national nonpr<strong>of</strong>it (Hopk<strong>in</strong>s), a government representative, five<br />
academics (Cor, Gokiert, Poth, Spr<strong>in</strong>gett), <strong>an</strong>d one postdoctoral fellow (K<strong>in</strong>gsley). The Project<br />
M<strong>an</strong>agement Team met weekly <strong>an</strong>d was responsible for <strong>the</strong> development <strong>an</strong>d monitor<strong>in</strong>g <strong>of</strong><br />
<strong>the</strong> pl<strong>an</strong>s, schedules, budgets, <strong>an</strong>d deliverables <strong>of</strong> <strong>the</strong> ECN with<strong>in</strong> <strong>the</strong> established time frames<br />
<strong>an</strong>d quality guidel<strong>in</strong>es approved by <strong>the</strong> Steer<strong>in</strong>g Committee. The Project M<strong>an</strong>agement Team<br />
comprised <strong>the</strong> pr<strong>in</strong>cipal <strong>in</strong>vestigator (Gokiert), a postdoctoral fellow (K<strong>in</strong>gsley), graduate<br />
research assist<strong>an</strong>ts (El Hassar, T<strong>in</strong>k, Tremblay), practicum students, <strong>an</strong>d student volunteers.<br />
Figure 1: <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> Govern<strong>an</strong>ce<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
65<br />
Engag<strong>in</strong>g Partners through Focus Groups<br />
To develop guid<strong>in</strong>g goals, common pr<strong>in</strong>ciples, <strong>in</strong>dicators <strong>of</strong> success for <strong>the</strong> partnership, <strong>an</strong>d<br />
a framework for build<strong>in</strong>g a prov<strong>in</strong>cial evaluation agenda, we hosted two focus groups with<br />
16 <strong>in</strong>dividuals across <strong>the</strong> levels <strong>of</strong> govern<strong>an</strong>ce. The questions posed were: What was <strong>the</strong><br />
orig<strong>in</strong>al reason you agreed to participate <strong>in</strong> <strong>the</strong> ECN <strong>an</strong>d what do you hope to ga<strong>in</strong> from<br />
your <strong>in</strong>volvement? What are <strong>the</strong> essential elements <strong>of</strong> <strong>the</strong> community-university partnership<br />
that will contribute to develop<strong>in</strong>g a successful ECN? What would success look like for <strong>the</strong><br />
ECN? The focus groups were audio-recorded, <strong>an</strong>d three graduate research assist<strong>an</strong>ts took<br />
extensive notes. Members <strong>of</strong> <strong>the</strong> Project M<strong>an</strong>agement Team completed a <strong>the</strong>matic <strong>an</strong>alysis <strong>of</strong><br />
this data <strong>an</strong>d org<strong>an</strong>ized <strong>the</strong> <strong>in</strong>formation <strong>in</strong>to core partnership pr<strong>in</strong>ciples, project outcomes <strong>an</strong>d<br />
particular actions that partners described w<strong>an</strong>t<strong>in</strong>g from <strong>the</strong> ECN.<br />
The pr<strong>in</strong>ciples that arose from <strong>the</strong> focus groups were highly reflective <strong>of</strong> <strong>the</strong> guid<strong>in</strong>g<br />
pr<strong>in</strong>ciples <strong>of</strong> a CBPR approach (Israel et al., 1998). They <strong>in</strong>cluded <strong>the</strong> need for trust, mutual<br />
benefit, equity, co-creation, accessibility, collaboration, strong communication, a commitment<br />
to action, <strong>an</strong>d engagement at all levels. The focus groups provided <strong>an</strong> import<strong>an</strong>t opportunity<br />
not only to identify key pr<strong>in</strong>ciples to guide <strong>the</strong> partnership, but to also ascribe me<strong>an</strong><strong>in</strong>g to<br />
<strong>the</strong>se pr<strong>in</strong>ciples <strong>in</strong> <strong>the</strong> context <strong>of</strong> <strong>the</strong> ECN.<br />
The partners identified several immediate <strong>an</strong>d <strong>in</strong>termediate outcomes that were import<strong>an</strong>t<br />
to guide <strong>the</strong> ECN towards <strong>the</strong> ultimate outcome articulated through focus groups: children are<br />
provided <strong>the</strong> best possible start through evaluation-<strong>in</strong>formed policies <strong>an</strong>d practices. Immediate<br />
outcomes <strong>in</strong>cluded <strong>the</strong> need to <strong>in</strong>crease awareness <strong>an</strong>d underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> <strong>the</strong> purpose <strong>of</strong><br />
<strong>the</strong> ECN, <strong>in</strong>crease <strong>the</strong> perceived import<strong>an</strong>ce <strong>of</strong> evaluation, <strong>an</strong>d <strong>in</strong>crease evaluation capacity<br />
through elevated evaluative knowledge <strong>an</strong>d practice. Intermediate outcomes <strong>in</strong>cluded <strong>the</strong> need<br />
for a culture that values evaluation, risk-tak<strong>in</strong>g <strong>an</strong>d tr<strong>an</strong>sparency, <strong>an</strong> <strong>in</strong>crease time <strong>an</strong>d f<strong>in</strong><strong>an</strong>cial<br />
<strong>in</strong>vestments <strong>in</strong> evaluation, evaluations <strong>in</strong> ECD that are relev<strong>an</strong>t <strong>an</strong>d me<strong>an</strong><strong>in</strong>gful, <strong>an</strong>d <strong>the</strong><br />
<strong>in</strong>tegration <strong>of</strong> evaluation <strong>in</strong>to org<strong>an</strong>izational processes through a utilization focused approach.<br />
F<strong>in</strong>ally, a number <strong>of</strong> actions were identified that partners felt would contribute to <strong>the</strong><br />
success <strong>of</strong> <strong>the</strong> ECN. Actions <strong>in</strong>cluded <strong>the</strong> need to engage <strong>an</strong>d connect current <strong>an</strong>d future<br />
evaluators, users <strong>of</strong> evaluation, <strong>an</strong>d evaluation capacity builders; provide evaluation mentorship<br />
<strong>an</strong>d expertise; develop education tools <strong>an</strong>d resources; develop common evaluation l<strong>an</strong>guage<br />
through effective communication mech<strong>an</strong>isms; provide pr<strong>of</strong>essional development <strong>an</strong>d<br />
learn<strong>in</strong>g opportunities; <strong>an</strong>d share best practices <strong>in</strong> evaluation. For <strong>the</strong>se actions to occur, it<br />
was determ<strong>in</strong>ed that <strong>the</strong>re would be a need to mobilize <strong>an</strong>d create synergy between <strong>the</strong> levels<br />
<strong>of</strong> govern<strong>an</strong>ce <strong>an</strong>d <strong>the</strong> broader membership <strong>of</strong> <strong>the</strong> <strong>Network</strong>.<br />
Challenges <strong>an</strong>d Opportunities <strong>in</strong> Partner Engagement<br />
The focus group process was beneficial as a me<strong>an</strong>s <strong>of</strong> engag<strong>in</strong>g partners <strong>in</strong> a dialogue about<br />
shared pr<strong>in</strong>ciples, outcomes, <strong>an</strong>d actions that <strong>the</strong>y w<strong>an</strong>ted for <strong>the</strong> ECN. The focus groups also<br />
generated ideas, enthusiasm, <strong>an</strong>d a shared commitment for <strong>the</strong> ECN that o<strong>the</strong>rwise might not<br />
have happened. Despite <strong>the</strong> positive impacts <strong>of</strong> <strong>the</strong> focus groups, ma<strong>in</strong>ta<strong>in</strong><strong>in</strong>g a concerted<br />
level <strong>of</strong> engagement over time <strong>an</strong>d adher<strong>in</strong>g to <strong>the</strong> pr<strong>in</strong>ciples that were identified by <strong>the</strong><br />
Volume 3/Issue 2/Fall 2017
66 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
partners has been challeng<strong>in</strong>g for a number <strong>of</strong> reasons. For one, <strong>the</strong> breadth <strong>of</strong> <strong>the</strong> project<br />
– engag<strong>in</strong>g partners from across so m<strong>an</strong>y contexts (universities, fund<strong>in</strong>g agencies, nonpr<strong>of</strong>its,<br />
government, policy <strong>an</strong>d research centres <strong>an</strong>d consult<strong>in</strong>g firms) – has, to some extent, affected<br />
<strong>the</strong> depth <strong>of</strong> engagement that we were able to achieve. Hav<strong>in</strong>g 19 people on <strong>the</strong> steer<strong>in</strong>g<br />
committee alone limited <strong>the</strong> extent to which face-to-face meet<strong>in</strong>gs could be held <strong>an</strong>d rich<br />
discussion could be facilitated. In addition, m<strong>an</strong>y <strong>of</strong> <strong>the</strong> steer<strong>in</strong>g committee members were<br />
high-level decision makers <strong>an</strong>d leaders <strong>in</strong> <strong>the</strong> field. As a result, f<strong>in</strong>d<strong>in</strong>g meet<strong>in</strong>g times when<br />
everyone was available <strong>an</strong>d expect<strong>in</strong>g signific<strong>an</strong>t engagement <strong>in</strong> <strong>the</strong> ECN was difficult beyond<br />
meet<strong>in</strong>g once per year <strong>an</strong>d communicat<strong>in</strong>g project milestones through email.<br />
Although we <strong>in</strong>itially <strong>in</strong>tended <strong>the</strong> Steer<strong>in</strong>g Committee to be <strong>the</strong> primary decision mak<strong>in</strong>g<br />
body for <strong>the</strong> ECN, <strong>the</strong> Core Research Team took on m<strong>an</strong>y <strong>of</strong> <strong>the</strong> responsibilities <strong>of</strong> <strong>the</strong><br />
Steer<strong>in</strong>g Committee <strong>an</strong>d provided signific<strong>an</strong>t expertise <strong>in</strong> measurement, evaluation, <strong>an</strong>d<br />
engagement. Although not <strong>in</strong>itially <strong>in</strong>tended, <strong>the</strong> Core Research Team members suggested<br />
meet<strong>in</strong>g monthly to ga<strong>in</strong> momentum on <strong>the</strong> project. The Steer<strong>in</strong>g Committee still functioned,<br />
however, with a more distal role th<strong>an</strong> we orig<strong>in</strong>ally <strong>in</strong>tended, with strategic th<strong>in</strong>k<strong>in</strong>g, fund<strong>in</strong>g<br />
opportunities, <strong>an</strong>d prov<strong>in</strong>cial connection be<strong>in</strong>g <strong>the</strong> ma<strong>in</strong> areas <strong>of</strong> focus. To compensate for<br />
this limited engagement, we have <strong>in</strong>stead engaged with members <strong>of</strong> <strong>the</strong> Steer<strong>in</strong>g Committee<br />
on <strong>an</strong> <strong>in</strong>dividual basis depend<strong>in</strong>g on <strong>the</strong> task at h<strong>an</strong>d or <strong>the</strong> stage <strong>of</strong> <strong>the</strong> project, reflect<strong>in</strong>g a<br />
need to connect on a more personal level.<br />
Stakeholder Engagement: Underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong> <strong>Evaluation</strong> Needs <strong>an</strong>d Capacities <strong>of</strong> <strong>the</strong><br />
<strong>Field</strong><br />
<strong>Build<strong>in</strong>g</strong> <strong>an</strong> agenda for adv<strong>an</strong>c<strong>in</strong>g evaluation practices <strong>in</strong> <strong>the</strong> field <strong>of</strong> ECD required determ<strong>in</strong><strong>in</strong>g<br />
evaluation needs <strong>an</strong>d assets, which <strong>in</strong> turn required extensive stakeholder engagement.<br />
Stakeholder, <strong>in</strong> this sense, refers to <strong>in</strong>dividuals who might be affected by decisions about <strong>an</strong><br />
evaluation agenda (Freem<strong>an</strong>, 2010) – namely funders, evaluators, nonpr<strong>of</strong>it <strong>an</strong>d government<br />
employees, <strong>an</strong>d academics. Consistent with a CBPR approach, we w<strong>an</strong>ted to develop <strong>an</strong> agenda<br />
that had mutual benefit across <strong>the</strong> ECD field (Israel et al., 1998). It was <strong>the</strong>refore essential to<br />
underst<strong>an</strong>d <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> field from <strong>the</strong> perspectives <strong>of</strong> stakeholders work<strong>in</strong>g<br />
across a r<strong>an</strong>ge <strong>of</strong> contexts. A subsequent goal <strong>of</strong> stakeholder engagement was to ga<strong>in</strong> a wider<br />
sense <strong>of</strong> ownership <strong>an</strong>d momentum across <strong>the</strong> ECD field towards work<strong>in</strong>g collectively to<br />
address <strong>the</strong> evaluation needs that exist.<br />
To accomplish <strong>the</strong> needs <strong>an</strong>d assets assessment, we used three primary methods <strong>of</strong><br />
stakeholder engagement across <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta: survey<strong>in</strong>g stakeholders, prim<strong>in</strong>g<br />
stakeholders <strong>an</strong>d consult<strong>in</strong>g stakeholders. The methods were complementary, <strong>in</strong> that<br />
<strong>an</strong> evaluation capacity build<strong>in</strong>g survey <strong>an</strong>d stimulus paper were sent to <strong>the</strong> particip<strong>an</strong>ts <strong>in</strong><br />
adv<strong>an</strong>ce <strong>of</strong> attend<strong>in</strong>g <strong>the</strong> forums, <strong>an</strong>d <strong>the</strong> survey results <strong>an</strong>d stimulus paper were used to spark<br />
discussion throughout <strong>the</strong> forums. We will describe each method <strong>of</strong> engagement <strong>in</strong> more<br />
detail with its contribution to underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong> needs <strong>an</strong>d assets <strong>of</strong> <strong>the</strong> ECD field.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
67<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> Survey<br />
It is import<strong>an</strong>t to recognize that evaluation is not simply about develop<strong>in</strong>g <strong>in</strong> people <strong>an</strong>d<br />
org<strong>an</strong>izations <strong>the</strong> capacity to do evaluation but also about develop<strong>in</strong>g <strong>the</strong> capacity to use<br />
evaluation (Cous<strong>in</strong>s et al., 2014). The construct <strong>of</strong> evaluation capacity has been conceptualized<br />
<strong>an</strong>d def<strong>in</strong>ed <strong>in</strong> <strong>the</strong> literature <strong>an</strong>d operationalized <strong>in</strong> <strong>the</strong> form <strong>of</strong> assessment tools (e.g., Lab<strong>in</strong>,<br />
2014; Preskill & Boyle, 2008; Taylor-Ritzler, Suarez-Balcazar, Garcia-Iriarte, Henry, & Balcazar,<br />
2013). However, m<strong>an</strong>y <strong>of</strong> <strong>the</strong>se frameworks <strong>an</strong>d tools are not context specific, mak<strong>in</strong>g <strong>the</strong>m less<br />
reflective <strong>of</strong> <strong>the</strong> complex field <strong>of</strong> ECD <strong>an</strong>d thus were not appropriate for underst<strong>an</strong>d<strong>in</strong>g <strong>the</strong><br />
<strong>in</strong>dividual <strong>an</strong>d org<strong>an</strong>izational capacity <strong>of</strong> our stakeholders. As a result, it was necessary to draw<br />
on <strong>the</strong>se frameworks <strong>an</strong>d tools, through <strong>an</strong> extensive review <strong>of</strong> <strong>the</strong> literature <strong>an</strong>d to develop a<br />
survey that better reflected our context. A doctoral student (El Hassar) took a leadership role<br />
<strong>in</strong> develop<strong>in</strong>g <strong>the</strong> survey as part <strong>of</strong> her doctoral <strong>the</strong>sis. She engaged <strong>the</strong> Core Research Team at<br />
several po<strong>in</strong>ts <strong>in</strong> <strong>the</strong> development process such as determ<strong>in</strong><strong>in</strong>g <strong>the</strong> most contextually relev<strong>an</strong>t<br />
survey components <strong>an</strong>d items, measurement scales, <strong>an</strong>d survey format (onl<strong>in</strong>e). The f<strong>in</strong>al<br />
survey consisted <strong>of</strong> items across three ma<strong>in</strong> areas <strong>of</strong> evaluation capacity: <strong>in</strong>dividual capacity<br />
(e.g., attitudes, motivation, knowledge, <strong>an</strong>d skills), org<strong>an</strong>izational capacity (e.g., leadership,<br />
org<strong>an</strong>izational processes, <strong>an</strong>d available resources), <strong>an</strong>d tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d pr<strong>of</strong>essional development<br />
(e.g., tra<strong>in</strong><strong>in</strong>g experiences, desires for capacity build<strong>in</strong>g). Stakeholders that were <strong>in</strong>vited to <strong>the</strong><br />
evaluation capacity build<strong>in</strong>g forums (described below) were purposefully selected based on<br />
experience, expertise, or leadership <strong>in</strong> evaluation <strong>an</strong>d/or early childhood development <strong>in</strong> <strong>the</strong><br />
prov<strong>in</strong>ce <strong>of</strong> Alberta. The survey was sent out to <strong>the</strong>se stakeholders prior to <strong>the</strong>ir attend<strong>an</strong>ce<br />
at <strong>the</strong> prov<strong>in</strong>cial forums. Approximately 164 surveys were sent to <strong>in</strong>vitees, <strong>an</strong>d a total <strong>of</strong> 101<br />
surveys were returned. As mentioned, <strong>the</strong> result<strong>in</strong>g data were presented back to particip<strong>an</strong>ts<br />
at <strong>the</strong> forums.<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> Forums<br />
In total, <strong>the</strong> ECN hosted four forums across <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta <strong>in</strong> W<strong>in</strong>ter/Spr<strong>in</strong>g 2016<br />
as a way to fur<strong>the</strong>r underst<strong>an</strong>d <strong>the</strong> collective evaluation needs <strong>an</strong>d capacities <strong>of</strong> <strong>the</strong> ECD field.<br />
The specific purpose <strong>of</strong> <strong>the</strong> forums was to engage <strong>in</strong>fluential stakeholders <strong>in</strong> conversation<br />
about <strong>the</strong> evaluation barriers, facilitators, <strong>an</strong>d needs experienced <strong>in</strong> <strong>the</strong>ir work <strong>an</strong>d to generate<br />
<strong>in</strong>novative <strong>an</strong>d collective solutions for address<strong>in</strong>g <strong>the</strong>m. A total <strong>of</strong> 164 leaders from 78 different<br />
org<strong>an</strong>izations (fund<strong>in</strong>g, government, nonpr<strong>of</strong>it, university, <strong>an</strong>d consult<strong>in</strong>g) were <strong>in</strong>vited, <strong>an</strong>d a<br />
total <strong>of</strong> 122 attended. A large facilitation team guided <strong>an</strong>d stimulated discussion dur<strong>in</strong>g each<br />
forum <strong>an</strong>d comprised a mixed group that mirrored <strong>the</strong> diversity <strong>of</strong> <strong>the</strong> stakeholders. The team<br />
<strong>in</strong>cluded current <strong>an</strong>d retired leaders <strong>in</strong> <strong>the</strong> ECD field from school boards, fund<strong>in</strong>g agencies,<br />
<strong>an</strong>d nonpr<strong>of</strong>its; academics (Gokiert & K<strong>in</strong>gsley) <strong>an</strong>d graduate students from CUP (El Hassar,<br />
T<strong>in</strong>k, & Tremblay); <strong>an</strong>d a highly-respected consult<strong>an</strong>t <strong>in</strong> <strong>the</strong> social sector. For each forum,<br />
<strong>the</strong>re was a primary moderator who led <strong>the</strong> agenda <strong>an</strong>d several co-facilitators sat at each<br />
discussion table to guide conversation, take notes, <strong>an</strong>d report back to <strong>the</strong> larger group. The<br />
agenda for <strong>the</strong> forums was co-created by <strong>the</strong> Core Research Team, <strong>the</strong> Project M<strong>an</strong>agement<br />
Team, <strong>an</strong>d <strong>the</strong> same external consult<strong>an</strong>t who aided <strong>the</strong> facilitation.<br />
Volume 3/Issue 2/Fall 2017
68 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
To provide contextual ground<strong>in</strong>g, shared l<strong>an</strong>guage <strong>an</strong>d def<strong>in</strong>itions for <strong>the</strong> forum, <strong>an</strong>d to<br />
spark some <strong>in</strong>itial ideas, we developed <strong>an</strong>d sent a stimulus paper to all <strong>the</strong> <strong>in</strong>vitees (<strong>Evaluation</strong><br />
<strong>Capacity</strong> <strong>Network</strong>, 2015). The stimulus paper comprised five ma<strong>in</strong> sections that situated<br />
evaluation capacity build<strong>in</strong>g <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood <strong>an</strong>d asked particip<strong>an</strong>ts to th<strong>in</strong>k<br />
about <strong>the</strong> possibilities for adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation <strong>in</strong> <strong>the</strong> field. Discussion questions<br />
were <strong>in</strong>tegrated throughout <strong>an</strong>d provided <strong>in</strong> <strong>an</strong> accomp<strong>an</strong>y<strong>in</strong>g worksheet to generate ideas <strong>in</strong><br />
preparation for <strong>the</strong> forum (e.g., What ECD evaluation framework, if <strong>an</strong>y, have you found most<br />
useful <strong>an</strong>d why? What characteristics does your org<strong>an</strong>ization have that support <strong>an</strong> evaluation<br />
learn<strong>in</strong>g culture?). The stimulus paper was one <strong>of</strong> several engagement techniques implemented<br />
with <strong>the</strong> aim <strong>of</strong> stimulat<strong>in</strong>g <strong>in</strong>novative, ch<strong>an</strong>ge-focused ideas. We also used a comb<strong>in</strong>ation<br />
<strong>of</strong> small <strong>an</strong>d larger table discussions, group plenaries, a world café (www.<strong>the</strong>worldcafe.com),<br />
<strong>an</strong>d a design th<strong>in</strong>k<strong>in</strong>g challenge (St<strong>an</strong>ford University, 2017). The facilitation team made<br />
modifications to <strong>the</strong> agenda through a reflective debrief after each forum.<br />
The <strong>in</strong>formation shared dur<strong>in</strong>g each forum was captured on particip<strong>an</strong>t worksheets that<br />
were given to stakeholders at <strong>the</strong> start <strong>of</strong> <strong>the</strong> day, <strong>in</strong> addition to sticky notes used dur<strong>in</strong>g<br />
some <strong>of</strong> <strong>the</strong> group activities. This <strong>in</strong>formation was electronically <strong>in</strong>putted <strong>an</strong>d a basic <strong>the</strong>matic<br />
<strong>an</strong>alysis was conducted to org<strong>an</strong>ize <strong>the</strong> ideas. These ideas were presented as three action areas<br />
for <strong>the</strong> ECN to focus on to adv<strong>an</strong>ce evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood <strong>an</strong>d were<br />
reported back to stakeholders <strong>in</strong> a “What We Heard” summary document (T<strong>in</strong>k, K<strong>in</strong>gsley, &<br />
Gokiert, 2016). In addition, feedback from stakeholders dur<strong>in</strong>g <strong>the</strong> forums <strong>in</strong>dicated that <strong>the</strong>y<br />
w<strong>an</strong>ted us to send <strong>the</strong> survey out more broadly <strong>in</strong> order to reach all levels <strong>of</strong> <strong>an</strong> org<strong>an</strong>ization,<br />
from frontl<strong>in</strong>e staff through to <strong>the</strong> leadership. With this advice, we modified <strong>the</strong> survey to be<br />
reflective <strong>of</strong> a more diverse audience <strong>an</strong>d redistributed it. We sent <strong>the</strong> survey to all particip<strong>an</strong>ts<br />
<strong>an</strong>d asked that <strong>the</strong>y forward it through <strong>the</strong>ir org<strong>an</strong>ization to staff us<strong>in</strong>g a snowball technique<br />
<strong>of</strong> sampl<strong>in</strong>g. The second adm<strong>in</strong>istration generated over 329 responses. The survey data was<br />
reported back to stakeholders <strong>in</strong> a report that is posted on <strong>the</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong><br />
website (T<strong>in</strong>k, Gokiert, K<strong>in</strong>gsley, & El Hassar, 2017).<br />
Au<strong>the</strong>ntic stakeholder engagement is not <strong>an</strong> easy or straightforward process, even with<br />
subst<strong>an</strong>tial experience. For this reason, we relied heavily on pre-exist<strong>in</strong>g, well-established,<br />
<strong>an</strong>d trust<strong>in</strong>g relationships with pivotal stakeholders <strong>an</strong>d org<strong>an</strong>izations across <strong>the</strong> prov<strong>in</strong>ce<br />
(developed over <strong>the</strong> past decade <strong>of</strong> CUP’s history). One <strong>of</strong> <strong>the</strong> ECN partners, <strong>the</strong> Muttart<br />
Foundation, had carried out a series <strong>of</strong> forums across Alberta <strong>an</strong>d Saskatchew<strong>an</strong> with <strong>the</strong><br />
ECD sector to determ<strong>in</strong>e a system <strong>of</strong> early learn<strong>in</strong>g <strong>an</strong>d care (Muttart Foundation, 2013). We<br />
reached out to <strong>the</strong> Muttart Foundation for advice about <strong>the</strong>ir engagement process, costs, ideal<br />
locations <strong>an</strong>d venues, skilled facilitators, <strong>an</strong>d <strong>the</strong> leaders <strong>the</strong>y had previously <strong>in</strong>vited. It was<br />
under <strong>the</strong>ir guid<strong>an</strong>ce that we recruited facilitators <strong>in</strong> each region as <strong>the</strong>y had <strong>the</strong> relationships<br />
necessary to <strong>in</strong>crease <strong>the</strong> likelihood that stakeholders would participate. The Foundation used<br />
stimulus papers for each <strong>of</strong> <strong>the</strong>ir forums, <strong>an</strong>d so we adopted this idea <strong>an</strong>d created a stimulus<br />
paper to contextually ground our forums. They also provided <strong>in</strong>sight <strong>in</strong>to <strong>the</strong> pr<strong>of</strong>essional<br />
cultures <strong>of</strong> each location so we could adapt our style <strong>of</strong> engagement. For example, one <strong>of</strong> <strong>the</strong><br />
larger cities <strong>in</strong> which we hosted a forum had more <strong>of</strong> a ‘corporate’ culture, which was quite<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
69<br />
different from <strong>the</strong> ‘relaxed’ culture <strong>of</strong> one <strong>of</strong> <strong>the</strong> smaller rural locations. We made subsequent<br />
decisions about <strong>the</strong> locations based on <strong>the</strong>se differences, with <strong>the</strong> larger urb<strong>an</strong> forums held<br />
at a conference centre <strong>an</strong>d hotel <strong>in</strong> <strong>the</strong> downtown core, compared to <strong>the</strong> rural forums which<br />
were held at a public library <strong>an</strong>d a school.<br />
We w<strong>an</strong>ted <strong>the</strong> forums to be ch<strong>an</strong>ge-oriented ra<strong>the</strong>r th<strong>an</strong> deficit-focused. With this <strong>in</strong><br />
m<strong>in</strong>d, we purposefully <strong>in</strong>vited leaders who we believed were driven to f<strong>in</strong>d unique solutions to<br />
complex problems, <strong>an</strong>d possessed <strong>the</strong> <strong>in</strong>fluence to effect such ch<strong>an</strong>ge. Despite <strong>the</strong>se <strong>in</strong>tentions,<br />
<strong>the</strong> extent to which this made a difference to <strong>the</strong> quality <strong>of</strong> <strong>the</strong> discussion is difficult to gauge.<br />
In focus<strong>in</strong>g on <strong>in</strong>vit<strong>in</strong>g ch<strong>an</strong>ge-oriented leaders, we also restricted <strong>the</strong> number <strong>an</strong>d breadth<br />
<strong>of</strong> perspectives shap<strong>in</strong>g <strong>the</strong> direction <strong>of</strong> <strong>the</strong> ECN, particularly those <strong>of</strong> frontl<strong>in</strong>e staff <strong>an</strong>d<br />
service providers.<br />
Position<strong>in</strong>g <strong>the</strong> forums around ch<strong>an</strong>ge created expectations from our stakeholders for<br />
action to result from <strong>the</strong> discussions. This was exacerbated by <strong>the</strong> fact that a number <strong>of</strong> o<strong>the</strong>r<br />
consultations had been recently conducted <strong>in</strong> <strong>the</strong> field. For this reason, it was import<strong>an</strong>t to<br />
provide time-sensitive reports to summarize learn<strong>in</strong>g, <strong>an</strong>d to f<strong>in</strong>d a bal<strong>an</strong>ce between community<br />
need <strong>an</strong>d academic def<strong>in</strong>itions <strong>of</strong> rigour (K<strong>in</strong>gsley & Chapm<strong>an</strong>, 2013). In this process, our<br />
ability to respond to dem<strong>an</strong>ds for action has been impacted by our own limited resource, time<br />
<strong>an</strong>d expertise, which must grow signific<strong>an</strong>tly if we are to adequately respond to <strong>the</strong> needs<br />
presented dur<strong>in</strong>g <strong>the</strong> forums. This is fur<strong>the</strong>r tested by <strong>the</strong> <strong>in</strong>terest <strong>an</strong>d awareness generated<br />
through <strong>the</strong> engagement process. Rais<strong>in</strong>g <strong>the</strong> pr<strong>of</strong>ile <strong>of</strong> <strong>the</strong> ECN through public dialogue has<br />
also resulted <strong>in</strong> <strong>an</strong> <strong>in</strong>creased number <strong>of</strong> requests for evaluation support <strong>an</strong>d resources. In <strong>an</strong><br />
attempt to meet this need, we have drawn upon graduate students as a source <strong>of</strong> capacity for<br />
<strong>the</strong> ECN, <strong>the</strong> details <strong>of</strong> which are described <strong>in</strong> <strong>the</strong> next section.<br />
Student Engagement: Tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d Mentorship<br />
Graduate student engagement has been fundamental to <strong>the</strong> success <strong>of</strong> <strong>the</strong> ECN, as <strong>the</strong>y<br />
have provided signific<strong>an</strong>t capacity. Some students have completed <strong>the</strong>ir masters’ <strong>an</strong>d doctoral<br />
research with <strong>the</strong> ECN, while o<strong>the</strong>rs contributed through <strong>in</strong>dependent studies, research<br />
assist<strong>an</strong>tships, practicums <strong>an</strong>d course placements, <strong>an</strong>d volunteer<strong>in</strong>g. For example, two students<br />
jo<strong>in</strong>ed <strong>the</strong> project m<strong>an</strong>agement team to fulfill a 150-hour practicum as part <strong>of</strong> <strong>an</strong> embedded<br />
graduate certificate <strong>in</strong> community-based research <strong>an</strong>d evaluation. Two graduate students from<br />
<strong>the</strong> Master <strong>of</strong> Arts <strong>in</strong> Communication <strong>an</strong>d Technology program <strong>in</strong> <strong>the</strong> Faculty <strong>of</strong> Extension<br />
developed a communications pl<strong>an</strong> <strong>an</strong>d website for <strong>the</strong> ECN as part <strong>of</strong> <strong>an</strong> <strong>in</strong>dependent study<br />
course. One <strong>of</strong> <strong>the</strong>se students became fur<strong>the</strong>r <strong>in</strong>volved <strong>in</strong> <strong>the</strong> ECN <strong>an</strong>d completed her f<strong>in</strong>al<br />
capstone project with <strong>the</strong> ECN. Two doctoral c<strong>an</strong>didates are pursu<strong>in</strong>g <strong>the</strong>ir dissertation<br />
research focused on different research questions relat<strong>in</strong>g to <strong>the</strong> ECN (El Hassar, Poth, Gokiert,<br />
K<strong>in</strong>gsley, & Krishn<strong>an</strong>, 2016; Gokiert et al., 2017; Tremblay et al., 2016). In <strong>the</strong> three-year sp<strong>an</strong><br />
<strong>of</strong> <strong>the</strong> ECN thus far, eleven graduate students <strong>an</strong>d one postdoctoral fellow have been mentored<br />
through, <strong>an</strong>d provided support to, <strong>the</strong> ECN. Mentorship is generally provided by academics<br />
that are community-engaged scholars <strong>the</strong>mselves <strong>an</strong>d/or who specialize <strong>in</strong> evaluation.<br />
Volume 3/Issue 2/Fall 2017
70 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Emerg<strong>in</strong>g Community-Engaged Scholars<br />
The <strong>in</strong>tent <strong>of</strong> engag<strong>in</strong>g students <strong>in</strong> <strong>the</strong> ECN is to support <strong>the</strong>ir development as communityengaged<br />
scholars <strong>an</strong>d evaluators. The focus on such enh<strong>an</strong>ced learn<strong>in</strong>g opportunities is based<br />
on a grow<strong>in</strong>g need for pr<strong>of</strong>essionals who are equipped to navigate a social sector with complex<br />
challenges (Armitage & Levac, 2015; C<strong>an</strong>tor, DeLauer, Mart<strong>in</strong>, & Rog<strong>an</strong>, 2015). Students seem<br />
to be recogniz<strong>in</strong>g this need to exp<strong>an</strong>d <strong>the</strong>ir learn<strong>in</strong>g <strong>an</strong>d build new skills, which has resulted <strong>in</strong><br />
<strong>an</strong> <strong>in</strong>creas<strong>in</strong>g number seek<strong>in</strong>g community-based research <strong>an</strong>d evaluation opportunities. These<br />
emerg<strong>in</strong>g scholars are acquir<strong>in</strong>g skills, knowledge <strong>an</strong>d experiences that would not be available<br />
through conventional academic tra<strong>in</strong><strong>in</strong>g. For example, students are exposed to several research<br />
methods <strong>in</strong> <strong>an</strong> <strong>in</strong>terdiscipl<strong>in</strong>ary context, <strong>an</strong>d developed skills <strong>in</strong> knowledge mobilization <strong>an</strong>d<br />
engagement with academic <strong>an</strong>d non-academic audiences. More specifically, <strong>the</strong>y have facilitated<br />
meet<strong>in</strong>gs <strong>an</strong>d forums, produced grey literature <strong>an</strong>d scholarly m<strong>an</strong>uscripts, <strong>an</strong>d developed <strong>an</strong>d<br />
ma<strong>in</strong>ta<strong>in</strong>ed <strong>the</strong> ECN’s onl<strong>in</strong>e presence through email campaigns <strong>an</strong>d blog posts.<br />
Community Service Learn<strong>in</strong>g<br />
In addition to prepar<strong>in</strong>g students as community-engaged scholars, <strong>the</strong> ECN also supports<br />
students to develop <strong>the</strong>ir skills as evaluators. As mentioned, <strong>the</strong>re is a subst<strong>an</strong>tial need for<br />
evaluation support <strong>in</strong> <strong>the</strong> social sector, which was reiterated by <strong>the</strong> <strong>in</strong>formation ga<strong>the</strong>red<br />
dur<strong>in</strong>g <strong>the</strong> forums. To meet this need, while provid<strong>in</strong>g experiential learn<strong>in</strong>g <strong>in</strong> evaluation for<br />
students, service learn<strong>in</strong>g placements were established. Service learn<strong>in</strong>g is a method to enh<strong>an</strong>ce<br />
students’ learn<strong>in</strong>g <strong>an</strong>d development through org<strong>an</strong>ized service experiences <strong>in</strong> <strong>the</strong> community,<br />
<strong>an</strong>d is <strong>in</strong>tegrated <strong>in</strong>to academic curriculum (Taylor et al., 2015). Service learn<strong>in</strong>g, particularly <strong>in</strong><br />
community-based participatory projects, is widely considered to exp<strong>an</strong>d <strong>the</strong> student learn<strong>in</strong>g,<br />
provid<strong>in</strong>g opportunities to develop critical th<strong>in</strong>k<strong>in</strong>g <strong>an</strong>d a sense <strong>of</strong> civic responsibility (Taylor<br />
et al., 2015). Service learn<strong>in</strong>g also provides additional capacity to <strong>the</strong> org<strong>an</strong>izations <strong>in</strong> which<br />
<strong>the</strong> placements occur. Learn<strong>in</strong>g opportunities are <strong>the</strong>refore <strong>in</strong>tended to be <strong>of</strong> equal benefit to<br />
both <strong>the</strong> student <strong>an</strong>d <strong>the</strong> recipient <strong>of</strong> <strong>the</strong> service (Furco, 1996).<br />
The service learn<strong>in</strong>g placements <strong>in</strong> this project are embedded with<strong>in</strong> three graduate-level<br />
evaluation courses at <strong>the</strong> University <strong>of</strong> Alberta: Health Promotion Pl<strong>an</strong>n<strong>in</strong>g <strong>an</strong>d <strong>Evaluation</strong><br />
<strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health, Program <strong>Evaluation</strong> <strong>in</strong> <strong>the</strong> Department <strong>of</strong> Educational<br />
Psychology, <strong>an</strong>d Program Pl<strong>an</strong>n<strong>in</strong>g <strong>an</strong>d <strong>Evaluation</strong> <strong>in</strong> <strong>the</strong> Department <strong>of</strong> Hum<strong>an</strong> Ecology.<br />
To fulfill <strong>the</strong>ir course requirements, graduate students are asked to work with community<br />
org<strong>an</strong>izations to build contextually relev<strong>an</strong>t evaluation pl<strong>an</strong>s. To build a comprehensive<br />
pl<strong>an</strong>, <strong>the</strong>y meet with org<strong>an</strong>izations several times to ga<strong>in</strong> a contextual underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong> <strong>the</strong>ir<br />
evaluation needs, develop a logic model <strong>an</strong>d ga<strong>in</strong> feedback from staff. The course <strong>in</strong>structors<br />
mentor <strong>the</strong> students to enh<strong>an</strong>ce learn<strong>in</strong>g <strong>an</strong>d serve as a form <strong>of</strong> quality assur<strong>an</strong>ce for <strong>the</strong><br />
community <strong>an</strong>d government agencies. This approach has been generally positive; however,<br />
<strong>the</strong>re is a need to fur<strong>the</strong>r extend this service learn<strong>in</strong>g to assist org<strong>an</strong>izations with evaluation<br />
beyond <strong>the</strong> creation <strong>of</strong> <strong>an</strong> evaluation pl<strong>an</strong>. To this end, adv<strong>an</strong>ced courses <strong>in</strong> evaluation are<br />
required that <strong>in</strong>volve more extensive <strong>an</strong>d concerted student engagement over time. This need<br />
to build <strong>the</strong> evaluation capacity <strong>of</strong> university students <strong>in</strong> C<strong>an</strong>ada is well supported <strong>in</strong> <strong>the</strong><br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
71<br />
literature (e.g. McDavid & Dev<strong>in</strong>e, 2009; McSh<strong>an</strong>e et al., 2015), <strong>an</strong>d will become a focus for<br />
<strong>the</strong> ECN mov<strong>in</strong>g forward.<br />
Fund<strong>in</strong>g Graduate Students<br />
Due to limited funds to engage graduate students <strong>in</strong> <strong>the</strong> ECN, we have sought o<strong>the</strong>r fund<strong>in</strong>g<br />
opportunities to supplement student <strong>in</strong>volvement. Several funders have supported <strong>the</strong> ECN’s<br />
student engagement <strong>an</strong>d created various learn<strong>in</strong>g opportunities. The Women <strong>an</strong>d Children’s<br />
Health Research Institute (WCHRI) based at <strong>the</strong> University <strong>of</strong> Alberta has several student<br />
fund<strong>in</strong>g streams <strong>in</strong>clud<strong>in</strong>g <strong>the</strong> Patient <strong>an</strong>d Community Engagement Tra<strong>in</strong><strong>in</strong>g program<br />
<strong>an</strong>d conference travel. Through this program, graduate students <strong>an</strong>d postdoctoral fellows<br />
conduct<strong>in</strong>g engaged research are funded for 8 months to support <strong>the</strong>ir work. In each fund<strong>in</strong>g<br />
year, a community <strong>of</strong> practice is formed to discuss issues relat<strong>in</strong>g to engagement. One doctoral<br />
student (El Hassar) <strong>an</strong>d one postdoctoral fellow (K<strong>in</strong>gsley) have participated <strong>in</strong> <strong>the</strong> tra<strong>in</strong><strong>in</strong>g<br />
program <strong>an</strong>d received travel gr<strong>an</strong>ts to present <strong>the</strong>ir research at conferences. Mitacs, a C<strong>an</strong>adi<strong>an</strong><br />
fund<strong>in</strong>g agency that builds partnerships between academia <strong>an</strong>d nonpr<strong>of</strong>it or <strong>in</strong>dustry partners,<br />
has also provided subst<strong>an</strong>tial graduate student fund<strong>in</strong>g support. Mitacs funded a postdoctoral<br />
fellow (K<strong>in</strong>gsley) for three years, two doctoral (Tremblay) <strong>an</strong>d one masters’ student up to one<br />
year, to assist community agencies with evaluation <strong>an</strong>d evaluation capacity build<strong>in</strong>g.<br />
Graduate students have added signific<strong>an</strong>t capacity to <strong>the</strong> ECN; however, we have also run<br />
<strong>in</strong>to some challenges with a project reli<strong>an</strong>t on such extensive student <strong>in</strong>volvement. F<strong>in</strong>d<strong>in</strong>g<br />
fund<strong>in</strong>g for students has been difficult at times <strong>an</strong>d as mentioned, it was necessary to seek<br />
alternative forms <strong>of</strong> fund<strong>in</strong>g. Apply<strong>in</strong>g for this fund<strong>in</strong>g consumes available resources such as<br />
time. In addition, students – no matter <strong>the</strong>ir level <strong>of</strong> experience – require mentorship, or at<br />
<strong>the</strong> very least, m<strong>an</strong>agement <strong>an</strong>d supervision. Allocat<strong>in</strong>g university mentors or supervisors who<br />
have sufficient time to work with students, above <strong>the</strong>ir regular academic responsibilities, is also<br />
challeng<strong>in</strong>g. For this reason, much <strong>of</strong> this workload falls on <strong>the</strong> project m<strong>an</strong>agement team, <strong>an</strong>d<br />
is <strong>the</strong> reason that hav<strong>in</strong>g a postdoctoral fellow <strong>in</strong>volved <strong>in</strong> <strong>the</strong> project has been essential for<br />
shar<strong>in</strong>g <strong>the</strong> supervision load <strong>of</strong> students.<br />
An additional issue relates to student retention, as most students are only available on a<br />
short-term basis, ei<strong>the</strong>r because <strong>the</strong>y f<strong>in</strong>ish <strong>the</strong>ir degrees, accept alternative opportunities, or<br />
because our ability to fund <strong>the</strong>m dim<strong>in</strong>ishes. Although we have had some students volunteer<br />
for us <strong>in</strong> <strong>the</strong> past, we would prefer to be able to pay <strong>in</strong>dividuals for <strong>the</strong>ir time. F<strong>in</strong>ally, as<br />
with all CBPR projects, <strong>the</strong>re c<strong>an</strong> <strong>of</strong>ten be a disson<strong>an</strong>ce between community <strong>an</strong>d academic<br />
timel<strong>in</strong>es. Graduate students have milestones <strong>the</strong>y need to achieve before <strong>the</strong>y c<strong>an</strong> progress<br />
with <strong>the</strong>ir research (e.g., a c<strong>an</strong>didacy exam), yet <strong>the</strong> project needs to progress regardless. This<br />
c<strong>an</strong> put additional pressure on <strong>the</strong> student dur<strong>in</strong>g what is already a stressful process <strong>an</strong>d has <strong>the</strong><br />
potential to prevent <strong>the</strong> project from mov<strong>in</strong>g ahead as pl<strong>an</strong>ned. Similarly, academic st<strong>an</strong>dards<br />
<strong>an</strong>d expectations that <strong>in</strong>form student processes <strong>an</strong>d products are sometimes at odds with<br />
<strong>the</strong> flexible, responsive approaches needed when partner<strong>in</strong>g with community. For example,<br />
one <strong>of</strong> <strong>the</strong> students (El Hassar) took <strong>the</strong> lead on <strong>the</strong> development <strong>of</strong> <strong>the</strong> evaluation capacity<br />
survey as part <strong>of</strong> her dissertation <strong>an</strong>d engaged partners dur<strong>in</strong>g this process. In addition<br />
Volume 3/Issue 2/Fall 2017
72 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
to reflect<strong>in</strong>g conventional st<strong>an</strong>dards <strong>of</strong> validity, this engagement (<strong>an</strong>d <strong>the</strong> tool itself) also<br />
reflects <strong>an</strong> alternative form <strong>of</strong> rigour that more closely reflected a community-based research<br />
approach (K<strong>in</strong>gsley & Chapm<strong>an</strong>, 2013). To allow for a broader conceptualization <strong>of</strong> rigour,<br />
it is import<strong>an</strong>t to form graduate committees with faculty who have some underst<strong>an</strong>d<strong>in</strong>g <strong>of</strong><br />
community-based <strong>an</strong>d engaged research to avoid potential friction between academic <strong>an</strong>d<br />
community needs <strong>an</strong>d expectations.<br />
Conclud<strong>in</strong>g Remarks<br />
The value <strong>of</strong> co-created underst<strong>an</strong>d<strong>in</strong>gs is highlighted by <strong>the</strong> evaluation-focused efforts <strong>of</strong><br />
McSh<strong>an</strong>e <strong>an</strong>d colleagues (2015) who state, “community engagement is <strong>of</strong>ten touted as a goal<br />
for universities <strong>an</strong>d community collaboration is <strong>in</strong>creas<strong>in</strong>gly viewed as favourable <strong>in</strong> research”<br />
(p.149). <strong>Evaluation</strong> experts have long recognized <strong>the</strong> import<strong>an</strong>ce <strong>of</strong> <strong>the</strong> <strong>in</strong>terests, views,<br />
<strong>in</strong>volvement, needs, <strong>an</strong>d roles <strong>of</strong> all stakeholders <strong>in</strong> evaluation practice <strong>an</strong>d <strong>the</strong>ory (e.g., Alk<strong>in</strong>,<br />
2004; Cockerill, Myers, & Allm<strong>an</strong>, 2000; Cous<strong>in</strong>s & Earl, 1992). Us<strong>in</strong>g a CBPR approach <strong>an</strong>d<br />
through multiple forms <strong>of</strong> engagement, <strong>the</strong> ECN was developed to br<strong>in</strong>g toge<strong>the</strong>r m<strong>an</strong>y<br />
voices from across academic, government, nonpr<strong>of</strong>it <strong>an</strong>d consult<strong>in</strong>g contexts around <strong>the</strong> issue<br />
<strong>of</strong> evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> ECD.<br />
The purpose <strong>of</strong> this reflective essay was to provide <strong>an</strong> <strong>in</strong>-depth account <strong>of</strong> <strong>the</strong> development<br />
<strong>of</strong> <strong>the</strong> network <strong>an</strong>d reflect on our engagement processes. Illustrat<strong>in</strong>g <strong>the</strong> context for <strong>the</strong><br />
development <strong>of</strong> <strong>the</strong> ECN, we provided a detailed description <strong>of</strong> <strong>the</strong> purpose <strong>an</strong>d process<br />
<strong>of</strong> engagement with partners, stakeholders, <strong>an</strong>d students <strong>in</strong> <strong>the</strong> development <strong>of</strong> a prov<strong>in</strong>cial<br />
agenda for <strong>the</strong> ECN, reflect<strong>in</strong>g on each method <strong>an</strong>d its impact on <strong>the</strong> outcomes we were able<br />
to achieve. In <strong>the</strong>se conclud<strong>in</strong>g remarks, we <strong>of</strong>fer some f<strong>in</strong>al considerations for communityengaged<br />
scholars relat<strong>in</strong>g to <strong>the</strong>se three forms <strong>of</strong> engagement, provide details <strong>of</strong> our <strong>in</strong>tended<br />
next steps for <strong>the</strong> ECN, <strong>an</strong>d <strong>in</strong>vite <strong>in</strong>dividuals <strong>an</strong>d org<strong>an</strong>izations to contact us for more detailed<br />
<strong>in</strong>formation.<br />
In a summary <strong>of</strong> learn<strong>in</strong>g from a conference co-hosted by Community-Campus<br />
Partnerships for Health (CCPH) <strong>an</strong>d <strong>the</strong> University <strong>of</strong> Guelph, Wenger, Hawk<strong>in</strong>s, <strong>an</strong>d Seifer<br />
(2011, 2012) articulated <strong>the</strong> need for flexibility <strong>in</strong> community-engaged scholarship. Through<br />
our engagement with ECN partners, we also learned early on that we needed to be flexible<br />
<strong>in</strong> our approach. Despite a comprehensive pl<strong>an</strong> for govern<strong>an</strong>ce – three committees with<br />
specific roles, responsibilities, <strong>an</strong>d expectations – it did not play out <strong>in</strong> practice as we orig<strong>in</strong>ally<br />
<strong>in</strong>tended. This did not appear to negatively impact <strong>the</strong> quality <strong>of</strong> <strong>the</strong> process, but did require<br />
us to adjust our engagement expectations <strong>an</strong>d modify our collaboration processes to align<br />
with <strong>the</strong>se. To be respectful <strong>of</strong> partners’ time, we only convened <strong>the</strong> entire group for specific<br />
<strong>an</strong>d necessary purposes <strong>an</strong>d <strong>in</strong>stead met one-on-one to access specific expertise on <strong>an</strong> ad hoc<br />
basis. It is difficult to know <strong>the</strong> effects <strong>of</strong> this shift, <strong>an</strong>d it may have led to limited ownership<br />
<strong>an</strong>d learn<strong>in</strong>g amongst those less engaged. To access this <strong>in</strong>formation, it would be beneficial<br />
to ask our partners if this reflected <strong>the</strong>ir preferences <strong>an</strong>d how it may have impacted <strong>the</strong>ir<br />
<strong>in</strong>volvement.<br />
Relat<strong>in</strong>g to stakeholder engagement, <strong>the</strong> role <strong>of</strong> CUP was pivotal <strong>in</strong> provid<strong>in</strong>g a stable<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
73<br />
foundation for reach<strong>in</strong>g out to community <strong>an</strong>d government agencies <strong>an</strong>d receiv<strong>in</strong>g such a<br />
positive <strong>an</strong>d enthusiastic response. With CUP’s 17-year history as <strong>an</strong> effective relationshipbuilder,<br />
<strong>in</strong> develop<strong>in</strong>g <strong>the</strong> ECN we drew heavily on CUP’s exist<strong>in</strong>g relationships as a trusted<br />
org<strong>an</strong>ization with<strong>in</strong> <strong>the</strong> prov<strong>in</strong>ce’s social sector to determ<strong>in</strong>e scope <strong>an</strong>d pool resources. The<br />
benefits <strong>of</strong> this are unsurpris<strong>in</strong>g s<strong>in</strong>ce trust is identified as a fundamental pillar <strong>of</strong> au<strong>the</strong>ntic<br />
community-based research partnerships (e.g., Cargo & Mercer, 2008). Key partners, such as <strong>the</strong><br />
Muttart Foundation, which we relied on for guid<strong>an</strong>ce <strong>in</strong> our prov<strong>in</strong>cial forums, were critical <strong>in</strong><br />
avoid<strong>in</strong>g missteps, ga<strong>in</strong><strong>in</strong>g momentum quickly, <strong>an</strong>d <strong>in</strong>creas<strong>in</strong>g our reach subst<strong>an</strong>tially. Through<br />
our stakeholder engagement process, we also identified a number <strong>of</strong> assets that have led to<br />
new evaluation-related opportunities <strong>in</strong> <strong>the</strong> community.<br />
F<strong>in</strong>ally, engag<strong>in</strong>g students was a mutually beneficial process for <strong>the</strong> ECN <strong>an</strong>d for <strong>the</strong><br />
students <strong>the</strong>mselves. Students, through research assist<strong>an</strong>tships, practicums, course-based<br />
learn<strong>in</strong>g, <strong>an</strong>d volunteer<strong>in</strong>g, provided much needed capacity. In return, students <strong>the</strong>mselves<br />
were mentored as community-engaged, <strong>in</strong>terdiscipl<strong>in</strong>ary scholars who were better equipped<br />
to respond to “wicked problems” (C<strong>an</strong>tor et al., 2015, p. 407). Engag<strong>in</strong>g students to <strong>the</strong> extent<br />
that we did helped to highlight <strong>the</strong> press<strong>in</strong>g need to build better <strong>in</strong>frastructure to support<br />
student engagement. Although <strong>the</strong>re are a number <strong>of</strong> courses through which students are<br />
connected to <strong>the</strong> ECN, <strong>the</strong>re is currently a signific<strong>an</strong>t lack <strong>of</strong> evaluation courses <strong>in</strong> C<strong>an</strong>adi<strong>an</strong><br />
universities generally (McDavid & Dev<strong>in</strong>e, 2009), <strong>an</strong>d <strong>the</strong> University <strong>of</strong> Alberta specifically<br />
(Bis<strong>an</strong>z et al., 2013). This makes both <strong>the</strong> tra<strong>in</strong><strong>in</strong>g <strong>an</strong>d recruitment <strong>of</strong> students who have<br />
evaluation experience difficult <strong>an</strong>d limits <strong>the</strong> supply <strong>of</strong> students available to <strong>the</strong> ECN. In<br />
addition, hir<strong>in</strong>g students or facilitat<strong>in</strong>g practicum placements requires mentorship, supervision,<br />
<strong>an</strong>d m<strong>an</strong>agement. This is currently <strong>an</strong> area <strong>of</strong> lack for <strong>the</strong> ECN, with most <strong>of</strong> <strong>the</strong> supervision<br />
fall<strong>in</strong>g on <strong>the</strong> pr<strong>in</strong>ciple <strong>in</strong>vestigator <strong>an</strong>d a postdoctoral fellow on <strong>the</strong> project. This will not be<br />
susta<strong>in</strong>able over time <strong>an</strong>d is <strong>an</strong> area that requires attention.<br />
To our knowledge, <strong>the</strong> ECN is <strong>the</strong> first <strong>in</strong>itiative <strong>of</strong> its k<strong>in</strong>d to use a systems approach to<br />
build evaluation capacity that extends beyond <strong>the</strong> scope <strong>of</strong> a particular org<strong>an</strong>ization. To effect<br />
ch<strong>an</strong>ge, we believe a broad systems approach is essential to mobilize <strong>in</strong>fluential players from<br />
across <strong>the</strong> entire early childhood field (S<strong>an</strong>derson, 2000; Suárez-Herrera, Spr<strong>in</strong>gett, & Kag<strong>an</strong>,<br />
2009; Waldrop, 1992). Do<strong>in</strong>g so will not only foster a collective <strong>an</strong>d coord<strong>in</strong>ated effort to build<br />
capacity, it will <strong>in</strong>crease <strong>the</strong> likelihood that <strong>the</strong> questions asked <strong>of</strong> evaluation are valuable to <strong>the</strong><br />
field as a whole <strong>an</strong>d will be more effectively used to support improvements to early childhood<br />
policies <strong>an</strong>d practices. In addition, while this broad reach is necessary, we must also develop<br />
resources <strong>an</strong>d opportunities that are tailored to different users <strong>an</strong>d <strong>the</strong>refore contextually<br />
relev<strong>an</strong>t. As such, <strong>the</strong> ECN aims to be, simult<strong>an</strong>eously, a broad yet localized approach. This<br />
has required us to draw on <strong>the</strong> various forms <strong>of</strong> engagement described <strong>in</strong> this paper. However,<br />
foster<strong>in</strong>g deep relationships with partners rema<strong>in</strong>s a challenge with a project <strong>of</strong> this scale <strong>an</strong>d<br />
<strong>an</strong> area requir<strong>in</strong>g fur<strong>the</strong>r attention.<br />
Volume 3/Issue 2/Fall 2017
74 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Next Steps<br />
The <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN) was developed to support dialogue among<br />
<strong>in</strong>tersectoral stakeholders <strong>an</strong>d create a central po<strong>in</strong>t through which stakeholders could<br />
access coord<strong>in</strong>ated evaluation capacity build<strong>in</strong>g resources tailored to <strong>the</strong> field <strong>of</strong> ECD, <strong>an</strong>d<br />
ensure high quality tra<strong>in</strong><strong>in</strong>g, practice, <strong>an</strong>d research <strong>in</strong> evaluation. The ECN has provided,<br />
<strong>an</strong>d cont<strong>in</strong>ues to provide a mech<strong>an</strong>ism for foster<strong>in</strong>g dialogue among academics, funders,<br />
government representatives, evaluation consult<strong>an</strong>ts, <strong>an</strong>d nonpr<strong>of</strong>it representatives. It also<br />
provides a way to more easily share <strong>an</strong>d develop capacity build<strong>in</strong>g resources, expertise <strong>an</strong>d<br />
opportunities. Through our engagement with partners, stakeholders, <strong>an</strong>d students, we have<br />
developed a prov<strong>in</strong>cial agenda for <strong>the</strong> ECN that we <strong>in</strong>tend to implement over <strong>the</strong> next several<br />
years, while cont<strong>in</strong>u<strong>in</strong>g to exp<strong>an</strong>d <strong>the</strong> network <strong>an</strong>d establish new partnerships <strong>in</strong> C<strong>an</strong>ada <strong>an</strong>d<br />
<strong>in</strong>ternationally. It is also through engagement that our learn<strong>in</strong>g <strong>an</strong>d those we engaged with<br />
rema<strong>in</strong> <strong>in</strong> motion <strong>an</strong>d challenge us to adapt <strong>in</strong> new ways. In light <strong>of</strong> this, it is import<strong>an</strong>t to<br />
acknowledge that engagement is not <strong>an</strong> <strong>in</strong>herently necessary or equally adv<strong>an</strong>tageous process.<br />
Although <strong>the</strong> various forms <strong>of</strong> engagement through <strong>the</strong> ECN have helped to generate <strong>in</strong>terest<br />
<strong>an</strong>d develop a mutual agenda, it will be essential as we move forward to cont<strong>in</strong>ually assess<br />
whe<strong>the</strong>r <strong>the</strong> ways we are engag<strong>in</strong>g are me<strong>an</strong><strong>in</strong>gful <strong>an</strong>d appropriate. To evaluate our partnership<br />
<strong>an</strong>d our engagement processes systematically, we will use multiple methods <strong>an</strong>d tools (e.g.<br />
<strong>the</strong> PARTNERtool; Varda, Ch<strong>an</strong>dra, Stern, & Lurie, 2008), which will allow us to respond to<br />
un<strong>an</strong>ticipated challenges throughout <strong>the</strong> life <strong>of</strong> <strong>the</strong> partnership.<br />
With <strong>an</strong> <strong>in</strong>creas<strong>in</strong>g number <strong>of</strong> <strong>in</strong>stitutions attempt<strong>in</strong>g to dist<strong>an</strong>ce <strong>the</strong>mselves from <strong>an</strong> ivory<br />
tower status (Furco, 2001), <strong>the</strong> concept <strong>of</strong> engagement appears to have ga<strong>in</strong>ed signific<strong>an</strong>t<br />
momentum across <strong>the</strong> academy. There is a subsequent need for caution when mak<strong>in</strong>g decisions<br />
about <strong>the</strong> forms <strong>an</strong>d extent <strong>of</strong> engagement appropriate <strong>in</strong> each research project to avoid<br />
tokenism (at best) <strong>an</strong>d tyr<strong>an</strong>ny (at worst) (Cooke & Kothari, 2001). We commit ourselves to<br />
this ongo<strong>in</strong>g deliberation <strong>an</strong>d <strong>in</strong>vite o<strong>the</strong>rs to jo<strong>in</strong> us <strong>in</strong> a collective effort to engage carefully.<br />
About <strong>the</strong> Authors<br />
Rebecca Gokiert, PhD, R.Psych, (correspond<strong>in</strong>g author) is <strong>an</strong> Associate Pr<strong>of</strong>essor <strong>an</strong>d Associate<br />
Director, Community-University Partnership (CUP), University <strong>of</strong> Alberta. Her teach<strong>in</strong>g <strong>an</strong>d<br />
research is focused <strong>in</strong> community engagement <strong>an</strong>d participatory research <strong>in</strong> cross-cultural<br />
early childhood contexts. She is a registered psychologist <strong>in</strong> <strong>the</strong> prov<strong>in</strong>ce <strong>of</strong> Alberta <strong>an</strong>d works<br />
<strong>in</strong> partnership with m<strong>an</strong>y school divisions. Email: rgokiert@ualberta.ca<br />
Beth<strong>an</strong> K<strong>in</strong>gsley, PhD, is a postdoctoral fellow with <strong>the</strong> Community-University Partnership<br />
(CUP) at <strong>the</strong> University <strong>of</strong> Alberta. Beth<strong>an</strong>’s current focus is to build <strong>an</strong>d support <strong>the</strong> <strong>Evaluation</strong><br />
<strong>Capacity</strong> <strong>Network</strong>, with a keen <strong>in</strong>terest <strong>in</strong> community-based research <strong>an</strong>d evaluation. Her<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
75<br />
doctoral dissertation explored marg<strong>in</strong>aliz<strong>in</strong>g practices <strong>in</strong> recreation for young people liv<strong>in</strong>g<br />
with lower <strong>in</strong>comes.<br />
Cheryl Poth, PhD, CE, is <strong>an</strong> Associate Pr<strong>of</strong>essor <strong>in</strong> <strong>the</strong> Centre for Research <strong>an</strong>d Applied<br />
Measurement <strong>an</strong>d <strong>Evaluation</strong> (CRAME), University <strong>of</strong> Alberta. Cheryl has expertise<br />
conduct<strong>in</strong>g research <strong>an</strong>d teach<strong>in</strong>g <strong>in</strong> <strong>the</strong> areas <strong>of</strong> program evaluation, mixed methods <strong>an</strong>d<br />
qualitative research, classroom assessment <strong>an</strong>d health sciences education.<br />
Karen Edwards, Med, is <strong>the</strong> Director <strong>of</strong> <strong>the</strong> Community-University Partnership (CUP) at <strong>the</strong><br />
University <strong>of</strong> Alberta. In her role, she fosters ongo<strong>in</strong>g network <strong>an</strong>d partnership development,<br />
facilitates <strong>the</strong> implementation <strong>of</strong> <strong>the</strong> org<strong>an</strong>izational strategic pl<strong>an</strong>, brokers requests from<br />
network partners, <strong>an</strong>d pursues fund<strong>in</strong>g <strong>an</strong>d resource development.<br />
Btissam El Hassar, MPP, is a doctoral c<strong>an</strong>didate <strong>in</strong> Measurement <strong>an</strong>d <strong>Evaluation</strong> through<br />
<strong>the</strong> Centre for Research <strong>an</strong>d Applied Measurement <strong>an</strong>d <strong>Evaluation</strong> (CRAME), University<br />
<strong>of</strong> Alberta. Her dissertation is on <strong>the</strong> development <strong>an</strong>d validation <strong>of</strong> <strong>an</strong> evaluation capacity<br />
<strong>in</strong>strument that is contextually relev<strong>an</strong>t to <strong>the</strong> early childhood development field <strong>in</strong> Alberta.<br />
Lisa T<strong>in</strong>k, MA, is a doctoral student <strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health at <strong>the</strong> University <strong>of</strong><br />
Alberta. Prior to purs<strong>in</strong>g a doctorate degree, Lisa held numerous positions <strong>in</strong> <strong>the</strong> non-pr<strong>of</strong>it<br />
sector, prov<strong>in</strong>cial government, <strong>an</strong>d <strong>the</strong> academy. Lisa’s work focuses on how historical social<br />
<strong>an</strong>d political arr<strong>an</strong>gements create <strong>the</strong> conditions for policy development.<br />
Melissa Tremblay, MSc., is a doctoral c<strong>an</strong>didate <strong>in</strong> <strong>the</strong> School <strong>an</strong>d Cl<strong>in</strong>ical Child Psychology<br />
Program at <strong>the</strong> University <strong>of</strong> Alberta. Melissa has been a graduate research assist<strong>an</strong>t with <strong>the</strong><br />
ECN project m<strong>an</strong>agement team. Through her doctoral research, Melissa is exam<strong>in</strong><strong>in</strong>g <strong>the</strong><br />
impact <strong>of</strong> a supportive hous<strong>in</strong>g program for teen parents.<br />
J<strong>an</strong>e Spr<strong>in</strong>gett, PhD, is a Pr<strong>of</strong>essor <strong>in</strong> <strong>the</strong> School <strong>of</strong> Public Health at <strong>the</strong> University <strong>of</strong><br />
Alberta. She has expertise <strong>in</strong> participatory practice, network <strong>an</strong>d system development, <strong>an</strong>d<br />
realist approaches to syn<strong>the</strong>sis <strong>an</strong>d evaluation. She br<strong>in</strong>gs over 25 years <strong>of</strong> <strong>in</strong>ternational<br />
evaluation experience <strong>an</strong>d graduate supervision to <strong>the</strong> ECN.<br />
References<br />
Akbari, E., & McCuaig, K. (2017). <strong>Early</strong> <strong>Childhood</strong> Education Report. Retrieved from http://ecereport.<br />
ca/media/uploads/2017-report-pdfs/ece-report2017-en-feb6.pdf<br />
Volume 3/Issue 2/Fall 2017
76 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Alk<strong>in</strong>, M. C. (2004). <strong>Evaluation</strong> roots: Trac<strong>in</strong>g <strong>the</strong>orists’ views <strong>an</strong>d <strong>in</strong>fluences. Thous<strong>an</strong>d Oaks, CA: Sage<br />
Publications.<br />
Armitage, T., & Levac, L. (2015). The development <strong>of</strong> community-engaged scholars through coursebased<br />
learn<strong>in</strong>g: A student perspective. Engaged Scholar Journal: Community-Engaged Research,<br />
Teach<strong>in</strong>g <strong>an</strong>d Learn<strong>in</strong>g, 1(1), 148-163.<br />
Bakken, L. L., Núñez, J., & Couture, C. (2014). A course model for build<strong>in</strong>g evaluation capacity<br />
through a university-community partnership. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 35(4), 579-593.<br />
Bis<strong>an</strong>z, J., Edwards, K., & Shaw, K. (2013). <strong>Develop<strong>in</strong>g</strong> support for evaluation at CUP. Edmonton,<br />
AB: Community-University Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families,<br />
University <strong>of</strong> Alberta.<br />
Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Def<strong>in</strong><strong>in</strong>g <strong>an</strong>d teach<strong>in</strong>g evaluative<br />
th<strong>in</strong>k<strong>in</strong>g: Insights from research on critical th<strong>in</strong>k<strong>in</strong>g. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 36(3), 375-<br />
388.<br />
Cabrera, N. J. (2013). Positive development <strong>of</strong> m<strong>in</strong>ority children. Social Policy Report, 27(2), 1-30.<br />
C<strong>an</strong>adi<strong>an</strong> Institute for Health Information (CIHI). (2014). Children vulnerable <strong>in</strong> areas <strong>of</strong> early<br />
development: A determ<strong>in</strong><strong>an</strong>t <strong>of</strong> child health. Retrieved from https://secure.cihi.ca/free_products/<br />
Children_Vulnerable_<strong>in</strong>_Areas_<strong>of</strong>_<strong>Early</strong>_<strong>Development</strong>_EN.pdf<br />
C<strong>an</strong>tor, A., DeLauer, V., Mart<strong>in</strong>, D., & Rog<strong>an</strong>, J. (2015). Interdiscipl<strong>in</strong>ary “wicked problem” solvers:<br />
Apply<strong>in</strong>g lessons from HERO <strong>in</strong> community-based research experiences for undergraduates.<br />
Geography <strong>in</strong> Higher Education, 39(3), 407-419.<br />
Cargo, M. & Mercer, S. L. (2008). The value <strong>an</strong>d challenges <strong>of</strong> participatory research: Streng<strong>the</strong>n<strong>in</strong>g<br />
its practice. Annual Review <strong>of</strong> Public Health, 29, 325-350.<br />
Carm<strong>an</strong>, J. G. (2010). The accountability movement: What’s wrong with this <strong>the</strong>ory <strong>of</strong> ch<strong>an</strong>ge?<br />
Nonpr<strong>of</strong>it <strong>an</strong>d Voluntary Sector Quarterly, 39(2), 256-274.<br />
Carm<strong>an</strong>, J. G., & Millesen, J. L. (2005). Nonpr<strong>of</strong>it program evaluation: Org<strong>an</strong>izational challenges <strong>an</strong>d<br />
resource needs. Journal <strong>of</strong> Volunteer Adm<strong>in</strong>istration, 23(3), 36-43.<br />
Chapm<strong>an</strong>, S. (2015). Community-University Partnerships: A Case Study. In G. Higg<strong>in</strong>bottom & P.<br />
Liamputtong (Eds.), Participatory Qualitative Research Methodologies <strong>in</strong> Health (pp. 200-221). Los<br />
Angeles, US: SAGE.<br />
Cockerill, R., Myers, T., & Allm<strong>an</strong>, D. (2000). Pl<strong>an</strong>n<strong>in</strong>g for Community-based <strong>Evaluation</strong>. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 21(3), 351-357.<br />
Cooke, B., & Kothari, U. (2001). The case for participation as tyr<strong>an</strong>ny. In B. Cooke & U. Kothari<br />
(Eds.), Participation: The new tyr<strong>an</strong>ny? (pp. 1-15). New York, NY: Zed Books.<br />
Council <strong>of</strong> M<strong>in</strong>isters <strong>of</strong> Education, C<strong>an</strong>ada (CMEC). (2014). CMEC early learn<strong>in</strong>g <strong>an</strong>d development<br />
framework. Retrieved from http://www.cmec.ca/Publications/Lists/ Publications/<br />
Attachments/ 327/2014-07-<strong>Early</strong>-Learn<strong>in</strong>g-Framework-EN.pdf<br />
Cous<strong>in</strong>s, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational <strong>Evaluation</strong> <strong>an</strong>d<br />
Policy Analysis, 14(4), 397-418.<br />
Cous<strong>in</strong>s, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Fram<strong>in</strong>g <strong>the</strong> capacity to do <strong>an</strong>d use<br />
evaluation. New Directions for <strong>Evaluation</strong>, 141, 7-23.<br />
Doherty, G., Friendly, M., & Beach, J. (2003). OECD <strong>the</strong>matic review <strong>of</strong> early childhood education <strong>an</strong>d<br />
care: C<strong>an</strong>adi<strong>an</strong> background report. Retrieved from http://www.oecd.org/education/school/<br />
33852192.pdf<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
77<br />
Dunc<strong>an</strong>, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Kleb<strong>an</strong>ov, P., … Japel, C.<br />
(2007). School read<strong>in</strong>ess <strong>an</strong>d later achievement. <strong>Development</strong>al Psychology, 43(6), 1428-1446.<br />
<strong>Early</strong> <strong>Childhood</strong> Mapp<strong>in</strong>g Project (ECMap). (2014). How are our young children do<strong>in</strong>g? F<strong>in</strong>al report <strong>of</strong> <strong>the</strong><br />
early child development mapp<strong>in</strong>g project (ECMap). Edmonton, AB: ECMap, Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth <strong>an</strong>d Families, University <strong>of</strong> Alberta. Retrieved<br />
from http://www.ecmap.ca/images/results/ECMap_F<strong>in</strong>al_Report_20141118.pdf<br />
El Hassar, B., Poth, C., Gokiert, R. K<strong>in</strong>gsley, B., & Krishn<strong>an</strong>, V. (2016, October). Design <strong>an</strong>d validation<br />
<strong>of</strong> <strong>an</strong> evaluation capacity build<strong>in</strong>g <strong>in</strong>strument for <strong>the</strong> early childhood sector. Presentation delivered at <strong>the</strong><br />
Americ<strong>an</strong> <strong>Evaluation</strong> Association Conference. Atl<strong>an</strong>ta, GA, U.S.<br />
<strong>Evaluation</strong> <strong>Capacity</strong> <strong>Network</strong> (ECN). (2015). Adv<strong>an</strong>c<strong>in</strong>g evaluation practices <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood<br />
development. Edmonton, AB: ECN, Community-University Partnership for <strong>the</strong> Study <strong>of</strong><br />
Children, Youth <strong>an</strong>d Families, University <strong>of</strong> Alberta. Retrieved from http://www.cup.<br />
ualberta.ca/wp-content/uploads/2015/12/ECN-Community-Primer-December-17.pdf<br />
Family <strong>an</strong>d Community Support Services, City <strong>of</strong> Edmonton. (2015). 2015 common outcomes report<br />
summary. Retrieved from https://www.edmonton.ca/programs_services/documents/PDF/<br />
CommonOutcomesDataSummary2015.pdf<br />
Freem<strong>an</strong>, R. E. (2010). Strategic m<strong>an</strong>agement: A stakeholder approach. Cambridge, UK: Cambridge<br />
University Press.<br />
Furco, A. (1996). Service learn<strong>in</strong>g: A bal<strong>an</strong>ced approach to experiential education. In B.Taylor, (Ed.),<br />
Exp<strong>an</strong>d<strong>in</strong>g boundaries: Serv<strong>in</strong>g <strong>an</strong>d learn<strong>in</strong>g (pp. 2-6). Wash<strong>in</strong>gton, DC: Corporation for National<br />
Service.<br />
Furco, A. (2001). Adv<strong>an</strong>c<strong>in</strong>g service-learn<strong>in</strong>g at research universities. New Directions for Higher<br />
Education, 114, 67-78.<br />
Gauthier, B., Barr<strong>in</strong>gton, G. V., Bozzo, S. L., Chaytor, K., Dignard, A., Lahey, R., … Roy, S. (2010).<br />
The lay <strong>of</strong> <strong>the</strong> l<strong>an</strong>d: <strong>Evaluation</strong> practice <strong>in</strong> C<strong>an</strong>ada <strong>in</strong> 2009. The C<strong>an</strong>adi<strong>an</strong> Journal <strong>of</strong> Program<br />
<strong>Evaluation</strong>, 24(1), 1-49.<br />
Georgis, R., Gokiert, R.J., & Kirova, A. (2018). Research considerations <strong>in</strong> early childhood<br />
partnerships with immigr<strong>an</strong>t <strong>an</strong>d refugee communities: Researcher reflections. In S. Madrid,<br />
M.J. Mor<strong>an</strong>, R. Brookshire, & M. Buch<strong>an</strong><strong>an</strong> (Eds.), Collaborative research methodologies <strong>in</strong> diverse<br />
early care <strong>an</strong>d education contexts. New York, NY: Routledge<br />
Gokiert, R.J., Tremblay, M., K<strong>in</strong>gsley, B., Mottershead, K., Appleyard, R., & Spr<strong>in</strong>gett, J. (2017). The<br />
life we deserve: A model <strong>of</strong> supportive hous<strong>in</strong>g for teen families. [technical report]. Homeward Trust<br />
Edmonton, Edmonton, Alberta, C<strong>an</strong>ada.<br />
Gokiert, R. J., Georgis, R., Rob<strong>in</strong>son, T., Alex<strong>an</strong>der First Nation, Alexis Nakota Sioux Nation,<br />
O’Chiese First Nation, Sunchild First Nation, & Yellowhead Tribal College. (2014). First<br />
Nation child development: Collective community report. Edmonton, AB: Community-University<br />
Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families, University <strong>of</strong> Alberta.<br />
Government <strong>of</strong> Alberta. (2013). Toge<strong>the</strong>r we raise tomorrow: An Alberta approach to early childhood<br />
development. Retrieved from https://open.alberta.ca/publications/6881618<br />
Government <strong>of</strong> C<strong>an</strong>ada (2011). The Wellbe<strong>in</strong>g <strong>of</strong> C<strong>an</strong>ada’s Young Children. Government <strong>of</strong> C<strong>an</strong>ada.<br />
Government <strong>of</strong> M<strong>an</strong>itoba. (2013). Start<strong>in</strong>g early, start<strong>in</strong>g strong: M<strong>an</strong>itoba’s early childhood development<br />
framework. Retrieved from http://www.gov.mb.ca/healthychild/ecd/ ecd_sessframework.pdf<br />
Volume 3/Issue 2/Fall 2017
78 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
Government <strong>of</strong> Northwest Territories. (2013). A framework for early childhood development <strong>in</strong> <strong>the</strong> Northwest<br />
Territories: Right from <strong>the</strong> start. Retrieved from https://www.ece.gov.nt.ca/sites/ www.ece.gov.<br />
nt.ca/files/resources/ecd_framework_<strong>an</strong>d_action_pl<strong>an</strong>.pdf<br />
Heckm<strong>an</strong>, J. J. (2008). The case for <strong>in</strong>vest<strong>in</strong>g <strong>in</strong> disadv<strong>an</strong>taged young children. In First Focus (Ed.),<br />
Big ideas for children: Invest<strong>in</strong>g <strong>in</strong> our nation’s future (pp. 49-58). Wash<strong>in</strong>gton, DC: First Focus.<br />
Hertzm<strong>an</strong>, C., & Boyce, T. (2010). How experience gets under <strong>the</strong> sk<strong>in</strong> to create gradients <strong>in</strong><br />
developmental health. Annual Review <strong>of</strong> Public Health, 31(1), 329-347.<br />
Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (1998). Review <strong>of</strong> community-based<br />
research: Assess<strong>in</strong>g partnership approaches to improve public health. Annual Review <strong>of</strong> Public<br />
Health, 19(1), 173-202.<br />
Jagosh, J., Bush, P., Salsberg, J., Macaulay, A.C., Greenhalgh, T., Wong, G., Cargo, M., Green, L.W.,<br />
Herbert, C.P., & Pluye, P. (2015). A realist evaluation <strong>of</strong> community-based participatory<br />
research: partnership synergy, trust build<strong>in</strong>g <strong>an</strong>d related ripple effects. BMC Public Health, 15,<br />
725.<br />
J<strong>an</strong>zen, R., Ochocka, J., Turner, L., Cook, T., Fr<strong>an</strong>kl<strong>in</strong>, M., & Deichert, D. (2017). <strong>Build<strong>in</strong>g</strong> a<br />
community-based culture <strong>of</strong> evaluation. <strong>Evaluation</strong> <strong>an</strong>d Program Pl<strong>an</strong>n<strong>in</strong>g, 65, 163-170.<br />
Kajner, T., Fletcher, F., & Makokis, P. (2011). Bal<strong>an</strong>c<strong>in</strong>g head <strong>an</strong>d heart: The import<strong>an</strong>ce <strong>of</strong> relational<br />
accountability <strong>in</strong> community-university partnerships. Innovative Higher Education, 37, 257-270.<br />
K<strong>in</strong>gsley, B. C., & Chapm<strong>an</strong>, S-A. (2013). Question<strong>in</strong>g <strong>the</strong> me<strong>an</strong><strong>in</strong>gfulness <strong>of</strong> rigour <strong>in</strong> communitybased<br />
research: Navigat<strong>in</strong>g a dilemma. International Journal <strong>of</strong> Qualitative Methods, 12, 551-569.<br />
Lab<strong>in</strong>, S. N. (2014). <strong>Develop<strong>in</strong>g</strong> Common Measures <strong>in</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong> An Iterative<br />
Science <strong>an</strong>d Practice Process. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 35(1), 107-115.<br />
Leviton, L., (2014). Some underexam<strong>in</strong>ed aspects <strong>of</strong> evaluation capacity build<strong>in</strong>g. Americ<strong>an</strong> Journal <strong>of</strong><br />
<strong>Evaluation</strong>, 35(1), 90-94.<br />
Liket, K. C., Rey-Garcia, M., & Maas, K. E. H. (2014). Why aren’t evaluations work<strong>in</strong>g <strong>an</strong>d what to<br />
do about it: A framework for negotiat<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluations <strong>in</strong> nonpr<strong>of</strong>its. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 35(2), 171-188.<br />
McCaffrey, L. (2007). The History <strong>of</strong> <strong>the</strong> Community-University Partnership for <strong>the</strong> Study <strong>of</strong><br />
Children, Youth, <strong>an</strong>d Families (CUP). Edmonton, AB: McCaffery Consult<strong>in</strong>g. Retrieved<br />
from https://d1pbog36rugm0t.cloudfront.net/-/media/ualberta/faculties-<strong>an</strong>d-programs/<br />
centres-<strong>in</strong>stitutes/community-university-partnership/about-us/cup-history.pdf<br />
McDavid, J., & Dev<strong>in</strong>e, H. (2009). Consortium <strong>of</strong> Universities for <strong>Evaluation</strong> Education (CUEE) project:<br />
Research on evaluation education at <strong>the</strong> graduate level <strong>in</strong> C<strong>an</strong>adi<strong>an</strong> universities. F<strong>in</strong>al report.<br />
McSh<strong>an</strong>e, K., Usher, A. M., T<strong>an</strong>don, R., & Steel, J. (2015). Chart<strong>in</strong>g <strong>the</strong> trajectory <strong>of</strong> a flexible<br />
community-university collaboration <strong>in</strong> <strong>an</strong> applied learn<strong>in</strong>g ecosystem. Engaged Scholar Journal:<br />
Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g, 1(2), 149-165.<br />
M<strong>in</strong>kler, M., & Wallerste<strong>in</strong>, N. (Eds.). (2008). Community-based participatory research for health: From process<br />
to outcomes (2nd ed.). S<strong>an</strong> Fr<strong>an</strong>cisco, CA: Jossey-Bass.<br />
Munro, C. (2006). BC evaluation project report: C<strong>an</strong> we agree on common child outcomes for BC? Retrieved<br />
from http://www.successby6bc.ca/sites/default/files/BC%20<strong>Evaluation</strong>%20Project%20<br />
Report%202006.pdf<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g
79<br />
Muttart Foundation. (2013). What We Heard: A Report on Fall 2012 Consultations with <strong>Early</strong><br />
Learn<strong>in</strong>g <strong>an</strong>d Care Stakeholders. Retrieved from http://www.muttart.org/sites/default/<br />
files/report/Toward%20A%20Prov<strong>in</strong>cial%20ELC%20Framework%20-%20What%20<br />
We%20Heard%20Report%20-%20f<strong>in</strong>al.pdf<br />
Org<strong>an</strong>isation for Economic Cooperation <strong>an</strong>d <strong>Development</strong>. (2012). Education at a gl<strong>an</strong>ce 2012: OECD<br />
<strong>in</strong>dicators. Retrieved from https://www.oecd.org/edu/EAG%202012_e-book_EN_200912.<br />
pdf<br />
Pei, N., Felth<strong>an</strong>, J., Ford, I., Schwartz, K. (2015). Best practices for implement<strong>in</strong>g a liv<strong>in</strong>g wage policy<br />
<strong>in</strong> C<strong>an</strong>ada: Us<strong>in</strong>g Community-Campus Partnerships to Fur<strong>the</strong>r <strong>the</strong> Community’s Goal.<br />
Engaged Scholar Journal, 1(2), 149-153.<br />
Preskill, H., & Boyle, S. (2008). A Multidiscipl<strong>in</strong>ary Model <strong>of</strong> <strong>Evaluation</strong> <strong>Capacity</strong> <strong>Build<strong>in</strong>g</strong>. Americ<strong>an</strong><br />
Journal <strong>of</strong> <strong>Evaluation</strong>, 29(4), 443-459.<br />
Reed, E., & Morariu, J. (2010). State <strong>of</strong> evaluation 2010: <strong>Evaluation</strong> practice <strong>an</strong>d capacity <strong>in</strong> <strong>the</strong> nonpr<strong>of</strong>it sector.<br />
Retrieved from Innovation <strong>Network</strong> Website: http://www.po<strong>in</strong>tk.org/client_docs/<strong>in</strong>nonetstate-<strong>of</strong>-evaluation-2010.pdf<br />
S<strong>an</strong>derson, I. (2000). <strong>Evaluation</strong> <strong>in</strong> complex policy systems. <strong>Evaluation</strong>, 6(4), 433-454.<br />
Shonk<strong>of</strong>f, J. P., & Levitt, P. (2010). Neuroscience <strong>an</strong>d <strong>the</strong> future <strong>of</strong> early childhood policy: Mov<strong>in</strong>g<br />
from why to what <strong>an</strong>d how. Neuron, 67(5), 689-691.<br />
Shonk<strong>of</strong>f, J. P. (2017). Reth<strong>in</strong>k<strong>in</strong>g <strong>the</strong> def<strong>in</strong>ition <strong>of</strong> evidence-based <strong>in</strong>terventions to promote early<br />
childhood development. Pediatrics, 140(6), 1-2.<br />
Snibbe, A. C. (2006). Drown<strong>in</strong>g <strong>in</strong> data. St<strong>an</strong>ford Social Innovation Review, 4(3), 39-45.<br />
St<strong>an</strong>ford University (2017). A virtual crash course <strong>in</strong> design th<strong>in</strong>k<strong>in</strong>g. Retrieved from: https://<br />
dschool.st<strong>an</strong>ford.edu/resources-collections/a-virtual-crash-course-<strong>in</strong>-design-th<strong>in</strong>k<strong>in</strong>g<br />
Statistics C<strong>an</strong>ada, Government <strong>of</strong> C<strong>an</strong>ada. (2011). 2011 National Household Survey: Data Tables<br />
[C<strong>an</strong>ada]. Retrieved from http://www12.statc<strong>an</strong>.gc.ca/nhs-enm/2011/dp-pd/dt-td/Rp-eng.<br />
cfm?LANG=E&APATH=3&DETAIL=0&DIM=0&FL=A&FREE=0&GC=0&GID=0<br />
&GK=0&GRP=0&PID=106715&PRID=0&PTYPE=105277&S=0&SHOWALL=0&SUB<br />
=0&Temporal=2013&THEME=98&VID=0&VNAMEE=&VNAMEF=<br />
Suárez-Herrera, J. C., Spr<strong>in</strong>gett, J., & Kag<strong>an</strong>, C. (2009). Critical connections between<br />
participatory evaluation, org<strong>an</strong>izational learn<strong>in</strong>g <strong>an</strong>d <strong>in</strong>tentional ch<strong>an</strong>ge <strong>in</strong> pluralistic<br />
org<strong>an</strong>izations. <strong>Evaluation</strong>, 15(3), 321-342.<br />
Taylor, A., Butterwick, S., Raykov, M., Glick, S., Peikazadi, N., & Mehrabi, S. (2015). Community<br />
Service-Learn<strong>in</strong>g <strong>in</strong> C<strong>an</strong>adi<strong>an</strong> Higher Education. University <strong>of</strong> British Columbia. Retrieved from<br />
https://d1pbog36rugm0t.cloudfront.net/-/media/arts/departments-<strong>in</strong>stitutes-<strong>an</strong>d-centres/<br />
community-service-learn<strong>in</strong>g/documents/reports/ks-report-31-oct-2015-f<strong>in</strong>al.pdf<br />
Taylor-Ritzler, T., Suarez-Balcazar, Y., Garcia-Iriarte, E., Henry, D. B., & Balcazar, F. E. (2013).<br />
Underst<strong>an</strong>d<strong>in</strong>g <strong>an</strong>d Measur<strong>in</strong>g <strong>Evaluation</strong> <strong>Capacity</strong>: A Model <strong>an</strong>d Instrument Validation<br />
Study. Americ<strong>an</strong> Journal <strong>of</strong> <strong>Evaluation</strong>, 34(2), 190-206.<br />
T<strong>in</strong>k, L.N., Gokiert, R.J., K<strong>in</strong>gsley, B.C., & El Hassar, B. (2017). <strong>Evaluation</strong> capacity build<strong>in</strong>g survey<br />
results: Adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation <strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood. Edmonton, AB, Community-<br />
University Partnership for <strong>the</strong> Study <strong>of</strong> Children, Youth, <strong>an</strong>d Families, University <strong>of</strong> Alberta.<br />
Retrieved from www.evaluationcapacitynetwork.com.<br />
Volume 3/Issue 2/Fall 2017
80 Rebecca Gokiert, Beth<strong>an</strong> K<strong>in</strong>gsley, Cheryl Poth, Karen Edwards, Btissam El Hassar, Lisa T<strong>in</strong>k, Melissa Tremblay, Ken Cor, J<strong>an</strong>e Spr<strong>in</strong>gett, Sus<strong>an</strong> Hopk<strong>in</strong>s<br />
T<strong>in</strong>k, L. N., K<strong>in</strong>gsley, B. C., & Gokiert, R. J. (2016). What we heard: Adv<strong>an</strong>c<strong>in</strong>g me<strong>an</strong><strong>in</strong>gful evaluation<br />
<strong>in</strong> <strong>the</strong> field <strong>of</strong> early childhood. Retrieved from http://www.evaluationcapacitynetwork.com/<br />
wp-content/uploads/2017/03/ECN-Dialogues_What-we-heard_Sept-2016.pdf<br />
Tremblay, M., Gokiert, R. J., K<strong>in</strong>gsley, B. C., Mottershead, K., Ca<strong>in</strong>e, K., Appleyard, R., & Appleyard,<br />
R. (2016, March). Creat<strong>in</strong>g a research agenda to measure <strong>the</strong> impact <strong>of</strong> a supportive wraparound hous<strong>in</strong>g<br />
program for teen families. B<strong>an</strong>ff International Conference for Behavioural Sciences, B<strong>an</strong>ff, AB.<br />
UNICEF, Office <strong>of</strong> Research. (2016). Fairness for children (Innocenti Report Card 13). Retrieved from<br />
https://www.unicef.ca/en/unicef-report-card-13-fairness-for-children<br />
Varda, D. M., Ch<strong>an</strong>dra, A., Stern, S., & Lurie, L. (2008). Core dimensions <strong>of</strong> connectivity <strong>in</strong> public<br />
health collaboratives. Journal <strong>of</strong> Public Health M<strong>an</strong>agement Practice, 14, 1-7.<br />
Visw<strong>an</strong>ath<strong>an</strong>, M., Ammerm<strong>an</strong>, A., Eng, E., Garlehner, G., Lohr, K.N., Griffith, D., Rhodes, S.,<br />
Samuel-Hodge, C., Maty, S., Lux, L., Webb, L., Sutton, S.F., Sw<strong>in</strong>son, T., Jackm<strong>an</strong>, A., &<br />
Whitener, L. (2004). Community-Based Participatory Research: Assess<strong>in</strong>g <strong>the</strong> Evidence.<br />
Evidence Report/Technology Assessment No. 99 (Prepared by RTI-University <strong>of</strong> North<br />
Carol<strong>in</strong>a Evidence-based Practice Center under Contract No. 290-02-0016). AHRQ<br />
Publication 04-E022- 2. Rockville, MD: Agency for Healthcare Research <strong>an</strong>d Quality. July<br />
2004.<br />
Waddell, C., Shepherd, C. A., Shwartz, C., & Baric<strong>an</strong>, J. (2014). Child <strong>an</strong>d youth mental disorders: Prevalence<br />
<strong>an</strong>d evidence-based <strong>in</strong>terventions. V<strong>an</strong>couver, BC: Children’s Health Policy Centre, Simon Fraser<br />
University. Retrieved from http://childhealthpolicy.ca/wp-content/uploads/2015/11/2015-<br />
10-05-Waddell-et-al-Report-2014.06.16-w-errata.pdf<br />
Waldrop, M. M. (1992). Complexity: The emerg<strong>in</strong>g science at <strong>the</strong> edge <strong>of</strong> order <strong>an</strong>d chaos. New York, NY:<br />
Simon & Schuster.<br />
Wenger, L., Hawk<strong>in</strong>s, L., & Seifer, S. D. (Eds.). (2011). Community-engaged scholarships: Critical<br />
junctures <strong>in</strong> research, practice, <strong>an</strong>d policy: Conference report. Community-Campus<br />
Partnerships for Health & <strong>the</strong> University <strong>of</strong> Guelph. Retrieved from http://criticaljunctures.<br />
ca<br />
Wenger, L., Hawk<strong>in</strong>s, L., & Seifer, S. D. (2012). Community-engaged scholarships: Critical junctures<br />
<strong>in</strong> research, practice, <strong>an</strong>d policy. Journal <strong>of</strong> Higher Education Outreach <strong>an</strong>d Engagement, 16, 171-<br />
181.<br />
Engaged Scholar Journal: Community-Engaged Research, Teach<strong>in</strong>g, <strong>an</strong>d Learn<strong>in</strong>g