28.01.2020 Views

Advancing Evaluation Practices: Stimulus Paper

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong><br />

in the Field of Early Childhood Development


1 Introduction:<br />

SETTING THE STAGE<br />

3<br />

<strong>Evaluation</strong><br />

Capacity Building:<br />

WHY NOW?<br />

6<br />

<strong>Evaluation</strong><br />

Capacity Building:<br />

WHERE ARE WE CURRENTLY?<br />

9<br />

<strong>Evaluation</strong><br />

Capacity Building:<br />

WHAT DO WE KNOW?<br />

16<br />

Looking Forward:<br />

WHAT ARE THE POSSIBILITIES?<br />

December 2015<br />

<strong>Evaluation</strong> Capacity Network<br />

University of Alberta


Introduction: SETTING THE STAGE<br />

As you read, we<br />

invite you to draw<br />

on your own<br />

experiences with<br />

evaluation.<br />

An overview of this<br />

stimulus paper.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> in the Field of Early Childhood<br />

Development is intended to generate ideas and discussion about<br />

the role of evaluation in the field of early childhood development<br />

(ECD). We invite you to engage with the content and the questions<br />

we have posed in a way that is personal to your own experiences<br />

with evaluation. This stimulus paper provides a foundation for<br />

celebrating previous evaluative efforts, challenging current<br />

practices, and forging new pathways for evaluation in ECD.<br />

We begin the paper by outlining the need for, and potential of,<br />

evaluation in ECD. Next, we present the need to advance current<br />

evaluation practices by building capacity to engage in evaluation.<br />

We provide an overview of evaluation strategies in ECD in Alberta<br />

and beyond, and turn to the academic literature for a conceptual<br />

understanding about evaluation capacity building more broadly.<br />

Finally, we conclude with a list of possibilities for evaluation<br />

capacity building to encourage discussion and the flow of<br />

creative ideas as we come together to collectively advance our<br />

evaluation practices.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 1


<strong>Evaluation</strong> in the field<br />

of ECD offers many<br />

opportunities and<br />

challenges.<br />

The <strong>Evaluation</strong><br />

Capacity Network<br />

aims to build<br />

evaluation capacity<br />

& promote learning<br />

in the ECD field.<br />

Early childhood is a time of rapid development, learning,<br />

and opportunity for children, as well as for their parents,<br />

administrators, front-line staff, policymakers, and funders. We<br />

all want to be confident that the time and energy we devote to<br />

raising young children is enhancing their development. We want<br />

to know whether programs are making a meaningful difference<br />

in their lives. Using evaluation, we can begin to respond to<br />

these pressing questions. However, evaluation comes with<br />

many opportunities and challenges that are experienced by<br />

organizations in different ways.<br />

The <strong>Evaluation</strong> Capacity Network (the Network) formed out of an<br />

awareness of these opportunities and challenges, with the goal<br />

of advancing evaluation practices in the ECD field. Supporting<br />

children in the early years requires expertise and input across<br />

multiple sectors and disciplines, many of which have differing<br />

perspectives on ECD and evaluation. The Network connects<br />

this diverse community to enhance evaluation capacity and to<br />

promote learning through the shared practices, perspectives,<br />

and approaches of diverse ECD stakeholders. We need to<br />

engage in open dialogue to align resources, come to a common<br />

understanding of the impacts we are having on ECD, and<br />

collectively reflect on how to make informed decisions to best<br />

support young children. As the first step, this paper provides an<br />

information base to begin this dialogue.<br />

2


<strong>Evaluation</strong> Capacity Building: WHY NOW?<br />

Healthy experiences in<br />

the early years create<br />

a foundation for a<br />

lifetime of positive<br />

outcomes.<br />

Strong, well–rounded<br />

ECD programming<br />

supports long–term<br />

healthy outcomes, and<br />

can reduce expensive<br />

interventions in later<br />

years.<br />

It is an opportune time for enhancing our evaluation capacity<br />

in ECD. Decades of research show that the early years are a<br />

time of both promise and risk, setting the foundation for healthy<br />

functioning throughout the lifespan. [1] The environments that<br />

children encounter from the prenatal period onward have a<br />

cumulative effect on all aspects of their development. [2] In order<br />

to learn effectively and grow into successful adults, children<br />

need supportive and responsive nurturing in stimulating<br />

environments. Negative experiences during early childhood can<br />

put children at risk for many challenges, which in turn have<br />

significant social and economic costs. [2]<br />

Early interventions can help lessen the impacts of negative<br />

experiences. Providing children with early supports prevents<br />

negative outcomes and promotes long–term positive<br />

development in health and cognitive, social, behavioural,<br />

and academic domains. [3, 4] Organizations and funders have<br />

recognized the cost–effectiveness of earlier interventions over<br />

those implemented at later ages [5] , and are increasingly directing<br />

their efforts toward ECD.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 3


There are multiple<br />

reasons for<br />

strengthening<br />

evaluation:<br />

strategic learning,<br />

accountability to<br />

funders, and building<br />

the field of ECD.<br />

There is a need for<br />

evaluation alignment<br />

and capacity building<br />

at the individual,<br />

organizational, and<br />

system levels.<br />

As investment in the field of ECD grows, so too does the need<br />

for organizations to be able to participate in multiple types of<br />

evaluation. <strong>Evaluation</strong> can be used for:<br />

• Strategic learning, to inform practices and enhance<br />

programming at the organizational level to better meet the<br />

needs of children and families.<br />

• Accountability to funders, to show how resources invested in<br />

ECD programming are used, demonstrate quality of service, and<br />

track the satisfaction and outcomes for children and families.<br />

• Field–building, to provide an evidence base for the field of<br />

early childhood development, and contribute to our broader<br />

knowledge of needs, appropriate programs, and services.<br />

It is critical that ECD organizations are able to participate effectively<br />

in each of these forms of evaluation.<br />

A review of the current state of program evaluation practice<br />

in Canada reported that there are not enough evaluators and<br />

evaluation resources to meet the increasing demand for program<br />

evaluations. [6] Organizations often lack the capacity, knowledge, and<br />

skills to collect the evidence they need to demonstrate their impacts<br />

and justify continued support. [7] In addition, different language<br />

and methods are used to describe and measure child outcomes<br />

across organizations, sectors, disciplines, and systems, highlighting<br />

the need for evaluation alignment and capacity building at the<br />

individual, organizational, and system levels.<br />

4


At the beginning of this section, we posed the question, Why<br />

Now? The current focus on evaluation capacity building is<br />

important because we now know that the early years are critical<br />

in setting the stage for long–term health and development, and<br />

we know that ECD programs are increasingly required to provide<br />

evidence that their programs are working. New and innovative<br />

evaluation approaches are therefore necessary to capitalize<br />

and build on the existing capacity in the ECD field, and to fully<br />

support the needs of children and families.<br />

Discussion Questions:<br />

What is your experience with evaluation?<br />

How is evaluation generally viewed in your<br />

organization?<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 5


<strong>Evaluation</strong> Capacity Building: WHERE ARE WE CURRENTLY?<br />

Consistent ways of<br />

measuring, reporting,<br />

and comparing<br />

the effectiveness of<br />

ECD programs are<br />

essential.<br />

CMEC recently<br />

developed a common<br />

vision for early<br />

learning in the<br />

Early Learning<br />

and Development<br />

Framework.<br />

In 2003, a report on early childhood education and care in<br />

Canada showed that ECD programs collect considerable<br />

amounts of data. [8] However, different provinces and territories<br />

often collect different information at different time intervals,<br />

using different methods. It is not often possible to compare<br />

data nationally, provincially, or even regionally. We therefore<br />

lack a solid understanding of the need for early childhood<br />

programs, the effectiveness of these programs, and the<br />

children and families they serve. Consistent ways of measuring,<br />

reporting, and comparing the effectiveness of ECD programs<br />

are essential. [9]<br />

At a national level, the Council of Ministers of Education, Canada<br />

(CMEC) has developed an Early Learning and Development<br />

Framework. This national framework presents a vision for early<br />

learning that could be applied Canada–wide to foster continuity<br />

across jurisdictions and ECD settings. The framework offers six<br />

principles to promote a common understanding of a continuum<br />

of learning and development, as well as shared values regarding<br />

what is most important in the early years. [10] (www.cmec.ca)<br />

6


Provincial<br />

frameworks can<br />

support measurement<br />

of common outcomes<br />

at regional, provincial,<br />

and national levels.<br />

There are also a number of frameworks in use at the provincial<br />

level. For example, British Columbia’s Early Childhood<br />

Development <strong>Evaluation</strong> Project brought together community and<br />

government stakeholders to address the lack of coordinated ECD<br />

evaluation. The project identified four long–term outcomes, as<br />

well as shared measurement tools and technology, which have<br />

been adopted across the province. [11] (www.successby6bc.ca)<br />

In 2013, the Government of the Northwest Territories<br />

departments of Health and Social Services (HSS) and Education,<br />

Culture and Employment (ECE) released Right from the Start, an<br />

updated framework for early childhood development informed by<br />

an intensive public engagement process. The framework includes<br />

seven commitments, each supported by specific areas for action,<br />

deliverables, and target objectives. [12] (www.rightfromthestart.ca)<br />

Healthy Child Manitoba has also released an ECD framework<br />

that includes guiding principles, ECD building blocks, and<br />

recommendations for measuring success and reporting on<br />

progress. [13) (www.gov.mb.ca)<br />

Common across each of these provincial frameworks is a<br />

commitment to empower families, strengthen communities,<br />

improve access to programs and services, and foster safe,<br />

secure, and supportive environments during the early years.<br />

These frameworks can support the measurement of common<br />

outcomes at regional, provincial, and national levels to inform<br />

decisions around policy, funding, and programming.<br />

The Play, Participation,<br />

and Possibilities<br />

framework for early<br />

learning draws on<br />

stakeholder consultations<br />

to guide decision–making<br />

and align policy in the<br />

province, alongside the<br />

Alberta Social Policy<br />

Framework and Together<br />

We Raise Tomorrow<br />

initiative.<br />

In Alberta, there have been several ongoing efforts to guide<br />

decision-making and align policies and practices. For example,<br />

Play, Participation, and Possibilities is a provincial early learning<br />

and child care curriculum framework that was produced<br />

through a partnership between the Government of Alberta,<br />

faculty at MacEwan Early Learning and Child Care, and Mount<br />

Royal Department of Child and Youth Studies program, via<br />

consultations with stakeholders across Alberta. [14]<br />

(www.childcareframework.com)<br />

The Government of Alberta also engaged in consultations to<br />

create a vision for social policy in the province. The Social Policy<br />

Framework outlines eight priority initiatives, one of which is early<br />

childhood development. [15] These conversations also led to the<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 7


Together We Raise Tomorrow initaitive to support the well–being,<br />

safety, security, education, and health of children in Alberta, and<br />

included a preliminary outcomes measurement framework. [16)<br />

(www.humanservices.alberta.ca/department/previousconsultations.html)<br />

“Are we making a difference with our ECD programs?”<br />

“Yes, I think we are.”<br />

“But how do we know that we are making a difference?”<br />

Overall, everyone working in different sectors across the ECD<br />

field aims to make a difference. The question remains: how<br />

do we know we are doing the best we can for children and<br />

families in our province? By setting out shared language and<br />

common outcomes, these frameworks are a useful starting<br />

point. However, organizations still need the capacity to conduct<br />

quality evaluation to contribute to these frameworks. Community<br />

agencies, funders, and policymakers need to identify collectively<br />

the capacity that they each require to contribute to healthy ECD.<br />

This is where evaluation capacity building becomes important.<br />

In response to the question, Where Are We Currently?, there are<br />

frameworks available to help us understand how we are making<br />

a difference for ECD on national, provincial, and territorial levels.<br />

These frameworks identify outcomes that will be essential to<br />

consider when examining how we can build evaluation capacity in<br />

ECD.<br />

Discussion Questions:<br />

What ECD evaluation framework—if any—<br />

have you found most useful and why?<br />

How can these frameworks inform<br />

evaluation in the field of ECD?<br />

8


<strong>Evaluation</strong> Capacity Building: WHAT DO WE KNOW?<br />

There are multiple definitions of evaluation capacity building. For<br />

the purpose of this discussion, we have chosen to use Labin’s [17) ,<br />

which describes evaluation capacity building as:<br />

“an intentional process to increase individual motivation,<br />

knowledge, and skills, and to enhance a group or<br />

organization’s ability to conduct or use evaluation.”<br />

In this section, we present some research findings related to<br />

evaluation capacity building, and what they might mean for an<br />

organization. However, academic literature does not offer a<br />

complete picture. We also need to know more from you as an<br />

individual or organization working directly in the ECD field.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 9


“I got an email from our funder about annual reporting.”<br />

“What do they want us to report on this year?”<br />

“Seems they have a new framework for us to use.”<br />

“What kind of information are they looking for?”<br />

“I’ll have to take a closer look. We may need to collect some<br />

new information.”<br />

“Who’s going to take that on? My plate is already full.”<br />

“We may need to bring someone in to help us, depending on<br />

how complex it is.”<br />

“Well, once it’s done, we can get back to business.”<br />

Research tells us:<br />

Using hands–on<br />

strategies promotes<br />

the positive benefits<br />

of evaluation and<br />

improves buy–in.<br />

Mutually–Beneficial <strong>Evaluation</strong><br />

The above conversation demonstrates how demands for<br />

evaluation can, at times, lack relevance in our work, and place<br />

additional stress on staff. Research shows that organizations<br />

need to directly experience the benefits of evaluation before<br />

staff will fully buy into evaluation as a tool for program<br />

improvement. [18) This suggests that efforts to build evaluation<br />

capacity should be highly participatory, with opportunities for<br />

hands–on, practical learning, and reflection throughout.<br />

Discussion Questions:<br />

Can you think of a positive experience your<br />

organization has had with evaluation?<br />

What made it positive?<br />

10


Research tells us:<br />

Fostering a strong<br />

evaluative learning<br />

culture leads to<br />

effective evaluation<br />

capacity building.<br />

A Strong <strong>Evaluation</strong> Learning Culture<br />

Having an evaluative learning culture is a fundamental<br />

component of effective evaluation capacity building in an<br />

organization. [18) In organizations with a strong learning<br />

culture, staff freely share information, trust one another and<br />

feel comfortable asking questions and taking risks. Leaders<br />

encourage collaboration, seek information from others when<br />

making decisions, are open to feedback, and reward employees<br />

for learning, taking part in professional development, and<br />

engaging in evaluation. [19) In this environment, evaluation can<br />

be a system for organizational learning [18) that staff use to make<br />

informed decisions that can enhance programming.<br />

Discussion Questions:<br />

What characteristics does your organization<br />

have that support an evaluation learning<br />

culture?<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 11


Research tells us:<br />

Participants’<br />

motivations,<br />

assumptions, and<br />

expectations should<br />

inform evaluation<br />

capacity building<br />

activities.<br />

Motivations, Assumptions, and<br />

Expectations<br />

In line with a hands–on participatory approach, participants’<br />

motivations, assumptions, and expectations should inform the<br />

design of evaluation capacity building activities. [19) Understanding<br />

motivation helps determine if timing is appropriate, and provides<br />

insight into which staff members should participate and which<br />

teaching and learning strategies are likely to succeed. It is also<br />

important to discuss participants’ assumptions and expectations<br />

for evaluation in their organization. For example, some staff<br />

members may not believe that evaluation leads to effective<br />

decision–making. Sharing these assumptions ensures that all<br />

participants are starting evaluation capacity building activities on<br />

the same page.<br />

Discussion Question:<br />

What might be the motivation for you or<br />

your organization to engage in evaluation?<br />

12


Research tells us:<br />

Understanding<br />

current capacity and<br />

future goals for your<br />

organization shapes an<br />

effective approach.<br />

Current Capacity, Future Goals, and<br />

Specific Needs<br />

Assessing an organization’s existing capacity levels before<br />

implementing capacity building initiatives will help shape an<br />

effective approach. Goals for capacity building should also be<br />

identified at the beginning of initiatives. Some organizations may<br />

start with the objective to address staff beliefs about evaluation,<br />

or to foster a stronger organizational learning culture. Other<br />

organizations may prioritize building evaluation skills and<br />

knowledge. Regardless of the specific objectives, evaluation<br />

capacity building initiatives are more likely to succeed if these<br />

objectives are clearly defined and understood.<br />

Discussion Question:<br />

What might be some objectives for building<br />

evaluation capacity in your organization?<br />

Research tells us:<br />

Understanding the<br />

specific need(s) for<br />

evaluation in your<br />

organization will<br />

streamline evaluation<br />

efforts.<br />

When implementing evaluation capacity building initiatives, it<br />

is important to clarify between the need for building capacity to<br />

do evaluation, the need for building capacity to use evaluation<br />

results, or the need for both. [18)<br />

Capacity to do evaluation is built primarily from formal and<br />

informal training to build knowledge, skills, and abilities<br />

and apply them to the workplace setting. This type of<br />

capacity building should happen on both an individual and<br />

an organizational level. On an individual level, this includes<br />

developing the abilities to plan evaluations, develop and<br />

use evaluation instruments, collect data, and analyze and<br />

interpret data, as well as a number of “soft skills” that are<br />

developed through practical experience, such as conflict<br />

resolution, collaboration, and cooperative communication. At an<br />

organizational level, evaluation capacity can be measured by the<br />

frequency and quality of evaluation efforts.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 13


We can also consider individual and organizational capacity<br />

to use evaluation results in planned and intentional ways. For<br />

example, organizations can use evaluation results to make<br />

decisions on program continuation, revision, and improvement;<br />

to reaffirm program impacts with staff and stakeholders; and to<br />

more closely align with an organizational or funders’ mandate.<br />

Factors that determine whether evaluation results are used<br />

can include the extent to which evaluation results are timely,<br />

constructive, relevant, credible, and understandable to primary<br />

evaluation users. Building capacity for evaluation use involves<br />

developing an awareness of these conditions and how they shape<br />

how evaluation results are used. [18)<br />

Discussion Questions:<br />

How is the information gained<br />

through evaluation used by your<br />

organization?<br />

14


Research tells us:<br />

Dedicating time and<br />

effort after a learning<br />

opportunity supports<br />

sustainable evaluation<br />

capacity building.<br />

Sustaining Your Efforts<br />

<strong>Evaluation</strong> capacity needs to be sustained in order to make<br />

a lasting difference. [19) Capacity building participants need<br />

support to transfer and use their learning once they have<br />

completed training. [17) Organizations can also develop evaluation–<br />

specific policies and procedures, such as including evaluation<br />

responsibilities in job descriptions and resource allocation<br />

documents. Other strategies to support long–term sustainability<br />

of evaluation capacity building efforts can include making an<br />

explicit commitment to integrate evaluation into decision–<br />

making, goal–setting, and resource allocation, as well as<br />

communicating and celebrating how evaluation findings are used<br />

in reports and communications.<br />

Discussion Questions:<br />

How realistic are these suggested practices<br />

for building evaluation capacity in your<br />

organization?<br />

In response to the question, What Do We Know?, we understand<br />

that there are a number of elements important to effective<br />

evaluation capacity building. According to research, we need to<br />

• employ hands–on strategies,<br />

• foster a strong learning culture,<br />

• consider motivations, assumptions, and expectations,<br />

• understand current capacity, future goals, and specific needs,<br />

and<br />

• invest time and effort post–learning.<br />

It will be useful to consider these elements when we begin to<br />

explore the possibilities for building evaluation capacity in ECD.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 15


Looking Forward: WHAT ARE THE POSSIBILITIES?<br />

We need to explore<br />

ways to advance<br />

how we collect and<br />

examine information.<br />

If we work together, we can generate solutions to the evaluation<br />

challenges we currently face in the ECD field. There are many<br />

possibilities for evaluation capacity building, and we want to know<br />

what would help you in your work.<br />

There are already evaluation capacity building efforts in place<br />

across Canada. For example, the University of Manitoba<br />

offers a Summer Institute in Program <strong>Evaluation</strong> (www.<br />

thesummerinstitute.ca). The Institute brings together<br />

practitioners and university students to exchange information and<br />

improve evaluation skills using lectures, case studies, and group<br />

work. [20) Another example is the First Steps First: A Community–<br />

Based Workbook for Evaluating Substance Abuse and Mental<br />

Health Programs in Saskatchewan, created by the Saskatchewan<br />

Team for Research and <strong>Evaluation</strong> of Addictions Treatment<br />

and Mental Health Services (www.addictionresearchchair.ca/<br />

creating-knowledge/provincial/s-t-r-e-a-m/first-steps-first).<br />

This workbook provides information to program staff on the steps<br />

needed to prepare for an effective evaluation. [21) The Canadian<br />

<strong>Evaluation</strong> Society (CES) also offers a number of regional and<br />

national professional development tools. The CES Essential Skills<br />

Series provides an introduction to program evaluation skills and<br />

16


is offered on demand. CES webinars cover areas of interest within<br />

each region, and the CES Annual Conference provides a forum to<br />

discuss major theoretical, philosophical, and practical evaluation<br />

issues. [22) (www.evaluationcanada.ca/professional-learning)<br />

There are many<br />

ways we can build<br />

evaluation capacity<br />

together.<br />

When we consider the question, What Are the Possibilities?, it<br />

is evident that there are many potential options for evaluation<br />

capacity building strategies. Examples from across Canada<br />

can stimulate ideas for implementing strategies through the<br />

<strong>Evaluation</strong> Capacity Network. Ultimately, there is a need to<br />

generate solutions that will advance how we collect and examine<br />

information, to ensure that children in our communities are<br />

thriving. It is our hope that this stimulus paper provides a starting<br />

point to open dialogue and stimulate creative ideas among ECD<br />

stakeholders to better meet the evaluation capacity building needs<br />

across sectors and disciplines.<br />

Discussion Question:<br />

What would help you to build evaluation<br />

capacity in your organization?<br />

The time is right to come together and work toward the advancement<br />

of evaluation practices in the ECD field. Meaningful evaluation can<br />

enhance our collective ability to engage in reflective practices that<br />

best support the well–being of young children. <strong>Advancing</strong> <strong>Evaluation</strong><br />

<strong>Practices</strong> provides an overview of the need for building evaluation<br />

capacity, and offers some examples for doing so. However, this paper<br />

is only a starting point. For the <strong>Evaluation</strong> Capacity Network to advance<br />

evaluation practices in ECD, we need to engage in collective discussion<br />

across the province to ensure steps taken are relevant to a range of<br />

stakeholders working across varied contexts.<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 17


Through our collective ideas, we can forge new pathways for<br />

advancing evaluation practices in the field of ECD. In collaboration,<br />

we can ensure that the evaluation we engage in is meaningful<br />

to our work and helps us to know that we are truly making a<br />

difference.<br />

Consider the<br />

possibilities...<br />

The examples below are some ideas for how evaluation capacity<br />

could be enhanced in the field of ECD. Consider these as a starting<br />

point as you ask yourself, What Are the Possibilities? Explore<br />

what would be useful for you and your organization to advance<br />

evaluation practices and better understand how you are making a<br />

difference for children and families.<br />

courses<br />

events<br />

tools<br />

dialogue<br />

interacting<br />

together<br />

learning<br />

from<br />

experts<br />

mentoring<br />

18


REFERENCES<br />

1. Shonkoff, J. P., & Levitt, P. (2010). Neuroscience and the future of early childhood policy:<br />

Moving from why to what and how. Neuron, 67(5), 689-691.<br />

2. Shonkoff, J. P., & Phillips, D. A. (2000). From neurons to neighbourhoods: The science of<br />

Early Childhood Development. Washington, DC: National Academy Press.<br />

3. Burger, K. (2010). How does early childhood care and education affect cognitive<br />

development? An international review of the effects of early interventions for children<br />

from different social backgrounds. Early Childhood Research Quarterly, 25, 140-165.<br />

4. Sylva, K., Melhuish, E. C., Sammons,P., Siraj-Blatchford, I., & Taggart, B. (2004). The<br />

effective provision of pre-school education (EPPE) project: Final report. London: DfES<br />

Institute of Education.<br />

5. Heckman, J. J. (2006). Skill formation and the economics of investing in disadvantaged<br />

children. Science, 312(5782), 1900-1902.<br />

6. Gauthier, B., Barrington, G. V., Bozzo, S. L., Chaytor, K., Dignard, A., Lahey, R., Malatest,<br />

R., McDavid, J. C., Mason, G., Mayne, J., Porteous, N. L., & Roy, S. (2010). The lay of the<br />

land: <strong>Evaluation</strong> practice in Canada in 2009. The Canadian Journal of Program <strong>Evaluation</strong>,<br />

24(1), 1-49.<br />

7. Bakken, L. L., Nunez, J., & Couture, C. (2014). A course model for building evaluation<br />

capacity through a university-community partnership. American Journal of <strong>Evaluation</strong>,<br />

35(4), 579-593.<br />

8. Doherty, G., Friendly, M., & Beach, J. (2003). OECD thematic review of early childhood<br />

education and care: Canadian background report. Ottawa: Government of Canada.<br />

9. Mustard, J. F., & Young, M. E. (2007). Measuring child development to leverage<br />

ECD policy and investment. In M. E. Young and L. M. Richardson (Eds.), Early child<br />

development from measurement to action: A priority for growth and equity, (253-291).<br />

Washington: The World Bank.<br />

10. Council of Ministers of Education, Canada (CMEC). (2014). CMEC early learning and<br />

development framework. Canada: CMEC Early Childhood Learning and Development<br />

Working Group. Retrieved from www.cmec.ca<br />

11. Munro, C. (2006). BC evaluation project report: Can we agree on common child outcomes<br />

for BC? Retrieved from www.successby6bc.ca<br />

12. Government of the Northwest Territories. (2013). Right from the start. Retrieved from<br />

www.rightfromthestart.ca<br />

13. Government of Manitoba. (2013). Starting early, starting strong: Manitoba’s early childhood<br />

development framework. Retrieved from www.gov.mb.ca<br />

<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 19


14. Makovichuk, L., Hewes, J., Lirette, P., & Thomas, N. (2014). Play, participation, and<br />

possibilities: An early learning and child care curriculum framework for Alberta. Retrieved<br />

from www.childcareframework.com<br />

15. Government of Alberta Human Services. (2013). Alberta’s Social Policy Framework.<br />

Referenced in www.humanservices.alberta.ca/department/previous-consultations.html<br />

16. Alberta Government. (2013). Together we raise tomorrow: An Alberta approach to early<br />

childhood development. Referenced in www.humanservices.alberta.ca/department/<br />

previous-consultations.html<br />

17. Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A<br />

research synthesis of the evaluation capacity building literature. American Journal of<br />

<strong>Evaluation</strong>, 33(3), 307-338.<br />

18. Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the capacity to do<br />

and use evaluation. In J. B. Cousins and I. Bourgeois (Eds.), Organizational capacity to do<br />

and use evaluation. New Directions for <strong>Evaluation</strong>, 141, 7-23.<br />

19. Preskill, H., & Boyle,S. (2008). A multidisciplinary model of evaluation capacity building.<br />

American Journal of <strong>Evaluation</strong>, 29(4) 443-459.<br />

20. University of Manitoba (2014). Summer institute in program evaluation. Retrieved from<br />

www.thesummerinstitute.ca<br />

21. Addiction Research Chair (2012). First Steps First: A community-based workbook for<br />

evaluating substance abuse and mental health programs in Saskatchewan. Retrieved from<br />

www.addictionresearchchair.ca/creating-knowledge/provincial/s-t-r-e-a-m/firststeps-first<br />

22. Canadian <strong>Evaluation</strong> Society (2015). Professional learning. Retrieved from www.<br />

evaluationcanada.ca/professional-learning<br />

20


<strong>Advancing</strong> <strong>Evaluation</strong> <strong>Practices</strong> 21


December 2015<br />

<strong>Evaluation</strong> Capacity Network<br />

University of Alberta<br />

This research was supported by the Social Sciences and Humanities Research Council of Canada.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!