27.04.2022 Views

2020-2021_AllenES_21stCenturyGrant_Year2

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

Evaluation Report 2021

21 st Century Learning Center Grant:

Boys and Girls Clubs of Skagit County

Allen Elementary School

YEAR 2 EVALUATION REPORT

STACY MEHLBERG, Ph.D.

RONAK PATEL, Ph.D


Members of The BERC Group have K–20, experiences as

teachers, counselors, psychologists, building administrators,

district administrators, and college professors. The team is

currently working on research and evaluation projects at the

national, state, regional, district, school, classroom, and

student levels in over 1000 schools in Washington State and

nationally.

COPYRIGHT © 2021 BY THE BERC GROUP INC. ALL RIGHTS RESERVED

ADDITIONAL COPIES OF THIS REPORT MAY BE OBTAINED THROUGH THE BERC GROUP

(www.bercgroup.com).

THE BERC GROUP


Table of Contents

INTRODUCTION.................................................................................1

Year 2 Focus .....................................................................................................3

Grant Performance Objectives .......................................................................4

EVALUATION DESIGN.......................................................................4

Evaluation Questions ......................................................................................4

Data Sources ....................................................................................................5

Implementation Questions ...............................................................5

1. To what extent were components of the grant implemented with fidelity

during Year 2? ..................................................................................................5

2. What challenges emerged during Year 2 implementation? .....................7

3. What promising practices emerged during Year 2 implementation?......8

Outcome Questions ..........................................................................8

4. To what extent does program participation relate to improved

academic and social outcomes? ....................................................................9

RECOMMENDATIONS .....................................................................11

THE BERC GROUP


21 st Century Learning Center Grant:

Allen Elementary School

Year 2 Evaluation Report

INTRODUCTION

In 2019, the Boys and Girls Clubs of Skagit County (BGCSC) extended their 21st

Century Community Learning Centers (21 st CCLC) program into Allen Elementary

School in the Burlington-Edison School District. Allen Elementary had previously been

the site of a 21 st CCLC successfully run by the school district from 2014-2019. Based on

the continued needs of the students in this community, the school district and BGCSC

collaborated to develop a continuous service model to extend programming an additional

five years.

The mission of the BGCSC is to “” To enable all young people, especially those who

need us most, to reach their full potential, as productive, caring, responsible citizens.”

(BGCSC Website, 2020). The organization prioritizes three areas: Academic Success,

Healthy Lifestyles, and Good Character and Citizenship (Learn, Live, and Lead). The

main goals of the program include providing services and promoting academic growth for

“students who attend high-poverty and low-performing schools” (Department of

Education website, 2018). The grant proposal states a range of expected outcomes,

including:

• Improved academic performance

• Enriched learning experiences

• Increased family engagement

The director of the BGCSC manages the grant, while a site coordinator manages 3-4 part

time staff members. The program serves a diverse population of students, particularly

those with socio-economic, cultural and language barriers. Families at Allen Elementary

School are impacted by poverty at a higher rate than families within the district, and state.

Based on results from the 2018-2019 school year, less than 20% of students are passing

their standardized assessments in math, ELA, and science, with scores trending

negatively over time (Figure 1).

THE BERC GROUP

1


Allen Elementary Assessment Proficiency

Over Time

ELPA

10%

Math SBA

16%

18%

ELA SBA

20%

20%

Figure 1

2019-2020 2018-2019 2017-2018

Allen Elementary Demographics

According to OSPI, Allen Elementary School served approximately 409 students during

the 2020-2021 school year, with 34% of the population identifying as English Language

Learners, 79% meeting criteria for low-income status, and 9.8% identifying as homeless.

Additionally, approximately 18% of the students are from migrant families, and 11.7%

are students identified with learning/behavioral disabilities. Ethnicity demographics are

included in (Figure 2).

2

THE BERC GROUP


70%

64%

Allen Elementary Student Ethnicity

70%

67%

53%

35%

27%

22%

25%

18%

0%

Figure 2

1%

2% 1% 1%

1%

2017-2018 2018-2019 2019-2020

American Indian/ Alaskan Native

Black/ African American

Two or more races

1%

Asian

Hispanic/Latino of any race

White

1%

Year 2 Focus

During Year 2, site-based leadership was committed to helping staff develop a mission

and vision to drive programming. One staff member shared,

Something that Cesar did with us was to collect staff mission statements to

compile into one big mission statement. We have shared values; a lot of us want

to build whole child development and attend to the emotional and physical needs.

Honesty with the kids is really important to us. We want to be transparent as an

organization. It reflects with everyone else- we have a great team- bosses who

care about their employees and the kids. I feel supported in my personal life as

well as my professional life.

The site coordinator responded by sharing that he wanted to create a unified team, even

though they had different backgrounds and goals. He worked to help his staff understand

that they had a shared vision for what they wanted to see for the kids, and how they could

impact their lives. Students and staff agreed that they were a community focused on

connection and kindness. Each member of the 21 st CCLC community brought a unique

perspective and set of experiences, and the program remained dedicated to helping

students develop empathy and learn to support one another through challenges.

THE BERC GROUP

3


Grant Performance Objectives

The grant goals identified in the original proposal are listed below.

• Improve academic outcomes

• Improve positive student behavior, engagement, and healthy choices

• Provide family engagement development and education strategies that

ensure positive role models for students by increasing the self-sufficiency

of families

• Provide an enriching program environment that demonstrates a

commitment to a formal continuous quality improvement process

The purpose of this report is to provide an evaluation of implementation progress and

outcomes during Year 2 of the Allen Elementary 21 st Century Community Learning

Center grant cycle. The evaluation design and methodology, evaluation findings, and

recommendations are included within this report.

EVALUATION DESIGN

Researchers used a multiple measures, mixed methodology approach to conduct this

evaluation. The collection of both quantitative and qualitative data adds scope and

breadth to the study in addition to providing the ability to triangulate findings (Creswell,

1994). Additionally, researchers followed a utilization-focused model of evaluation. This

model of evaluation is intended to provide useful feedback that can be implemented in

real time (Patton, 2013). Qualitative data was gathered during program observations,

interviews with staff, and document review. The BGCSC provided a database of student

attendance in programming. Due to the pandemic, student assessment data were not

available for the 2020-2021 school year.

Evaluation Questions

The evaluation questions were designed to provide program leaders formative feedback

for continuous improvement, as well as summative feedback about progress toward Boys

& Girls Clubs of Skagit County 21 st Century After-School Program grant goals. The

questions align with OSPI’s local evaluation guidelines for 21 st Century Community

Learning Center programs and specific performance objectives of the BGCSC. The

evaluation questions are:

Implementation Questions (Focused on the effectiveness of plans, processes, and

program strategies related to the grant vision and goals):

4

THE BERC GROUP


1. To what extent were components of the grant implemented with fidelity?

2. What challenges emerged during Year 2 implementation?

3. What promising practices emerged during Year 2 implementation?

Outcome Questions (Focused on meeting local, state, and federal performance goals and

objectives):

4. To what extent does program participation relate to improved academic and social

outcomes?

5. To what extent are program components sustainable and continuously improving?

Data Sources

To address these questions researchers gathered data from multiple sources throughout

the evaluation cycle 1. BGCSC staff contributed attendance data to the AIR Portal.

Additionally, BERC researchers conducted:

• Collection and analysis of interviews in Spring 2021

• Analysis of After-school program student attendance data

• Observation of the program

• Initiative documents and materials review

Implementation Questions

The following evaluation questions focus on program implementation during Year 2,

including the effectiveness of plans, processes, and program strategies relating to the

grant vision and goals. These questions were primarily assessed through focus groups and

ongoing correspondence with program stakeholders.

1. To what extent were components of the grant implemented with fidelity during

Year 2?

During the 2020-2021 school year, schools and communities faced many challenges

related to the pandemic. At Allen Elementary, the 21 st CCLC was able to operate inperson,

implementing a cohort model to comply with covid policies and maintain safe

social-distancing practices. Allen 21 st CCLC staff identified the cohort model as a

successful way to meet the needs of students, although they acknowledged there were

1 Researchers will conduct and analyze the Program Quality Assessment (PQA) during future evaluation

reports. This assessment was not required during the 2020-2021 school year.

THE BERC GROUP

5


many things they just could not do that they had done before, specifically related to

program choice. Despite the challenges, the staff and leadership felt that they were able to

deliver a robust academic program for their students, and worked to develop enrichment

opportunities that would encourage engagement and provide social, emotional, and

academic support.

At the start of the school year, the 21 st CCLC supported student academics throughout the

day, offering on-site programming from 7 a.m. to 6 p.m., and working with students from

Kindergarten to Seventh grade. One staff member shared,

We have gone above and beyond, really helping them with schoolwork, trying to

help teach them different content areas. We have tried to support their in-class

learning. We made sure they went to ZOOM lessons, and completed school day

work. We do school stuff in the morning; we were in the room with them while

they logged into their computers…everyone was on different ZOOM calls. We

had to help create an environment for the students to be able to learn.

Once students returned to school for a more consistent in-person school schedule, the

Allen ES 21 st CCLC staff continued to support students in the afternoons, providing

Power Hour and enrichment opportunities. One staff member shared, “During power

hour, we would do reading sometimes, either individually or I would read to them. We

built educational aspects into all of the enrichment programs. We tried to have them learn

something without them knowing.” The site-coordinator also noted that staff tried to

maintain relationships with teachers in order to better support students, and used SeeSaw

to identify missing assignments.

In addition to a focus on improving academic outcomes for students, Allen ES 21 st CCLC

staff were committed to providing enrichment activities that promoted social-emotional

well-being and connection to others. Staff were acutely aware of the challenges their

students were facing as a result of limited interaction with peers and a disrupted schedule.

Program staff developed several activities that would be successful with the cohort

model, and encouraged student engagement, voice, and leadership whenever possible.

They acknowledged, however, that student choice became more difficult as a result of

staffing and restrictions to movement.

Staff felt that their STEM programming was successful during the 2020-2021 school

year, introducing STEM challenges, STEM Tech, and STEM Explorers. Activities

included building bridges with toothpicks, 3-D shapes with marshmallows, building a

robot and making it follow a path, making slime, a mock oil spill activity, crystal

6

THE BERC GROUP


growing, and working together to develop survival skills, including building a shelter and

filtering water to make it potable.

Art was also a successful program throughout the year, and staff incorporated art history,

art appreciation, and social-emotional expression into lesson topics. Students used

untraditional materials to make art, incorporating elements of nature, and studied Monet,

Warhol and Mattisse. One staff member shared that they also brought in a local artist to

make relevant connections for the students. This artist brought in ideas and examples, and

talked about being a working artist. Students were then able to make art in a similar style,

but from their own perspectives.

In addition to art and STEM, the 21 st CCLC continued to provide Triple Play, a B&G

Club program that encouraging activity and peer collaboration through sports. Students

played freeze tag, navigated an obstacle course, and played Capture the Flag. Students

were also given unstructured play time to encourage some choice and autonomy over

their program day.

Finally, program staff offered a popular Cooking Club on Fridays, which they identified

as something the students consistently looked forward to. The goal of the program was to

create healthy and delicious snacks using ingredients the students could access at home.

Staff shared that it was sometimes challenging to come up with recipes that could be

made without an oven or stovetop, but they enjoyed being creative and trying new things.

2. What challenges emerged during Year 2 implementation?

COVID.

The pandemic was a persistent challenge throughout Year 2. It was difficult to keep up

with district and state level changes, and 21 st CCLC staff needed to remain flexible to

comply with expectations while also providing high-quality programming that would

support and engage students. As a result of an altered delivery model, staff felt they were

somewhat limited in what they could provide. The site coordinator agreed with this

perspective, sharing, “This is where COVID has impacted us the most- we are not able to

give the students the same options as before COVID. With the cohort model, the students

need to work together, and they need to follow the programming that is offered- which is

why we switch it up a lot… [these was] just not as much individual choice.”

Staffing.

Staffing the Allen ES 21 st CCLC has continued to remain a challenge. During Year 2,

program leaders had to hire several staff members simultaneously, making it difficult to

clearly assign roles and responsibilities for each aspect of programming. The program

THE BERC GROUP

7


director shared that despite this, the Allen ES 21 st CCLC leadership was able to use their

strengths to make the best of a difficult situation.

Student Behaviors and Engagement.

Staff shared that while many students seemed eager to participate in programming, there

were times when it was challenging to manage behaviors and build engagement,

particularly around academics. One staff member shared, “I have had a couple of

challenges with specific kids with behavioral needs I was not used to, so I had to work

through that, but I was supported, and never left work feeling too stressed.” Staff also

noted that students seemed to struggle with risk taking, and were sometimes reluctant to

try new things. Staff shard concerns about student development, as several noted more

immature behaviors than they had previously experienced with students of the same age.

3. What promising practices emerged during Year 2 implementation?

Interviews with program staff and leadership revealed promising practices that emerged

during Year 2, including building relationships with students, developing a strong 21 st

CCLC team, and remaining flexible in order to serve as many students as possible during

a particularly challenging year. Staff acknowledged, however, that many of the practices

implemented during the COVID response were not intended to remain forever, and they

hoped that program would return to pre-pandemic operations during the 2021-2022

school year. Despite this, staff felt proud of their ability to meet the changing needs of

their students by spending time with them during their school day and supporting them

with academics. Staff also appreciated the time and effort their site-coordinator made to

provide training, build comradery amongst the staff, and adopt a team approach to

programming that helped staff feel supported throughout the year.

Outcome Questions

The following evaluation questions focus on outcomes of program implementation

relating to grant objectives and goals. These questions were primarily assessed through

quantitative analysis of student data, while focus group responses helped provide context

around the outcomes. During Year 2, student state assessments were not administered, so

there is no outcome data to share. Quantitative assessment data will be included in future

reports.

8

THE BERC GROUP


4. To what extent does program participation relate to improved academic and

social outcomes?

During interviews staff were asked to share their perceptions of student outcomes during

Year 2. Overall, staff shared that students were relearning the rules and expectations for

behavior, and seemed excited to be with others again after so much time in isolation. One

staff member noted that as the year progressed, her students were willing to try new

things, made more efforts to get to know new people, and developed stronger

communication skills. Students began to show a preference for program, and would

choose to stay later, or make positive comments about the program to staff. Students also

expressed their excitement for summer program, which gave them something to look

forward to.

Attendance. One outcome measure for the 21 st Century program is student attendance.

Program leaders collected and shared program attendance for all participating students.

Students that were in attendance for more than 30 days were labeled “Regular

Attendees”. There was a total of 33 students enrolled at Allen Elementary during Year 2,

62.5% of which attended at least 30 days of the program. Figure 1 shows consistent

attendance from September 2020 to January 2021, with a slight drop off between

February 2021 to the end of Year 2 in June 2021. Of the 33 total students attending the

program, about half attended at least 90 days (Figure 5).

Figure 3

THE BERC GROUP

9


Figure 4

Figure 5

Assessment Scores

10

THE BERC GROUP


Due to school closures related to the COVID19 pandemic, standardized assessments were

not administered. Evaluators made attempts to collect any formative data gathered during

the school year, but were unable to acquire the data at the time of this report.

5. To what extent are program components sustainable?

This year programming looked different due to the timing of the grant, and pandemic

restrictions and regulations. Sustainability of program components will be assessed

during further evaluation cycles.

RECOMMENDATIONS

Recommendations are based on evidence from data presented throughout the report, and

from results of the external Program Quality Assessments (PQAs) conducted in

2019-2020. In future reports, researchers will use the quantitative data and PQA results to

guide new recommendations for continuous improvement.

Identify incentives to help attract and retain quality program staff. Difficulty in

recruiting and retaining program staff remained a challenge during Year 2. One possible

way to increase recruitment would be to partner with local colleges and universities to

offer incentives, such as credit, for students who want experience with youth.

Specifically, students in Education programs may benefit from experience with

elementary and middle school students and be willing to serve as program staff for the

right incentives.

Continue to address student behaviors, and communicate with families and students

regarding the safety of the program space. It is important for staff members to work

together to clearly communicate behavior expectations for students and to consistently

reinforce those expectations. When the afterschool staff are using the same language and

terms, especially around behavior expectations, as teachers and other school staff, it sends

a message that the school and program are aligned in providing student support.

Increasing interactions with parents regarding student behavior would also help to

provide continuous behavior management between school times, after school

programming, and at home. Recommendations from SA-PQA include, “when done well,

structure and clear limits can provide the space kids need to explore and express

themselves…and can ensure a safe program climate” (SA-QA, 2017).

Increase student reflection in projects. Recommendations from the YPQA suggest that

the Allen site should increase opportunities for student reflection into regular

THE BERC GROUP

11


programming. One way to increase student reflection is to provide a dedicated time at the

start or end of an activity. Students may be asked to verbally share their thoughts with

peers, or draw or write in a journal. Additionally, exit tickets provide opportunities for

reflection, as well as offering feedback that program staff can use to make adjustments to

lessons based on student interests. Students may feel more invested in the program if they

are regularly asked to share their insights about projects and are encouraged to plan future

activities.

The BERC Group, Inc.

P.O. Box 3552

Redmond, WA 98073

Phone: 425.327.2801

www.bercgroup.com

12

THE BERC GROUP

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!