2020-2021_AllenES_21stCenturyGrant_Year2

Evaluation Report 2021
21 st Century Learning Center Grant:
Boys and Girls Clubs of Skagit County
Allen Elementary School
YEAR 2 EVALUATION REPORT
STACY MEHLBERG, Ph.D.
RONAK PATEL, Ph.D

Members of The BERC Group have K–20, experiences as
teachers, counselors, psychologists, building administrators,
district administrators, and college professors. The team is
currently working on research and evaluation projects at the
national, state, regional, district, school, classroom, and
student levels in over 1000 schools in Washington State and
nationally.
COPYRIGHT © 2021 BY THE BERC GROUP INC. ALL RIGHTS RESERVED
ADDITIONAL COPIES OF THIS REPORT MAY BE OBTAINED THROUGH THE BERC GROUP
(www.bercgroup.com).
THE BERC GROUP

Table of Contents
INTRODUCTION.................................................................................1
Year 2 Focus .....................................................................................................3
Grant Performance Objectives .......................................................................4
EVALUATION DESIGN.......................................................................4
Evaluation Questions ......................................................................................4
Data Sources ....................................................................................................5
Implementation Questions ...............................................................5
1. To what extent were components of the grant implemented with fidelity
during Year 2? ..................................................................................................5
2. What challenges emerged during Year 2 implementation? .....................7
3. What promising practices emerged during Year 2 implementation?......8
Outcome Questions ..........................................................................8
4. To what extent does program participation relate to improved
academic and social outcomes? ....................................................................9
RECOMMENDATIONS .....................................................................11
THE BERC GROUP

21 st Century Learning Center Grant:
Allen Elementary School
Year 2 Evaluation Report
INTRODUCTION
In 2019, the Boys and Girls Clubs of Skagit County (BGCSC) extended their 21st
Century Community Learning Centers (21 st CCLC) program into Allen Elementary
School in the Burlington-Edison School District. Allen Elementary had previously been
the site of a 21 st CCLC successfully run by the school district from 2014-2019. Based on
the continued needs of the students in this community, the school district and BGCSC
collaborated to develop a continuous service model to extend programming an additional
five years.
The mission of the BGCSC is to “” To enable all young people, especially those who
need us most, to reach their full potential, as productive, caring, responsible citizens.”
(BGCSC Website, 2020). The organization prioritizes three areas: Academic Success,
Healthy Lifestyles, and Good Character and Citizenship (Learn, Live, and Lead). The
main goals of the program include providing services and promoting academic growth for
“students who attend high-poverty and low-performing schools” (Department of
Education website, 2018). The grant proposal states a range of expected outcomes,
including:
• Improved academic performance
• Enriched learning experiences
• Increased family engagement
The director of the BGCSC manages the grant, while a site coordinator manages 3-4 part
time staff members. The program serves a diverse population of students, particularly
those with socio-economic, cultural and language barriers. Families at Allen Elementary
School are impacted by poverty at a higher rate than families within the district, and state.
Based on results from the 2018-2019 school year, less than 20% of students are passing
their standardized assessments in math, ELA, and science, with scores trending
negatively over time (Figure 1).
THE BERC GROUP
1

Allen Elementary Assessment Proficiency
Over Time
ELPA
10%
Math SBA
16%
18%
ELA SBA
20%
20%
Figure 1
2019-2020 2018-2019 2017-2018
Allen Elementary Demographics
According to OSPI, Allen Elementary School served approximately 409 students during
the 2020-2021 school year, with 34% of the population identifying as English Language
Learners, 79% meeting criteria for low-income status, and 9.8% identifying as homeless.
Additionally, approximately 18% of the students are from migrant families, and 11.7%
are students identified with learning/behavioral disabilities. Ethnicity demographics are
included in (Figure 2).
2
THE BERC GROUP

70%
64%
Allen Elementary Student Ethnicity
70%
67%
53%
35%
27%
22%
25%
18%
0%
Figure 2
1%
2% 1% 1%
1%
2017-2018 2018-2019 2019-2020
American Indian/ Alaskan Native
Black/ African American
Two or more races
1%
Asian
Hispanic/Latino of any race
White
1%
Year 2 Focus
During Year 2, site-based leadership was committed to helping staff develop a mission
and vision to drive programming. One staff member shared,
Something that Cesar did with us was to collect staff mission statements to
compile into one big mission statement. We have shared values; a lot of us want
to build whole child development and attend to the emotional and physical needs.
Honesty with the kids is really important to us. We want to be transparent as an
organization. It reflects with everyone else- we have a great team- bosses who
care about their employees and the kids. I feel supported in my personal life as
well as my professional life.
The site coordinator responded by sharing that he wanted to create a unified team, even
though they had different backgrounds and goals. He worked to help his staff understand
that they had a shared vision for what they wanted to see for the kids, and how they could
impact their lives. Students and staff agreed that they were a community focused on
connection and kindness. Each member of the 21 st CCLC community brought a unique
perspective and set of experiences, and the program remained dedicated to helping
students develop empathy and learn to support one another through challenges.
THE BERC GROUP
3

Grant Performance Objectives
The grant goals identified in the original proposal are listed below.
• Improve academic outcomes
• Improve positive student behavior, engagement, and healthy choices
• Provide family engagement development and education strategies that
ensure positive role models for students by increasing the self-sufficiency
of families
• Provide an enriching program environment that demonstrates a
commitment to a formal continuous quality improvement process
The purpose of this report is to provide an evaluation of implementation progress and
outcomes during Year 2 of the Allen Elementary 21 st Century Community Learning
Center grant cycle. The evaluation design and methodology, evaluation findings, and
recommendations are included within this report.
EVALUATION DESIGN
Researchers used a multiple measures, mixed methodology approach to conduct this
evaluation. The collection of both quantitative and qualitative data adds scope and
breadth to the study in addition to providing the ability to triangulate findings (Creswell,
1994). Additionally, researchers followed a utilization-focused model of evaluation. This
model of evaluation is intended to provide useful feedback that can be implemented in
real time (Patton, 2013). Qualitative data was gathered during program observations,
interviews with staff, and document review. The BGCSC provided a database of student
attendance in programming. Due to the pandemic, student assessment data were not
available for the 2020-2021 school year.
Evaluation Questions
The evaluation questions were designed to provide program leaders formative feedback
for continuous improvement, as well as summative feedback about progress toward Boys
& Girls Clubs of Skagit County 21 st Century After-School Program grant goals. The
questions align with OSPI’s local evaluation guidelines for 21 st Century Community
Learning Center programs and specific performance objectives of the BGCSC. The
evaluation questions are:
Implementation Questions (Focused on the effectiveness of plans, processes, and
program strategies related to the grant vision and goals):
4
THE BERC GROUP

1. To what extent were components of the grant implemented with fidelity?
2. What challenges emerged during Year 2 implementation?
3. What promising practices emerged during Year 2 implementation?
Outcome Questions (Focused on meeting local, state, and federal performance goals and
objectives):
4. To what extent does program participation relate to improved academic and social
outcomes?
5. To what extent are program components sustainable and continuously improving?
Data Sources
To address these questions researchers gathered data from multiple sources throughout
the evaluation cycle 1. BGCSC staff contributed attendance data to the AIR Portal.
Additionally, BERC researchers conducted:
• Collection and analysis of interviews in Spring 2021
• Analysis of After-school program student attendance data
• Observation of the program
• Initiative documents and materials review
Implementation Questions
The following evaluation questions focus on program implementation during Year 2,
including the effectiveness of plans, processes, and program strategies relating to the
grant vision and goals. These questions were primarily assessed through focus groups and
ongoing correspondence with program stakeholders.
1. To what extent were components of the grant implemented with fidelity during
Year 2?
During the 2020-2021 school year, schools and communities faced many challenges
related to the pandemic. At Allen Elementary, the 21 st CCLC was able to operate inperson,
implementing a cohort model to comply with covid policies and maintain safe
social-distancing practices. Allen 21 st CCLC staff identified the cohort model as a
successful way to meet the needs of students, although they acknowledged there were
1 Researchers will conduct and analyze the Program Quality Assessment (PQA) during future evaluation
reports. This assessment was not required during the 2020-2021 school year.
THE BERC GROUP
5

many things they just could not do that they had done before, specifically related to
program choice. Despite the challenges, the staff and leadership felt that they were able to
deliver a robust academic program for their students, and worked to develop enrichment
opportunities that would encourage engagement and provide social, emotional, and
academic support.
At the start of the school year, the 21 st CCLC supported student academics throughout the
day, offering on-site programming from 7 a.m. to 6 p.m., and working with students from
Kindergarten to Seventh grade. One staff member shared,
We have gone above and beyond, really helping them with schoolwork, trying to
help teach them different content areas. We have tried to support their in-class
learning. We made sure they went to ZOOM lessons, and completed school day
work. We do school stuff in the morning; we were in the room with them while
they logged into their computers…everyone was on different ZOOM calls. We
had to help create an environment for the students to be able to learn.
Once students returned to school for a more consistent in-person school schedule, the
Allen ES 21 st CCLC staff continued to support students in the afternoons, providing
Power Hour and enrichment opportunities. One staff member shared, “During power
hour, we would do reading sometimes, either individually or I would read to them. We
built educational aspects into all of the enrichment programs. We tried to have them learn
something without them knowing.” The site-coordinator also noted that staff tried to
maintain relationships with teachers in order to better support students, and used SeeSaw
to identify missing assignments.
In addition to a focus on improving academic outcomes for students, Allen ES 21 st CCLC
staff were committed to providing enrichment activities that promoted social-emotional
well-being and connection to others. Staff were acutely aware of the challenges their
students were facing as a result of limited interaction with peers and a disrupted schedule.
Program staff developed several activities that would be successful with the cohort
model, and encouraged student engagement, voice, and leadership whenever possible.
They acknowledged, however, that student choice became more difficult as a result of
staffing and restrictions to movement.
Staff felt that their STEM programming was successful during the 2020-2021 school
year, introducing STEM challenges, STEM Tech, and STEM Explorers. Activities
included building bridges with toothpicks, 3-D shapes with marshmallows, building a
robot and making it follow a path, making slime, a mock oil spill activity, crystal
6
THE BERC GROUP

growing, and working together to develop survival skills, including building a shelter and
filtering water to make it potable.
Art was also a successful program throughout the year, and staff incorporated art history,
art appreciation, and social-emotional expression into lesson topics. Students used
untraditional materials to make art, incorporating elements of nature, and studied Monet,
Warhol and Mattisse. One staff member shared that they also brought in a local artist to
make relevant connections for the students. This artist brought in ideas and examples, and
talked about being a working artist. Students were then able to make art in a similar style,
but from their own perspectives.
In addition to art and STEM, the 21 st CCLC continued to provide Triple Play, a B&G
Club program that encouraging activity and peer collaboration through sports. Students
played freeze tag, navigated an obstacle course, and played Capture the Flag. Students
were also given unstructured play time to encourage some choice and autonomy over
their program day.
Finally, program staff offered a popular Cooking Club on Fridays, which they identified
as something the students consistently looked forward to. The goal of the program was to
create healthy and delicious snacks using ingredients the students could access at home.
Staff shared that it was sometimes challenging to come up with recipes that could be
made without an oven or stovetop, but they enjoyed being creative and trying new things.
2. What challenges emerged during Year 2 implementation?
COVID.
The pandemic was a persistent challenge throughout Year 2. It was difficult to keep up
with district and state level changes, and 21 st CCLC staff needed to remain flexible to
comply with expectations while also providing high-quality programming that would
support and engage students. As a result of an altered delivery model, staff felt they were
somewhat limited in what they could provide. The site coordinator agreed with this
perspective, sharing, “This is where COVID has impacted us the most- we are not able to
give the students the same options as before COVID. With the cohort model, the students
need to work together, and they need to follow the programming that is offered- which is
why we switch it up a lot… [these was] just not as much individual choice.”
Staffing.
Staffing the Allen ES 21 st CCLC has continued to remain a challenge. During Year 2,
program leaders had to hire several staff members simultaneously, making it difficult to
clearly assign roles and responsibilities for each aspect of programming. The program
THE BERC GROUP
7

director shared that despite this, the Allen ES 21 st CCLC leadership was able to use their
strengths to make the best of a difficult situation.
Student Behaviors and Engagement.
Staff shared that while many students seemed eager to participate in programming, there
were times when it was challenging to manage behaviors and build engagement,
particularly around academics. One staff member shared, “I have had a couple of
challenges with specific kids with behavioral needs I was not used to, so I had to work
through that, but I was supported, and never left work feeling too stressed.” Staff also
noted that students seemed to struggle with risk taking, and were sometimes reluctant to
try new things. Staff shard concerns about student development, as several noted more
immature behaviors than they had previously experienced with students of the same age.
3. What promising practices emerged during Year 2 implementation?
Interviews with program staff and leadership revealed promising practices that emerged
during Year 2, including building relationships with students, developing a strong 21 st
CCLC team, and remaining flexible in order to serve as many students as possible during
a particularly challenging year. Staff acknowledged, however, that many of the practices
implemented during the COVID response were not intended to remain forever, and they
hoped that program would return to pre-pandemic operations during the 2021-2022
school year. Despite this, staff felt proud of their ability to meet the changing needs of
their students by spending time with them during their school day and supporting them
with academics. Staff also appreciated the time and effort their site-coordinator made to
provide training, build comradery amongst the staff, and adopt a team approach to
programming that helped staff feel supported throughout the year.
Outcome Questions
The following evaluation questions focus on outcomes of program implementation
relating to grant objectives and goals. These questions were primarily assessed through
quantitative analysis of student data, while focus group responses helped provide context
around the outcomes. During Year 2, student state assessments were not administered, so
there is no outcome data to share. Quantitative assessment data will be included in future
reports.
8
THE BERC GROUP

4. To what extent does program participation relate to improved academic and
social outcomes?
During interviews staff were asked to share their perceptions of student outcomes during
Year 2. Overall, staff shared that students were relearning the rules and expectations for
behavior, and seemed excited to be with others again after so much time in isolation. One
staff member noted that as the year progressed, her students were willing to try new
things, made more efforts to get to know new people, and developed stronger
communication skills. Students began to show a preference for program, and would
choose to stay later, or make positive comments about the program to staff. Students also
expressed their excitement for summer program, which gave them something to look
forward to.
Attendance. One outcome measure for the 21 st Century program is student attendance.
Program leaders collected and shared program attendance for all participating students.
Students that were in attendance for more than 30 days were labeled “Regular
Attendees”. There was a total of 33 students enrolled at Allen Elementary during Year 2,
62.5% of which attended at least 30 days of the program. Figure 1 shows consistent
attendance from September 2020 to January 2021, with a slight drop off between
February 2021 to the end of Year 2 in June 2021. Of the 33 total students attending the
program, about half attended at least 90 days (Figure 5).
Figure 3
THE BERC GROUP
9

Figure 4
Figure 5
Assessment Scores
10
THE BERC GROUP

Due to school closures related to the COVID19 pandemic, standardized assessments were
not administered. Evaluators made attempts to collect any formative data gathered during
the school year, but were unable to acquire the data at the time of this report.
5. To what extent are program components sustainable?
This year programming looked different due to the timing of the grant, and pandemic
restrictions and regulations. Sustainability of program components will be assessed
during further evaluation cycles.
RECOMMENDATIONS
Recommendations are based on evidence from data presented throughout the report, and
from results of the external Program Quality Assessments (PQAs) conducted in
2019-2020. In future reports, researchers will use the quantitative data and PQA results to
guide new recommendations for continuous improvement.
Identify incentives to help attract and retain quality program staff. Difficulty in
recruiting and retaining program staff remained a challenge during Year 2. One possible
way to increase recruitment would be to partner with local colleges and universities to
offer incentives, such as credit, for students who want experience with youth.
Specifically, students in Education programs may benefit from experience with
elementary and middle school students and be willing to serve as program staff for the
right incentives.
Continue to address student behaviors, and communicate with families and students
regarding the safety of the program space. It is important for staff members to work
together to clearly communicate behavior expectations for students and to consistently
reinforce those expectations. When the afterschool staff are using the same language and
terms, especially around behavior expectations, as teachers and other school staff, it sends
a message that the school and program are aligned in providing student support.
Increasing interactions with parents regarding student behavior would also help to
provide continuous behavior management between school times, after school
programming, and at home. Recommendations from SA-PQA include, “when done well,
structure and clear limits can provide the space kids need to explore and express
themselves…and can ensure a safe program climate” (SA-QA, 2017).
Increase student reflection in projects. Recommendations from the YPQA suggest that
the Allen site should increase opportunities for student reflection into regular
THE BERC GROUP
11

programming. One way to increase student reflection is to provide a dedicated time at the
start or end of an activity. Students may be asked to verbally share their thoughts with
peers, or draw or write in a journal. Additionally, exit tickets provide opportunities for
reflection, as well as offering feedback that program staff can use to make adjustments to
lessons based on student interests. Students may feel more invested in the program if they
are regularly asked to share their insights about projects and are encouraged to plan future
activities.
The BERC Group, Inc.
P.O. Box 3552
Redmond, WA 98073
Phone: 425.327.2801
www.bercgroup.com
12
THE BERC GROUP