CIS of Texas Evaluation Fiinal Technical Report - TEA - Home ...

Texas Education Agency, Evaluation of Communities in Schools (CIS) of Texas
3. Individual Study Designs and Samples
Each of the study designs used for the evaluation and the study samples are described in this
section.
3.1 Student-Level Studies
There are two parts to the student-level design, both intended to demonstrate the impact of CIS
on at-risk students over time—the within-CIS case-managed students study and the between
case-managed and non-case-managed students study. Each is described below.
Within-CIS Case-Managed Students Study
The within-CIS case-managed student study is intended to examine trends in outcomes for
case-managed students over time and to examine the relationship between service type,
dosage, and outcomes. The within-CIS student design identifies not only whether CIS is having
a positive impact on case-managed students, but also points to what factors influence outcomes
and for which subgroups CIS has the greatest impact. The outcomes examined included:
achievement (as measured by proficiency in reading and math and course completion for high
school students), attendance, discipline, graduation (high school only), dropout (high school
only), and promotion. Actual course grades were not included in the analysis because this
information was not available in PEIMS or CISTMS. Only information on pass/fail was reported
in both systems. For CISTMS, more than 50 percent of the pass/fail data were missing across
courses and students.
Within-Student Sample. Students in grades 4, 7, and 10 who were first enrolled in CIS in
2004-05 were selected for the within-CIS student analyses. School year 2003-04 served as the
baseline, with 2004-05 through 2006-07 serving as post years. The within-student sample
cohorts included:
●
●
Elementary school students first enrolled in CIS in 2004-05 as fourth graders (N=3,767).
Middle school students first enrolled in CIS in 2004-05 as seventh graders (N=5,270).
● High school students first enrolled in CIS in 2004-05 as tenth graders (N=2,989). 3
This creation of cohorts enabled the evaluation team to track students over the course of four
years. More importantly, these cohorts allowed the evaluation team to examine both immediate
and longer-term outcomes for case-managed students. Specifically, changes between 2004-05
(year of enrollment in CIS) and 2005-06 represent the period of immediate CIS impact. For
elementary and middle school students, changes between 2005-06 and 2006-07 represent
longer-term outcomes and more importantly, possible sustained outcomes during a critical
transition period for students (elementary to middle school and middle school to high school). 4
This time period is necessary to highlight the potential benefits and need for the continuation of
CIS services across grade levels (i.e., feeder patterns within school districts) and in particular,
the continuation of services to case-managed students during transition when existing
challenges are often compounded (Pinkus, 2008). Finally, for high school students, examining
outcomes in 2006-07 allows us to assess change through to Grade 12 for this cohort.
3 It is important to note that the final sample sizes for each outcome may vary as a result of missing data.
4 While not all middle schools in Texas (with or without CIS) included Grades 6 through 8, the majority followed this structure. Therefore, viewing Grade 6 as a
transition period to middle school and Grade 9 as a transition to high school is appropriate for this evaluation.
Final Technical Report December 2008 8