Technical Manual - Renaissance Learning

ValidityPost-Publication Study DataDemographic data are not available on the students themselves. However, dataare available on the demographic makeup of the schools they attended. Theaverage school size was about 490 with a standard deviation of about 210students. The ethnic distribution of the sample was about 59% EuropeanAmerican, 14% African American, 22% Hispanic American, 3% Native American,and 2% Asian American.To minimize the cohort effect on any one estimate, the two comparable cohortsfrom successive school years were combined. For example, the students startingkindergarten in the 2001–2002 school year were combined with those startingkindergarten in the 2002 school year. This first cohort had second grade readingscores two years later; that is, for the 2003–2004 school year. The second cohorthad second grade reading scores for the 2004–2005 school year. Thus, the sixcohorts were combined into three age-similar groups defined by their initial gradelevel: a K-grade 2 group, a grade 1-grade 3 group, and a grade 2-grade 4 group.Each cohort’s early literacy skills were assessed during the fall, winter, and springof the beginning school year using STAR Early Literacy. The fall assessment periodtook place in August–September, the winter assessment in December–January,and the spring assessment in April–May. The final criterion variable was the scoreon the STAR Reading test taken during the fall assessment period two years afterthe initial early literacy assessment. Two complete school years separated the firstpredictor variable measurement and this criterion measurement.Since the students were assessed using STAR Early Literacy at three time pointswithin their initial grade levels, and if that test has developmental validity, thenscores should increase at each successive time point and the correlations acrossoccasions should be substantial. In addition to each cohort’s scores increasingwith time, STAR Early Literacy scores should also increase from grade to gradeacross cohorts.The use of the aggregated results will provide a measure of the general validity ofthe scores in predicting later reading scores. This permits the assessment ofpredictive validity across multiple time frames. Breaking the total sample downinto three groups by initial grade level status, one can analyze the predictivevalidity of previous years’ fall, winter, and spring STAR Early Literacy scoresrelative to STAR Reading scores at the second grade.Means and standard deviations of scale scores at each measurement time pointare provided in Table 27. For STAR Early Literacy tests in the initial year, overallaverage scale scores across the school year increased, indicating that the earlyliteracy assessment appears to follow an assumed developmental trajectory. Thisis also evident for scores within each initial grade level: kindergarten and first andsecond grades.STAR Early LiteracyTechnical Manual73