Technical Manual - Renaissance Learning

More documents

Recommendations

Info

$Accelerated Math for Interventionâ¢ - Renaissance Learning$

ValiditySummary of STAR Early Literacy Validity DataCorrelations between the Est. ORF and DORF WCPM are displayed in Table 30along with the mean difference, standard deviation of differences, and the 95% CIof the mean difference between the estimated score (Est. ORF) and the observedscore (WCPM). Correlations were moderately high, ranging from 0.64 to 0.69. Meandifferences between Est. ORF and WCPM ranged from 3.99 to –7.29, indicating thatat grade 1 the Est. ORF tended to over-estimate the students’ reading fluency byabout 4 words per minute, whereas it tended to underestimate WCPM in grade 3by about 7 words. These differences were small but statistically significant (allp < 0.001).Table 30:Relations STAR Early Literacy Est. ORF and DIBELS Oral Reading FluencyGradeCorrelationMean Difference(95% CI)SDDifferencet-test1 0.69 3.99 (3.07, 4.90) 20.99 t(2027) = 8.56, p < 0.0012 0.64 –3.67 (–5.37, –1.97) 23.40 t(729) = –4.23, p < 0.0013 0.64 –7.29 (–9.82, –4.76) 27.77 t(463) = –5.67, p < 0.001Summary of STAR Early Literacy Validity DataIn the aggregate, the data presented in the Validity section above are evidence ofSTAR Early Literacy’s concurrent, retrospective, predictive, and construct validity.The majority of the validity evidence presented in this chapter was specific to theversions of the assessment that preceded development of the Enterprise version.However, because of the similarity of the pre-Enterprise and Enterprise versions,and the high degree of correlation between them reported here, there is amplereason to consider the Enterprise version of STAR Early Literacy to be equivalentto other versions, and therefore to accept evidence of the validity of those earlierversions as being applicable to the Enterprise version as well. Indeed, theobserved correlation between the Enterprise and earlier “service” version,corrected for attenuation due to measurement error, is nearly perfect. Along withthe close correspondence of Enterprise-specific validity data to that of previousversions, this disattenuated correlation is evidence that the two versions aremeasuring a common underlying attribute, and doing so with equivalent degreesof measurement precision. We can confidently treat all of the evidence of thevalidity of the earlier STAR Early Literacy as applying perforce to STAR EarlyLiteracy Enterprise, and can accept all of the summary statements here as equallyapplicable to the Enterprise version.As the data presented in this chapter attests, scores on STAR Early Literacyincrease systematically and substantially with age and school grade, reaching aSTAR Early LiteracyTechnical Manual80
ValidityValidation Research Study Proceduresplateau at grade 3, by which time the overwhelming majority of children havemastered the early literacy skills the test measures.Scores on STAR Early Literacy were also shown to be strongly related to teachers’ratings of children’s skills, with easier skills mastered by children at relatively lowlevels on the STAR Early Literacy score scale, and more difficult skills mastered bychildren with scores at higher levels.In concurrent test administrations, STAR Early Literacy was found to correlate 0.78with STAR Reading in a sample of first- to third-grade students. Correlations withnumerous other tests were presented in Table 21 on page 61 and Table 22 onpage 63. These showed that STAR Early Literacy correlated an average of 0.59 witha wide range of measures of early literacy, readiness, and reading administered ingrades K through 3. The Meta-Analysis section showed the average uncorrectedcorrelation between STAR Early Literacy and all of the other tests to be 0.60. (Manymeta-analyses adjust the correlations for range restriction and attenuation to lessthan perfect reliability; had we done that here, the average correlation would haveexceeded 0.84.) Correlations with specific measures of reading ability were oftenhigher than this average.Research subsequent to publication shows relationships with othertests—including DIBELS, GRADE, the Michigan Literacy Progress Profile, RunningRecord, and TPRI—to be consistent with the Validity Study data: STAR EarlyLiteracy scores are moderately to highly correlated with scores on a wide range ofmeasures of early literacy skills. Perhaps most importantly, research shows STAREarly Literacy to be a valid predictor of children’s later reading development, asmeasured by scores on reading tests administered two years later.Validation Research Study ProceduresThe Validation Research StudyThe technical results of the STAR Early Literacy Calibration Study were excellent,with the tests showing good measurement properties, a high degree of reliability,and high correlation with an independent measure of reading ability. However,the Calibration Study was conducted using conventional tests, while STAR EarlyLiteracy was designed to be an adaptive test.Because the technical properties of the adaptive version may be somewhatdifferent from those found in the Calibration Study, additional psychometricresearch data were collected in the Spring of 2001 with the firstcomputer-adaptive version of STAR Early Literacy. Data from this ValidationResearch Study were intended to assess a number of technical characteristics ofthe adaptive version, including the following:STAR Early LiteracyTechnical Manual81
Page 1 and 2:
STAR Early LiteracyTechnical Manual
Page 3:
ContentsIntroduction . . . . . . .
Page 8 and 9:
STAR Early LiteracyTechnical Manual
Page 10 and 11:
IntroductionOverviewEnterprise inco
Page 12 and 13:
IntroductionDesign of STAR Early Li
Page 14 and 15:
IntroductionDesign of STAR Early Li
Page 16 and 17:
IntroductionTest SecurityRepeating
Page 18 and 19:
IntroductionPsychometric Characteri
Page 20 and 21:
IntroductionPsychometric Characteri
Page 22 and 23:
IntroductionSTAR Early Literacy Ent
Page 24 and 25:
Content and Item DevelopmentThe STA
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Page 32 and 33:
Content and Item DevelopmentItem De
Page 34 and 35:
Page 36 and 37:
Page 38 and 39: Content and Item DevelopmentItem De
Page 44 and 45: Core Progress Learning Progression
Page 46 and 47: Psychometric Research Supporting ST
Page 52 and 53: Reliability and Measurement Precisi
Page 64 and 65: ValidityRelationship of STAR Early
Page 74 and 75: ValidityPost-Publication Study Data
Page 86 and 87: ValidityConcurrent Validity of Esti
Page 90 and 91: ValidityValidation Research Study P
Page 92 and 93: ValidityValidation Research Study P
Page 94 and 95: ValiditySTAR Early Literacy Enterpr
Page 100 and 101: ValidityThe Validity of Early Numer
Page 102 and 103: ValidityThe Validity of Early Numer
Page 108 and 109: NormingSTAR Early Literacy Enterpri
Page 110 and 111: NormingDevelopment of Norms for STA
Page 112 and 113: NormingData AnalysisTable 49:Gender
Page 114 and 115: NormingGrowth Normswere smoothed us
Page 116 and 117: Score DistributionsScaled Scores: S
Page 118 and 119: Score DefinitionsFor its internal c
Page 120 and 121: Score DefinitionsEstimated Oral Rea
Page 122 and 123: STAR Early Literacy Enterprise in t
Page 138 and 139:
STAR Early Literacy Enterprise in t
Page 140 and 141:
Page 142 and 143:
Page 144 and 145:
Page 146 and 147:
Page 148 and 149:
Page 150 and 151:
Page 152 and 153:
Page 154 and 155:
Page 156 and 157:
Page 158 and 159:
Page 160 and 161:
AppendixTable 61:STAR Early Literac
Page 162 and 163:
AppendixTable 61:STAR Early Literac
Page 164 and 165:
ReferencesAdams, M. J. (1990). Begi
Page 166 and 167:
IndexAAbsolute growth, 134Access le
Page 168 and 169:
IndexScaled Score norms, 100Scaled
Page 170:
IndexTest item design guidelinesans
show all

Technical Manual - Renaissance Learning

Create successful ePaper yourself

Delete template?

Save as template?