Pearson Successmaker Math Efficacy Study 2009-10 Final Report

SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10Operations and Computation ***Process and Applications 1.01*** Indicates group means are not statistically significantly different1. effect size = estimated adjusted group difference / comparison sample standard deviation2. The average effect size for studies with large samples (i.e., more than 250 students) hasbeen recently estimated at 0.13 standard deviations.The effects for GMADE total score were consistently large across usage levels when separatedout by blocks of ten hours. At all three grades, ten to nineteen hours of program usage wasenough to see SuccessMaker users outperform nonusers by large amounts. Increasing usage forthe 3 rd and 5 th grade samples did not statistically increase the amount those users outperformedthe comparison group. The 7 th grade sample, however, continued to increase their generallylarge comparative effects from blocks two to three to four.3 rd Grade Usage 1 Ave. Hours 2 GMADE Effect Size 3less than 10 hours 9 (3) ***10 to 19 hours 17 (147) 1.2320 to 29 hours 25 (106) 1.1830 or more hours 32 (26) 1.21*** Indicates group means are not statistically significantly different.1. usage time rounded down to nearest hour2. Ave. Hours = average of students’ usage in hours, parentheses indicate sample size3. Effect Size = estimated adjusted group difference / comparison sample standard deviation5 th Grade Usage 1 Ave. Hours 2 GMADE Effect Size 3less than 10 hours 7 (11) 0.7410 to 19 hours 15 (94) 0.7020 to 29 hours 23 (54) 0.6430 or more hours 35 (65) 0.551. usage time rounded down to nearest hour2. Ave. Hours = average of students’ usage in hours, parentheses indicate sample size3. Effect Size = estimated adjusted group difference / comparison sample standard deviation7 th Grade Usage 1 Ave. Hours 2 GMADE Effect Size 3less than 10 hours 9 (4) 0.7010 to 19 hours 16 (72) 0.7520 to 29 hours 24 (51) 0.93 (2)30 or more hours 31 (9) 1.14 (1,2,3)1. usage time rounded down to nearest hour2. Ave. Hours = average of students’ usage in hours, parentheses indicate sample size3. Effect Size = estimated adjusted group difference / comparison sample standard deviation- 39 -