Pearson Successmaker Math Efficacy Study 2009-10 Final Report

More documents

Recommendations

Info

$Math Review - Pearson$

SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10program was overwhelmingly positive, with 80% of the 646 recorded comments coded aspositive in nature. Teachers appreciated the reporting system for informing classroominstruction, identifying students for remediation, monitoring student progress, and as a tool toshare student progress with curriculum specialists and parents. A majority of teachers felt theinitial placement and the adaptive motion of sequencing students through the program waseffective. In addition, the learning activities were rated as well-differentiated and aligned tocurrent curricula and state educational objectives, and the program challenged both lower andhigher achieving student populations. Teachers reported rare minor technical issues (ex., loggingin, activities loading), primarily the result of district and school infrastructure.Teachers firmly believed that their students enjoyed using the math version of SuccessMaker,and felt that the program made the learning process more fun. When formally interviewed,teachers were overwhelmingly positive about their students’ interactions with the program. Ofthe 170 recorded comments, 79% were positive in nature. When students were surveyed, 93% of3 rd grade, 79% of 5 th grade, and 88% of 7 th grade students indicated they liked using theSuccessMaker program. Third grade students responded most positively to the characters andanimation, and found the learning activities engaging. Fifth and 7 th grade students more oftenperceived the characters as immature and the animation sometimes excessive and distracting(i.e., only 9% of 3 rd grade versus 28% of 5 th grade and 35% of 7 th grade students indicated theydisliked the animation).RQ: How was the SuccessMaker Math program utilized?The majority of study teachers implemented SuccessMaker Math in a computer laboratoryenvironment, typically implementing the program 2-3 days per week for an average of 24minutes per session. Ten teachers implemented SuccessMaker in the lab more than three times aweek. Three teachers utilized a joint-usage model, implementing SuccessMaker in the classroomfor 30% to 40% of the total usage, and the remainder in the computer lab. One 3 rd grade teacherchose not to utilize the computer laboratory after a couple months of implementation, andimplemented SuccessMaker the remainder of the year in the classroom with laptop stations(accounting for 75% of total usage minutes in the classroom). SuccessMaker students in 3 rd and5 th grade generally used the program in addition to their regular math block, while 7 th gradestudents used SuccessMaker during their daily math block.The three grade levels were similar in their usage time with medians (i.e., 50 th percentile or thosestudents with usage in the center of the distribution) of 19, 18, and 17 hours logged on theprogram for 3 rd , 5 th , and 7 th grade respectively. The three grade levels also demonstratedsimilarly good productivity and success rates with their assigned tasks. Students at the center ofthe distribution completed well over one exercise per minute indicating, as a group, that studentwere on-task. All three grade levels also had median success rates in the 60% to 80% rangeindicating that students as a group were continuously and appropriately challenged as theyprogressed through the program. Students in the 3 rd grade SuccessMaker classrooms attempted43 exercises every thirty minutes with a success rate of 68%, while students in 5 th gradeclassrooms attempted 44 exercises every thirty minutes with a success rate of 68%. Seventhgrade SuccessMaker students attempted 38 exercises every thirty minutes with a success rate of63%.- 2 -
SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10The program’s reporting feature was well-received by the teachers. Individual preference andteacher expectations dictated how teachers utilized information gained from the reports. Theaverage teacher reported utilizing the program’s reporting system in an educationally significantway during 57% of the usage weeks (P 25 = 22%, P 75 =92%). A majority of the SuccessMakerteachers used the reporting system at least once to check students’ progress, determine whichstudents needed help while using the program, and/or to inform additional classroom instructionor practice on specific topics. To a lesser extent several teachers used information from thereporting system to evaluate students on state testing goals, to ability group students duringclassroom instruction, or to provide data to parents.RQ: Do 3 rd , 5 th , and 7 th grade students making regular use of the SuccessMaker Math programdemonstrate higher mathematics achievement as compared to students that did not utilizeSuccessMaker Math?SuccessMaker students in 3 rd , 5 th , and 7 th grades statistically significantly outperformed theircomparison group counterparts on the GMADE Total score. The magnitude of the difference inperformance observed at all three grades was remarkable, with standard deviations of 1.00, 0.53,and 0.61 for 3 rd , 5 th , and 7 th grade respectively. These effects were consistently large acrossusage levels with only ten to nineteen hours on the program enough to see large differences overthe comparison group. SuccessMaker students in 3 rd , 5 th , and 7 th grade statistically significantlyoutperformed their comparison group counterparts on the Process and Applications subtest. Themagnitude of the difference in performance observed at all three grades was again very large,with standard deviations of 1.32, 0.59, and 1.01 for 3 rd , 5 th , and 7 th grade respectively. Theseeffects were also consistently large across usage levels.SuccessMaker students in 3 rd and 5 th grade statistically significantly outperformed theircomparison group counterparts on the Operations and Computation subtest. The magnitude ofthe differences in performance observed at both grades were equivalently very large, 0.75standard deviations. And yet again, these effects were consistently large across usage levels.The 7 th grade SuccessMaker students outperformed their comparison group peers though notstatistically so. Though the SuccessMaker students in 3 rd and 7 th grade performed statisticallysimilar to the comparison group on the Concepts and Communications subtest, the 5 th gradecomparison group statistically significantly outperformed the SuccessMaker group on thissubtest.When the data was broken out for student subpopulations, 3 rd grade Hispanic, low SES, non-English proficient, female, and lower-achieving SuccessMaker students all statisticallysignificantly outperformed their comparison group peers on GMADE Total score (i.e., 0.50 to1.31 standard deviations), as well as the Process and Applications (i.e., 0.91 to 1.65 standarddeviations) and the Operations and Computation subtests (i.e., 0.49 to 1.19 standard deviations).Low SES, non-English proficient and female 5 th grade SuccessMaker students statisticallysignificantly outperformed their comparison group peers on GMADE Total score (i.e., 0.48 to0.53 standard deviations), as well as, both the Process and Applications (i.e., 0.49 to 0.63standard deviations) and Operations and Computation subtests (i.e., 0.55 to 0.73 standarddeviations).Seventh grade low SES, non-English proficient, and female students all dramaticallyoutperformed their comparison group counterparts on GMADE Total score (i.e., 0.57 to 0.66- 3 -
Page 1 and 2: PEARSON SUCCESSMAKER MATHEFFICACY S
Page 3 and 4: SuccessMaker Math RCT Gatti Evaluat
Page 5: SuccessMaker Math RCT Gatti Evaluat
Page 57 and 58:
SuccessMaker Math RCT Gatti Evaluat
Page 59 and 60:
Page 61 and 62:
Page 63:
show all

Pearson Successmaker Math Efficacy Study 2009-10 Final Report

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?