Pearson Successmaker Math Efficacy Study 2009-10 Final Report

SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10IV. DISCUSSIONTeachers and students quickly became comfortable with the SuccessMaker program, and felt theprogram was a good educational investment. When interviewed, the teacher response to theprogram was overwhelmingly positive. Teachers appreciated the reporting system, felt the initialplacement and adaptive motion of students through the program were effective, the learningactivities were well-differentiated and aligned to their current curricula and state educationalobjectives, the program challenged both their lower and higher achieving student populations,and that the audio and graphics allowed ELL and lower reading achieving populations to learn.Teachers firmly believe that their students like using SuccessMaker Math andfeel that the program makes the learning process more fun for students.Students themselves reported positive attitudes towards the program as well asmore positive academic attitudes than non-users.Teachers also firmly believe that their students like using the program and feel that the programmakes the learning process more fun. Students appreciate the capacity of the program to allowthem to laugh and interact with their own virtual learning environment. When surveyed, only asmall minority of students indicated they disliked the program. Further evidence that theprogram resonated positively with students can be seen in the math attitude survey results whereSuccessMaker students had higher scores than did their comparison group counterparts. The 3 rdand 7 th grade differences were both statistically significant, very large (i.e., 3 rd 0.99 standarddeviations, 7 th 0.62 standard deviations) and also seen for several at risk populations.Teachers came up with creative solutions to get all students on the program each week,overcoming packed classroom lesson plans and filled computer lab schedules. Most teacherswent to the lab 2 or 3 times a week for an average of 24 minutes. Ten teachers went to the labmore than three times a week. Only four teachers had their students use the program in theclassroom for 30% or more of their total usage. Total program usage was a median of 19, 18,and 17 hours, for 3 rd , 5 th , and 7 th grade respectively.The final study sample was diverse and very large at 1,186 students. Three districts have ahighly transient population and thus had comparatively high attrition. Though diverse, thesample was specifically heavily Hispanic, low SES, and overall low achieving, including thetype of at-risk students that would benefit from a well-conceived and implemented mathematicsintervention.The data indicates clearly that diverse populations of students receivingSuccessMaker Math can be successful in significantly increasing achievement.The achievement data indicates clearly that diverse populations of students receivingSuccessMaker Math can be successful when receiving as little as ten to nineteen hours on theprogram. After holding confounding factors constant for both groups (i.e., baseline scores,student demographic information, and classroom environment indicators) and estimating end-ofyearraw score group mean differences SuccessMaker students in all three grades statistically- 58 -