Pearson Successmaker Math Efficacy Study 2009-10 Final Report

SuccessMaker Math RCT Gatti Evaluation Inc. 9-15-10II. METHODOLOGYThe SuccessMaker Math program was evaluated in sixty-three diverse 3 rd , 5 th and 7 th gradeclassrooms from eight urban and suburban school districts in seven different states (i.e., AZ, AR,CA, IN, KS, NY, PA) during the 2009-10 school year. The program was evaluated via a twogroup,classroom level randomized, baseline to post observation assessment research design.Teachers or sections within each school were randomly assigned to one of two study groups (i.e.,comparison v. SuccessMaker Math). Students in classrooms randomly assigned to implementSuccessMaker Math made regular use of the program for one hour a week in two or threesessions while students in comparison classrooms generally received supplemental instructionfrom non-computerized mathematics programs currently in use at their school.Gatti Evaluation provided research schools all data collection materials, maintainedcommunication with the study sites, and followed clear data collection procedures throughout thestudy to ensure that both study and program implementation ran smoothly and effectively. Thefollowing sections provide information on study procedures, including; student and teacher leveldata collection, site recruitment and selection, the nature of math instruction at the study sites,program training and implementation, detail on educational settings at each study site,demographic information for study participants, and the statistical methodologies used to analyzeoutcomes.Student Outcome MeasuresA challenging assessment battery was group administered to students to measureachievement and academic attitude growth during the school year.An assessment battery was group administered to students, proctored by their teachers, at thestart of program use (i.e., baseline testing) and again in the last month of the school year (i.e.,end-of-year testing). The assessment battery consisted of the Group Mathematics Assessmentand Diagnostic Evaluation (GMADE) and a mathematics academic attitude survey. Theassessment battery was intended to challenge the students; attempting to adequately assessincoming mathematics knowledge for a wide range of abilities while providing room for growthas knowledge was acquired during the school year.The GMADE is a standardized, nationally norm-referenced mathematics achievement testpublished by Pearson Assessments. The GMADE was constructed with all fifty states’ standardsin mind, covering a wide range of content topics and skills. The GMADE includes 9 levels thatspan grades K-12, each with two parallel forms (i.e., level 3 for 3 rd grade, level 5 for 5 th grade,level M for 7 th grade). Form A was administered at baseline and form B was administered at theend of the school year. The GMADE is not a timed test, but generally takes between 60 and 90minutes to administer. Sites returned completed student tests to the site coordinators, who thenshipped the tests to the research team for hand-scoring.- 7 -