Improving Mathematical Problem Solving in Grades 4 Through 8

EDUCATOR’S PRACTICE GUIDEWHAT WORKS CLEARINGHOUSEImproving Mathematical ProblemSolving in Grades 4 Through 8NCEE 2012-4055U.S. DEPARTMENT OF EDUCATION

The Institute of Education Sciences (IES) publishes practice guides in education to bring the bestavailable evidence and expertise to bear on current challenges in education. Authors of practiceguides combine their expertise with the findings of rigorous research, when available, to developspecific recommendations for addressing these challenges. The authors rate the strength of theresearch evidence supporting each of their recommendations. See Appendix A for a full descriptionof practice guides.The goal of this practice guide is to offer educators specific, evidence-based recommendationsthat address the challenge of improving mathematical problem solving in grades 4 through 8.The guide provides practical, clear information on critical topics related to improving mathematicalproblem solving and is based on the best available evidence as judged by the authors.Practice guides published by IES are available on our website at http://ies.ed.gov/ncee/wwc/publications_reviews.aspx#pubsearch.

IES Practice GuideImproving Mathematical ProblemSolving in Grades 4 Through 8May 2012PanelJohn Woodward (Chair)University of Puget SoundSybilla BeckmannUniversity of GeorgiaMark DriscollEducation Development CenterMegan FrankeUniversity of California, Los AngelesPatricia HerzigIndependent Math ConsultantStaffMadhavi JayanthiRebecca Newman-GoncharKelly HaymondInstructional Research GroupJoshua FurgesonMathematica Policy ResearchProject OfficerJoy LesnickInstitute of Education SciencesAsha JitendraUniversity of MinnesotaKenneth R. KoedingerCarnegie Mellon UniversityPhilip OgbuehiLos Angeles Unified School DistrictNCEE 2012-4055U.S. DEPARTMENT OF EDUCATION

This report was prepared for the National Center for Education Evaluation and Regional Assistance,Institute of Education Sciences under Contract ED-07-CO-0062 by the What Works Clearinghouse,which is operated by Mathematica Policy Research.DisclaimerThe opinions and positions expressed in this practice guide are those of the authors and do notnecessarily represent the opinions and positions of the Institute of Education Sciences or the U.S.Department of Education. This practice guide should be reviewed and applied according to thespecific needs of the educators and education agency using it, and with full realization that itrepresents the judgments of the review panel regarding what constitutes sensible practice, basedon the research that was available at the time of publication. This practice guide should be usedas a tool to assist in decision making rather than as a “cookbook.” Any references within thedocument to specific education products are illustrative and do not imply endorsement of theseproducts to the exclusion of other products that are not referenced.U.S. Department of EducationArne DuncanSecretaryInstitute of Education SciencesJohn Q. EastonDirectorNational Center for Education Evaluation and Regional AssistanceRebecca A. MaynardCommissionerMay 2012This report is in the public domain. Although permission to reprint this publication is not necessary,the citation should be:Woodward, J., Beckmann, S., Driscoll, M., Franke, M., Herzig, P., Jitendra, A., Koedinger, K. R., &Ogbuehi, P. (2012). Improving mathematical problem solving in grades 4 through 8: A practiceguide (NCEE 2012-4055). Washington, DC: National Center for Education Evaluation and RegionalAssistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc/publications_reviews.aspx#pubsearch/.What Works Clearinghouse practice guide citations begin with the panel chair, followed by thenames of the panelists listed in alphabetical order.This report is available on the IES website at http://ies.ed.gov/ncee and http://ies.ed.gov/ncee/wwc/publications_reviews.aspx#pubsearch/.Alternate FormatsOn request, this publication can be made available in alternate formats, such as Braille, large print, orCD. For more information, contact the Alternate Format Center at (202) 260–0852 or (202) 260–0818.

ContentsImproving Mathematical ProblemSolving in Grades 4 Through 8Table of ContentsReview of Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . 1Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2Institute of Education Sciences Levels of Evidence for Practice Guides . . . . . . . 3Introduction to the Improving Mathematical Problem Solvingin Grades 4 Through 8 Practice Guide . . . . . . . . . . . . . . . . . . . . . .6Recommendation 1. Prepare problems and use them in whole-class instruction . . . . . 10Recommendation 2. Assist students in monitoring and reflecting on theproblem-solving process . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Recommendation 3. Teach students how to use visual representations . . . . . . . . . 23Recommendation 4. Expose students to multiple problem-solving strategies . . . . . . 32Recommendation 5. Help students recognize and articulate mathematicalconcepts and notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Appendix A. Postscript from the Institute of Education Sciences. . . . . . . . . . . . 47Appendix B. About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . 49Appendix C. Disclosure of Potential Conflicts of Interest. . . . . . . . . . . . . . . 52Appendix D. Rationale for Evidence Ratings. . . . . . . . . . . . . . . . . . . . 53Endnotes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81( iii )

Table of Contents (continued)List of TablesTable 1. Institute of Education Sciences levels of evidence for practice guides . . . . . . . . 4Table 2. Recommendations and corresponding levels of evidence . . . . . . . . . . . . . 9Table D.1. Studies of interventions that involved problem selection and contributeto the level of evidence rating . . . . . . . . . . . . . . . . . . . . . . . . . . 55Table D.2. Studies of interventions that involved monitoring and contributeto the level of evidence rating. . . . . . . . . . . . . . . . . . . . . . . . . 58Table D.3. Supplemental evidence supporting the effectiveness of Recommendation 2 . . . . 59Table D.4. Studies of interventions that used visual representations and contributeto the level of evidence rating. . . . . . . . . . . . . . . . . . . . . . . . . . 61Table D.5. Supplemental evidence supporting the effectiveness of Recommendation 3 . . . . 62Table D.6. Studies of interventions that involved multiple problem-solving strategiesand contribute to the level of evidence rating . . . . . . . . . . . . . . . . . . . . 65Table D.7. Studies of interventions that helped students recognize and articulate conceptsand contribute to the level of evidence rating . . . . . . . . . . . . . . . . . . . . 69List of ExamplesExample 1. Sample routine problems . . . . . . . . . . . . . . . . . . . . . . . 12Example 2. Sample non-routine problems. . . . . . . . . . . . . . . . . . . . . 12Example 3. One teacher’s efforts to clarify vocabulary and context. . . . . . . . . . . 14Example 4. What mathematical content is needed to solve the problem?. . . . . . . . . 14Example 5. Mathematical language to review with students . . . . . . . . . . . . . . 15Example 6. Sample question list. . . . . . . . . . . . . . . . . . . . . . . . . 19Example 7. Sample task list . . . . . . . . . . . . . . . . . . . . . . . . . . 19Example 8. One way to model monitoring and reflecting using questions . . . . . . . . . 20Example 9. Using student ideas to clarify and refine the monitoring and reflecting process. . . 21Example 10. Sample table, strip diagram, percent bar, and schematic diagram . . . . . . . 24Example 11. One way of thinking aloud . . . . . . . . . . . . . . . . . . . . . . 27Example 12. Variations in story contexts for a proportion problem. . . . . . . . . . . 29Example 13. Diagrams with relevant and irrelevant details. . . . . . . . . . . . . . 30Example 14. Two ways to solve the same problem. . . . . . . . . . . . . . . . . 33Example 15. A comparison of strategies . . . . . . . . . . . . . . . . . . . . . 35Example 16. How students solved a problem during a small-group activity . . . . . . . . 36Example 17. Two students share their strategies for solving a fractions problem. . . . . . 37Example 18. Students’ intuitive understanding of formal mathematical concepts . . . . . . 41Example 19. Sample student explanations: How mathematically valid are they?. . . . . . 42Example 20. How to make sense of algebraic notation: Solve a problem arithmeticallybefore solving it algebraically . . . . . . . . . . . . . . . . . . . . . . . . . . 43Example 21. How to make sense of algebraic notation: Link components of theequation to the problem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43( iv )

Review of RecommendationsRecommendation 1.Prepare problems and use them in whole-class instruction.1. Include both routine and non-routine problems in problem-solving activities.2. Ensure that students will understand the problem by addressing issues students mightencounter with the problem’s context or language.3. Consider students’ knowledge of mathematical content when planning lessons.Recommendation 2.Assist students in monitoring and reflecting on the problem-solving process.1. Provide students with a list of prompts to help them monitor and reflect during the problemsolvingprocess.2. Model how to monitor and reflect on the problem-solving process.3. Use student thinking about a problem to develop students’ ability to monitor and reflect.Recommendation 3.Teach students how to use visual representations.1. Select visual representations that are appropriate for students and the problems theyare solving.2. Use think-alouds and discussions to teach students how to represent problems visually.3. Show students how to convert the visually represented information into mathematical notation.Recommendation 4.Expose students to multiple problem-solving strategies.1. Provide instruction in multiple strategies.2. Provide opportunities for students to compare multiple strategies in worked examples.3. Ask students to generate and share multiple strategies for solving a problem.Recommendation 5.Help students recognize and articulate mathematical concepts and notation.1. Describe relevant mathematical concepts and notation, and relate them to theproblem-solving activity.2. Ask students to explain each step used to solve a problem in a worked example.3. Help students make sense of algebraic notation.( 1 )

AcknowledgmentsThe panel appreciates the efforts of Madhavi Jayanthi, Rebecca Newman-Gonchar, and KellyHaymond from the Instructional Research Group, and of Joshua Furgeson and Robin Patfieldfrom Mathematica Policy Research, who participated in the panel meetings, described the researchfindings, and drafted the guide. We also thank Russell Gersten, Alison Wellington, and AnnalisaMastri for helpful feedback and reviews of earlier versions of the guide.John WoodwardSybilla BeckmannMark DriscollMegan FrankePatricia HerzigAsha JitendraKenneth R. KoedingerPhilip Ogbuehi( 2 )

Levels of Evidence for Practice GuidesInstitute of Education Sciences Levels of Evidence for Practice GuidesThis section provides information about the role of evidence in Institute of Education Sciences’(IES) What Works Clearinghouse (WWC) practice guides. It describes how practice guide panelsdetermine the level of evidence for each recommendation and explains the criteria for each of thethree levels of evidence (strong evidence, moderate evidence, and minimal evidence).The level of evidence assigned to each recommendationin this practice guide represents thepanel’s judgment of the quality of the existingresearch to support a claim that, when thesepractices were implemented in past research,positive effects were observed on studentoutcomes. After careful review of the studiessupporting each recommendation, panelistsdetermine the level of evidence for each recommendationusing the criteria in Table 1. Thepanel first considers the relevance of individualstudies to the recommendation and thendiscusses the entire evidence base, taking thefollowing into consideration:• the number of studies• the design of the studies• the quality of the studies• whether the studies represent the rangeof participants and settings on which therecommendation is focused• whether findings from the studies can beattributed to the recommended practice• whether findings in the studies are consistentlypositiveA rating of strong evidence refers to consistentevidence that the recommended strategies,programs, or practices improve studentoutcomes for a wide population of students. 1In other words, there is strong causal andgeneralizable evidence.A rating of moderate evidence refers either toevidence from studies that allow strong causalconclusions but cannot be generalized withassurance to the population on which a recommendationis focused (perhaps because thefindings have not been widely replicated) or toevidence from studies that are generalizablebut have some causal ambiguity. It also mightbe that the studies that exist do not specificallyexamine the outcomes of interest in thepractice guide, although they may be related.A rating of minimal evidence suggests that thepanel cannot point to a body of research thatdemonstrates the practice’s positive effect onstudent achievement. In some cases, this simplymeans that the recommended practices wouldbe difficult to study in a rigorous, experimentalfashion; 2 in other cases, it means that researchershave not yet studied this practice, or thatthere is weak or conflicting evidence of effectiveness.A minimal evidence rating does notindicate that the recommendation is any lessimportant than other recommendations witha strong evidence or moderate evidence rating.In developing the levels of evidence, the panelconsiders each of the criteria in Table 1. Thelevel of evidence rating is determined as thelowest rating achieved for any individual criterion.Thus, for a recommendation to get astrong rating, the research must be rated asstrong on each criterion. If at least one criterionreceives a rating of moderate and none receivea rating of minimal, then the level of evidenceis determined to be moderate. If one or morecriteria receive a rating of minimal, then thelevel of evidence is determined to be minimal.( 3 )

Levels of Evidence for Practice Guides (continued)Table 1. Institute of Education Sciences levels of evidence for practice guidesCriteriaSTRONGEvidence BaseMODERATEEvidence BaseMINIMALEvidence BaseValidityHigh internal validity (highqualitycausal designs).Studies must meet WWCstandards with or withoutreservations. 3ANDHigh external validity(requires multiple studieswith high-quality causaldesigns that represent thepopulation on which therecommendation is focused).Studies must meet WWCstandards with or withoutreservations.High internal validity butmoderate external validity(i.e., studies that supportstrong causal conclusions butgeneralization is uncertain).ORHigh external validity butmoderate internal validity(i.e., studies that support thegenerality of a relation butthe causality is uncertain). 4The research may includeevidence from studies thatdo not meet the criteriafor moderate or strongevidence (e.g., case studies,qualitative research).Effects onrelevantoutcomesConsistent positive effectswithout contradictoryevidence (i.e., no statisticallysignificant negativeeffects) in studies with highinternal validity.A preponderance of evidenceof positive effects. Contradictoryevidence (i.e., statisticallysignificant negativeeffects) must be discussedby the panel and consideredwith regard to relevance tothe scope of the guide andintensity of the recommendationas a component of theintervention evaluated.There may be weak orcontradictory evidenceof effects.Relevance toscopeDirect relevance to scope(i.e., ecological validity)—relevant context (e.g.,classroom vs. laboratory),sample (e.g., age and characteristics),and outcomesevaluated.Relevance to scope (ecologicalvalidity) may vary, includingrelevant context (e.g.,classroom vs. laboratory),sample (e.g., age and characteristics),and outcomesevaluated. At least someresearch is directly relevantto scope (but the researchthat is relevant to scope doesnot qualify as strong withrespect to validity).The research may beout of the scope of thepractice guide.Relationshipbetweenresearch andrecommendationsDirect test of the recommendationin the studiesor the recommendationis a major component ofthe intervention tested inthe studies.Intensity of the recommendationas a component ofthe interventions evaluatedin the studies may vary.Studies for which theintensity of the recommendationas a component ofthe interventions evaluatedin the studies is low; and/orthe recommendationreflects expert opinionbased on reasonable extrapolationsfrom research.( 4 )(continued)

Levels of Evidence for Practice Guides (continued)Table 1. Institute of Education Sciences levels of evidence for practice guides (continued)CriteriaSTRONGEvidence BaseMODERATEEvidence BaseMINIMALEvidence BasePanel confidencePanel has a high degree ofconfidence that this practiceis effective.The panel determines thatthe research does not riseto the level of strong butis more compelling than aminimal level of evidence.Panel may not be confidentabout whether the researchhas effectively controlledfor other explanations orwhether the practice wouldbe effective in most or allcontexts.In the panel’s opinion, therecommendation must beaddressed as part of thepractice guide; however, thepanel cannot point to a bodyof research that rises to thelevel of moderate or strong.Role of expertopinionNot applicable Not applicable Expert opinion based ondefensible interpretationsof theory (theories). (In somecases, this simply meansthat the recommendedpractices would be difficultto study in a rigorous,experimental fashion; inother cases, it means thatresearchers have not yetstudied this practice.)When assessmentis thefocus of therecommendationFor assessments, meets thestandards of The Standardsfor Educational and PsychologicalTesting. 5For assessments, evidenceof reliability that meets TheStandards for Educationaland Psychological Testingbut with evidence of validityfrom samples not adequatelyrepresentative ofthe population on which therecommendation is focused.Not applicableThe panel relied on WWC evidence standards to assess the quality of evidence supporting educationalprograms and practices. WWC evaluates evidence for the causal validity of instructional programsand practices according to WWC standards. Information about these standards is available athttp://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19. Eligible studies that meet WWC evidencestandards or meet evidence standards with reservations are indicated by bold text in the endnotesand references pages.( 5 )

IntroductionIntroduction to the Improving Mathematical Problem Solvingin Grades 4 Through 8 Practice GuideThis section outlines the importance of improving mathematical problem solving for studentsin grades 4 through 8 and explains key parameters considered by the panel in developing thepractice guide. It also summarizes the recommendations for readers and concludes with a discussionof the research supporting the practice guide.Students who develop proficiency in mathematicalproblem solving early are better preparedfor advanced mathematics and other complexproblem-solving tasks. 6 Unfortunately, whencompared with students in other countries,students in the U.S. are less prepared to solvemathematical problems. 7 For example, recentTrends in International Mathematics and ScienceStudy (TIMSS) data suggest that, whencompared to other industrialized countriessuch as the Netherlands, China, and Latvia, U.S.4th-graders rank tenth and 8th-graders rankseventh out of 41 countries in problem solving. 8Problem solving involves reasoning and analysis,argument construction, and the developmentof innovative strategies. These abilitiesare used not only in advanced mathematicstopics—such as algebra, geometry and calculus—butalso throughout the entire mathematicscurriculum beginning in kindergarten, aswell as in subjects such as science. Moreover,these skills have a direct impact on students’achievement scores, as many state andnational standardized assessments and collegeentrance exams include problem solving. 9Traditional textbooks 10 often do not providestudents rich experiences in problem solving.11 Textbooks are dominated by sets ofproblems that are not cognitively demanding,particularly when assigned as independentseatwork or homework, and teachers oftenreview the answers quickly without discussingwhat strategies students used to solvethe problems or whether the solutions canbe justified. 12 The lack of guidance in textbooksis not surprising, given that state anddistrict standards are often less clear in theirguidelines for process skills, such as problemsolving, than they are in their wording ofgrade-level content standards. 13The goal of this practice guide is to giveteachers and administrators recommendationsfor improving mathematical problem-solvingskills, regardless of which curriculum is used.The guide offers five recommendations thatprovide teachers with a coherent approachfor regularly incorporating problem solvinginto their classroom instruction to achieve thisend. It presents evidence-based suggestionsfor putting each recommendation into practiceand describes roadblocks that may be encountered,as well as possible solutions.Scope of the practice guideAudience and grade level. The need foreffective problem-solving instruction is particularlycritical in grades 4 through 8, whenthe mathematics concepts taught becomemore complicated and when various formsof assessments—from class tests to state andnational assessments—begin incorporatingproblem-solving activities. In this guide, thepanel provides teachers with five recommendationsfor instructional practices thatimprove students’ problem-solving ability.Math coaches and other administrators alsomay find this guide helpful as they prepareteachers to use these practices in their classrooms.Curriculum developers may find theguide useful in making design decisions, andresearchers may find opportunities to extendor explore variations in the evidence base.Content. The literature reviewed for thisguide was restricted to mathematical problem-solvingtopics typically taught in grades4 through 8. The panelists reviewed a numberof definitions of problem solving as partof the process of creating this guide, but asingle, prevalent definition of problem solvingwas not identified. This is understandable,( 6 )

Introduction (continued)given the different contexts in which theterm problem solving is used in mathematics.Some definitions are exceedingly broad andapplied to a general level of problem solvingthat goes beyond mathematics into everydayhuman affairs. For example, problem solvingis often defined as the “movement from agiven state to a goal state with no obviousway or method for getting from one to theother.” 14 This kind of definition underscoresthe non-routine nature of problem solvingand the fact that it is not the execution ofmemorized rules or shortcuts, such as usingkey words, to solve math word problems.More contemporary definitions of problemsolving focus on communication, reasoning, andmultiple solutions. In addition to the non-routinenature of the process, this kind of mathematicalproblem solving is portrayed as the opportunityto engage in mathematics and derive a reasonableway or ways to solve the problem. 15 In lightof the long-standing historical variations anddisputes over definitions of problem solving, thepanel ultimately decided that it was not in theirpurview to resolve this issue. The panel definedthe characteristics of problem solving thatapplied to this guide as follows:• First, students can learn mathematicalproblem solving; it is neither an innate talentnor happenstance that creates skilledproblem solvers.• Second, mathematical problem solving isrelative to the individual. What is challengingor non-routine for one student may becomparatively straightforward for a moreadvanced student.• Third, mathematical problem solving neednot be treated like just another topic in thepacing guide; instead, it can serve to supportand enrich the learning of mathematicsconcepts and notation.• Fourth, often more than one strategycan be used to solve a problem. Learningmultiple strategies may help students seedifferent ideas and approaches for solvingproblems and may enable students to thinkmore flexibly when presented with a problemthat does not have an obvious solution.Problem solving includes more than workingword problems. While word problemshave been the mainstay of mathematicstextbooks for decades, they are only onetype of math problem. Other types of mathproblems appropriate to grades 4 through8, such as algebraic and visual-spatial problems(e.g., “How many squares are there on acheckerboard?”), are addressed in this guide.The panel excluded whole number additionand subtraction, which are typically taughtin kindergarten through grade 3, as well asadvanced algebra and advanced geometry,which are typically taught in high school.When developing recommendations, the panelincorporated several effective instructionalpractices, including explicit teacher modelingand instruction, guided questions, and effortsto engage students in conversations abouttheir thinking and problem solving. The panelbelieves it is important to include the varietyof ways problem solving can be taught.There are several limitations to the scope of thisguide. The literature reviewed for this guide waslimited to studies pertaining to mathematicalproblem solving; therefore, it did not includecognitive or psychological dimensions ofproblem solving that fell outside of this topicarea. 16 While the panel considered studies thatincluded students with disabilities and studentswho were learning English, this guide does notaddress specific instructional practices for thesegroups. Instead, this guide is intended for use byall teachers, including general education, specialeducation teachers, and teachers of Englishlearners, of mathematics in grades 4 through 8.Summary of the recommendationsThe five recommendations in this guide canbe used independently or in combinationto help teachers engage students in problemsolving on a regular basis. To facilitateusing the recommendations in combination,the panel provided a discussion of how the( 7 )

Introduction (continued)recommendations can be combined in thelesson-planning process. This discussion is presentedin the conclusion section of the guide.Recommendation 1 explains how teachersshould incorporate problem-solving activitiesinto daily instruction, instead of saving them forindependent seatwork or homework. The panelstresses that teachers must consider their unitgoals and their students’ background and interestswhen preparing problem-solving lessons.Recommendation 2 underscores the importanceof thinking through or reflecting on theproblem-solving process. Thinking throughthe answers to questions such as “What isthe question asking me to do?” and “Why didthese steps in solving the problem work ornot work?” will help students master multistepor complex problems.Recommendations 3, 4, and 5 focus on specificways to teach problem solving.Recommendation 3 covers instruction invisual representations, such as tables, graphs,and diagrams. Well-chosen visual representationshelp students focus on what is centralto many mathematical problems: the relationshipbetween quantities.Recommendation 4 encourages teachersto teach multiple strategies that can be usedto solve a problem. Sharing, comparing, anddiscussing strategies afford students theopportunity to communicate their thinkingand, by listening to others, become increasinglyflexible in the way they approach andsolve problems. Too often students becomewedded to just one approach and then flounderwhen it does not work on a different ormore challenging problem.Recommendation 5 encourages teachers tohelp students recognize and articulate mathematicalconcepts and notation during problemsolvingactivities. The key here is for teachersto remember that students’ problem solving willimprove when students understand the formalmathematics at the heart of each problem.Of the five recommendations the panel sharesin this guide, the panel chose to present therecommendation (Recommendation 1) thatprovides guidance for preparing problemsolvingactivities first. Even though the levelof evidence supporting this recommendationis not strong, the panel believes teachersshould plan before undertaking these activities.The first two recommendations can beused regularly when preparing and implementingproblem-solving lessons; in contrast,the panel does not think recommendations3 through 5 must be used in every lesson.Instead, teachers should choose the recommendationsthat align best with their goals fora given lesson and its problems. For example,there are occasions when visual representationsare not used as part of problem-solvinginstruction, such as when students solve anequation by considering which values of thevariable will make both sides equal.Use of researchThe evidence used to create and supportthe recommendations in this practice guideranges from rigorous experimental studies toexpert reviews of practices and strategies inmathematics education; however, the evidenceratings are based solely on high-quality groupdesignstudies (randomized controlled trialsand rigorous quasi-experimental designs) thatmeet What Works Clearinghouse (WWC) standards.Single-case design studies that meetWWC pilot standards for well-designed singlecasedesign research are also described, butdo not affect the level of evidence rating. Thepanel paid particular attention to a set of highqualityexperimental and quasi-experimentalstudies that meets the WWC criteria, includingboth national and international studies of strategiesfor teaching problem solving to studentsin grades 4 through 8. 17 This body of researchincluded strategies and curricular materialsdeveloped by researchers or ones commonlybeing used by teachers in classrooms. Thepanel also considered studies recommendedby panel members that included students ingrades 3 and 9.( 8 )

Introduction (continued)Studies of problem-solving interventions inthe past 20 years have yielded few causalevaluations of the effectiveness of the varietyof approaches used in the field. For example,as much as the panel believes that teachingstudents to persist in solving challengingproblems is important to solving math problems,it could not find causal research thatisolated the impact of persistence. The panelalso wanted to include studies of teachersusing their students’ culture to enhanceproblem-solving instruction; however, panelistscould not find enough research that metWWC standards and isolated this practice.The panel was able to include suggestions forteaching the language of mathematics and foradapting problems so that contexts are morerelevant to students—but these suggestionsare supported by limited evidence.The research base for this guide was identifiedthrough a comprehensive search forstudies evaluating instructional practices forimproving students’ mathematical problemsolving. An initial search for literature related toproblem-solving instruction in the past 20 yearsyielded more than 3,700 citations; the panelrecommended an additional 69 citations. Peerreviewers suggested several additional studies.Of these studies, only 38 met the causal validitystandards of the WWC and were related tothe panel’s recommendations. 18The supporting research provides a stronglevel of evidence for two of the recommendations,a moderate level of evidence foranother two of the recommendations, anda minimal level of evidence for one recommendation.Despite the varying levels ofevidence, the panel believes all five recommendationsare important for promotingeffective problem-solving skills in students.The panel further believes that even thoughthe level of evidence for Recommendation1 is minimal, the practice holds promise forimproving students’ mathematical problemsolving. Very few studies examine the effectsof teacher planning on student achievement;therefore, few studies are available to supportthis recommendation. Nonetheless, thepanel believes that the practice of intentionallypreparing problem-solving lessons canlead to improvement in students’ problemsolvingabilities.Table 2 shows each recommendation andthe strength of the evidence that supportsit as determined by the panel. Following therecommendations and suggestions for carryingout the recommendations, Appendix Dpresents more information on the researchevidence that supports each recommendation.It also provides details on how studieswere assessed as showing positive, negative,or no effects.Table 2. Recommendations and corresponding levels of evidenceLevels of EvidenceRecommendationStrongEvidenceModerateEvidenceMinimalEvidence1. Prepare problems and use them in whole-class instruction. 2. Assist students in monitoring and reflecting on the problemsolvingprocess.3. Teach students how to use visual representations. 4. Expose students to multiple problem-solving strategies. 5. Help students recognize and articulate mathematical conceptsand notation.( 9 )

Recommendation 1Prepare problems and use them in whole-class instruction.A sustained focus on problem solving is often missing in mathematics instruction, in large partdue to other curricular demands placed on teachers and students. 19 Daily math instructionis usually limited to learning and practicing new skills, leaving problem-solving time toindependent seatwork or homework assignments. 20 The panel believes instruction in problemsolving must be an integral part of each curricular unit, with time allocated for problemsolvingactivities with the whole class. In this recommendation, the panel provides guidance forthoughtful preparation of problem-solving lessons. Teachers are encouraged to use a varietyof problems intentionally and to ensure that students have the language and mathematicalcontent knowledge necessary to solve the problems.Summary of evidence: Minimal EvidenceFew studies directly tested the suggestionsof this recommendation, leading the panelto assign a level of evidence rating for thisrecommendation of “minimal evidence.”Although the panel believes teacher planningshould incorporate both routine and nonroutineproblems, no studies meeting WWCstandards directly examined this issue.One study found that students performedbetter when teacher planning consideredstudents’ mathematical content weaknessesand understanding of language and context. 21However, this intervention included additionalinstructional components that mayhave caused the positive results. Similarly,while another study found that incorporatingfamiliar contexts into instruction can improveproblem-solving skills, the interventionincluded other instructional components thatmay have caused these positive results. 22On a related issue, a few well-designed studiesdid find that students who have practiced withword problems involving contexts (people,places, and things) they like and know do( 10 )

Recommendation 1 (continued)better on subsequent word-problem teststhan do students who have practiced withgeneric contexts. 23 These achievement gainsoccurred when computer programs were usedto personalize problem contexts for individualstudents or when contexts were based on thecommon preferences of student groups. 24The panel identified three suggestions forhow to carry out this recommendation.How to carry out the recommendation1. Include both routine and non-routine problems in problem-solving activities.Definitions of routine and non-routineproblemsRoutine problems can be solved usingmethods familiar to students 25 by replicatingpreviously learned methods in astep-by-step fashion. 26Non-routine problems are problems for“which there is not a predictable, wellrehearsedapproach or pathway explicitlysuggested by the task, task instructions,or a worked-out example.” 27Teachers must consider students’ previousexperience with problem solvingto determine which problems will beroutine or non-routine for them. A seeminglyroutine problem for older studentsor adults may present surprising challengesfor younger students or thosewho struggle in mathematics.Teachers should choose routine problems iftheir goal is to help students understand themeaning of an operation or mathematicalidea. Collections of routine problems canhelp students understand what terms such asmultiplication and place value mean, and howthey are used in everyday life. 28 For example,6 ÷ 2/3 may become more meaningful whenincorporated into this word problem: “Howmany 2/3-foot long blocks of wood can a carpentercut from a 6-foot long board?” (See Example1 for additional sample routine problems.)Routine problems are not only the one- andtwo-step problems students have solvedmany times, but they can also be cognitivelydemanding multistep problems that requiremethods familiar to students. For example,see problem 3 in Example 1. The typical challengeof these problems is working throughthe multiple steps, rather than determiningnew ways to solve the problem. Thus, sixthorseventh-grade students who have beentaught the relevant geometry (e.g., types oftriangles, area formulas for triangles) andbasic features of coordinate graphs should beable to solve this problem by following a setof steps that may not differ significantly fromwhat they may have already been shown orpracticed. In this instance, it would be reasonablefor an average student to draw a linebetween (0,4) and (0,10), observe that thelength of this distance is 6, and then use thisinformation in the area formula A = ½ × b × h.If the student substitutes appropriately forthe variables in the formula, the next set is tosolve for the height: 12 = ½ × 6 × h. This stepyields a height of 4, and a student could thenanswer the question with either (4,0) or (4,10).The routine nature of the problem solvingis based on the fact that this problem mayrequire little or no transfer from previouslymodeled or worked problems.( 11 )

Recommendation 1 (continued)Example 1. Sample routine problemsExample 2. Sample non-routine problems1. Carlos has a cake recipe that callsfor 23/4 cups of flour. He wants tomake the recipe 3 times. How muchflour does he need?This problem is likely routine for a studentwho has studied and practiced multiplicationwith mixed numbers.2. Solve 2y + 15 = 29This problem is likely routine for a studentwho has studied and practiced solvinglinear equations with one variable.3. Two vertices of a right triangle arelocated at (0,4) and (0,10). The area ofthe triangle is 12 square units. Find apoint that works as the third vertex.This problem is likely routine for a studentwho has studied and practiced determiningthe area of triangles and graphing incoordinate planes.When the primary goal of instruction is todevelop students’ ability to think strategically,teachers should choose non-routine problemsthat force students to apply what they havelearned in a new way. 29 Example 2 providessamples of problems that are non-routine formost students. For students who have not hadfocused instruction on geometry problems likeproblem 1 in Example 2, the task presents aseries of challenges. Much more time needs tobe spent interpreting the problem and determiningwhat information is relevant, as well ashow it should be used. Time also needs to bespent determining or inferring if information notpresented in the problem is relevant (e.g., whatthe measures of the supplementary angles arein the problem, how this information might beused to solve the problem). All of these featuresincrease the cognitive demands associatedwith solving this problem and make it nonroutine.Finally, competent students who solvethis problem would also spend additional timedouble-checking the correctness of the solution.1. Determine angle x without measuring.Explain your reasoning.parallel155°110°This problem is likely non-routine for astudent who has only studied simple geometryproblems involving parallel linesand a transversal.2. There are 20 people in a room. Everybodyhigh-fives with everybodyelse. How many high-fives occurred?This problem is likely non-routine forstudents in beginning algebra.3. Solve for the variables a through f inthe equations below, using the digitsfrom 0 through 5. Every digit shouldbe used only once. A variable has thesame value everywhere it occurs, andno other variable will have that value.a + a + a = a 2b + c = bd × e = da – e = bb 2 = dd + e = fThe problem is likely non-routine for astudent who has not solved equations byreasoning about which values can makean equation true.4. In a leap year, what day and time areexactly in the middle of the year?This problem is likely non-routine for astudent who has not studied problems inwhich quantities are subdivided into unequalgroups.x( 12 )

Recommendation 1 (continued)2. Ensure that students will understand the problem by addressing issues studentsmight encounter with the problem’s context or language.Given the diversity of students in today’sclassrooms, the problems teachers select forlesson plans may include contexts or vocabularythat are unfamiliar to some. 30 Thesestudents may then have difficulty solvingthe problems for reasons unrelated to thestudents’ understanding of concepts or theirability to compute answers. 31 This is a particularlysignificant issue for English languagelearners and for students with disabilities.The goal of ensuring that students understandthe language and context of problemsis not to make problems less challenging.Instead, it is to allow students to focus on themathematics in the problem, rather than onthe need to learn new background knowledgeor language. The overarching point is thatstudents should understand the problem andits context before attempting to solve it. 32Here are some ways teachers can prepare lessonsto ensure student understanding:• Choose problems with familiar contexts.Students can often solve problemsmore successfully when they are familiarwith the subject matter presented. 33• Clarify unfamiliar language and contexts.Identify the language and contextsthat need to be clarified in order forstudents to understand the problem. 34Teachers should think about students’experiences as they make these determinations.For example, an island in thekitchen, a yacht, the Iditarod dog sled race,a Laundromat, or sentence structures suchas “if…, then…” 35 might need clarification,depending on students’ backgrounds.Example 3 shows how one 5th-gradeteacher identified vocabulary and contextualterms that needed to be clarified forher students.• Reword problems, drawing uponstudents’ experiences. Reword problemsso they are familiar to students bydrawing upon students’ personal, familial,and community experiences. 36 Teacherscan replace unfamiliar names, objects, andactivities that appear in the problem withfamiliar ones, so as to create a problemsolvingcontext that is more aligned withstudents’ experiences. In this way, theproblem becomes more culturally relevantand easier for students to respond to. Forexample, soccer may be more familiar tosome students than hockey, apples morefamiliar than soybeans, and the guitarmore familiar than the oboe. By rewordinga problem to meet the needs of theirstudents, teachers may not only increasecomprehension levels, but also motivatetheir students to become more involvedin the problem-solving activity. 37 Pleasenote that teachers need not always rewordthe problems themselves. They can discusswith their students how to make theproblems more familiar, interesting, andcomprehensible. For instance, teachers canask students questions such as “Can wechange Ms. Inoye to the name of a personyou know?” or “This problem says we haveto measure the shag carpet in the livingroom. Can we use other words or just theword carpet in this problem to make iteasier to understand?”( 13 )

Recommendation 1 (continued)Example 3. One teacher’s efforts to clarify vocabulary and contextMary, a 5th-grade teacher, identified the following vocabulary, contextual terms, and content forclarification, based on the background of her students.Example Problem Vocabulary ContextIn a factory, 54,650 parts weremade. When they were tested,4% were found to be defective.How many parts were working?Students need to understandthe term defective as being theopposite of working and thesymbol % as percent to correctlysolve the problem.What is a factory?What does parts mean in thiscontext?At a used-car dealership, a carwas priced at $7,000. The salespersonthen offered a discountof $350.What percent discount, appliedto the original price, gives theoffered price?Students need to know whatoffered and original price meanto understand the goal of theproblem, and they need toknow what discount and percentdiscount mean to understandwhat mathematical operatorsto use.What is a used-car dealership?Example 4. What mathematical content is needed to solve the problem?ProblempostrailingSarah Sanchez is planningto build a corral on herranch for her two horses.She wants to build thecorral in the shape of arectangle. Here is a drawingof one side of the corral,and as you can see,this side is 20 yards wide.10 feet20 yards (i.e., 60 feet)It will take 480 yards of railing to build the corral based on Sarah’s plan.1. What will be the perimeter of the corral?2. What will be the area of the corral?3. Can you show a way that Sarah can use the same amount of railing and build acorral with a bigger area for her horses?Mathematical content needed for solution• Addition, subtraction, multiplication and division.• Opposite sides of rectangles are equal.• The perimeter of a shape can stay the same, but its area can change.( 14 )

Recommendation 1 (continued)3. Consider students’ knowledge of mathematical content when planning lessons.As teachers plan problem-solving instruction,they should identify the mathematical contentneeded to solve the included problems. It isimportant to remember that problems thatalign with the current unit often draw onskills taught in prior units or grade levels. Inthe problems listed in Example 3, studentsneed to understand (1) what percent means(i.e., that 4% is 4 out of 100 or 4/100 or 0.04),and (2) how to calculate the value of a percentof a quantity or percent of a change. Strugglingstudents are likely to benefit from aquick review of the relevant skills needed tounderstand and solve the problem, especiallyif the mathematical content has not beendiscussed recently or if a non-routine problemis presented. 38 A brief review of skills learnedearlier also helps students see how thisknowledge applies to challenging problems. 39For example, teachers may need to reviewthe difference between fractions and ratiosbefore students are asked to solve ratio andproportion problems. Similarly, before solvingthe fourth-grade problem shown in Example4, students should know and be able to applyseveral facts about area and perimeter.Mathematical language in problems may alsoneed to be reviewed with students. For example,before presenting students with a problemsuch as the one in Example 5, a teacher mightneed to clarify the terms in the problem.Example 5. Mathematical languageto review with studentsProblemTwo vertices of a triangle are located at(0,4) and (0,10). The area of the triangleis 12 square units. What are all possiblepositions for the third vertex?Mathematical language that needsto be reviewed• vertices• area square units• triangle• vertexPotential roadblocks and solutionsRoadblock 1.1. Teachers are having troublefinding problems for the problem-solvingactivities.Suggested Approach. Textbooks usuallyinclude both routine and non-routine problems,but teachers often have a hard timefinding non-routine problems that fit theirlesson’s goals. In addition to the class text,teachers may need to use ancillary materials,such as books on problem solving andhandouts from professional-developmentactivities. Teachers also can ask colleaguesfor additional problem-solving activities orwork on teams with other teachers or withinstructional leaders using lesson study toprepare materials for problem-solving instruction.Teachers also can search the Internet forexamples. Helpful online resources includeIlluminations from the National Council ofTeachers of Mathematics, problems of theweek from the Math Forum at Drexel University,and practice problems from high-qualitystandardized tests such as the state assessments,the Trends in International Mathematicsand Science Study (TIMSS), the Programmefor International Student Assessment (PISA),and the Scholastic Assessment Test (SAT). 40Roadblock 1.2. Teachers have no time toadd problem-solving activities to their mathematicsinstruction.Suggested Approach. The panel believesthat including problem-solving activitiesthroughout each unit is essential. To maketime during instruction, teachers shouldconsider balancing the number of problems( 15 )

Recommendation 1 (continued)students are required to solve during seatworkactivities with worked examples studentscan simply study. Worked examplescould benefit student learning and decreasethe time necessary to learn a new skill. 41 Formore information on how to use workedexamples as a part of problem-solvinginstruction, see Recommendations 2 and 4of this practice guide or Recommendation2 of the Organizing Instruction and Study toImprove Student Learning practice guide. 42Overview of Recommendation 2 in theOrganizing Instruction and Study toImprove Student Learning practice guide 431. Teachers should give studentsassignments that provide alreadyworked solutions for students tostudy, interleaved with problemsfor them to solve on their own.2. As students develop greater problemsolvingabilities, teachers can reducethe number of worked problems theyprovide and increase the number ofproblems that students should solveindependently.Roadblock 1.3. Teachers are not sure whichwords to teach when teaching problem solving.Suggested Approach. The panel believesacademic language, including the languageused in mathematics, should be taughtexplicitly so that all students understandwhat is being asked in a problem and howthe problem should be solved. Identifying thelanguage used in a problem-solving task canguide lesson planning. Based on the scopeand sequence of the curricular material, mathcoaches or specialists can provide a list ofacademic words and phrases (e.g., addition,not greater than) 44 that are essential forteaching a given unit. The list can also focuson the language that will be necessary forstudents to know as they progress to thenext grade level. Teachers can work withcolleagues to solve problems and identifywords students need to understand to solvethe problem. They also can look for importantacademic terms and vocabulary in theclass textbook or the mathematics standardsfor the state.( 16 )

Recommendation 2Assist students in monitoring and reflecting on theproblem-solving process.Students learn mathematics and solve problems better when they monitor their thinking andproblem-solving steps as they solve problems. 45 Monitoring and reflecting during problemsolving helps students think about what they are doing and why they are doing it, evaluatethe steps they are taking to solve the problem, and connect new concepts to what theyalready know. The more students reflect on their problem-solving processes, the better theirmathematical reasoning—and their ability to apply this reasoning to new situations—will be. 46In this recommendation, the panel suggests that teachers help students learn to monitor andreflect on their thought process when they solve math problems. While the ultimate goal is forstudents to monitor and reflect on their own while solving a problem, teachers may need tosupport students when a new activity or concept is introduced. For instance, a teacher mayprovide prompts and use them to model monitoring and reflecting as the teacher solves aproblem aloud. In addition, a teacher can use what students say as a basis for helping thestudents improve their monitoring and reflecting. Teachers can use students’ ideas to helpstudents understand the problem-solving process.Summary of evidence: Strong EvidenceSeveral studies with diverse student samples directly tested this recommendation and consistentlyfound positive effects. As a result, the panel determined there was strong evidence to supportthis recommendation. 47( 17 )

Recommendation 2 (continued)The relevant studies examined students’mathematics achievement in different contentareas, including numbers and operations,data analysis and probability, algebra, andgeometry. Two studies found that providingstudents with a task list that identifiedspecific steps to solving problems resultedin better student achievement. 48 Two additionalstudies found that a self-questioningchecklist improved achievement, 49 and in onestudy, this effect persisted for at least fourmonths after instruction ended; 50 however,both studies included additional instructionalcomponents (visual aids and multiple-strategyinstruction) that may have produced the positiveresults. Similarly, five studies found thatstudent performance improved when teachersmodeled a self-questioning process and thenasked students to practice it. 51The panel identified three suggestions forhow to carry out this recommendation.How to carry out the recommendation1. Provide students with a list of prompts to help them monitor and reflect during theproblem-solving process.The prompts that teachers provide can eitherbe questions that students should ask andanswer as they solve problems (see Example 6)or task lists that help students complete stepsin the problem-solving process (see Example7). 52 The questions teachers provide shouldrequire students to think through the problemsolvingprocess, similar to the way in whichtask lists guide students through the process.Select a reasonable number of prompts, ratherthan an exhaustive list, as too many promptsmay slow down the problem-solving processor be ignored. Ensure that the prompts helpstudents evaluate their work at each stage ofthe problem-solving process, from initiallyreading and understanding the problem, todetermining a way to solve the problem, andthen to evaluating the appropriateness of thesolution given the facts in the problem. 53Encourage students to explain and justify theirresponse to each prompt, either orally 54 or inwriting. 55 Students can use the prompts whenworking independently, in small groups, 56 oreven when solving problems at a computer. 57When working in small groups, students cantake turns asking and answering questionsor reading each action aloud and respondingto it. As they share in small groups, studentsserve as models for others in their group,allowing all the students to learn from oneanother. Teachers may wish to post promptson the board, include them on worksheets, 58or list them on index cards for students. 59When students first use the prompts, theymay need help. Teachers can participate inthe questioning or refer to tasks in the tasklist when students work in small groups orduring whole-group discussions. If, for example,a student solves a problem incorrectly,ask him what questions he should have askedhimself to help him reason out loud, ratherthan providing him with the correct answer. 60Alternatively, provide the correct answer, butask the student to explain why it is right andwhy his original answer is not. 61 As studentsbecome more comfortable with their reasoningabilities and take greater responsibilityfor monitoring and reflecting during problemsolving, teachers can gradually withdraw theamount of support they provide. 62( 18 )

Recommendation 2 (continued)Example 6. Sample question listExample 7. Sample task list 63• What is the story in this problemabout?• What is the problem asking?• What do I know about the problem sofar ? What information is given to me?How can this help me?• Which information in the problemis relevant?• In what way is this problem similarto problems I have previously solved?• What are the various ways I mightapproach the problem?• Is my approach working? If I am stuck, isthere another way I can think about solvingthis problem?• Does the solution make sense? HowcanI verify the solution?• Why did these steps work or not work?• What would I do differently next time?• Identify the givens and goals ofthe problem.• Identify the problem type.• Recall similar problems to help solvethe current problem.• Use a visual to represent and solvethe problem.• Solve the problem.• Check the solution.Note: These are examples of the kinds of questionsthat a teacher can use as prompts tohelp students monitor and reflect during theproblem-solving process. Select those that areapplicable for your students, or formulate newquestions to help guide your students.2. Model how to monitor and reflect on the problem-solving process.Model how to monitor and reflect while solvinga problem using the prompts given tostudents. 64 This can be done when introducinga problem-solving activity or a new conceptto the whole class or as students workindependently or in small groups. 65 Say aloudnot only the response to each prompt, butalso the reasons why each step was taken.Alternatively, say which step was taken, butask students to explain why this would work.Make sure to use a prompt at each stage inthe problem-solving process, for example,when first reading the problem, whenattempting a strategy to solve the problem,and after solving the problem.Example 8 describes one teacher’s experiencewith modeling how to monitor and reflectusing questions. It illustrates the importanceof persistence if the student fails to understandthe problem or the appropriate methodto employ for solving it.( 19 )

Recommendation 2 (continued)Example 8. One way to model monitoring and reflecting using questionsProblemLast year was unusually dry in Colorado. Denver usually gets 60 inches of snow per year. Vail,which is up in the mountains, usually gets 350 inches of snow. Both places had 10 inches ofsnow less than the year before. Kara and Ramon live in Colorado and heard the weather report.Kara thinks the decline for Denver and Vail is the same. Ramon thinks that when youcompare the two cities, the decline is different. Explain how both people are correct.SolutionTEACHER: First, I ask myself, “What is this story about, and what do I need to findout?” I see that the problem has given me the usual amount of snowfall and the change insnowfall for each place, and that it talks about a decline in both cities. I know what declinemeans: "a change that makes something less." Now I wonder how the decline in snowfallfor Denver and Vail can be the same for Kara and different for Ramon. I know that a declineof 10 inches in both cities is the same, so I guess that’s what makes Kara correct. How isRamon thinking about the problem?I ask myself, “Have I ever seen a problem like this before?” As I think back to theassignments we had last week, I remember seeing a problem that asked us to calculatethe discount on a $20 item that was on sale for $15. I remember we had to determinethe percent change. This could be a similar kind of problem. This might be the wayRamon is thinking about the problem.Before I go on, I ask myself, “What steps should I take to solve this problem?”It looks like I need to divide the change amount by the original amount to find thepercent change in snowfall for both Denver and Vail.Denver: 10 ÷ 60 = 0.166 or 16.67% or 17% when we round it to the nearest whole numberVail: 10 ÷ 350 = 0.029 or 2.9% or 3% when we round it to the nearest whole numberSo the percent decrease in snow for Denver was much greater (17%) than for Vail (3%).Now I see what Ramon is saying! It’s different because the percent decrease for Vail ismuch smaller than it is for Denver.Finally, I ask myself, “Does this answer make sense when I reread the problem?”Kara’s answer makes sense because both cities did have a decline of 10 inches of snow.Ramon is also right because the percent decrease for Vail is much smaller than it is forDenver. Now, both of their answers make sense to me.( 20 )

Recommendation 2 (continued)3. Use student thinking about a problem to develop students’ ability to monitorand reflect.The panel believes that, by building onstudents’ ideas, teachers can help studentsclarify and refine the way they monitor andreflect as they solve a problem. Teachers canhelp students verbalize other ways to thinkabout the problem. The teacher-student dialoguecan include guided questioning to helpstudents clarify and refine their thinking andto help them establish a method for monitoringand reflecting that makes sense to them(see Example 9). This is helpful for studentswho dislike working with teacher-providedprompts or who are having difficulty understandingand using these prompts.Example 9. Using student ideas to clarify and refine the monitoring and reflecting processProblemFind a set of five different numbers whose average is 15.SolutionTEACHER: Jennie, what did you try?STUDENT: I’m guessing and checking. I tried 6, 12, 16, 20, 25 and they didn’t work. Theaverage is like 17.8 or something decimal like that.TEACHER: That’s pretty close to 15, though. Why’d you try those numbers?STUDENT: What do you mean?TEACHER: I mean, where was the target, 15, in your planning? It seems like it was in yourthinking somewhere. If I were choosing five numbers, I might go with 16, 17, 20, 25, 28.STUDENT: But they wouldn’t work—you can tell right away.TEACHER: How?STUDENT: Because they are all bigger than 15.TEACHER: So?STUDENT: Well, then the average is going to be bigger than 15.TEACHER: Okay. That’s what I meant when I asked “Where was 15 in your planning?”You knew they couldn’t all be bigger than 15. Or they couldn’t all be smaller either?STUDENT: Right.TEACHER: Okay, so keep the target, 15, in your planning. How do you think five numberswhose average is 15 relate to the number 15?STUDENT: Well, some have to be bigger and some smaller. I guess that is why I tried thefive numbers I did.TEACHER: That’s what I guess, too. So, the next step is to think about how much biggersome have to be, and how much smaller the others have to be. Okay?STUDENT: Yeah.TEACHER: So, use that thinking to come up with five numbers that work.( 21 )

Recommendation 2 (continued)Potential roadblocks and solutionsRoadblock 2.1. Students don’t want tomonitor and reflect; they just want to solvethe problem.Suggested Approach. The panel believesthat students need to develop the habit ofmonitoring and reflecting throughout theproblem-solving process, from setting up theproblem to evaluating whether their solutionis accurate. Ideally, whenever students solveproblems, they should practice monitoringand reflecting on their problem-solvingprocess. Acknowledge that simply solvinga problem may seem easier, but encouragestudents to incorporate monitoring andreflecting into their process every time theysolve a problem. Inform students that doingso will help them understand and solveproblems better, as well as help them conveytheir strategies to classmates. 66 Explain thatexpert problem solvers learn from unsuccessfulexplorations and conjectures by reflectingon why they were unsuccessful.Roadblock 2.2. Teachers are unclearon how to think aloud while solving a nonroutineproblem.Roadblock 2.3. Students take too muchtime to monitor and reflect on the problemsolvingprocess.Suggested Approach. It is likely that whenstudents initially learn the tasks of monitoringand reflecting, they will be slow in using theprompts. However, after a bit of practice, theyare likely to become more efficient at usingthe prompts.Roadblock 2.4. When students reflect onproblems they have already solved, they resortto using methods from those problems ratherthan adapting their efforts to the new problembefore them.Suggested Approach. While studentsshould consider whether they have seena similar problem before, sometimes theyoverdo it and simply solve the problem usingsimilar methods, rather than using methodsthat will work for the problem they are solving.To help students overcome this, try askingthem to explain why the solution methodworked for the previous problem and whatcomponents of it may or may not be usefulfor the new problem.Suggested Approach. Prepare ahead oftime using the list of prompts given to students.Outline responses to the prompts inadvance of the lesson. It might also help toanticipate how students would think aboutthe prompts as they solved the problem. Acolleague or math coach can help teachersthink through the prompts and the problemsolvingprocess if they get stuck.( 22 )

Recommendation 3Teach students how to use visual representations.A major task for any student engaged in problem solving is to translate the quantitativeinformation in a problem into a symbolic equation—an arithmetic/algebraic statement—necessary for solving the problem. Visual representations help students solve problems bylinking the relationships between quantities in the problem with the mathematical operationsneeded to solve the problem. Students who learn to visually represent the mathematicalinformation in problems prior to writing an equation are more effective at problem solving. 67Visual representations include tables, graphs, number lines, and diagrams such as stripdiagrams, percent bars, and schematic diagrams. Example 10 provides a brief explanationof how a few types of visual representations can be used to solve problems. 68 In thepanel’s opinion, teachers should consistently teach students to use a few types of visualrepresentations rather than overwhelming them with many examples. In this recommendation,the panel offers suggestions for selecting appropriate visual representations to teach andmethods for teaching students how to represent the problem using a visual representation.Definitions of strip diagrams, percent bars, and schematic diagramsStrip diagrams use rectangles to represent quantities presented in the problem.Percent bars are strip diagrams in which each rectangle represents a part of 100 in the problem.Schematic diagrams demonstrate the relative sizes and relationships between quantitiesin the problem.( 23 )

Recommendation 3 (continued)Example 10. Sample table, strip diagram, percent bar, and schematic diagramProblemCheese costs $2.39 per pound. Find the cost of 0.75 pounds of cheese. 69Sample table(a)(b)Cost of CheesePounds of Cheese2.39 1? 0.752.39 2.39 1x 2.39 0.75This table depicts the relationship between the weight of cheese and its cost. Every poundof cheese will cost $2.39, and this relationship can be used to determine the cost of 0.75pounds of cheese by using the rule “times 2.39,” which can be stated in an equation asx = 0.75 × 2.39.ProblemEva spent 2/5 of the money she had on a coat and then spent 1/3 of what was left on asweater. She had $150 remaining. How much did she start with?Sample strip diagramThis strip diagram depicts the money Eva spent on a coat and a sweater. It shows how theamount of money she originally had is divided into 5 equal parts and that 2 of the 5 partsare unspent. The problem states that the unspent amount equals $150. Several strategiescan then be employed to make use of this information in an equation, such as 2/5 × x = 150,to determine the original amount.(continued)( 24 )

Recommendation 3 (continued)Example 10. Sample table, strip diagram, percent bar, and schematic diagram (continued)ProblemDuring a sale, prices were marked down by 20%. The sale price of an item was $84. Whatwas the original price of the item before the discount? 70Sample percent barOriginal Decrease Final Amount100%x20%y80%$84These percent bars depict the relative values of the original, decrease, and final amountsas 100:20:80, which can be reduced to 5:1:4. The relationship between the original andfinal amount (5:4) can be used in an algebraic equation, such as x /84 = 5/4, to determinethe original amount when the final amount is $84.ProblemJohn recently participated in a 5-mile run. He usually runs 2 miles in 30 minutes. Becauseof an ankle injury, John had to take a 5-minute break after every mile. At each break hedrank 4 ounces of water. How much time did it take him to complete the 5-mile run?Sample schematic diagramStartEnd15 15 15 15 155 5 5 5This schematic diagram depicts the amount of time John needed to run 5 miles wheneach mile took him 15 minutes to run and he took a 5-minute break after every mile.The total time (x) it took him to complete the run is equal to the total number of minutesin this diagram, or x = (5 × 15) + (4 × 5).( 25 )

Recommendation 3 (continued)Summary of evidence: Strong EvidenceThe panel determined there is strong evidencesupporting this recommendation because sixstudies with middle school student samplesconsistently found that using visual representationsimproved achievement. 71 Both generaleducation students and students with learningdisabilities performed better when taught touse visual representations 72 such as identifyingand mapping relevant information onto schematicdiagrams. 73In four of the six studies, students were taughtto differentiate between types of math problemsand then to implement an appropriatediagram for the relevant type. 74 An additionalstudy involving an alternative problem-solvingapproach integrated with visual representationsalso resulted in higher achievement. 75Finally, one study showed that if teachers helpstudents design, develop, and improve theirown visual representations, student achievementimproves more than if students simplyuse teacher- or textbook-developed visuals. 76The panel identified three suggestions forhow to carry out this recommendation.How to carry out the recommendation1. Select visual representations that are appropriate for students and the problemsthey are solving.Sometimes curricular materials suggest usingmore than one visual representation for aparticular type of problem. Teachers shouldnot feel obligated to use all of these; instead,teachers should select the visual representationthat will work best for students and should useit consistently for similar problems. 77For example, suppose a teacher introduced aratio or proportion problem using a diagramthat students found helpful in arriving at theequation needed to solve the problem. Theteacher should continue using this same diagramwhen students work on additional ratioor proportion problems. Remember, studentsmay need time to practice using visual representationsand may struggle before achievingsuccess with them. 78 If, after a reasonableamount of time and further instruction, therepresentation still is not working for individualstudents or the whole class, considerteaching another type of visual representationto the students in the future. Teachers canalso consult a mathematics coach, other mathteachers, or practitioner publications to identifymore appropriate visual representations.Also keep in mind that certain visual representationsare better suited for certaintypes of problems. 79 For instance, schematicdiagrams work well with ratio and proportionproblems, percent bars are appropriatefor percent problems, and strip diagrams aresuited for comparison and fraction problems.2. Use think-alouds and discussions to teach students how to represent problems visually.When teaching a new visual representation ortype of problem, demonstrate how to representthe problem using the representation.Teachers should think aloud about the decisionsthey make as they connect the problemto the representation. 80 Thinking aloud ismore than just the teacher telling studentswhat he or she is doing. It also involves theteacher expressing his or her thoughts ashe or she approaches the problem, includingwhat decisions he or she is making andwhy he or she is making each decision (seeExample 11).Teachers should explain how they identifiedthe type of problem—such as proportion,ratio, or percent—based on mathematicalideas in the problem and why they think a( 26 )

Recommendation 3 (continued)certain visual representation is most appropriate.For example, proportion problemsdescribe equality between two ratios or ratesthat allows students to think about how bothare the same. Teachers should be careful notto focus on surface features such as storycontext. 81 In Example 12, the story contextsare different, but the problem type is thesame. Students who cannot articulate thetype of problem may struggle to solve it,even if they have the basic math skills. 82Demonstrate to students how to representthe information in a problem visually. 83Teach students to identify what informationis relevant or critical to solving the problem.Often, problems include information that isirrelevant or unnecessary. For instance, inExample 11, students need to determine howmany red roses are in Monica’s bouquet. Thenumber of pink roses in Bianca’s bouquetis irrelevant; students need only focus onMonica’s bouquet.Example 11. One way of thinking aloud 84ProblemMonica and Bianca went to a flower shop to buy some roses. Bianca bought a bouquetwith 5 pink roses. Monica bought a bouquet with two dozen roses, some red and someyellow. She has 3 red roses in her bouquet for every 5 yellow roses. How many red rosesare in Monica’s bouquet?SolutionTEACHER: I know this is a ratio problem because two quantities are being compared: thenumber of red roses and the number of yellow roses. I also know the ratio of the two quantities.There are 3 red roses for every 5 yellow roses. This tells me I can find more of eachkind of rose by multiplying.I reread the problem and determine that I need to solve the question posed in the last sentence:“How many red roses are in Monica’s bouquet?” Because the question is about Monica,perhaps I don’t need the information about Bianca. The third sentence says there are twodozen red and yellow roses. I know that makes 24 red and yellow roses, but I still don’t knowhow many red roses there are. I know there are 3 red roses for every 5 yellow roses. I think Ineed to figure out how many red roses there are in the 24 red and yellow roses.Let me reread the problem… That’s correct. I need to find out how many red roses thereare in the bouquet of 24 red and yellow roses. The next part of the problem talks about theratio of red roses to red and yellow roses. I can draw a diagram that helps me understandthe problem. I’ve done this before with ratio problems. These kinds of diagrams show therelationship between the two quantities in the ratio.ComparedRatio valueBase(continued)( 27 )

Recommendation 3 (continued)Example 11. One way of thinking aloud (continued)TEACHER: I write the quantities and units from the problem and an x for what must besolved in the diagram. First, I am going to write the ratio of red roses to yellow roseshere in the circle. This is a part-to-whole comparison—but how can I find the whole inthe part-to-whole ratio when we only know the part-to-part ratio (the number of red rosesto the number of yellow roses)?I have to figure out what the ratio is of red roses to red and yellow roses when the problemonly tells about the ratio of red roses to yellow roses, which is 3:5. So if there are3 red roses for every 5 yellow roses, then the total number of units for red and yellowroses is 8. For every 3 units of red roses, there are 8 units of red and yellow roses, whichgives me the ratio 3:8. I will write that in the diagram as the ratio value of red roses tored and yellow roses. There are two dozen red and yellow roses, and that equals 24 redand yellow roses, which is the base quantity. I need to find out how many red roses (x)there are in 24 red and yellow roses.Comparedxred roses24red and yellowrosesBase38Ratio valueI can now translate the information in this diagram to an equation like this:x red roses24 red-and-yellow roses38Then, I need to solve for x.x 324 8x324( 24) 24( 8 )72x8x 9( 28 )

Recommendation 3 (continued)Example 12. Variations in storycontexts for a proportion problemProblemSolve 2/10 = x /30Context 1Sara draws 2 trees for every 10 animals.How many trees will she need to draw ifshe has 30 animals?Context 2Sarah creates a tiled wall using 2 blacktiles for every 10 white tiles. If she has30 white tiles, how many black tiles willshe need?Promote discussions by asking students guidingquestions as they practice representingproblems visually. 85 For example, teacherscan ask the following questions:• What kind of problem is this? How doyou know?• What is the relevant information in thisproblem? Why is this information relevant?• Which visual representation did you usewhen you solved this type of problemlast time?• What would you do next? Why?Encourage students to discuss similaritiesand differences among the various visualsthey have learned or used. When studentsuse their own visual representations to solvea problem correctly, teachers can emphasizenoteworthy aspects of the representationsand ask the students to share their visualrepresentations with the class. They also mayask the students to explain how and why theyused a particular representation to solve aproblem. By sharing their work, students aremodeling how to use visual representationsfor other students, allowing them to learnfrom one another.3. Show students how to convert the visually represented information intomathematical notation.After representing the relevant informationin a problem visually, demonstrate how eachpart of the visual representation can be translatedinto mathematical notation. 86 Studentsmust see how each quantity and relationshipin the visual representation corresponds toquantities and relationships in the equation.Sometimes, the translation from representationto equation is as simple as rewritingthe quantities and relationships without theboxes, circles, or arrows in the visual representation(see Example 11). Other times,correspondence between the visual representationand the equation is not as simple,and teachers must illustrate the connectionsexplicitly. For example, a teacher using thetable in Example 10 should demonstratehow to represent the cost of 0.75 poundsof cheese as x and the rule “times 2.39” withthe correct notation in an equation.Potential roadblocks and solutionsRoadblock 3.1. Students do not capture therelevant details in the problem or includeunnecessary details when representing aproblem visually.Suggested Approach. Often, when representinga problem visually, students do notcapture the relevant details and relationshipsin the problem, or include unnecessary details,such as the dots on a ladybug or a picketfence around a house. Consequently, such( 29 )

Recommendation 3 (continued)representations do not correctly depict or helpidentify the mathematical nature of the problem.Teachers can help students improve theirrepresentations by building upon students’thinking. They can ask guiding questionsthat will help students clarify and refine theirrepresentations. Once a student has revisedhis or her representation, teachers can ask himor her to explain what was missing from therepresentation and why the representation didnot work initially. Teachers should be sure topoint out specific aspects of the representationthat the student did correctly. This will encouragestudents to keep trying.If necessary, teachers also can demonstratehow to alter the representation to representand solve the problem correctly, using students’representations as a springboard forrefinement. 87 Teachers can show studentshow their diagrams can be modified torepresent the relevant information withoutthe unnecessary details. It is important tohelp students improve their representations,because students who represent irrelevantinformation could be less effective problemsolvers than those who draw diagrams withrelevant details from the problem. 88 Teachersalso can help by explaining what thedifference is between relevant and irrelevantdetails and how a visual representation cancapture relevant details and relationships.They also should emphasize that a diagram’sgoal is to illustrate the relationships that areimportant for solving the problem.Example 13. Diagrams with relevantand irrelevant details 89ProblemThere are 4 adults and 2 children whoneed to cross the river. A small boat isavailable that can hold either 1 adult or1 or 2 small children. Everyone can rowthe boat. How many one-way trips doesit take for all of them to cross the river?Using a representation without relevantdetails and with irrelevant detailsto represent the problem:Using a schematic diagram withrelevant details and without irrelevantdetails to represent the problem:Consider the river-crossing problem detailedin Example 13. The first representation is asimple narrative description of the story thatdepicts the boat and river, and the adults andchildren waiting to cross the river. The schematicdiagram, on the other hand, outlinesthe sequence of trips and boat occupants thatare needed to take 4 adults and 2 childrenacross the river in a small boat. The schematicdiagram better represents the relevantinformation in the problem and is helpful inarriving at the symbolic equation.( 30 )

Recommendation 3 (continued)Roadblock 3.2. The class text does not usevisual representations.Suggested Approach. Teachers can askcolleagues or math coaches for relevant visualrepresentations, or they can develop someon their own. Teachers also can look throughprofessional-development materials theymay have collected or search the Internet formore examples. Using an overhead projector,a document reader, or an interactive whiteboard,teachers can incorporate these visualsinto their lessons.( 31 )

Recommendation 4Expose students to multiple problem-solving strategies.Problem solvers who know how to use multiple strategies to solve problems may be moresuccessful. 90 When regularly exposed to problems that require different strategies, studentslearn different ways to solve problems. As a result, students become more efficient in selectingappropriate ways to solve problems 91 and can approach and solve math problems with greaterease and flexibility. 92In this recommendation, the panel suggests ways to teach students that problems can besolved in more than one way and that they should learn to choose between strategies basedupon their ease and efficiency. The panel recommends that teachers instruct students in avariety of strategies for solving problems and provide opportunities for students to use, share,and compare the strategies. Teachers should consider emphasizing the clarity and efficiency ofdifferent strategies when they are compared as part of a classroom discussion.Summary of evidence: Moderate EvidenceEight studies found positive effects of teachingand encouraging multiple problem-solvingstrategies, although in some of these studiesthe effects were not discernible across alltypes of outcomes. Three additional studiesinvolving students with limited or no knowledgeof algebraic methods found negativeeffects for some algebra outcomes—two ofthese studies also found positive effects forsome outcomes. Consequently, the paneldetermined that there is moderate evidenceto support this recommendation.Six of the seven studies that included proceduralflexibility outcomes found that exposingstudents to multiple problem-solving strategies( 32 )

Recommendation 4 (continued)improved students’ procedural flexibility—their ability to solve problems in different waysusing appropriate strategies. 93 However, theestimated effects of teaching multiple strategieson students’ ability to solve problemscorrectly (procedural knowledge) and awarenessof mathematical concepts (conceptualknowledge) were inconsistent. 94Three studies found that when studentswere instructed in using multiple strategiesto solve the same problem, proceduralknowledge improved; however, all of thesestudies included additional instructionalcomponents (checklists and visual aids) thatmay have produced the positive results. 95Another study with an eight-minute strategydemonstration found no discernible effects. 96Providing students with worked examplesexplicitly comparing multiple-solution strategieshad positive effects on students’ proceduralflexibility in three of the four studiesthat examined this intervention; however,the three studies found inconsistent effectson students’ procedural knowledge andconceptual knowledge. 97 The fourth studyproviding students with worked examplesfound that the effects varied by baselineskills—the intervention had a negative effecton procedural knowledge, conceptual knowledge,and procedural flexibility for studentswho did not attempt algebra reasoning on apretest, but no discernible effect for studentswho had attempted algebraic reasoning onthe pretest (correctly or incorrectly). 98 Finally,when students attempted to solve problemsusing multiple strategies and then sharedand compared their strategies, their abilityto solve problems did improve. 99 Two additionalstudies involving students with no orminimal algebra knowledge found that askingstudents to re-solve an algebra problem usinga different method had negative effects onprocedural knowledge measures and positiveeffects on procedural flexibility. 100The panel identified three suggestions forhow to carry out this recommendation.How to carry out the recommendation1. Provide instruction in multiple strategies.Teach students multiple strategies for solvingproblems. 101 These can be problem-specific 102or general strategies for use with more thanone type of problem. 103 For instance, in Example14, a teacher shows his or her students twoways to solve the same problem.Example 14. Two ways to solve the same problemProblemRamona’s furniture store has a choice of 3-legged stools and 4-legged stools. There arefive more 3-legged stools than 4-legged stools. When you count the legs of the stools,there are exactly 29 legs. How many 3-legged and 4-legged stools are there in the store?Solution 1: Guess and check4 × 4 legs = 16 legs 9 × 3 legs = 27 legs total = 43 legs3 × 4 legs = 12 legs 8 × 3 legs = 24 legs total = 36 legs2 × 4 legs = 8 legs 7 × 3 legs = 21 legs total = 29 legsTEACHER: This works; the total equals 29, and with two 4-legged stools and seven3-legged stools, there are five more 3-legged stools than 4-legged stools.( 33 )(continued)

Recommendation 4 (continued)Example 14. Two ways to solve the same problem (continued)Solution 2TEACHER: Let’s see if we can solve this problem logically. The problem says that thereare five more 3-legged stools than 4-legged stools. It also says that there are 29 legs altogether.If there are five more 3-legged stools, there has to be at least one 4-legged stoolin the first place. Let’s see what that looks like.stoolstotal legs 4 × 1 = 4 +3 × 6 = 184 + 18 = 22TEACHER: We can add a stool to each group, and there will still be a difference of five stools.stoolstotal legs 4 × 2 = 8 +3 × 7 = 218 + 21 = 29TEACHER: I think this works. We have a total of 29 legs, and there are still five more3-legged stools than 4-legged stools. We solved this by thinking about it logically. Weknew there was at least one 4-legged stool, and there were six 3-legged stools. Then weadded to both sides so we always had a difference of five stools.When teaching multiple strategies, periodicallyemploy unsuccessful strategies and demonstratechanging to an alternate strategyto show students that problems are notalways solved easily the first time and thatsometimes problem solvers need to try morethan one strategy to solve a problem. Thiswill help students develop the persistence thepanel believes is necessary to complete challengingand non-routine problems.( 34 )

Recommendation 4 (continued)2. Provide opportunities for students to compare multiple strategies in worked examples.Worked examples allow for quick, efficientcomparisons between strategies. 104 Successfulstudents should be able to compare thesimilarities and differences among multiplestrategies. 105 They may reap more benefitsfrom comparing multiple strategies in workedexamples when they work with a partnerinstead of alone 106 and when they can activelyparticipate in the learning process. 107 Teachersshould provide opportunities for students towork together and should use worked examplesto facilitate the comparison of strategies.Teachers can use worked examples to facilitatecomparison of strategies with interestingcontrasts and not just minor differences. Anadded benefit of comparing strategies is thatcertain examples allow for concepts to behighlighted. The strategies used in Example15 allow for a discussion of treating (y + 1) asa composite variable.Teachers should present worked examplesside-by-side on the same page, rather than ontwo separate pages, to facilitate more effectivecomparisons (see Example 15). 108 Teachersalso should include specific questions tofacilitate student discussions. 109 For example,teachers can ask these questions:• How are the strategies similar? How arethey different?• Which method would you use to solvethe problem? Why would you choose thisapproach?• The problem was solved differently,but the answer is the same. How is thatpossible?Ask students to respond to the questions firstverbally and then in writing. 110 See Example15 for an example of how these questions canbe tailored to a specific problem.Example 15. A comparison of strategies 111Mandy’s solutionErica’s solution5(y + 1) = 3(y + 1) + 85y + 5 = 3y + 3 + 85y + 5 = 3y + 112y + 5 = 112y = 6y = 3DistributeCombineSubtract on bothSubtract on bothDivide on both5(y + 1) = 3(y + 1) + 82(y + 1) = 8 Subtract on bothy + 1 = 4Divide on bothy = 3Subtract on bothTEACHER: Mandy and Erica solved the problem differently, but they got the same answer.Why? Would you choose to use Mandy’s way or Erica’s way? Why?Worked examples, of course, should be usedalongside opportunities for students to solveproblems on their own. For instance, teacherscan provide worked examples for students tostudy with every couple of practice problems.Students who receive worked examples earlyon in a lesson may experience better learningoutcomes with less effort than students whoonly receive problems to solve. 112( 35 )

Recommendation 4 (continued)3. Ask students to generate and share multiple strategies for solving a problem.Encourage students to generate multiplestrategies as they work independently 113 orin small groups. 114 In Example 16, studentsshare their strategies in a small group. Provideopportunities for students to share their strategieswith the class. When students see the variousmethods employed by others, they learn toapproach and solve problems in different ways.Example 16. How students solved a problem during a small-group activityProblem 115Find the area of this pentagon.Solution strategiesAli and Maria each worked on this problem individually.After 20 minutes in a small-group activity, they talked toeach other about how they approached the problem.ALI: The pentagon is slanted, so first I looked for figures forwhich I knew how to compute the area. Look what I found:six right triangles inside; and they get rid of the slantedparts, so what’s left are rectangles.Then, I noticed that the right triangles are really three pairsof congruent right triangles. So together, the ones marked1 have an area of 2 × 3 = 6 square units. The ones marked2 combine for an area of 3 × 1 = 3 square units. The onesmarked 3 also combine for an area of 3 square units.What’s left inside is a 2-by-3 rectangle, with an area of 6square units; a 1-by-4 rectangle, with an area of 4 square units;and a 1-by-3 rectangle, with an area of 3 square units.So, the area of the pentagon is 6 + 3 + 3 + 6 + 4 + 3 = 25 square units.MARIA: You looked inside the pentagon, but I lookedoutside to deal with the slanted parts. I saw that I couldput the pentagon inside a rectangle. I colored in the pentagonand figured if I could subtract the area of the whitespace from the area of the rectangle, I’d have the area of thepentagon.I know the area of the rectangle is 6 × 7 = 42 square units.I saw that the white space was really five right triangles plus alittle rectangle. The little rectangle is 1 by 2 units, so its areais 1 × 2 = 2 square units. Then, I figured the areas of the five right triangles: 1.5 square units,1.5 square units, 3 square units, 3 square units, and 6 square units. So, the area of the whitespace is 2 + 1.5 + 1.5 + 3 + 3 + 6 = 17 square units.To get the area of the pentagon, I subtracted 17 from 42 and, like you, I got 25 square unitsfor the area of the pentagon.( 36 )

Recommendation 4 (continued)Rather than randomly calling on students toshare their strategies, select students purposefullybased on the strategies they haveused to solve the problem. For instance,teachers can call on students who solvedthe problem using a strategy other than theone demonstrated, or a strategy not used byothers in the class. Example 17 illustrates howtwo students might share their strategies forsolving a fractions problem.Example 17. Two students share their strategies for solving a fractions problemProblem 116What fraction of the whole rectangle is green?Solution strategiesSTUDENT 1: If I think of it as what’s to the left of the middle plus what’s to the right ofthe middle, then I see that on the left, the green part is 1/3 of the area; so that is 1/3 of ½of the entire rectangle. On the right, the green part is ½ of the area; so it is ½ of ½ ofthe entire rectangle. This information tells me that the green part is(1/3 × ½) + (½ × ½) = 1/6 + 1/4 = 2/12 + 3/12 = 5/12 of the entire rectangle.STUDENT 2: I see that the original green part and the part I’ve colored black have thesame area. So the original green part is ½ of the parts black and green, or ½ of 5/6 ofthe entire rectangle. This tells me that the green and black part is ½ × 5/6 = 5/12 of theentire rectangle.Ensure that students present not only theirstrategy but also an explanation for usingthe strategy. Engage students in a discussionabout the specifics of their strategy by questioningthem as they explain their thinking. Forexample, after the first student in Example17 said, “I see that on the left, the green partis 1/3,” the teacher could ask, “How do youknow?” The teacher also could ask, “How didyou know that the green part on the rightis half the area?” For the second student inExample 17, the teacher could ask, “How didyou know that the green part is the same asthe area colored black?”( 37 )

Recommendation 4 (continued)Potential roadblocks and solutionsRoadblock 4.1. Teachers don’t have enoughtime in their math class for students to presentand discuss multiple strategies.Suggested Approach. Teachers will needto purposefully select three to four strategiesfor sharing and discussing. To reduce the timestudents take to present their work to the class,ask them to use personal whiteboards or chartpaper, or to bring their work to the documentreader, so that they do not have to take timerewriting their strategies on the board. Teacherscan document the strategies that different studentsuse during independent or small-groupwork and summarize or display them for thewhole class. This is likely to take less time thanhaving students take turns sharing their ownstrategies. Teachers can also have students doa problem-solving task at the beginning of theschool day as they settle in or at the beginningof the math class as a warm-up activity anddevote 5–10 minutes to sharing and discussion.Roadblock 4.2. Not all students are willingto share their strategies.Suggested Approach. Fear of presentingwrong answers may limit the willingness ofsome students to share their strategies. It isimportant to create an environment in whichstudents feel supported and encouraged toshare, whether their strategy is correct or not.Teachers should emphasize that most problemscan be solved using a variety of strategies andthat each student may present a way to solvethe problem that other students in the classhad not thought to use. The panel believes thatstudents will be more willing to explain theirstrategies once they notice that sharing helpsthem understand and solve problems betterand gives students an opportunity to teachtheir classmates. Make sharing a regular part ofmathematics instruction. Point out the benefitsof sharing, such as how it enables students toteach their peers new ways to approach problemsand to learn potentially quicker, easier, andmore effective strategies from their peers.Roadblock 4.3. Some students struggle tolearn multiple strategies.Suggested Approach. For some studentswho lack the prerequisite knowledge or do notremember the required skills, exposure to multiplestrategies may be challenging. 117 For example,some students who do not remember their multiplicationand division facts will have difficultydetermining which strategy to use based on thenumbers in a problem. Teachers may need to askthese students to take a minute and write downtheir facts before asking them to solve problems.Teachers also may need to change the problemto include numbers that allow these students tofocus on the problem solving rather than on thearithmetic. For example, teachers could changethe number 89.5 to 90. Some students may alsoget confused when multiple strategies for solvinga problem are presented. Solving a problem oneway, erasing it, and then solving it in another waycan be difficult for students to comprehend ifno opportunity is given to compare the methodsside-by-side. 118 Students will also benefitfrom slowing down and having some time toget familiar with one strategy before beingexposed to another or comparing it to a previouslylearned strategy. 119Roadblock 4.4. Some of the strategiesstudents share are not clear or do not makesense to the class.Suggested Approach. Students may havetrouble articulating their strategies clearly sothey make sense to the class. As students solvea problem, circulate among them and ask themto privately explain how they are working it.This will give teachers a clearer idea of the students’strategies. Then, when a student shareswith the class, teachers will be better equippedto clarify the student’s thinking by asking guidingquestions or by carefully rewording what astudent shares. 120 Another way to help studentsshare is by asking another student to restatewhat the student has said. 121 Be careful not toevaluate what students share (e.g., “Yes, that isthe right answer.” or “No, that’s not correct.”)as students explain their thinking. Instead, askstudents questions to help them explain theirreasoning out loud (see Recommendation 2).( 38 )

Recommendation 5Help students recognize and articulate mathematicalconcepts and notation.Mathematical concepts and notation provide students with familiar structures for organizinginformation in a problem; they also help students understand and think about the problem. 122When students have a strong understanding of mathematical concepts and notation, they arebetter able to recognize the mathematics present in the problem, extend their understandingto new problems, 123 and explore various options when solving problems. 124 Building fromstudents’ prior knowledge of mathematical concepts and notation is instrumental in developingproblem-solving skills.In this recommendation, the panel suggests that teachers explain relevant concepts andnotation in the context of a problem-solving activity, prompt students to describe how workedexamples are solved using mathematically valid explanations, and introduce algebraic notationsystematically. The panel believes these actions will help students develop new ways ofreasoning, which in turn will help students successfully solve new mathematical challenges.Summary of evidence: Moderate EvidenceThree studies directly support two suggestionsfor implementing this recommendation,and although the findings for the other suggestionare inconsistent, the panel believesthere is moderate evidence supporting thisrecommendation. 125The first suggestion for implementing thisrecommendation, explaining relevant conceptsand notation, was supported by a study findingthat student achievement improved whenteachers discussed math problems conceptually(without numbers) and then representedthem visually. 126 Three studies examined thesecond suggestion for implementing this( 39 )

Recommendation 5 (continued)recommendation, providing students withworked examples and asking them to explainthe process used to solve a problem; twostudies reported positive effects, and onestudy reported no discernible effects. 127 Finally,two studies supported the third suggestionfor implementing this recommendation,algebraic notation. The first study found thatproviding students with concrete intermediatearithmetic problems before asking themto understand the algebraic notation for adifferent problem significantly improvedachievement. 128 The second study found thathaving students practice symbolic algebraicproblems (substituting one expression intoanother) improved performance on two-stepword problems more than practicing withone-step word problems. 129The panel identified three suggestions forhow to carry out this recommendation.How to carry out the recommendation1. Describe relevant mathematical concepts and notation, and relate them to theproblem-solving activity.Students tend to enter school with informal,personally constructed ways of making senseof math. 130 Students often use this informalunderstanding to solve problems. 131 Teacherscan turn problem-solving activities into learningopportunities by connecting students’intuitive understanding to formal mathematicalconcepts and notation. 132Teachers can watch and listen for opportunitiesto call attention to the mathematicalconcepts and notation that students use asthey solve problems. For example, if teachersnotice students informally using the commutativeproperty to solve a problem, teacherscan explain this concept, ask students if itwill always work in similar situations, anddescribe the property’s usefulness to theclass. 133 If teachers see students talking aboutformal mathematical notation in an informalway, teachers can connect students’ informallanguage to the formal notation or symbol,pointing out that there is often more thanone mathematically correct way to state it(e.g., “12 take away 3”, “12 minus 3”, and “12less 3” are all equal to 9). The teacher in Example18 uses this technique to help her studentbetter understand number theory. Thisexample illustrates how students can bettergrasp formal mathematical concepts whenteachers interpret the informal ideas andconcepts students use to solve problems. 134( 40 )

Recommendation 5 (continued)Example 18. Students’ intuitive understandingof formal mathematical conceptsProblemIs the sum of two consecutive numbersalways odd?SolutionSTUDENT: Yes.TEACHER: How do you know?STUDENT: Well, suppose you take anumber, like 5. The next number is 6.For 5, I can write five lines, like this:| | | | |For 6, I can write five lines and onemore line next to it, like this:| | | | | |Then, I can count all of them, and I get11 lines.See? It’s an odd number.TEACHER: When you say, “It’s an oddnumber,” you mean the sum of the twoconsecutive numbers is odd. So, can youdo that with any whole number, like n?What would the next number be?STUDENT: It would be n + 1.TEACHER: So, can you line them up likeyou did for 5 and 6?STUDENT: You mean, like this?nn + 1TEACHER: Right. So, what does that tellyou about the sum of n and n + 1?STUDENT: It’s 2 n’s and 1, so it’s odd.TEACHER: Very good. The sum, whichis n + n + 1 = 2n + 1, is always going tobe odd.Sometimes, teachers may need to draw attentionto mathematical ideas and concepts bydirectly instructing students in them beforeengaging the students in problem solving. 135For instance, some middle graders think ofarea as a product of numbers, rather than ameasure of an array of units. 136 Faced with aproblem such as the problem in Example 16that asks them to find the area of a pentagon,students might measure the lengths ofeach side of the pentagon and calculate areaby multiplying two or more of the numberstogether. To offset this mistaken approach,their teacher might explicitly define area as“the measure of the space inside a figure.”2. Ask students to explain each stepused to solve a problem in a workedexample.Routinely provide students with opportunitiesto explain the process used to solve a problemin a worked example and to explain whythe steps worked. 137 Students will develop abetter understanding of mathematical conceptswhen they are asked to explain thesteps used to solve a problem in a workedexample, and this understanding will helpthem solve problems successfully. 138 Studyingworked examples could help accelerate learningand improve problem solving. 139Use small-group activities to encouragestudents to discuss the process used to solvea problem in a worked example and thereasoning for each step. Alternatively, ask onestudent to repeat another student’s explanationand to then state whether he or sheagrees and why.Initially, students may not provide mathematicallyvalid explanations. For example,students may restate the correct steps butfail to provide good reasons or justifications.The panel believes that teachers should askstudents probing questions to help themarticulate mathematically valid explanations.Mathematically valid explanations are factuallyand mathematically correct, logical,( 41 )

Recommendation 5 (continued)thorough, and convincing. 140 See Example 19for sample student explanations that are notmathematically valid. Teachers also mightneed to provide students with examples ofmathematically valid explanations or helpreword their partially correct explanations.Example 19 also illustrates how teacherquestioning can help students better organizetheir thoughts when their explanation is notmathematically valid.Example 19. Sample student explanations: How mathematically valid are they?ProblemAre 2/3 and 8/12 equivalent fractions? Why or why not?An explanation that is not mathematicallyvalidSTUDENT: To find an equivalent fraction,whatever we do to the top of 2/3, we mustdo to the bottom.This description is not mathematically validbecause, using this rule, we might think wecould add the same number to the numeratorand denominator of a fraction and obtainan equivalent fraction. However, thatis not true. For example, if we add 1 to boththe numerator and denominator of 2/3, weget (2 + 1)/(3 + 1), which is 3/4. 3/4 and 2/3are not equivalent. Below is an explanationof how teacher questioning can clarify students’explanations and reasoning.TEACHER: What do you mean?STUDENT: It just works when you multiply.TEACHER: What happens when you multiplyin this step?STUDENT: The fraction stays…the same.TEACHER: That’s right. When you multiplya numerator and denominator by the samenumber, you get an equivalent fraction.Why is that?STUDENT: Before there were 3 parts, butwe made 4 times as many parts, so nowthere are 12 parts.TEACHER: Right, you had 2 parts of awhole of 3. Multiplying both by 4 gives you8 parts of a whole of 12. That is the samepart-whole relationship—the same fraction,as you said. Here’s another way to look at it:when you multiply the fraction by 4/4, youare multiplying it by a fraction equivalentto 1; this is the identify property of multiplication,and it means when you multiplyanything by 1, the number stays the same.A correct description, but still nota complete explanationSTUDENT: Whatever we multiply the top of2/3 by, we must also multiply the bottom by.This rule is correct, but it doesn’t explainwhy we get an equivalent fraction this way.A mathematically valid explanationSTUDENT: You can get an equivalent fractionby multiplying the numerator and denominatorof 2/3 by the same number. If wemultiply the numerator and denominatorby 4, we get 8/12.If I divide each of the third pieces in thefirst fraction strip into 4 equal parts, thenthat makes 4 times as many parts thatare shaded and 4 times as many parts inall. The 2 shaded parts become 2 × 4 = 8smaller parts and the 3 total parts become3 × 4 = 12 total smaller parts. So theshaded amount is 2/3 of the strip, but it isalso 8/12 of the strip.This explanation is correct, complete,and logical.( 42 )

Recommendation 5 (continued)3. Help students make sense of algebraic notation.Understanding the symbolic notation used inalgebra takes time. The panel suggests thatteachers introduce it early and at a moderatepace, allowing students enough time tobecome familiar and comfortable with it.Teachers should engage students in activitiesthat facilitate understanding and better or correctuse of symbols. 141 For instance, teacherscan provide familiar arithmetic problems asan intermediate step before asking studentsto translate a problem into an algebraic equation(see Example 20). 142 Simple arithmeticExample 20. How to make sense of algebraicnotation: Solve a problem arithmeticallybefore solving it algebraically 143ProblemA plumbing company charges $42 perhour, plus $35 for the service call.SolutionTEACHER: How much would you payfor a 3-hour service call?STUDENT: $42 × 3 + $35 = $161 fora 3-hour service call.TEACHER: What will the bill be for4.5 hours?STUDENT: $42 × 4.5 + $35 = $224 for4.5 hours.TEACHER: Now, I’d like you to assigna variable for the number of hours thecompany works and write an expressionfor the number of dollars required.STUDENT: I’ll choose h to represent thenumber of hours the company works.problems draw on students’ prior math experience,so the problems are more meaningful.By revisiting their earlier knowledge of simplearithmetic, students can connect what theyalready know (arithmetic) with new information(algebra).Teachers also can ask students to explain eachcomponent of an algebraic equation by havingthem link the equation back to the problemthey are solving (see Example 21). 144 This willhelp students understand how componentsin the equation and elements of the problemcorrespond, what each component of theequation means, and how useful algebra isfor solving the problem.Example 21. How to make sense of algebraicnotation: Link components of the equationto the problem 145ProblemJoseph earned money for selling 7 CDsand his old headphones. He sold theheadphones for $10. He made $40.31.How much did he sell each CD for?SolutionThe teacher writes this equation:10 + 7x = 40.31TEACHER: If x represents the numberof dollars he sold the CD for, what doesthe 7x represent in the problem? Whatdoes the 10 represent? What does the40.31 represent? What does the 10 + 7xrepresent?42h + 35 = $ requiredTEACHER: What is the algebraic equationfor the number of hours worked ifthe bill comes out to $140?STUDENT: 42h + 35 = 140( 43 )

Recommendation 5 (continued)Potential roadblocks and solutionsRoadblock 5.1. Students’ explanationsare too short and lack clarity and detail.It is difficult for teachers to identify whichmathematical concepts they are using.Suggested Approach. Many studentsin the United States are not given regularopportunities to explain why steps to solveproblems work; 146 without this experienceto draw from, students who are suddenlygiven sharing opportunities often providequick explanations that lack detail and clarity.They do not yet know how to explain theproblem-solving process, which informationto present, or how much detail to provide.To anticipate which mathematical conceptsstudents might use to solve the problem,teachers may need to prepare for each lessonby solving the problem themselves.To help students explain their thoughts inmore detail, teachers can ask them specificquestions about how a problem was solvedand how they thought about the problem.Teachers can also have students create a“reason sheet” of mathematical rules (e.g.,“the identity property of multiplication forfractions—multiplying a fraction by 1 keepsthe same value,” “2/2 and 3/3 are fractions equalto 1,” and so on). Students should only includea few key rules, as too many may make reasoningmore difficult. Some helpful ones mayappear in the student textbook as definitions,properties, rules, laws, or theorems. Ask studentsto use reasons from their reason sheetwhen composing explanations.Roadblock 5.2. Students may be confusedby mathematical notations used in algebraicequations.Suggested Approach. Students may havedifficulty interpreting mathematical notationsused as variables in algebraic equations whenthe notations relate to items in the problem.Students may misinterpret the notations aslabels for items in the problem (e.g., c standsfor cookies). Teachers should encouragestudents to use arbitrary variables, such asnon-mnemonic English letters (x and y) orGreek letters (Φ and Ω). 147 Arbitrary variablescan facilitate student understanding of theabstract role that variables play in representingquantities.( 44 )

ConclusionThe recommendations in this practice guide include research-based practices for teaching mathematicalproblem solving. Below is an example of how these recommendations can be incorporatedinto a lesson using a four-step process. It is important to consider all of the recommendedpractices when conducting a lesson from start to finish, even though you may not be able to applyall four steps for every problem-solving lesson.1. Plan by preparing appropriate problemsand using them in whole-class instruction(Recommendation 1) and by selectingvisual representations that are appropriatefor students and the problemsthey are solving (Recommendation 3).• Include a variety of problem-solvingactivities (Recommendation 1). Teachersshould ask themselves, “Is the purpose ofthese problems to help students understanda key concept or operation? To help studentslearn to persist with solving difficult problems?To use a particular strategy? To use avisual representation?” Teachers should selectproblems that fit the goal of the lesson.• Ensure that students will understandthe problem by addressing issues studentsmight encounter with the problem’scontext or language; considerstudents’ knowledge of mathematicalcontent when planning the lesson(Recommendation 1). Some problems mayhave complex or unusual vocabulary,depend on specific background knowledge,or reference ideas that are unfamiliar tosome students. In some cases, it may benecessary to modify problems based onthe learning or cultural needs of the students.In other cases, teachers may need toexplain the context or language to studentsbefore asking them to solve the problem.• Select visual representations thatare appropriate for students and theproblems they are solving (Recommendation3). If the lesson will include a visualrepresentation, teachers should considerthe types of problems they plan to presentand should select appropriate visualrepresentations. Also, teachers should considerstudents’ past experience with visualrepresentations to determine whether anew representation should be presented.2. Depending on the content or goal of thelesson, teach students how to use visualrepresentations (Recommendation 3),expose students to multiple problemsolvingstrategies (Recommendation 4),and/or help students recognize andarticulate mathematical concepts andnotation (Recommendation 5).If visual representations are featured in thelesson:• Use think-alouds and discussionsto teach students how to representproblems visually (Recommendation 3).Teachers should clarify the type of problemand how to determine which informationin the problem is relevant. Teachersshould then talk students through howto map the relevant information onto anappropriate visual representation and leada discussion to compare representations.Allow students to share their work sothat students can learn from others in theclass. When students use their own representations,have them explain why andhow they are using the representation. Ifany refinement is needed, build upon thestudent representation.• Show students how to convert thevisually represented information intomathematical notation (Recommendation3). Teachers should demonstrate howeach quantity and relationship in the visualrepresentation corresponds to componentsof the equation.If the goal is to teach students multiplestrategies:• Provide instruction in multiple strategies(Recommendation 4). Teachers shouldteach students a variety of ways to solveproblems. These can be generic strategies( 45 )

Conclusion (continued)that work for a wide range of problems orspecific strategies that work for a giventype of problem. Teachers can model theuse of strategies by thinking aloud aboutwhy they selected the particular strategyand how they would work a problem.• Ask students to generate multiplestrategies for solving a problem(Recommendation 4). Encourage studentsto generate strategies of their own as theywork through the problems they are given.If the goal is to help students recognize andarticulate mathematical concepts and notation:• Describe relevant mathematical conceptsand notation, and relate themto the problem-solving activity(Recommendation 5). Teachers should lookfor opportunities to explain the formalmathematical concepts and notation usedin the problem-solving activity.• Help students make sense of algebraicnotation (Recommendation 5). One wayto do this is to introduce similar arithmeticproblems before algebraic problems torevisit students’ earlier mathematical understanding.Another way is to help studentsexplain how the algebraic notation representseach component in the problem.3. Assist students in monitoring andreflecting on the problem-solvingprocess (Recommendation 2).• Provide students with a list of promptsto help them monitor and reflect duringthe problem-solving process(Recommendation 2). It may be necessaryto assist students as they begin to workwith prompts. Assisting means more thansimply telling students what to do next.Teachers can assist by asking studentsguiding questions to help them learn to usethe prompts when they solve problems.• Model how to monitor and reflect onthe problem-solving process (Recommendation2). Teachers can state a promptin front of the class and describe how theyused it to solve a problem. This will helpstudents see how prompts or items froma task list can be used to solve problems.Teacher modeling is a useful way to showhow people think as they solve problems.• Use student thinking about a problemto develop students’ ability to monitorand reflect on their thought processwhile solving a problem (Recommendation2). Teachers can ask guiding questionsto help students verbalize what they coulddo to improve their monitoring and reflectionduring the problem-solving process.4. Conduct discussions to help studentsrecognize and articulate mathematicalconcepts and notation (Recommendation5) and to expose students tomultiple problem-solving strategies(Recommendation 4).• Ask students to explain each step usedto solve a problem (Recommendation 5).Debriefing each step allows teachers toconnect the problem-solving activity to relevantmathematical concepts and notation.• Provide opportunities for studentsto compare multiple strategies inworked examples; ask students togenerate, share, and compare multiplestrategies for solving a problem(Recommendation 4). This approach allowsstudents to hear multiple problem-solvingstrategies, which is particularly beneficialif the strategies are more advanced thanthose used by most students in the classroom.It also affords students the chanceto present and discuss their strategies,thereby building their confidence levels.( 46 )

Appendix APostscript from the Institute of Education SciencesWhat is a practice guide?The Institute of Education Sciences (IES) publishes practice guides to share rigorous evidence andexpert guidance on addressing education-related challenges not solved with a single program,policy, or practice. Each practice guide’s panel of experts develops recommendations for a coherentapproach to a multifaceted problem. Each recommendation is explicitly connected to supportingevidence. Using standards for rigorous research, the supporting evidence is rated to reflect howwell the research demonstrates that the recommended practices are effective. Strong evidencemeans positive findings are demonstrated in multiple well-designed, well-executed studies, leavinglittle or no doubt that the positive effects are caused by the recommended practice. Moderateevidence means well-designed studies show positive impacts, but some questions remain aboutwhether the findings can be generalized or whether the studies definitively show the practice iseffective. Minimal evidence means data may suggest a relationship between the recommendedpractice and positive outcomes, but research has not demonstrated that the practice is the causeof positive outcomes. (See Table 1 for more details on levels of evidence.)How are practice guides developed?To produce a practice guide, IES first selects atopic. Topic selection is informed by inquiriesand requests to the What Works ClearinghouseHelp Desk, formal surveys of practitioners,and a limited literature search of the topic’sresearch base. Next, IES recruits a panel chairwho has a national reputation and expertisein the topic. The chair, working with IES, thenselects panelists to co-author the guide. Panelistsare selected based on their expertise in thetopic area and the belief that they can worktogether to develop relevant, evidence-basedrecommendations. IES recommends that thepanel include at least one practitioner withrelevant experience.The panel receives a general template fordeveloping a practice guide, as well as examplesof published practice guides. Panelistsidentify the most important research withrespect to their recommendations and augmentthis literature with a search of recentpublications to ensure that supporting evidenceis current. The search is designed tofind all studies assessing the effectiveness ofa particular program or practice. These studiesare then reviewed against the What WorksClearinghouse (WWC) standards by certifiedreviewers who rate each effectiveness study.WWC staff assist the panelists in compilingand summarizing the research and in producingthe practice guide.IES practice guides are then subjected torigorous external peer review. This reviewis done independently of the IES staff thatsupported the development of the guide. Acritical task of the peer reviewers of a practiceguide is to determine whether the evidencecited in support of particular recommendationsis up-to-date and that studies of similaror better quality that point in a different directionhave not been overlooked. Peer reviewersalso evaluate whether the level of evidencecategory assigned to each recommendation isappropriate. After the review, a practice guideis revised to meet any concerns of the reviewersand to gain the approval of the standardsand review staff at IES.A final note about IES practice guidesIn policy and other arenas, expert panels typicallytry to build a consensus, forging statementsthat all its members endorse. Practiceguides do more than find common ground;they create a list of actionable recommendations.Where research clearly shows whichpractices are effective, the panelists use thisevidence to guide their recommendations.However, in some cases, research does notprovide a clear indication of what works, and( 47 )

Appendix A (continued)panelists’ interpretation of the existing (butincomplete) evidence plays an important rolein guiding the recommendations. As a result,it is possible that two teams of recognizedexperts working independently to produce apractice guide on the same topic would cometo very different conclusions. Those who usethe guides should recognize that the recommendationsrepresent, in effect, the adviceof consultants. However, the advice mightbe better than what a school or district couldobtain on its own. Practice guide authorsare nationally recognized experts who collectivelyendorse the recommendations,justify their choices with supporting evidence,and face rigorous independent peer reviewof their conclusions. Schools and districtswould likely not find such a comprehensiveapproach when seeking the advice of individualconsultants.Institute of Education Sciences( 48 )

Appendix BAbout the AuthorsPanelJohn Woodward, Ph.D., is distinguishedprofessor and dean of the School of Educationat the University of Puget Sound in Tacoma,Washington. His research experience includesinstructional interventions for at-risk students,technology-based instruction in math andscience, and professional development inmathematics for elementary and middle gradeteachers. Dr. Woodward has conducted mathresearch and developed curricular materialsfor the U.S. Department of Education, Officeof Special Education Programs, for almost 20years. One of his recent projects was a collaborativefive-year research program thatexamined methods for helping students withdisabilities succeed in standards-based instructionin grades 4 through 8. Dr. Woodward isco-author of TransMath, which is designed tomeet the needs of academically low-achievingstudents in the late elementary and middleschool grades. His current professional interestsinclude models of professional developmentin mathematics that involve face-to-faceand technology-based interactions.Sybilla Beckmann, Ph.D., is Josiah MeigsDistinguished Teaching Professor of Mathematicsat the University of Georgia. Dr. Beckmann’scurrent interests are the mathematicaleducation of teachers and mathematicscontent for students at all levels, but especiallypre-K through the middle grades. Shedeveloped three mathematics content coursesfor prospective elementary school teachersat the University of Georgia and wrote a bookfor these courses, Mathematics for ElementaryTeachers. Dr. Beckmann was a member of thewriting team of the National Council of Teachersof Mathematics’ Curriculum Focal Pointsfor Pre-K Through Grade 8 Mathematics; wasa member of the National Research Council’sCommittee on Early Childhood Mathematicsand co-author of its report, MathematicsLearning in Early Childhood: Paths TowardExcellence and Equity; has helped developseveral state mathematics standards; and wasa member of the mathematics writing teamfor the Common Core State Standards Initiative.She has won several teaching awards,including the General Sandy Beaver TeachingProfessorship, awarded by the College of Artsand Sciences at the University of Georgia.Mark Driscoll, Ph.D., is a managing directorat the Education Development Center(EDC). He has directed project work in teacherenhancement, teacher leadership, and materialsdevelopment. Two recent examples ofprofessional development materials fromthese projects include the Fostering AlgebraicThinking Toolkit and Fostering GeometricThinking in the Middle Grades. Before joiningEDC, Dr. Driscoll developed a love of teachingmathematics at an alternative high schoolin St. Louis, Missouri, where he also grew toappreciate high-quality professional developmentfor teachers and administrators. He hasserved as a co-chair of the National Council ofTeachers of Mathematics (NCTM) Task Forceon Reaching All Students with Mathematicsand a member of the writing team for NCTM’sAssessment Standards for School Mathematics.From 2003 to 2007, Dr. Driscoll served aseditor of Mathematics Education Leadership,the journal of the National Council of Supervisorsof Mathematics (NCSM). In April 2010, hereceived the NSCM Ross Taylor/Glenn GilbertNational Leadership Award.Megan Franke, Ph.D., is chair of the educationdepartment and a professor in the UrbanSchooling Division at the Graduate Schoolof Education and Information Studies at theUniversity of California, Los Angeles (UCLA).Dr. Franke’s work focuses on understandingand supporting teacher learning throughprofessional development, particularly withinelementary mathematics. She works withUCLA’s Teacher Education Program, as well asthe Principal Leadership Institute, CaliforniaSubject Matter Projects, university researchers,community administrators, and teachers,to create and study learning opportunities forstudents in Los Angeles’s lowest-performingschools. Drawing on years of studying teacherswithin the Cognitively Guided Instruction( 49 )

Appendix B (continued)Project, Dr. Franke focuses on enhancingteacher knowledge of content, student thinking,and teaching techniques.Patricia Herzig, M.S., is a national mathconsultant in Virginia, New Jersey, andOregon. She was a classroom teacher for 20years in Ohio and Washington, and she hastaught mathematics in kindergarten through12th grades. Since 2001, Ms. Herzig hasplayed a leadership role as a math specialistfor Highline and Bremerton school districtsin the state of Washington. She helped writethe Mathematics Grade Level Expectations forthis state as well, and she has been active inwriting and scoring performance items on theWashington Assessment for Student Learning.Asha Jitendra, Ph.D., is the Rodney WallaceProfessor for the Advancement of Teachingand Learning at the University of Minnesota.Her research focuses on evaluating the variablesthat impact children’s ability to succeed inschool-related tasks. Dr. Jitendra has developedboth mathematics and reading interventionsand has tested their effectiveness with studentswith learning difficulties. She has scrutinizedtheoretical models that deal with the coherentorganization and structure of information as thebasis for her research on textbook analysis. Herresearch has evaluated the content of textbooksfor the degree to which the information adheresto a selected model. Based on her systematicline of research on mathematical problemsolving, Dr. Jitendra has published the curriculumtext Solving Math Word Problems: TeachingStudents with Learning Disabilities UsingSchema-Based Instruction. She has authored 75peer-reviewed publications, three book chapters,and six other papers, and she has beena principal investigator for four major grants.Kenneth R. Koedinger, Ph.D., is a professorof human-computer interaction and psychologyat Carnegie Mellon University. Dr. Koedinger’sresearch focuses on educational technologiesthat dramatically increase student achievement.He works to create computer simulations ofstudent thinking and learning that are usedto guide the design of educational materials,practices, and technologies. These simulationsprovide the basis for an approach toeducational technology called “CognitiveTutors” that offers rich problem-solvingenvironments in which students can workand receive just-in-time learning assistance,much like what human tutors offer. Dr. Koedingerhas developed Cognitive Tutors forboth mathematics and science and has testedthe technology in laboratory and classroomenvironments. His research has contributednew principles and techniques for the designof educational software and has producedbasic cognitive-science research results on thenature of mathematical thinking and learning.Dr. Koedinger has many publications, isa cofounder and board member of CarnegieLearning, and directs the Pittsburgh Scienceof Learning Center.Philip Ogbuehi, Ph.D., is a mathematicsspecialist with the pre-K–12 mathematicsprogram in the Los Angeles Unified SchoolDistrict, where he designs and facilitates professionaldevelopment as well as curriculumand assessment development. Dr. Ogbuehitaught 8th-grade algebra and 7th-grade prealgebrain middle school for seven years andled the Mathematics, Engineering, and ScienceAchievement program at his school site. Dr.Ogbuehi’s research has focused on evaluatinginnovative strategies for teaching algebra concepts,largely with regard to students’ perceptionsof classrooms as learning environments,attitudes toward mathematics, and conceptualdevelopment; his research has been presentedlocally and internationally, and alsohas been published. Dr. Ogbuehi was selectedas a content-review panel member for the2007 Mathematics Primary Adoption by theCalifornia Department of Education.( 50 )

Appendix B (continued)StaffMadhavi Jayanthi, Ed.D., is a senior researchassociate at the Instructional Research Group inLos Alamitos, California. Dr. Jayanthi has morethan 10 years of experience working on grantsfrom the Office of Special Education Programsand the Institute of Education Sciences (IES).She has published extensively in peer-reviewedjournals such as American Educational ResearchJournal, Review of Educational Research,Journal of Learning Disabilities, Remedial andSpecial Education, and Exceptional Children.Her research interests include effective instructionaltechniques for students with disabilitiesand at-risk learners, in the areas of readingand mathematics. Currently, she serves as aco–principal investigator for a federally fundedstudy of professional development.Rebecca Newman-Gonchar, Ph.D., is asenior research associate at the InstructionalResearch Group in Los Alamitos, California.She has experience in project management,study design and implementation, and quantitativeand qualitative analysis. She servesas a co–principal investigator for a randomizedfield trial of professional development invocabulary, funded by IES. She has workedextensively on developing observationsystems to assess math and reading instructionfor four major IES-funded studies. Dr.Newman-Gonchar serves as project managerfor the Center on Instruction–Math Strand,and she has contributed to several practiceguides and intervention reports for the WhatWorks Clearinghouse in the areas of math,reading, response to intervention, and Englishlanguage learners.Kelly Haymond, M.A., is a research associateat the Instructional Research Group in LosAlamitos, California, and a Ph.D. student inpsychology at Claremont Graduate Universityin Claremont, California. She currently servesas a reviewer of experimental and single-casedesigns covering a range of topics, includingEnglish language learners, reading and mathematicsinterventions, response to intervention,and adult education, for the What WorksClearinghouse. Ms. Haymond has experienceproviding research support and conductingdata analysis for various projects on topicsrelated to reading, mathematics, assessment,response to intervention, and professionaldevelopment. Currently, she contributes toseveral projects to improve reading and mathinstruction in elementary schools.Joshua Furgeson, Ph.D., is a researcherat Mathematica Policy Research and a formerhigh school teacher. He is the deputyprincipal investigator for the What WorksClearinghouse (WWC) Students with LearningDisabilities topic area, and has contributedto WWC quick reviews and several practiceguides and intervention reports. Currently, heis the project director for the evaluation of theEquity Project Charter School, and is leadingthe experimental impact analysis for theNational Study of Charter-School ManagementOrganization Effectiveness.( 51 )

Appendix CDisclosure of Potential Conflicts of InterestPractice guide panels are composed of individuals who are nationally recognized experts on thetopics about which they are making recommendations. IES expects the experts to be involved professionallyin a variety of matters that relate to their work as a panel. Panel members are asked todisclose these professional activities and institute deliberative processes that encourage critical examinationof their views as they relate to the content of the practice guide. The potential influence ofthe panel members’ professional activities is further muted by the requirement that they ground theirrecommendations in evidence that is documented in the practice guide. In addition, before all practiceguides are published, they undergo an independent external peer review focusing on whether theevidence related to the recommendations in the guide has been presented appropriately.The professional activities reported by eachpanel member that appear to be most closelyassociated with the panel recommendationsare noted below.John Woodward is an author of the followingmathematics curricula interventions:TransMath Level 1: Developing Number Sense;TransMath Level 2: Making Sense of RationalNumbers; and TransMath Level 3: UnderstandingAlgebraic Expressions.Sybilla Beckmann receives royalties on herbook Mathematics for Elementary Teachers,which is published by Pearson Education. Sheserves as a consultant for Tom Snyder Productionson its Math 180 program. She also isa member of the mathematics work team onthe Common Core State Standards Initiative.Kenneth R. Koedinger served as a developerfor Cognitive Tutor math courses, whichinclude practices for supporting better studentlearning of problem solving. He receives royaltiesfrom Carnegie Learning, which distributesand improves the courses, and he is a stockholderand founder of Carnegie Learning.Philip Ogbuehi has an intellectual propertyinterest in the 2007 publication on LearningEnvironments Research published by SpringerNetherlands, and he co-authored the CaliforniaMath Review Teacher’s Edition books for grade6, grade 7, and Algebra 1, published by TPSPublishing Company.Asha Jitendra receives royalties for theproblem-solving textbook entitled SolvingMath Word Problems: Teaching Students withLearning Disabilities Using Schema-BasedInstruction. She also conducts trainings onschema-based instruction.( 52 )

Appendix DRationale for Evidence Ratings aAppendix D provides further detail about studies that the panel used to determine the evidence basefor the five recommendations in this guide. Studies that examined the effectiveness of recommendedpractices using strong designs for addressing questions of causal inference including randomizedcontrolled trials and rigorous quasi-experimental designs and that met What Works Clearinghouse(WWC) standards (with or without reservations) were used to determine the level of evidence and arediscussed here. This appendix also includes one study with a strong correlational design. 148Four studies met the WWC pilot standards for well-designed single-case design research and areincluded as supplemental evidence for Recommendations 2 and 3 in this guide. Single-case designstudies do not contribute to the level of evidence rating. While the panel believes that qualitativestudies, case studies, and other correlational studies contribute to the literature, these studies werenot eligible for WWC review, did not affect the level of evidence, and are not included in this appendix.Some studies have multiple intervention groups; only interventions relevant to this guide’srecommendations are included. 149In this practice guide, a group-design study 150result is classified as having a positive ornegative effect when:• The result is statistically significant (p ≤0.05) or marginally statistically significant(0.05 < p ≤ 0.10)• The result is substantively important asdefined by the WWC (effect sizes largerthan 0.25 or less than –0.25) 151When a result meets none of these criteria, itis classified as having “no discernible effect.”Some studies meet WWC standards (with orwithout reservations) for causal designs butdo not adjust statistical significance for multiplecomparisons or student clusters wherethe unit of assignment is different from theunit of analysis (e.g., classrooms are assignedto conditions, but student test scores areanalyzed). When full information is available,the WWC adjusts for clustering and multiplecomparisons within a domain. 152The three outcome domains 153 for this practiceguide are as follows:• Procedural knowledge, which relatesto whether students choose mathematicaloperations and procedures that will helpthem solve the problem and to how wellthey carry out the operations and proceduresthey choose to use. When studentscorrectly solve a math problem, they havelikely chosen the appropriate operation orprocedure and executed it correctly.• Conceptual understanding, whichrelates to how well students understandmathematical ideas, operations, and procedures,as well as the language of mathematics.One way for students to express theirconceptual understanding is to accuratelyand completely explain the operations andideas used to solve a problem. Another wayto show conceptual understanding is forstudents to explain relationships betweenideas, operations, and/or procedures asthey relate to a problem.• Procedural flexibility, which relates towhether students can identify and carryout multiple methods to solve math problems.If students can adaptively choose themost appropriate strategy for a particularproblem and can attempt to solve a mathproblem in multiple ways, then they havelikely developed procedural flexibility, askill that may help them solve problemsmore efficiently in the future.aEligible studies that meet WWC evidence standards or meet evidence standards with reservations are indicated by bold text in theendnotes and references pages.( 53 )

Appendix D (continued)Most studies only examined outcomes inthe procedural knowledge domain, and thusstudent achievement in this appendix refers tooutcomes in the procedural knowledge domainexcept where otherwise specified. To facilitatecomparisons, the appendix text focuses on theoutcome closest to the end of the intervention;these are labeled posttests. All outcome measuresadministered after the posttest are labeledmaintenance in appendix tables. Measuresthe panel believes require students to applyknowledge or skills in a new context are labeledtransfer outcomes in appendix tables. Whenstudies have multiple posttest outcome measuresadministered within the same domain,effect sizes for each measure are averaged, 154and the overall average is reported.Recommendation 1.Prepare problems and use themin whole-class instruction.Level of evidence: Minimal EvidenceFew studies directly tested the suggestionsof this recommendation, leading the panel toassign a minimal level of evidence. Althoughthe panel believes teacher planning shouldincorporate both routine and non-routineproblems, no studies meeting WWC standardsdirectly examined this issue.One study did find higher student achievementwhen teacher planning consideredstudents’ mathematical content weaknessesand whether students would understandlanguage and context prior to instruction(see Table D.1). 155 Another study showed thatincorporating a variety of familiar contextsinto instruction also may improve problemsolvingskills. 156 The panel interpreted theseresults cautiously, however, since both of thesestudies included additional instructional components(e.g., student monitoring and reflection).The panel did find several well-designedstudies showing that Taiwanese and Americanstudents who solve word problems incorporatingwell-liked and well-known contexts do betteron subsequent word problems tests thanstudents presented with generic contexts. 157Routine and non-routine problems. Thepanel’s suggestion to integrate a variety oftargeted problem-solving activities into wholeclassinstruction is based primarily on theexpertise of its members. No studies meetingWWC standards directly tested the complementaryuses of routine and non-routine problems.Problem context and vocabulary. Overall,no studies included interventions that solelytested this recommendation suggestion.One study meeting WWC standards involvedteacher planning that considered whether studentswould have difficulty understanding thelanguage, context, or mathematical contentin word problems (see Table D.1). 158 Teachersalso reviewed the problems’ vocabulary withstudents during instruction. This planningand instruction were part of a broader interventionthat also involved teaching studentsto pose questions to themselves while problemsolving. The overall intervention had asignificant positive effect on students’ abilityto solve word problems.Another study involved word problems thatincorporated contexts familiar to the 5th-gradestudents in the study. 159 However, these contextswere only one component of an interventionthat also focused on teaching students afive-step strategy for solving word problems.Although the study reported a positive effect,the panel cannot isolate the separate effect ofposing familiar word problems.Four additional studies found that incorporatingfavorite contexts into practice word problemsimproved students’ ability to solve multiplicationand division word problems. 160 Three of thestudies were conducted in Taiwan with 4th- and5th-grade students, and one took place in theUnited States with students in grades 6 through8. None of these studies had transfer or maintenanceoutcomes. In all of the studies, researchersasked students about their favorite places,foods, friends, sports, and so on, and thenincorporated that information into the practiceproblems. In two of the studies, word problemsused during classroom instruction were basedon the most common survey responses. 161 In the( 54 )

Appendix D (continued)other two, students completed computer-basedword problems that incorporated their individualsurvey responses. 162These favorite-context interventions wereconducted in two to four sessions, each lastingbetween 40 and 50 minutes. Students in thecontrol group received the same word problemsbut without the personalized content. Three ofthe four studies found that personalizing thecontent of word problems improved students’subsequent performance on a posttest thatincluded both personalized and nonpersonalizedword problems. 163 In the study that foundno discernible effects, the panel believes theoutcome measure limited the study’s ability todetect differences between groups.Planning using students’ math knowledge.Teachers in the study on teacher vocabularyplanning 164 also used information from earlierstudent performances to help them understandand plan for difficulties with mathematicalcontent that students might have. Theoverall intervention had a positive effect onstudents’ ability to solve word problems.Table D.1. Studies of interventions that involved problem selection and contribute to thelevel of evidence ratingStudy Comparison Duration Students Math Content Outcomes 165 Effect SizeFamiliar Contexts in ProblemsVerschaffel et al.(1999)Quasi-experimentaldesignWord problems withfamiliar contexts forstudents 166 vs. traditionalinstruction andstandard textbookproblemsA total of20 sessions,each lasting50–60 minutesA total of 203students in the5th grade inBelgiumWord problemsinvolvingnumbers andoperationsGeneral mathachievementPosttest (average 0.31** 168of a posttestand a retentiontest) 167Transfer 0.38** 169Preferred Contexts in ProblemsChen and Liu(2007)Randomizedcontrolled trialKu and Sullivan(2000)Randomizedcontrolled trialWord problems featuringstudents’ preferencesvs. standardtextbook problemsWord problems featuringstudents’ preferencesvs. standardtextbook problemsA total offour sessions,each lasting50 minutesA total oftwo sessions,each lasting50 minutesA total of 165students in the4th grade inTaiwanA total of 72students in the5th grade inTaiwanWord problemsinvolvingnumbers andoperationsWord problemsinvolving numbersand operations(multiplicationand division)Word problemsinvolvingnumbers andoperationsPosttest 0.72**Posttest (averageof subtests withpersonalized andnonpersonalizedproblems)0.13, nsKu and Sullivan(2002)Randomizedcontrolled trialWord problems featuringstudents’ preferencesvs. standardtextbook problemsA total ofthree sessions,each lasting40 minutesA total of 136students in the4th grade inTaiwanPosttest (averageof subtests withpersonalized andnonpersonalizedproblems)0.34*Ku et al. (2007)Randomizedcontrolled trialWord problems featuringstudents’ preferencesvs. standardtextbook problemsA total oftwo sessions,each lasting42 minutesA total of 104students ingrades 6–8 inthe United StatesWord problemsinvolvingnumbers andoperationsPosttest0.28, nsClarifying Vocabulary and Math ContentCardelle-Elawar(1995)Randomizedcontrolled trialTeacher considerationfor whether studentswould understandproblems and reviewof vocabulary andmath content withstudents 170 vs. traditionalinstructionOne year A total of 463students ingrades 4–8in the UnitedStates 171Word problemsinvolvinggeneral mathachievementPosttest (average 2.18**of posttest andtwo retentiontests administeredover sevenmonths) 172** = statistically significant at 0.05 level* = statistically significant at 0.10 levelns = not statistically significant( 55 )

Appendix D (continued)Recommendation 2.Assist students in monitoring andreflecting on the problem-solving process.Level of evidence: Strong EvidenceThe panel assigned a rating of strong evidenceto this recommendation based onnine studies that met WWC standards withor without reservations (see Table D.2). 173All nine studies reported positive effects onstudents’ ability to solve word problems. Theoutcomes measured diverse mathematicalcontent, including numbers and operations,data analysis and probability, geometry, andalgebra. Researchers conducted the studiesin 4th- through 8th-grade classrooms, withthree of the studies taking place in countriesoutside the United States. All the interventionstaught students to monitor and structure theirproblem-solving process, although the specificprompts varied. Four studies providedstudents with a list of key tasks for solvingword problems, 174 while teachers in the otherfive studies taught students to use self-questioningand reflection. 175 Two of these studiescombined interventions—a task list alongwith visual aids—so the panel could not attributeits results solely to the task checklist. 176Prompting students with lists. Several studies,including some that also involved teachermodeling, prompted students to self-questionor to complete tasks or steps while problemsolving. In two studies, teachers guided theself-questioning process by including questionsin students’ practice workbooks. 177 Studentswould answer the questions verbally and thenin writing when using their workbooks. 178 Inanother study, students received index cardswith question prompts and then asked eachother questions while working in pairs to solveword problems in a commercial softwareprogram. 179 In all of these studies, the intervention’seffects were positive.Other studies examined using task lists toprompt students. In one study with 5th-gradestudents in Belgium, teachers discussed strategiesfor solving word problems across 20lessons in four months. 180 Teachers combinedwhole-class instruction with small-groupwork and individual assignments. The controlgroup received traditional instruction for wordproblems, which did not include using a tasklist. Results from the study showed that theintervention had a positive effect on students’ability to solve word problems.In two other studies, students received checklistswith four-step strategies and were encouragedto think aloud while solving problemsindependently. 181 The interventions in thesetwo studies were very similar and included theuse of both schematic diagrams and other nonpromptingcomponents. Comparison teachersused district-adopted textbooks that focusedon direct instruction, worked examples, andstudent practice. Instruction in the first studyoccurred during 10 classes, each 40 minuteslong. The intervention had a positive effect on7th-grade students’ ability to solve ratio andproportion word problems, both immediatelyafterward and four months later. 182 Instructionin the second study occurred during 29 classes,each 50 minutes long. The intervention had apositive effect on 7th-grade students’ abilityto solve ratio, proportion, and percent wordproblems immediately afterward. There wasno discernible effect on an identical test givenone month after the intervention ended or ona transfer test. 183In a fourth study, 4th- and 5th-grade studentsreceived a five-step strategy list. 184 Teachersdemonstrated how to use the strategy andthen asked students to complete two practiceproblems individually. Finally, studentsdiscussed in pairs how they implemented thestrategy. Comparison students solved thesame problems individually and in groups,but without the five-step strategy. The studyfound that use of the checklist strategyimproved students’ performance on a fouritemtest of word problems measuring generalmath achievement.Modeling monitoring and reflection.Five randomized controlled trials andquasi-experimental studies examined how( 56 )

Appendix D (continued)self-questioning could help students monitor,reflect, and structure their problem solving. 185In these studies, teachers modeled the selfquestioningprocess and taught students howto ask themselves questions while problemsolving. 186 For example, in one study, teachersmodeled the self-questioning process byreading the questions and verbalizing theirthoughts aloud. 187 Compared to students whocompleted the same problems but were nottold to question one another, students in thisintervention experienced positive effects ontheir ability to solve geometry word problems.Similarly, four studies found positiveresults when teachers modeled multistepstrategies using task lists and then askedstudents to structure their problem solvingaround those strategies. 188In two of the self-questioning studies, teachersmodeled desired problem solving byasking and answering questions derived froma problem-solving model. 189 Teachers alsowere taught to practice self-questioning whenpreparing lessons (e.g., asking, “What are thekey errors students might make?”). These twostudies provided similar instruction but differedin student samples and duration: onestudy involved six hours of oral feedback overthree weeks to low-achieving 6th-grade bilingualstudents, 190 and the other study followedstudents in grades 4 through 8, including severalbilingual classes, and took place over thecourse of an entire school year. 191 Both studiesfound positive effects on students’ ability tosolve word problems compared to studentswho received only traditional instruction withlectures and worksheets.Two additional studies in Israel had teachersmodel a self-questioning approach(the IMPROVE method) for the whole class.Teachers instructed 8th-grade students to askthemselves four types of questions while solvingword problems: (1) comprehension questions,to ensure they understood the task orconcept in the problem; (2) connection questions,to think through similarities betweenproblem types; (3) strategic questions, tofocus on how to tackle the problem; and (4)reflection questions, to think about what theywanted to do during the solution process. 192Comparison conditions differed for the studies:in one, students received worked examplesfollowed by practice problems, 193 while in theother, instruction involved whole-class lecturesand practice problems. Both studies found apositive effect of teaching and actively supportingstudents to use the questions. 194Supplemental evidence comes from threesingle-case design studies. The first study,involving 3rd- and 4th-grade students, foundthat teacher modeling of a self-questioningapproach improved achievement for studentswith learning disabilities or mild intellectualdisabilities. 195 In this study, students were firsttaught a nine-step problem-solving strategy,and the instructor and student discussedthe importance of self-questioning. Afterthe students generated statements applyingthe strategy, the instructor and student thenmodeled the self-questioning process. Thetwo other single-case design studies foundno evidence of positive effects. 196 However,in one study, students were already achievingnear the maximum score during baseline,and thus the outcome could not measure anyimprovement. 197 In the other study, middleschoolstudents with learning disabilities weretaught a seven-step self-questioning process.Based on the findings reported, there is noevidence that this intervention had a positiveimpact on student achievement.( 57 )

Appendix D (continued)Table D.2. Studies of interventions that involved monitoring and contribute to thelevel of evidence ratingStudy Comparison Duration Students Math Content Outcomes 198 Effect SizeCardelle-Elawar(1990)Randomizedcontrolled trialInstruction in monitoringand reflecting usingquestions vs. traditionalinstructionSix hoursA total of 80 lowachieving6thgradestudentsfrom bilingualclasses in theUnited StatesWord problemsinvolvinggeneral mathachievementPosttest 2.54** 199Cardelle-Elawar(1995)Randomizedcontrolled trialHohn and Frey(2002)Randomizedcontrolled trialJitendra et al.(2009)Randomizedcontrolled trialJitendra et al.(2010)Randomizedcontrolled trialKing (1991)Randomizedcontrolled trialwith high attritionand baselineequivalenceKramarski andMevarech (2003)Randomizedcontrolled trialwith unknownattrition andbaselineequivalenceMevarech andKramarski (2003)Randomizedcontrolled trialInstruction in monitoringand reflecting usingquestions vs. traditionalinstructionInstruction in monitoringand reflecting using atask list vs. no instructionin monitoring andreflectingInstruction in monitoringand reflecting usingquestions and a tasklist 203 vs. traditionalinstructionInstruction in monitoringand reflecting usingquestions and a tasklist 204 vs. traditionalinstructionInstruction in monitoringand reflecting usingquestions vs. noinstruction in monitoringor reflectingInstruction in monitoringand reflecting usingquestions vs. noinstruction in monitoringand reflectingInstruction in monitoringand reflecting usingquestions vs. instructionthat had studentsstudy worked examplesand then discuss theirsolutions to problemsOne school year A total of 463students ingrades 4–8 in theUnited States 200A total offour sessionspresentedevery two daysA total of10 sessions,each lasting40 minutesA total of29 sessions,each lasting50 minutesA total ofsix sessions,each lasting45 minutes, acrossthree weeksA total of10 sessions,each lasting45 minutesA total of 72students in the4th and 5thgrades (locationnot reported) 202A total of 148students in the7th grade in theUnited StatesA total of 472students in the7th grade in theUnited StatesA total of 30students in the5th grade in theUnited StatesA total of 384students in the8th grade inIsraelFour weeks A total of 122students in the8th grade inIsraelWord problemsinvolvinggeneral mathachievementWord problemsinvolvinggeneral mathachievementPosttest 2.18**(average of aposttest andtwo retentiontests given overseven months) 201Posttest0.79, nsWord problems Posttest 0.33, nsinvolvingMaintenance 0.38, nsnumbers and(four monthsoperationsafter posttest)State assesment Transfer 0.08, nsWord problems Posttest 0.21**involvingMaintenance 0.09, nsnumbers and(one monthoperationsafter posttest)Transfer –0.01, nsWord problems Posttest 0.98* 205and problemsolving involvinggeometryMultiple-choiceproblems andword problemsinvolving dataanalysisData analysisData analysisWord problemsinvolvingalgebraPosttest 0.48Posttest(flexibilitycompetency)Transfer(graphicalrepresentations)PosttestMaintenance(one year afterposttest)0.770.370.34, ns0.31, ns(continued)( 58 )

Appendix D (continued)Table D.2. Studies of interventions that involved monitoring and contribute to thelevel of evidence rating (continued)Study Comparison Duration Students Math Content Outcomes 198 Effect SizeVerschaffel et al.(1999)Quasi-experimentaldesignInstruction in monitoringand reflecting using atask list vs. traditionalinstruction** = statistically significant at 0.05 level* = statistically significant at 0.10 levelns = not statistically significantA total of20 sessions,each lasting50–60 minutesA total of 203students in the5th grade inBelgiumWord problemsinvolvingnumbers andoperationsGeneral mathachievementPosttest 0.31** 207(average of aposttest anda retentiontest) 206Transfer 0.38** 208Table D.3. Supplemental evidence supporting the effectiveness of Recommendation 2Study Comparison Duration Students Math Content Outcomes 209 Effect Size 210Cassel and Reid(1996)Single-casedesignInstruction in monitoringand reflecting usingquestions and a tasklist 211 vs. no instructionUnknown numberof sessions, eachlasting 35 minutesFour 3rd and 4thgrade studentswith mild mentalhandicaps in theUnited StatesWord problemsinvolvingnumbers andoperationsRepeatedmeasurementPositiveevidenceCase et al. (1992)Single-casedesignMontague (1992)Single-casedesignInstruction in monitoringand reflecting usinga task list 212 vs. noinstructionInstruction in monitoringand reflecting usinga task list 213 vs. noinstructionBetween fourand five sessions,each lastingabout 35 minutesA total ofthree sessions,each lasting55 minutesFour 5th and 6thgrade studentswith learningdisabilities in theUnited StatesThree studentswith learningdisabilities ingrades 6–8 inthe United StatesWord problemsinvolvingnumbers andoperationsWord problemsinvolvingnumbers andoperationsRepeatedmeasurementRepeatedmeasurementNo evidenceNo evidence( 59 )

Appendix D (continued)Recommendation 3. Teach students howto use visual representations.Level of evidence: Strong EvidenceThe panel determined there is strong evidencesupporting this recommendation; severalstudies with diverse student samples—mostlytaking place in the United States, with middleschool students—consistently found that usingvisual representations improved problem-solvingachievement (see Table D.3). 214Both general-education students and studentswith disabilities performed better when specificallytaught how to use different visual representationsfor different types of problems: 215for example, to identify and map relevantinformation onto schematic diagrams 216 and tointegrate visual representations. 217 One studyfurther suggested that student achievementincreases more when students learn how todesign, develop, and improve their own representationsthan when students use teacher- ortextbook-developed visuals. 218Selecting visuals. 219 Several studies consistentlyfound positive results when studentswere taught to use visual representations tosolve problems. 220 For example, four studiestaught students to solve numbers-and-operationsword problems with only schematicdiagrams. 221 In another study, the authorstaught 7th-grade students to use a linkingtable to overcome students’ mistaken intuitivebeliefs about multiplication and division. 222Each study found positive effects compared tostudents who received traditional instruction.Using visuals. Multiple studies with positiveresults involved teachers providing differenttypes of visual representations for differenttypes of problems. 223 For example, in onestudy, middle school students received paperslisting the prominent features of two differentkinds of problems (e.g., for proportionproblems, “There are two pairs of associationsbetween two things that involve four quantities”).224 Students then used type-specificdiagrams to represent these problems. Initially,student worksheets included just one kind ofproblem, but after students learned how tosolve both, worksheets with multiple problemtypes were presented, and students couldcompare them. Students receiving this instructionscored higher than comparison studentswho were taught using the textbook. Teachersin the comparison condition also modeled howto use representations to represent informationin problems.In another study, students practiced identifyingdifferent problem types and then mapping thefeatures of each on a schematic diagram. 225Practice problems were grouped by problemtype, with students identifying the criticalelements. Teachers also repeated explicitinstructions in order to provide strategy stepsand clarify misconceptions. Students receivingthis intervention had higher achievement thanstudents in the comparison group on both aposttest conducted one to two weeks afterthe intervention and a transfer test that usedproblems taken from a textbook. These positiveeffects persisted for four months after theintervention; however, the intervention had nodiscernible effects on a state standardized test.Supplemental evidence comes from twosingle-case design studies in which studentswere taught how to use visual representations;one study found evidence of a positiveeffect, and one study found no evidence of aneffect. 226 In the study finding a positive effect,8th-grade students with learning disabilitieswere taught to identify different types of multiplicationand division problems. 227 Teachersdemonstrated the diagram-mapping processand instructed students on how to representthe diagram information as a mathematicalsentence. In the study that found no evidenceof an effect, 6th- and 7th-grade students withlearning disabilities were taught to identifydifferent types of addition and subtractionproblems. 228 Teachers demonstrated the diagram-mappingprocess and provided studentswith rules on how to represent the diagraminformation as a mathematical sentence.( 60 )

Appendix D (continued)Translating visuals. In two studies involvingstudents with learning disabilities or mild disabilities,students were taught to diagram story situationsthat contained all necessary information.Later, teachers presented these students withword problems and asked them to representunknown information with question marks. 229In one of the studies, instruction emphasizedthat the ultimate mathematical equation couldbe derived directly from the word problem diagram.230 Similarly, instruction in the other studysuggested that the mathematical equation couldbe identified directly from a data table. 231 Bothstudies found positive effects.Table D.4. Studies of interventions that used visual representations and contributeto the level of evidence ratingStudy Comparison Duration Students Math Content Outcomes 232 Effect SizeJitendra et al. Instruction in the use of Between 17 andWord problems Posttest 0.56, ns(1998)a visual representation 20 sessions,involvingRandomized (schematic drawing) vs. each lastingnumbers and0.85**controlled trial traditional instruction 40–45 minutesoperationsJitendra et al.(2009)Randomizedcontrolled trialJitendra et al.(2010)Randomizedcontrolled trialSelke et al. (1991)Randomizedcontrolled trialXin et al. (2005)Randomizedcontrolled trialInstruction in the use ofa visual representation(schematic drawing) 234vs. traditional instructionInstruction in the use ofa visual representation(schematic drawing) 235vs. traditional instructionInstruction in the use ofa visual representation(data table) vs. instructionin a substitution strategyInstruction in the use ofa visual representation(schematic drawing)vs. traditional instruction,including the use of asemi-concrete representationto representinformationA total of10 sessions,each lasting40 minutesA total of29 sessions,each lasting50 minutesA total of10 sessionsduring regularmath classA total of12 sessions,each lastingone hourA total of 34students ingrades 2–5in the UnitedStates 233 (moststudents hadmild disabilities)A total of 148students in the7th grade in theUnited StatesA total of 472students in the7th grade in theUnited StatesA total of 107students in the7th grade in theUnited StatesA total of 22students ingrades 6–8in the UnitedStates (most studentshad learningdisabilities)Word problemsinvolvingnumbers andoperationsMaintenance(one to twoweeks afterintervention)Transfer(one day afterposttest)PosttestMaintenance(four monthsafter posttest)1.01**0.33, ns0.38, nsState assessment Transfer 0.08, nsWord problems Posttest 0.21**involvingMaintenance 0.09, nsnumbers and(one monthoperationsafter posttest)Transfer –0.01, nsWord problemsinvolvingnumbers andoperations 236Word problemsinvolvingnumbers andoperationsPosttest(average oftwo subtests)1.29**Posttest 1.87**Maintenance 3.03**(administered3–12 weeksafterintervention) 237Transfer 1.51**Terwel et al.(2009)Randomizedcontrolled trialInstruction to students ongenerating their own visualsvs. providing studentswith teacher-made visualsStudent-Generated Visual RepresentationsA total of13 sessions,each lastingone hour, acrossthree weeksA total of 238students in the5th grade in theNetherlandsWord problemsinvolving dataanalysis andprobabilityPosttest0.41, nsTransfer 0.64**** = statistically significant at 0.05 level* = statistically significant at 0.10 levelns = not statistically significant( 61 )

Appendix D (continued)Table D.5. Supplemental evidence supporting the effectiveness of Recommendation 3Study Comparison Duration Students Math Content 238 Outcome Effect Size 239Jitendra et al.(1999)Single-casedesignInstruction in the use ofa visual representation(schematic diagram) 240vs. no instructionUnknownnumber ofsessions,each lasting45 minutesFour 6th- and 7thgradestudentswith learningdisabilities in theUnited StatesWord problemsinvolvingnumbers andoperationsRepeatedmeasurementNo evidenceRecommendation 4. Expose students tomultiple problem-solving strategies.Level of evidence: Moderate EvidenceEight studies found positive effects of teachingand encouraging multiple problemsolvingstrategies, although in some studies,the effects were not consistent across alltypes of outcomes. 241 Three studies involvingstudents with no or limited knowledgeof algebra found negative effects for somealgebra outcomes, but these students had notdeveloped sufficient skills in a domain beforemultiple-strategies instruction. Consequently,the panel determined that there is moderateevidence to support this recommendation.Six of the seven studies that examined proceduralflexibility—the ability to apply multipleproblem-solving approaches—found thatteaching multiple problem-solving strategies,either by instruction, worked examples, orprompting, improved students’ proceduralflexibility (see Table D.4). 242 Yet teaching multiplestrategies had mixed effects on students’procedural knowledge, or ability to solveproblems correctly, with five studies reportingpositive effects on posttests, two studiesreporting no discernible effects, and thethree studies involving students with no orminimal algebra knowledge reporting negativeeffects on algebra procedural knowledgeoutcomes. 243 Effects on conceptual knowledgeof mathematics were inconsistent, withtwo studies finding no discernible effects, onestudy finding positive effects, and one studyfinding that the effects depended on baselineknowledge in the domain. 244Multiple-strategies instruction. Two studiesinvolving multiple-strategies instructionfound positive effects on procedural knowledge;however, because both of these interventionsincorporated multiple components,the panel could not attribute their resultssolely to multiple-strategies instruction.One of the studies included instruction thatemphasized different solution strategies. 245Teachers directly taught a variety of solutionmethods—unit-rate strategies, crossmultiplication, and equivalent fractions—tosolve ratio-and-proportion word problems,and students learned to identify when aparticular strategy was appropriate. Seventhgradestudents who received this instructionfor 10 sessions made greater achievementgains than students who received traditionallessons including direct instruction, workedexamples, and guided practice. The secondstudy used a similar intervention but withmore instruction time and professionaldevelopment. 246In another study, 8th-grade students inGermany were taught to work forward andbackward, as needed, to solve problemsduring an after-school tutoring program. 247These students performed significantly betterthan students who received no after-schooltutoring. However, because the comparisonstudents received no organized instruction,the panel is unsure whether this result appliesto classroom settings.A final study involving received a brief,eight-minute period of strategy instruction. 248An instructor solved three equations on ablackboard using different strategies. Foreach equation, the instructor used the strategythat led to the most efficient solution toeach problem. Students were not told whya particular strategy was selected. Prior to( 62 )

Appendix D (continued)this instruction, none of the participants hadreceived formal instruction on equation solving.There were no discernable effects for thisbrief intervention.Worked examples. Three of the four studiesexamining this intervention found that teachingstudents to compare multiple strategieson worked examples improved proceduralflexibility—but these studies also found thatthe effects on procedural and conceptualknowledge were sometimes positive andsometimes not discernible. 249 In each ofthese studies, students worked in pairs, andthe researchers manipulated only how theworked examples were presented. Both interventionand control students were exposedto multiple solution strategies; however, thecomparison of the multiple solution strategieswas only facilitated for intervention students.For example, all students in one studyreviewed packets of worked examples, inwhich half the worked examples presentedthe conventional solution method and halfpresented a shortcut solution method. 250On each page, two worked examples werepresented side-by-side as a pair. For the interventiongroup, each worked-example paircontained the same equation, solved usingboth the conventional and shortcut strategies.In the control group, each worked-examplepair contained two different problems solvedwith the same solution strategy. Thus, onlythe intervention condition facilitated comparisonsbetween different strategies. Studentswere also presented with practice problems.In the intervention condition, students wereasked to solve two practice problems usingtwo different strategies for each, while controlstudents received four different equationsand were not asked to use different strategies.Intervention students scored higher thancontrol students on measures of conceptualknowledge and procedural flexibility, andthis impact persisted for two weeks after theintervention ended. However, there were nodiscernible differences between groups onprocedural knowledge.The other two studies facilitated multiple-strategycomparison for intervention students byproviding worked-example packets with eachworked example solved in two different wayson the same page; control students receivedthe same number of worked examples, buteach of the problems was different and presentedon a separate page. 251 In the first study,7th-grade students solved algebra problems;this study found that facilitating multiplestrategiescomparison improved both proceduralknowledge and procedural flexibility,but that there was no impact on conceptualknowledge. The second study involved multiplicationestimation problems and found nodiscernible effects on procedural or conceptualknowledge, either immediately or after twoweeks; it did, however, find a positive effecton procedural flexibility that persisted for twoweeks after the intervention ended.The fourth study had a similar intervention,but the participants were students with noor limited pretest algebra knowledge. 252Specifically, the study involved two groups ofstudents: students who never used an algebraicproblem-solving approach on a pretest,and students who attempted an algebraicapproach, correctly or incorrectly. 253 (Eventhe second group of students had limitedalgebra problem-solving skills—roughlytwo-thirds used algebra incorrectly on thepretest.) All participants were 7th- and 8thgradestudents at a low-performing middleschool. The intervention facilitated multiplestrategycomparison for intervention studentsby providing worked-example packets witheach example solved in two different wayson the same page, while control studentsreceived packets with the worked exampleson a different page. Additionally, at the end ofthe three daily sessions, intervention studentswere presented with two problems and askedto solve each of the problems using two differentsolution methods, while control studentswere presented with four problems andallowed to choose their solution method. Forthe students who did not attempt algebraicproblem solving on the pretest, the studyfound the intervention had negative effects( 63 )

Appendix D (continued)on procedural knowledge, conceptual knowledge,and procedural flexibility. However,there were no discernible effects on any ofthese outcomes for students who attemptedan algebraic approach on the pretest. 254Generating and sharing multiple strategies.Four studies, with one study involving twocomparisons, examined this approach, withthree comparisons focusing on proceduralknowledge and finding mixed effects, andthree comparisons focusing on proceduralflexibility and finding positive effects.Of the three studies that examined proceduralknowledge, one study found a positiveeffect and two studies found negativeeffects. 255 The panel believes these differentfindings result from important differencesin the student samples and how studentswere prompted to generate and share multiplestrategies. 256 In the study with positiveresults, there are two comparisons related tothis recommendation: one involved individualstudents generating multiple strategies, andthe other considered pairs of students collaboratingto generate multiple strategies. 257Students in both groups also shared strategiesand answers after solving an initial problem,and they also used instructional componentssuch as manipulatives. In the comparisongroup, multiple strategies were not discussed,and students completed their work individuallyafter being provided with solution steps.Although both interventions had positiveeffects on procedural knowledge, the effectsize for the two-person group comparisonwas about twice as large. Pretest scoresindicate that the 3rd- and 4th-grade studentsample had some relevant problem-solvingknowledge, even though participants scoredin the lower half of pretest achievement.In the two studies with negative findings onprocedural knowledge, participants did nothave basic algebra knowledge, and the interventionsinvolved algebra. In the first study,intervention students were prompted to generatemultiple strategies by simply reorderingproblem-solving steps, while comparisonstudents were not prompted. 258 No teacherinstruction or student sharing took place.None of the participants had received formalinstruction on equation solving in school.Participants in the second study were 6thgraderswith no baseline algebra knowledgewho received a 30-minute lesson on equationsolving and then practiced solving algebraproblems. 259 Intervention students were givenalgebra problems they had previously solvedand were asked to re-solve them using adifferent ordering of steps, while comparisonstudents were not given instructions but wereprovided with additional, similar problems.The panel noted that both studies found positiveeffects on procedural flexibility—a morealigned outcome.This procedural-flexibility finding was supportedby another study, this one involving8- and 9-year-olds in the United Kingdom. 260Working on computers, intervention studentswere provided with a monetary amount(represented with coins) and asked to developother combinations of coins that would totalthe same amount. These students scoredsignificantly higher on procedural flexibilitythan comparison students, who did not usethe computer program and were not asked togenerate multiple combinations.( 64 )

Appendix D (continued)Table D.6. Studies of interventions that involved multiple problem-solving strategiesand contribute to the level of evidence ratingStudy Comparison Duration StudentsMathContentInstruction in Multiple StrategiesJitendra et al.(2009)Randomizedcontrolled trialJitendra et al.(2010)Randomizedcontrolled trialPerels et al.(2005) 263Randomizedcontrolled trialStar and Rittle-Johnson (2008)Randomizedcontrolled trial 265Rittle-Johnsonand Star (2007)Randomizedcontrolled trialRittle-Johnsonand Star (2009)Randomizedcontrolled trialRittle-Johnsonet al. (2009)Randomizedcontrolled trialStudents not usingalgebra on pretestInstruction in problemspecificmultiple strategies 261vs. traditional instructionInstruction in problemspecificmultiple strategies 262vs. traditional instructionInstruction in generic multiplestrategies after school 264vs. no instructionDemonstration of mostefficient strategy to solvethree different equations vs.additional time to practicesolving equationsStudents comparing workedexamples solved with multiplestrategies vs. students studyingworked examples solvedwith multiple strategiesStudents comparing workedexamples of one problemsolved using multiple strategiesvs. students comparingworked examples of equivalentproblems solved withthe same strategyStudents comparing workedexamples of one problemsolved using multiple strategiesvs. students studyingworked examples solvedwith multiple strategiesA total of10 dailysessions,each lasting40 minutesA total of29 sessions,each lasting50 minutesA total ofsix weeklysessions,each lasting90 minutesA total offive sessions,each lasting onehour, conductedon consecutivedays during thesummerA total of 148students inthe 7th gradein the UnitedStatesA total of 472students inthe 7th gradein the UnitedStatesApproximately116students in the8th grade inGermanyA total of 66students in the6th and 7thgrades in theUnited StatesWord problemsinvolvingnumbers andoperationsStateassessmentWord problemsinvolvingnumbers andoperationsWord problemsinvolvinggeneral mathachievementAlgebraequationsWorked Examples with Students Comparing StrategiesA total oftwo sessions,each lasting45 minutes,across two daysThreeconsecutiveclass periodsA total of threedaily sessions,each lastingapproximately45 minutesA total of 70students inthe 7th gradein the UnitedStatesA total of 98students in the7th and 8thgrades in theUnited States55 studentsin the 7th and8th gradesin the UnitedStatesAlgebraequationsAlgebraequationsAlgebraequationsDomain andOutcomeProcedural posttestProcedural maintenance(four monthsafter posttest)Procedural transferEffectSize0.33, ns0.38, ns0.08, nsProcedural posttest 0.21**Procedural maintenance0.09, ns(one monthafter posttest)Transfer–0.01, nsProcedural posttest 0.46**Procedural posttestProcedural transferFlexibility posttest(average of threemeasures)0.02, ns0.06, ns0.23, nsProcedural posttest 0.08**Conceptual posttest–0.04, nsFlexibility posttest 0.10**Procedural–0.14,posttest 266 ns 267Procedural maintenance(two weeks0.01, nsafter posttest)Conceptual posttest 0.36*Conceptual maintenance(two weeksafter posttest)Flexibility posttest(average of twomeasures)Flexibility maintenance(two weeksafter posttest, averageof two measures)ProceduralposttestConceptualposttestFlexibility posttest0.29, ns0.36*0.50**–0.45* 268–0.33*–0.35, ns(continued)( 65 )

Appendix D (continued)Table D.6. Studies of interventions that involved multiple problem-solving strategiesand contribute to the level of evidence rating (continued)Study Comparison Duration StudentsRittle-Johnson etal. (2009)Randomizedcontrolled trialStudents usingsome algebra onpretestStar and Rittle-Johnson (2009a)Randomizedcontrolled trialAinsworth et al.(1998)Randomizedcontrolled trialGinsburg-Blockand Fantuzzo(1998)Randomizedcontrolled trialGinsburg-Blockand Fantuzzo(1998)Additionalcomparison 274Star and Rittle-Johnson (2008)Randomizedcontrolled trial 279Star and Seifert(2006)Randomizedcontrolled trialStudents comparing workedexamples of one problemsolved using multiple strategiesvs. students studyingworked examples solvedwith multiple strategiesStudents comparing workedexamples solved with multiplestrategies vs. studentsstudying worked examplessolved with multiplestrategiesStudents generating multiplestrategies vs. no treatmentStudents solving a problemwith a partner, sharing theirstrategy and solution with thelarger group, and then generatingmultiple strategies witha partner vs. students solvingproblems individually withoutgenerating or sharing multiplestrategiesStudents solving a problem,sharing their strategy and solutionswith the larger group,and then generating multiplestrategies individually 275 vs.students solving problems individuallywithout generatingor sharing multiple strategiesStudents being prompted togenerate multiple strategiesby resolving problems usingdifferent ordering of stepsvs. students solving similarproblems without beingprompted to generatemultiple strategiesStudents were given problemsthey had previouslysolved and were asked tore-solve using a differentordering of steps vs. studentssolving similar problemsA total ofthree dailysessions,each lastingapproximately45 minutesA total ofthree sessions,each lasting40 minutes55 studentsin the 7th and8th gradesin the UnitedStatesA total of 157students in the5th and 6thgrades in theUnited StatesMathContentAlgebraequationsNumbers andoperationsestimationStudents Generating and Sharing Multiple StrategiesA total oftwo sessions,with a totaltime of 60–90minutesA total of14 sessions,each lasting30 minutesA total of14 sessions,each lasting30 minutesA total of fivesessions, eachlasting one hour,conducted onconsecutivedays duringthe summerA total ofthree one-hoursessionsA total of 48students(average age 9)in the UnitedKingdomA total of 52students inthe 3rd and4th gradesin the UnitedStates 271A total of 52students inthe 3rd and4th gradesin the UnitedStates 276A total of 63students in the6th and 7thgrades in theUnited StatesA total of32 studentsin the 6thgrade in theUnited StatesProblem solvinginvolvingnumbers andoperationsWord problemsinvolvingnumbers andoperationsWord problemsinvolvingnumbers andoperationsAlgebraequationsAlgebraproblemsDomain andOutcomeProceduralposttestConceptualposttestFlexibility posttestEffectSize0.19,ns 269–0.13, ns0.12, nsConceptual –0.06, nsposttest 270Conceptual maintenance(two weeks0.00, nsafter posttest)Flexibility posttest 0.43**Flexibility maintenance(two weeks0.30*after posttest)Flexibility posttest 1.30**Flexibility1.01**maintenance(delay after posttestnot reported)Procedural0.76* 273posttest 272Procedural0.32,posttest 277 ns 278Procedural posttest –0.35Procedural transferFlexibility posttest(average of threemeasures)Procedural posttestFlexibility posttest–0.11, ns0.44*–0.35, ns0.43, ns** = statistically significant at 0.05 level* = statistically significant at 0.10 levelns = not statistically significant( 66 )

Appendix D (continued)Recommendation 5. Help studentsrecognize and articulate mathematicalconcepts and notation.Level of evidence: Moderate EvidenceThree studies directly support two suggestionsof this recommendation (see Table D.7),and non-robust findings exist for anothersuggestion; overall, the panel believes amoderate level of evidence supports the fullrecommendation. 280 The first suggestion,relating problem solving to mathematicalconcepts, was supported by a study findingthat student achievement improved whenteachers discussed mathematics problemsconceptually (without numbers) and then representedthem visually before focusing on themathematical operations and notation. 281 Twoother studies that tested the impact of teachingalgebra notation to students, anothersuggestion, found positive effects. 282 Finally,three studies examined student self-explanation,the second suggestion. The resultswere inconsistent across the studies, with twostudies reporting positive effects and onereporting no discernible effects. 283Relating mathematics to problem solving.One study meeting WWC standards foundthat discussing problem structure and visualrepresentation prior to formulating and computingmath problems had positive effectson student achievement. 284 In the intervention,teachers discussed word problems with 4thgradestudents without using numbers, toencourage students to think about problemstructure and apply their informal mathematicalknowledge. Students then visually representedthe problems; teachers also modeled therepresentation and discussed it with students.Only after these conceptual processes didstudents write a number sentence and solvethe problem. Comparison students receivedpractice worksheets with the same problems.Intervention students scored higher thancomparison students on multiplication anddivision word problems, and this positiveeffect persisted for at least two months afterthe intervention ended. 285Student explanation. Three studies usedworked examples to examine student selfexplanationof the solution process. Resultswere not robust, with two studies findingpositive results and one study finding nodiscernible effects. 286 These four studies haddiverse student samples and were conductedin four different countries, with studentsranging from 4th to 9th grade. 287 The mathematicalcontent also was varied, rangingfrom numbers and operations to algebra. 288The intervention details varied as well.In one study, teachers had students completefour in-class assignments. 289 For interventionstudents, these assignments included 5 to 6worked examples, some solved correctly andsome solved incorrectly, and students wereasked to explain why the solutions were corrector incorrect. Intervention students alsoreceived 5 to 6 practice problems to solve.Comparison students received 10 to 12 practiceproblems only. The authors found positiveeffects on the conceptual knowledge posttest.In the two other studies, all students receivedworked examples, but only students in theintervention group were asked to explaineach step in the process. Intervention studentsin the first of these were asked toself-explain each step in the problem-solvingprocess. These students were able to solvesignificantly more word problems than studentswho were asked to review only theproblems and learn each step. 290 This positiveeffect persisted for one month after theintervention ended. Intervention studentsin the second study were asked to pretendthey were explaining each step in the workedexamples to another student; students in thecomparison condition were asked to study theworked examples until they understood howthe problems were solved. 291 This interventionhad no discernible effects.Algebraic problem solving. Two studiesmeeting WWC standards directly tested theeffect of helping students make sense of algebraicnotation. The first study used an algebratutoring program to change the order of steps( 67 )

Appendix D (continued)high school students took to solve algebraproblems—a slight change that had a statisticallysignificant effect on later achievement. 292The authors proposed that solving intermediatearithmetic problems before representingthem with algebraic notation helps studentsunderstand problem structure using the mathematicalknowledge (arithmetic) they alreadypossess, and students can then use this existingknowledge to more easily determine algebraicnotation. 293 In the intervention condition,students were asked to solve two intermediatearithmetic problems before providing algebraicnotation, while in the comparison conditionstudents were asked to provide algebraic notationfirst. Students in both groups then solvednew algebra word problems, and the authorsreported that the intervention students solvedmore problems correctly. 294The second study examined two-step wordproblems in which students had to substituteone algebraic expression into another. 295 Studentsin the intervention condition were givenfour symbolic problems that required themto substitute one expression into another (forexample, “Substitute 62 – f for b in 62 + b.”).Comparison students were asked to solvefour one-step word problems; these wordproblems were very similar in format to onestep of the two-step word problems thatwere the outcomes. The authors found thatstudents in the intervention group correctlysolved more two-step word problems.( 68 )

Appendix D (continued)Table D.7. Studies of interventions that helped students recognize and articulate conceptsand contribute to the level of evidence ratingStudy Comparison Duration Students Math Content Outcome 296 Effect SizeHuinker (1992)Randomizedcontrolled trialBooth et al.(2010)Randomizedcontrolled trialMwangi andSweller (1998) 301Randomizedcontrolled trialTajika et al.(2007)Quasi-experimentaldesignKoedinger andAnderson (1998)Randomizedcontrolled trialKoedinger andMcLaughlin(2010)Randomizedcontrolled trialStudent and teacherdiscussion of problemrepresentation andconnection to a mathematicaloperation priorto formal mathematics 297vs. solving practiceproblemsStudents presented withworked examples andasked to explain why thesolution was correct orincorrect vs. studentsgiven similar problemsto solveStudents asked to explaineach step in workedexamples vs. studentsasked to study workedexamples until they understandthe solutionStudents asked toexplain each step inworked examples vs.students provided withexplanations for eachstep in worked examplesand told to study themStudents asked to solverelated arithmetic questionsbefore being askedto represent the problemalgebraically vs. studentsasked to representproblems algebraicallybefore solving relatedarithmetic questionsStudents given practiceword problems that involvedsubstituting onealgebraic expressioninto another vs. studentsgiven practice wordproblems that did notinvolve substitution** = statistically significant at 0.05 level* = statistically significant at 0.10 levelns = not statistically significantRelating to Conceptual UnderstandingA total of18 lessonsA total of 128students in the4th grade in theUnited StatesWord problemsinvolvingnumbers andoperationsStudent Explanation of Worked ExamplesFoursessionsOnesessionOne20-minutesessionPosttest 298 1.21**Retention(two to threemonths afterposttest)A total of 51high schoolstudents in theUnited States 299 Algebra Conceptualposttest 300A total of 48students in the4th grade inAustraliaA total of 53students in the6th grade inJapanMaking Sense of Algebra NotationTwosessions,each lastingone to twohoursA total of 20high schoolstudents in theUnited States 302One session A total of 303middle schoolstudents in theUnited StatesWord problemsinvolvingnumbers andoperationsWord problemsinvolvingnumbers andoperationsWord problemsinvolvinggeneral mathachievement(algebra andnumbers andoperations)Word problemsinvolvingalgebraPosttestRetention(10 days afterposttest)Transfer(10 days afterposttest)Posttest(one week afterintervention)Transfer(one monthafter posttest)Posttest1.24**0.50*0.00, ns–0.21, ns–0.11, ns0.93**0.58**Not reported(p < 0.05)Posttest 303 0.26**( 69 )

Endnotes a1. Following WWC guidelines, improved outcomesare indicated by either a positive,statistically significant effect or a positive,substantively important effect size. TheWWC defines substantively important, orlarge, effects on outcomes to be thosewith effect sizes greater than 0.25 standarddeviations. See the WWC guidelines athttp://ies.ed.gov/ncee/wwc/DocumentSum.aspx?sid=19.2. For more information, see the WWC FrequentlyAsked Questions page for practiceguides, http://ies.ed.gov/ncee/wwc/Document.aspx?sid=15.3. This includes randomized control trials(RCTs) and quasi-experimental designs(QEDs). Studies not contributing to levels ofevidence include single-case designs (SCDs)evaluated with WWC pilot SCD standardsand regression discontinuity designs (RDDs)evaluated with pilot RDD standards.4. The research may include studies generallymeeting WWC standards and supportingthe effectiveness of a program, practice, orapproach with small sample sizes and/orother conditions of implementation or analysisthat limit generalizability. The researchmay include studies that support the generalityof a relation but do not meet WWCstandards; however, they have no majorflaws related to internal validity other thanlack of demonstrated equivalence at pretestfor QEDs. QEDs without equivalence mustinclude a pretest covariate as a statisticalcontrol for selection bias. These studiesmust be accompanied by at least one relevantstudy meeting WWC standards.5. American Educational Research Association,American Psychological Association, andNational Council on Measurement in Education,1999.6. McCloskey (2007); National Academy ofSciences (2007).7. Lemke et al. (2004); Perie, Grigg, and Dion(2005); Gonzales et al. (2008).8. Gonzales et al. (2008).9. Kirsch et al. (2007); Lesh, Hamilton, andKaput (2007); Levy and Murnane (2004);Uhalde, Strohl, and Simkins (2006).10. Traditional mathematics textbooks focusinstruction on standard arithmetic methodsand procedural aspects of learningmathematics, rather than on conceptualunderstanding and having students generalizetheir understanding of mathematics tovaried contexts.11. Jones and Tarr (2007).12. Ibid.13. Reys et al. (2006).14. Mayer and Hegarty (1996).15. Hiebert et al. (1997).16. For example, Mayer (1992).17. While instructional methods may vary, thecontent of mathematical problem solvingis quite similar internationally. In fact, boththe Trends in International Mathematicsand Science Study (TIMSS) and Programmefor International Student Assessment (PISA)assessments include problem-solving items,allowing for international comparisons ofstudents’ ability to solve problems. Since4th- and 8th-grade students in the UnitedStates are not as proficient at solving problemsas students from other countries (Gonzaleset al., 2008), it makes sense to includeinternational studies in an attempt to learnwhat approaches are effective at teachingproblem solving.18. Reviews of studies for this practice guideapplied WWC Version 2.0 standards. Seehttp://ies.ed.gov/ncee/wwc/pdf/wwc_procedures_v2_standards_handbook.pdf.19. Stodolsky (1988).20. Porter (1989); Weiss et al. (2003).21. Cardelle-Elawar (1995).22. Verschaffel et al. (1999).23. None of these studies estimated impacts ontransfer or maintenance outcomes: Chenand Liu (2007); Ku and Sullivan (2002);Ku et al. (2007). While Ku and Sullivan(2000) found that incorporating class favoriteshad no statistically significant effecton student word-problem performance,students in this study had limited room forgrowth because they had already scoredhigh on the test.aEligible studies that meet WWC evidence standards or meet evidence standards with reservations are indicated by bold text in theendnotes and references pages. For more information about these studies, please see Appendix D.( 70 )

Endnotes (continued)24. Chen and Liu (2007); Ku and Sullivan(2002); Ku et al. (2007).25. National Research Council (2001).26. Pólya (1945).27. Stein and Lane (1996, p. 58).28. Schoenfeld (1992).29. Stein and Lane (1996).30. National Council of Teachers of Mathematics(2000).31. Cummins et al. (1988).32. Mayer (1998).33. Cardelle-Elawar (1995); Chen and Liu(2007); Ku et al. (2007); Ku and Sullivan(2000); Ku and Sullivan (2002);Verschaffel et al. (1999); Koedinger andNathan (2004).34. Cardelle-Elawar (1995).35. Schleppegrell (2007).36. Cardelle-Elawar (1995); Chen and Liu(2007); Ku et al. (2007); Ku and Sullivan(2000); Ku and Sullivan (2002);Verschaffel et al. (1999); Koedinger andNathan (2004).37. Cordova and Lepper (1996).38. Woodward, Monroe, and Baxter (2001).39. Donovan and Bransford (2006).40. For more information on the TIMSS, seehttp://nces.ed.gov/timss/. For more informationon PISA, see http://www.oecd.org. Formore information on Illuminations, see http://illuminations.nctm.org. For more informationon the Math Forum, see http://mathforum.org. For more information on the SAT, seehttp://sat.collegeboard.org/practice.41. Atkinson et al. (2000); Sweller and Cooper(1985); Cooper and Sweller (1987); Kirschner,Sweller, and Clark (2006); Renkl (1997); Wardand Sweller (1990).42. Pashler et al. (2007).43. Ibid.44. Schleppegrell (2007).45. Hohn and Frey (2002); Jitendra et al.(2009); Jitendra et al. (2010); Mevarechand Kramarski (2003); Kramarski andMevarech (2003); King (1991); Cardelle-Elawar (1990); Cardelle-Elawar (1995);Verschaffel et al. (1999).46. Kramarski and Mevarech (2003); Siegler(2003).47. Mevarech and Kramarski (2003);Cardelle-Elawar (1995); King (1991);Kramarski and Mevarech (2003); Verschaffelet al. (1999); Hohn and Frey(2002); Jitendra et al. (2009); Jitendraet al. (2010).48. Verschaffel et al. (1999); Hohn and Frey(2002).49. Jitendra et al. (2009); Jitendra et al.(2010).50. Jitendra et al. (2009).51. Mevarech and Kramarski (2003);Cardelle-Elawar (1995); Cardelle-Elawar(1990); King (1991); Kramarski andMevarech (2003).52. Horn and Frey (2002); Verschaffel et al.(1999); Jitendra et al. (2009); Jitendra etal. (2010).53. Refer to mathematics textbooks at theelementary and middle school levels foradditional sources of questions or task lists.54. King (1991); Jitendra et al. (2009); Jitendraet al. (2010). As Kramarski and Mevarech(2003) show, while students canmonitor and reflect on their problem-solvingprocesses verbally or in writing, they appearto do better when using verbal skills.55. Kramarski and Mevarech (2003).56. As Kramarski and Mevarech (2003) show,while students can monitor and reflect ontheir problem-solving processes individuallyor in small groups, they appear to do betterin small-group settings.57. King (1991).58. Kramarski and Mevarech (2003).59. King (1991).60. Cardelle-Elawar (1995).61. Siegler (2002).62. Verschaffel et al. (1999).63. Adapted from Hohn and Frey (2002).64. Hohn and Frey (2002); Jitendra et al.(2009); Jitendra et al. (2010); Mevarechand Kramarski (2003); Kramarski andMevarech (2003); King (1991); Cardelle-Elawar (1990); Cardelle-Elawar (1995).65. Kramarski and Mevarech (2003).66. King (1991).( 71 )

Endnotes (continued)67. Jitendra et al. (1998); Xin, Jitendra, andDeatline-Buchman (2005); Jitendra et al.(2009); Jitendra et al. (2010); Terwel et al.(2009); Selke, Behr, and Voelker (1991).68. Refer to Diezmann and English (2001) foradditional schematic diagrams.69. Adapted from Selke et al. (1991).70. Adapted from Parker (2004).71. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al.(1998); Selke et al. (1991); Jitendra,DiPipi, and Perron-Jones (2002).72. Xin et al. (2005); Jitendra et al. (1998);Selke et al. (1991); Jitendra et al. (2009);Jitendra et al. (2010).73. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al.(1998).74. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al.(1998).75. Selke et al. (1991).76. Terwel et al. (2009).77. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al.(2010); Selke et al. (1991).78. Rittle-Johnson and Koedinger (2005).79. Diezmann and English (2001); Ng and Lee(2009); Xin et al. (2005).80. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al.(2010).81. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al.(2010); Selke et al. (1991).82. Ng and Lee (2009).83. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al.(2010); Selke et al. (1991).84. Adapted from Jitendra and Star (2009).85. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al. (2010).86. Jitendra et al. (1998); Xin et al. (2005);Jitendra et al. (2009); Jitendra et al. (2010);Selke et al. (1991).87. Terwel et al. (2009).88. Hegarty and Kozhevnikov (1999).89. Adapted from Driscoll (1999).90. Dowker (1992); Siegler (2003).91. Star and Rittle-Johnson (2008).92. Siegler (2003).93. Star and Rittle-Johnson (2008); Ainsworth,O’Malley, and Wood (1998); Rittle-Johnsonand Star (2007); Star and Rittle-Johnson(2009a); Rittle-Johnson and Star (2009);Star and Seifert (2006). The seventh study,Rittle-Johnson et al. (2009), found negativeeffects for students with minimal or nopretest algebra knowledge and no discernibleeffects for students who had attemptedalgebraic reasoning on a pretest.94. The following studies reported positiveeffects for students’ ability to correctly solveproblems: Perels, Gurtler, and Schmitz(2005); Rittle-Johnson and Star (2007);Ginsburg-Block and Fantuzzo (1998);Jitendra et al. (2009); Jitendra et al.(2010). Rittle-Johnson and Star (2009)found no discernible effects. Three studies,each involving students with no or minimalalgebra knowledge, reported negativeeffects for algebra procedural knowledgeoutcomes: Star and Rittle-Johnson (2008);Star and Seifert (2006); Rittle-Johnsonet al. (2009). On conceptual knowledge,Rittle-Johnson and Star (2007) and Starand Rittle-Johnson (2009a) found nodiscernible effects, while Rittle-Johnsonand Star (2009) found a positive effect.Rittle-Johnson et al. (2009) found negativeeffects on understanding of mathematicalconcepts among students who attempted noalgebraic reasoning on a pretest, but therewere no discernible effects for students whohad attempted algebraic reasoning at pretest.95. Jitendra et al. (2009); Jitendra et al.(2010); Perels et al. (2005).96. Star and Rittle-Johnson (2008).97. Rittle-Johnson and Star (2007); Star andRittle-Johnson (2009a); Rittle-Johnsonand Star (2009).98. Rittle-Johnson et al. (2009). Roughly twothirdsof the students who attempted algebraicreasoning often used it incorrectly, meaningthat even these students did not have welldevelopedalgebra problem-solving skills.99. Ginsburg-Block and Fantuzzo (1998).( 72 )

Endnotes (continued)100. Star and Seifert (2006); Star and Rittle-Johnson (2008).101. Jitendra et al. (2009); Jitendra et al.(2010); Perels et al. (2005).102. Jitendra et al. (2009); Jitendra et al. (2010).103. Perels et al. (2005).104. Rittle-Johnson and Star (2007); Rittle-Johnson and Star (2009); Star and Rittle-Johnson (2009a).105. Silver et al. (2005); Stigler and Hiebert (1999);National Council of Teachers of Mathematics(2000).106. Johnson and Johnson (1994).107. Bielaczyc, Pirolli, and Brown (1995); Gentner,Loewenstein, and Thompson (2003).108. Rittle-Johnson and Star (2007); Rittle-Johnson and Star (2009); Star and Rittle-Johnson (2009a).109. Ibid.110. Rittle-Johnson and Star (2009).111. Adapted from Rittle-Johnson and Star(2007).112. Atkinson et al. (2000); Sweller and Cooper(1985); Cooper and Sweller (1987); Kirschneret al. (2006); Renkl (1997); Ward and Sweller(1990).113. Ginsburg-Block and Fantuzzo (1998);Ainsworth et al. (1998); Star and Rittle-Johnson (2008).114. As Ginsburg-Block and Fantuzzo (1998)show, generating multiple strategies in peertutoringdyads appears to be more beneficialto students than having them generatemultiple strategies individually.115. Adapted from Driscoll (2007).116. Adapted from Driscoll (2007).117. Rittle-Johnson et al. (2009).118. Star and Rittle-Johnson (2009b).119. Rittle-Johnson, Star, and Durkin (2009).120. Chapin, O’Connor, and Anderson (2003).121. Chapin et al. (2003).122. Schwartz and Martin (2004).123. Ibid.124. Schwartz, Martin, and Pfaffman (2005).125. Huinker (1992); Koedinger and Anderson(1998); Koedinger and McLaughlin (2010).126. Huinker (1992).127. Tajika et al. (2007); Booth, Koedinger,and Pare-Blagoev (2010); Mwangi andSweller (1998).128. Koedinger and Anderson (1998).Although the panel believes students shouldalso explain each component of an algebraicequation, no study directly tested the importanceof this.129. Koedinger and McLaughlin (2010).130. Garnett (1998); Yetkin (2003); NationalResearch Council (2001).131. National Research Council (2001).132. Huinker (1992).133. Jacobs et al. (2007).134. Garnett (1998); Carpenter et al. (2004);Yetkin (2003).135. Huinker (1992).136. Lehrer (2003).137. Tajika et al. (2007); Mwangi and Sweller(1998); Booth et al. (2010).138. Tajika et al. (2007); Booth et al. (2010).139. Atkinson et al. (2000). For more informationon worked examples, refer to Recommendation2 of the Organizing Instruction and Studyto Improve Student Learning practice guide(Pashler et al., 2007).140. Beckmann (2011).141. Koedinger and Anderson (1998); Koedingerand McLaughlin (2010).142. Koedinger and Anderson (1998).143. Adapted from Koedinger and Anderson(1998).144. Corbett et al. (2006).145. Adapted from Corbett et al. (2006).146. Ma (1999); Hiebert et al. (2003).147. McNeil et al. (2010).148. The following are considered strong correlationalstudies for this practice guide: (1)an analytical model that includes the pretestscore as a statistical covariate (e.g., ANCOVA),(2) a student fixed-effects analytical model,and (3) a two-stage least-squares model thatuses a valid instrumental variable.149. In some studies with multiple relevant interventionand comparison groups, our analysiscombined groups where appropriate tocreate more statistical power. See footnotesin Tables D.1–D.5 for further information.( 73 )

Endnotes (continued)150. For single-case designs, the effect is basedon visual analysis. For more information onvisual analysis, please see the WWC singlecasedesign technical documentation, availableon the IES website at http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.151. Recognizing that some studies lack the statisticalpower to classify practically importanteffects as statistically significant, thepanel also accepts substantively importanteffects as evidence of effectiveness.152. For multiple comparison adjustments andcluster corrections, see the WWC Handbook.153. The panel used Adding It Up (NationalResearch Council, 2001) as a basis for determiningthe three outcome domains, butfurther defined and operationalized them bytaking into account the outcome measuresused in contemporary mathematics research(e.g., Star and Rittle-Johnson, 2007; Rittle-Johnson and Star, 2007).154. When the authors report unadjusted meansand a parallel form pretest, the WWCreportedeffect size incorporates the differences-in-differencesapproach (see theWWC Handbook). When the authors report apretest that is not a parallel form of the outcome,the difference-in-difference approachis not used.155. Cardelle-Elawar (1995). This study had arandomized controlled trial design and didnot report baseline sample sizes for eachcondition (and the author was unable to providethis information). However, based onthe information reported, the panel believesthere was not high attrition.156. Verschaffel et al. (1999). This study hada quasi-experimental design and did notreport standard deviations for the pretest(and the authors were unable to providethis information). However, based on theinformation reported, the panel believesthere is baseline equivalence.157. None of these studies estimated impacts ontransfer or maintenance outcomes: Chenand Liu (2007); Ku and Sullivan (2002);Ku et al. (2007). One other study foundthat incorporating class favorites had nostatistically significant effect on studentword-problem performance, but students inboth conditions scored so high on the posttestthat the outcome may have precludedmeasurement of additional growth (Ku andSullivan, 2000).158. Cardelle-Elawar (1995).159. Verschaffel et al. (1999).160. Chen and Liu (2007); Ku and Sullivan(2000); Ku and Sullivan (2002); Ku et al.(2007).161. Ku and Sullivan (2000); Ku and Sullivan(2002).162. Chen and Liu (2007); Ku et al. (2007).163. Chen and Liu (2007); Ku and Sullivan(2002); Ku et al. (2007).164. Cardelle-Elawar (1995).165. All outcomes for this recommendationinvolved the procedural knowledge domain.166. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.167. Outcomes were not reported separately.168. The WWC could not compute effect sizesbased on the information provided in thepaper. These effect sizes and p-values werereported by the authors.169. The WWC could not compute effect sizesbased on the information provided in thepaper. These effect sizes and p-values werereported by the authors.170. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.171. While the study also included 3rd-gradestudents, those students are not includedin the results reported here.172. Outcomes were not reported separately.173. While the text in this section focuses on theoutcome closest to the end of the intervention,two studies found positive effects withmaintenance outcomes (Jitendra et al., 2009;Mevarech and Kramarski, 2003), and twostudies found positive effects with transferoutcomes (Verschaffel et al. 1999 and( 74 )

Endnotes (continued)Kramarski and Mevarech, 2003). There werealso three single-case design studies thatmet WWC single-case design pilot standardswith reservations (Cassel and Reid, 1996;Montague, 1992; Case et al., 1992). Oneof these studies found strong evidence thatprompting had a positive effect on studentachievement (Cassel and Reid, 1996). Theother two studies found no evidence of aneffect, although the panel believes that oneof these studies (Case et al., 1992) was notan appropriate test of the intervention asstudents were already achieving at very highlevel during baseline (there were “ceilingeffects”). These three studies do not affectthe evidence level.174. Hohn and Frey (2002); Jitendra et al.(2009); Jitendra et al. (2010); Verschaffelet al. (1999).175. Mevarech and Kramarski (2003);Cardelle-Elawar (1990); Cardelle-Elawar(1995); King (1991); Kramarski andMevarech (2003).176. Jitendra et al. (2009); Jitendra et al. (2010).177. Kramarski and Mevarech (2003); Mevarechand Kramarski (2003).178. Ibid.179. King (1991).180. Verschaffel et al. (1999).181. Jitendra et al. (2009); Jitendra et al. (2010).182. Jitendra et al. (2009).183. Jitendra et al. (2010).184. The study by Hohn and Frey (2002) alsoincludes 3rd-grade students, who wereexcluded from the WWC review because thepractice guide targets grades 4 through 8.185. Mevarech and Kramarski (2003);Cardelle-Elawar (1990); Cardelle-Elawar(1995); King (1991); Kramarski andMevarech (2003).186. Cardelle-Elawar (1990); Cardelle-Elawar(1995); King (1991); Kramarski and Mevarech(2003); Mevarech and Kramarski(2003).187. King (1991).188. Hohn and Frey (2002); Jitendra etal. (2009); Jitendra et al. (2010); Verschaffelet al. (1999). To accurately reflectwhat occurred in studies, this appendixdistinguishes between self-questioningand task lists outlining multistep strategies.While different in form, the practicaleffects on student problem solving are oftenindistinguishable. For example, in Mevarechand Kramarski (2003, p. 469), aself-questioning intervention, students areasked, “How is this problem/task differentfrom/similar to what teachers have alreadysolved?” One of the steps in the task-listintervention Hohn and Frey (2002, p. 375)is “Links to the past remind the student torecall similar problems completed earlierthat might suggest a solution plan.”189. Cardelle-Elawar (1990); Cardelle-Elawar(1995). The problem-solving model is presentedin Mayer (1985, 1987).190. Cardelle-Elawar (1990).191. The study Cardelle-Elawar (1995) alsoincludes 3rd-grade students, who wereexcluded from the WWC review because thepractice guide targets grades 4 through 8.192. Kramarski and Mevarech (2003); Mevarechand Kramarski (2003).193. Mevarech and Kramarski (2003).194. Kramarski and Mevarech (2003).195. Cassel and Reid (1996).196. Montague (1992); Case et al. (1992).197. Case et al. (1992).198. All outcomes for this recommendationinvolved the procedural knowledge domain.199. The authors only report outcomes by gender.The WWC reports pooled results thatassume half the students were girls and halfwere boys.200. While the study also includes 3rd-gradestudents, those students are not includedin the results reported here.201. Outcomes were not reported separately.202. While the study also includes 3rd-gradestudents, those students are not includedin the results reported here.203. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.( 75 )

Endnotes (continued)204. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.205. These effect sizes were calculated usingdyad-level (two-person) standard deviationsthat are smaller than the student-levelstandard deviations used to calculate othereffect sizes in this practice guide. Theseeffect sizes are larger than they would beif student-level standard deviations wereused. Following the WWC Handbook, theeffect size, although reported here, was notused when determining the level of evidencefor this recommendation. However, thestatistical significance of this finding wasconsidered when determining the level ofevidence, because cluster-level findings tendto be statistically underpowered.206. Outcomes were not reported separately.207. The WWC could not compute effect sizesbased on the information provided in thepaper. These effect sizes and p-values werereported by the authors.208. The WWC could not compute effect sizesbased on the information provided in thepaper. These effect sizes and p-values werereported by the authors.209. All outcomes for this recommendationinvolved the procedural knowledge domain.210. The WWC does not compute effect sizes forsingle-case design studies. These studies donot affect the evidence level because thesingle-case design studies related to thisrecommendation, as a whole do not meetWWC requirements for combining studies ina summary rating.211. This practice guide only reviewed the instructioninvolving change/equalize word problems.The panel believed that this providedthe most rigorous approach of reviewingthe study design, and the instruction wassufficient to appropriately test the impactof monitoring and reflecting for change/equalize problems.212. This practice guide only reviewed the instructioninvolving addition word problems. Thepanel believed that this provided the mostrigorous approach of reviewing the studydesign, and the instruction was sufficient toappropriately test the impact of monitoringand reflecting for addition problems.213. The findings listed here are from the “metacognitivestrategy instruction.” The studyincludes another single-case design evaluationof a similar intervention (“cognitivestrategy instruction”). The findings aresimilar.214. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al.(1998); Selke et al. (1991). There wasalso one single-case design study that metWWC single-case design pilot standards withreservations (Jitendra et al., 1999). Thisstudy found no evidence that schematicdiagram instruction improved achievement(Jitendra et al., 1999). This study does notaffect the evidence level.215. Xin et al. (2005); Jitendra et al. (1998).216. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al. (1998).217. Selke et al. (1991).218. Terwel et al. (2009).219. Note that a study in the Netherlands foundhigher achievement when students designedand developed their own representations,with assistance from their teachers, thanwhen students simply used visuals providedby their teachers and textbooks (Terwel etal., 2009).220. Selke et al. (1991); Xin et al. (2005); Jitendraet al. (2009); Jitendra et al. (2010);Jitendra et al. (1998).221. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al. (1998).222. Selke et al. (1991).223. Xin et al. (2005); Jitendra et al. (2009);Jitendra et al. (2010); Jitendra et al. (1998).224. Xin et al. (2005, p. 185).225. Jitendra et al. (1998).226. Jitendra et al., 2002; Jitendra et al., 1999.227. Jitendra et al., 2002.228. The study examined instruction on both onestepand two-step word problems (Jitendraet al., 1999). However, this practice guideonly reviewed the instruction involving onestepword problems. The panel believed that( 76 )

Endnotes (continued)this approach enabled the most rigorousreview of study design and that the instructionwas sufficient to appropriately test the impactof visuals. The visual instruction in the studydid have a positive effect on two-step wordproblem achievement, but this outcome didnot meet WWC single-case design standards.229. Xin et al. (2005); Jitendra et al. (1998).230. Xin et al. (2005).231. Selke et al. (1991).232. All outcomes involved the procedural knowledgedomain.233. Eighty-two percent of participants were ingrade 4 or 5 at the time of the study.234. Other important components of the interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.235. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.236. Students were asked to provide the correctarithmetic sentence but not to solve theproblem.237. Another maintenance test was administeredone to two weeks after the intervention.Results from that test are not reported herebecause they do not meet WWC standards;there was high attrition, and the authorsfailed to adjust for baseline differences.238. All outcomes involved the procedural knowledgedomain.239. The WWC does not compute effect sizes forsingle-case design studies. This study doesnot affect the evidence level because thesingle-case design studies related to thisrecommendation, as a whole do not meetWWC requirements for combining studies ina summary rating.240. This practice guide only reviewed the instructioninvolving one-step word problems. Thepanel believed that this approach enabled themost rigorous review of the study design,and the instruction was sufficient to appropriatelytest the impact of visuals.241. With a few exceptions, studies relevant tothe other recommendations in this guideonly measured student improvement in proceduralknowledge. In contrast, many of thestudies relevant to this recommendation alsoexamined outcomes measuring conceptualknowledge and procedural flexibility. Thepanel believes multiple-strategy interventionsare most likely to immediately affectstudents’ procedural flexibility, which is themost closely aligned competency.242. Star and Rittle-Johnson (2008); Ainsworthet al. (1998); Star and Seifert (2006); Rittle-Johnsonand Star (2007); Star andRittle-Johnson (2009a); Rittle-Johnsonand Star (2009). The exception is Rittle-Johnson et al. (2009).243. The following studies reported positiveeffects for procedural knowledge: Perelset al. (2005); Rittle-Johnson and Star(2007); Ginsburg-Block and Fantuzzo(1998); Jitendra et al. (2009); Jitendra etal. (2010). The following studies found noeffects for procedural knowledge: Star andRittle-Johnson (2009a); Rittle-Johnsonand Star (2009). Star and Rittle-Johnson(2008), Star and Seifert (2006), and Rittle-Johnson and Star (2009) found negativeeffects; each of these three studies involvedstudents with no or minimal algebra knowledgesolving algebra problems. Rittle-Johnsonet al. (2009) found that the negativeeffects did not exist for students who haddemonstrated minimal algebra knowledge.244. Rittle-Johnson and Star (2007) and Starand Rittle-Johnson (2009a) found noeffects on conceptual knowledge, whileRittle-Johnson and Star (2009) found apositive effect. Rittle-Johnson et al. (2009)found negative effects for students with nodemonstrated pretest abilities, but no discernibleeffects for students who had at leastminimal algebra knowledge on the pretest.245. Jitendra et al. (2009).246. Jitendra et al. (2010).247. Perels et al. (2005).248. Star and Rittle-Johnson (2008).249. Rittle-Johnson and Star (2007); Rittle-Johnson and Star (2009); Star and Rittle-Johnson (2009a).( 77 )

Endnotes (continued)250. Rittle-Johnson and Star (2009).251. Rittle-Johnson and Star (2007); Starand Rittle-Johnson (2009a). The controlconditions in these studies are only slightlydifferent than the control condition in Rittle-Johnsonand Star (2009), where thedifferent worked-example problems werepresented on the same page.252. Rittle-Johnson and Star (2009).253. Rittle-Johnson and Star (2009).254. The interaction term in the statistical modelwas always positive and was statisticallysignificant for procedural knowledge.255. Star and Rittle-Johnson (2008); Ginsburg-Blockand Fantuzzo (1998).256. Some minor differences also existed betweenstudy samples and content. The study withpositive findings involved 3rd- and 4thgraders(more than half were 4th-graders)solving numbers and operations problems,while the study with the negative resultinvolved students entering the 7th- and8th-grades solving algebra problems.257. Ginsburg-Block and Fantuzzo (1998).258. Star and Rittle-Johnson (2008).259. Star and Seifert (2006).260. Ainsworth et al. (1998).261. Other important components of the interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.262. Other important components of the interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.263. The authors do not report the number ofstudents or the attrition for this comparisonand did not respond to a WWC request forthis information. The WWC believes highattrition was unlikely but was unable toverify that information. To compute effectsizes and statistical significance, the WWCmade conservative assumptions regardingsample size.264. Other important components of the interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.265. A second comparison from this study, usinga different intervention but the same controlgroup, is reported in the Students Generatingand Sharing Multiple Strategies section ofthis table.266. This posttest includes some near-transferitems.267. The effect sizes and statistical significancereported here do not include the imputedstudent data reported in the study.268. The effect sizes and statistical significancereported here do not include the imputedstudent data reported in the study. Thisstudy also included maintenance outcomesthat are not reported here.269. The effect sizes and statistical significancereported here do not include the imputedstudent data reported in the study. Thisstudy also included maintenance outcomesthat are not reported here.270. The study also includes a computation test,but those results are not relevant for thispractice guide.271. Thirteen dyads (i.e., two people) made upeach group. While no attrition occurred atthe dyad level, some attrition took place atthe individual student level. The authors didnot respond to a request for this information;thus, the actual number of students islikely smaller than 52.272. This practice guide only reports results fromthe word problems posttest. A computationtest also exists, but those results are notrelevant for this practice guide.273. When only one member of the dyad took theposttest, the authors used that score as thedyad score. These effect sizes were calculatedusing dyad-level (two-person) standarddeviations that are smaller than the studentlevelstandard deviations used to calculateother effect sizes in this practice guide.These effect sizes are larger than they wouldbe if student-level standard deviations wereused. Consistent with the WWC Handbook,( 78 )

Endnotes (continued)the effect size, although reported here, wasnot used when determining the level of evidencefor this recommendation. However,the statistical significance of this findingwas considered when determining the levelof evidence, because cluster-level findingstend to be statistically underpowered.274. This study was a randomized controlled trialwith unknown attrition. (The author did notrespond to requests for more information.)For this comparison, baseline equivalencedid not meet WWC standards. However, thiscomparison meets strong correlational criteriabecause the statistical model includeda pretest covariate.275. Other important components of this interventionare not listed here. Because of theseother components, the panel cannot determinewhether one particular interventioncomponent or a combination of componentsproduced any differences between groups.276. Thirteen dyads (i.e., two people) made upeach group. While no attrition occurred atthe dyad level, some attrition took place atthe individual student level. The authors didnot respond to a request for this information;thus, the actual number of students islikely smaller than 52.277. This practice guide only reports results fromthe word problems posttest. A computationtest also exists, but those results are notrelevant for this practice guide.278. When only one member of the dyad took theposttest, the authors used that score as thedyad score. These effect sizes were calculatedusing dyad-level (two-person) standarddeviations that are smaller than the studentlevelstandard deviations used to calculateother effect sizes in this practice guide.These effect sizes are larger than they wouldbe if student-level standard deviations wereused. Consistent with the WWC Handbook,the effect size, although reported here, wasnot used when determining the level of evidencefor this recommendation. However,the statistical significance of this findingwas considered when determining the levelof evidence, because cluster-level findingstend to be statistically underpowered.279. A second comparison from this study, usinga different intervention but the same controlgroup, is reported in the Instruction in MultipleStrategies section of this table.280. Huinker (1992); Koedinger and Anderson(1998); Koedinger and McLaughlin(2010).281. Huinker (1992).282. Koedinger and Anderson (1998); Koedingerand McLaughlin (2010).283. Tajika et al. (2007); Booth et al. (2010);Mwangi and Sweller (1998).284. Huinker (1992).285. In two other outcomes, the study did notmeet WWC standards.286. Tajika et al. (2007); Booth et al. (2010);Mwangi and Sweller (1998).287. While 9th-grade students are outside thegrade range for this practice guide, thepanel believes results from this sample arelikely to apply to middle school studentsinvolved in algebra problem solving.288. The panel had concerns about the outcomemeasures. Only one of the four studies providedinformation on reliability of outcomemeasures, and none of the measures wasstandardized. The outcome measure in onestudy, Mwangi and Sweller (1998), hadonly three problems.289. Booth et al. (2010).290. Tajika et al. (2007).291. Mwangi and Sweller (1998).292. Koedinger and Anderson (1998). Whilehigh school students are outside the graderange for this practice guide, the panelbelieves results from this sample are likelyto apply to middle school students involvedin algebraic problem solving.293. Intermediate arithmetic problems are problemswith the same mathematical representationas an algebra problem but in whichstudents are provided with values for theunknown and are asked to solve for the result.294. The authors did not report enough informationto enable the WWC to confirm statisticalsignificance or calculate effect sizes.295. Koedinger and McLaughlin (2010).296. All outcomes, except for Booth et al. (2010),involved the procedural knowledge domain.297. Other important components of this interventionare not listed here. Because of these othercomponents, the panel cannot determine( 79 )

Endnotes (continued)whether one particular intervention componentor a combination of componentsproduced any differences between groups.298. Two other outcome measures did not meetWWC standards because the samples hadhigh attrition and did not demonstrate baselineequivalence.299. The students were all in Algebra I classrooms.300. The study also measured procedural knowledge.However, the internal consistency ofthat outcome did not meet WWC reliabilitystandards, and results are not reported here.301. Three experiments made up this study. Onlythe sample for the third experiment met theage requirements for this practice guide.302. The study was conducted the summer afterthe students completed an algebra course.303. Outcome items were interspersed withinstructional problems. Specifically, all participantsfirst received two practice problems,then one outcome item, then onepractice problem, then one outcome item,then one practice problem, then one outcomeitem.( 80 )

References aAinsworth, S., O’Malley, C., & Wood, D.(1998). There is more than one way tosolve a problem: Evaluating a learningenvironment that supports the developmentof children’s multiplication skills.Learning and Instruction, 8(2), 141–157.American Educational Research Association,American Psychological Association, & NationalCouncil on Measurement in Education. (1999).The standards for educational and psychologicaltesting. Washington, DC: American EducationalResearch Association Publications.American Psychological Association. (2002). Criteriafor practice guideline development andevaluation. American Psychologist, 57(12),1048–1051.Atkinson, R. K., Derry, S. J., Renkl, A., &Wortham, D. (2000). Learning fromexamples: Instructional principles fromthe worked examples research. Reviewof Educational Research, 70, 181–214.Beckmann, S. (2011). Mathematics for elementaryteachers (3rd ed.). Boston: Addison-Wesley.Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995).Training in self-explanation and self-regulationstrategies: Investigating the effects of knowledgeacquisition activities on problem solving.Cognition and Instruction, 13, 221–252.Booth, J. L., Koedinger, K. R., & Pare-Blagoev, J. (2010). Does practice reallymake perfect? Improving upon theubiquitous practice assignment. Unpublishedmanuscript.Cardelle-Elawar, M. (1990). Effects of feedbacktailored to bilingual students’ mathematicsneeds on verbal problem solving.Elementary School Journal, 91(2), 165.Cardelle-Elawar, M. (1995). Effects of metacognitiveinstruction on low achieversin mathematics problems. Teaching andTeacher Education, 11(1), 81–95.Carpenter, T. P., Blanton, M. L., Cobb, P., Franke,M. L., Kaput, J., & McClain, K. (2004). Scalingup innovative practices in mathematics andscience. Madison, WI: National Center forImproving Student Learning and Achievementin Mathematics and Science.Case, L. P., Harris, K. R., & Graham, S.(1992). Improving the mathematicalproblem solving skills of students withlearning disabilities: Self-regulatedstrategy development. The Journal ofSpecial Education, 26(1), 1–19.Cassel, J., & Reid, R. (1996). Use of aself-regulated strategy intervention toimprove word problem–solving skills ofstudents with mild disabilities. Journalof Behavioral Education, 6(2), 153–172.Chapin, S., O’Connor, C., & Anderson, N. (2003).Classroom discussions: Using math talk tohelp students learn, grades 1–6. Sausalito,CA: Math Solutions Publications.Chen, C., & Liu, P. (2007). Personalizedcomputer-assisted mathematics problem-solvingprogram and its impacton Taiwanese students. Journal ofComputers in Mathematics and ScienceTeaching, 26(2), 105–121.Common Core Standards Initiative. (2010). Commoncore state standards for mathematics.Retrieved from http://www.corestandards.org/assets/CCSSI_Math%20Standards.pdfCooper, G., & Sweller, J. (1987). The effects ofschema acquisition and rule automation onmathematical problem-solving transfer. Journalof Educational Psychology, 79, 347–362.Corbett, A., Wagner, A., Lesgold, S.,Ulrich, H., & Stevens, S. (2006). Theimpact on learning of generating vs.selecting descriptions in analyzingalgebra example solutions. In Proceedingsof the 7th InternationalConference on Learning Sciences(pp. 99–105). Mahwah, NJ: LawrenceErlbaum Associates.Cordova, D. I., & Lepper, M. R. (1996). Intrinsicmotivation and the process of learning: Beneficialeffects of contextualization, personalization,and choice. Journal of EducationalPsychology, 88(4), 715–730.Cummins, D. D., Kintsch, W., Reusser, K., &Weimer, R. (1988). The role of understandingin solving word problems. Cognitive Psychology,20, 405–438.aEligible studies that meet WWC evidence standards or meet evidence standards with reservations are indicated by bold text in theendnotes and references pages. For more information about these studies, please see Appendix D.( 81 )

References (continued)Diezmann, C., & English, L. (2001). The roles ofrepresentation in school mathematics: 2001yearbook (pp. 1–23). Reston, VA: NationalCouncil of Teachers of Mathematics.Donovan, M. S., & Bransford, J. D. (2006). Howstudents learn: Mathematics in the classroom.Washington, DC: National Academies Press.Dowker, A. (1992). Computational estimationstrategies of professional mathematicians.Journal for Research in Mathematics Education,23(1), 45–55.Driscoll, M. (1999). Fostering algebraic thinking:A guide for teachers, grades 6–10. New York:Heinemann.Driscoll, M. (2007). Fostering geometric thinking:A guide for teachers, grades 5–10. New York:Heinemann.Garnett, K. (1998). Math learning disabilities.Division for Learning Disabilities Journal ofCEC. Retrieved from http://www.ldonline.org/article/5896Gentner, D., Loewenstein, J., & Thompson, L.(2003). Learning and transfer: A general rolefor analogical encoding. Journal of EducationalPsychology, 95, 393–405.Ginsburg-Block, M., & Fantuzzo, J. W.(1998). An evaluation of the relativeeffectiveness of NCTM standards-basedinterventions for low-achieving urbanelementary students. Journal of EducationalPsychology, 90(3), 560–569.Gonzales, P., Williams, T., Jocelyn, L., Roey, S.,Kastberg, D., & Brenwald, S. (2008). Highlightsfrom TIMSS 2007: Mathematics andscience achievement of U.S. fourth- andeighth-grade students in an international context(NCES 2009-001 revised). Washington,DC: National Center for Education Statistics,Institute of Education Sciences, U.S. Departmentof Education.Hegarty, M., & Kozhevnikov, M. (1999). Types ofvisual-spatial representations and mathematicalproblem solving. Journal of EducationalPsychology, 91, 684–689.Hiebert, J., Carpenter, T., Fennema, E., Fuson, K.,Wearne, D., Murray, H., Oliver, A., & Human, P.(1997) Making sense: Teaching and learningmathematics with understanding. Portsmouth,NH: Heinemann.Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B.,Hollingsworth, H., Jacobs, J. Stigler, J. (2003).Teaching mathematics in seven countries:Results from the TIMSS 1999 video study.Washington, DC: National Center for EducationStatistics, Institute of Education Sciences,U.S. Department of Education.Hohn, R. L., & Frey, B. (2002). Heuristictraining and performance in elementarymathematical problem solving.Journal of Educational Research, 95(6),374–380.Huinker, D. M. (1992, April). Effects ofinstruction using part-whole conceptswith one-step and two-step word problems.Paper presented at the annualmeeting of the American EducationalResearch Association, San Francisco, CA.Jacobs, V. R., Franke, M. L., Carpenter, T. P., Levi,L., & Battey, D. (2007). Professional developmentfocused on children’s algebraic reasoningin elementary school. Journal for Researchin Mathematics Education, 38(3), 258–288.Jitendra, A., DiPipi, C. M., & Perron-Jones, N.(2002). An exploratory study of schemabasedword-problem-solving instructionfor middle school students with learningdisabilities: An emphasis on conceptualand procedural understanding. The Journalof Special Education, 36(1), 23–38.Jitendra, A. K., Griffin, C. C., McGoey, K.,Gardill, M. C., Bhat, P., & Riley, T. (1998).Effects of mathematical word problemsolving by students at risk or withmild disabilities. Journal of EducationalResearch, 91(6), 345–355.Jitendra, A. K., Hoff, K., & Beck, M. B.(1999). Teaching middle school studentswith learning disabilities to solveword problems using a schema-basedapproach. Remedial and Special Education,20(1), 50–64.Jitendra, A. K., & Star, J. R. (2009). Teachingratio and proportion word problem solvingusing schema-based instruction. Unpublishedcurriculum written for a study funded by theInstitute of Education Sciences: Enhancing themathematical problem solving performanceof seventh grade students using schemabasedinstruction (RFA: 12-01-OD-10).( 82 )

References (continued)Jitendra, A. K., Star, J. R., Rodriguez, M.,Lindell, M., & Someki, F. (2010). Improvingstudents’ proportional thinkingusing schema-based instruction. Manuscriptsubmitted for publication.Jitendra, A. K., Star, J. R., Starosta, K., Leh,J. M., Sood, S., Caskie, G., & Mack, T. R.(2009). Improving seventh grade students’learning of ratio and proportion:The role of schema-based instruction.Contemporary Educational Psychology,34(3), 250–264.Johnson, D. W., & Johnson, R. T. (1994). Learningtogether and alone: Cooperative, competitive,and individualistic learning (4th ed.). Boston,MA: Allyn and Bacon.Jones, D., & Tarr, J. (2007). An examination ofthe levels of cognitive demand required byprobability tasks in middle grades mathematicstextbooks. Statistics Education ResearchJournal, 6(2), 24–27.King, A. (1991). Effects of training in strategicquestioning on children’s problem-solvingperformance. Journal ofEducational Psychology, 83(3), 307–317.Kirsch, I., Braun, H., Yamamoto, K., & Sum, A.(2007). America’s perfect storm: Three forceschanging our nation’s future. Princeton, NJ:Educational Testing Service.Kirschner, P. A., Sweller, J., & Clark, R. E. (2006).Why minimal guidance during instructiondoes not work: An analysis of the failure ofconstructivist, discovery, problem-based,experiential, and inquiry-based teaching.Educational Psychologist, 41, 75–86.Koedinger, K. R., & Anderson, J. R. (1998).Illustrating principled design: The earlyevolution of a cognitive tutor for algebrasymbolization. Interactive LearningEnvironments, 5(1), 161–179.Koedinger, K. R. & McLaughlin, E. A. (2010).Seeing language learning inside themath: Cognitive analysis yields transfer.In S. Ohlsson & R. Catrambone(Eds.), Proceedings of the 32nd AnnualConference of the Cognitive ScienceSociety (pp. 471–476). Austin, TX: CognitiveScience Society.Koedinger, K. R., & Nathan, M. J. (2004). Thereal story behind story problems: Effects ofrepresentations on quantitative reasoning.Journal of the Learning Sciences, 13, 129–164.Kramarski, B., & Mevarech, Z. R. (2003).Enhancing mathematical reasoning inthe classroom: The effects of cooperativelearning and metacognitive training.American Educational ResearchJournal, 40(1), 281–310.Ku, H., Harter, C. A., Liu, P., Thompson, L.,& Cheng, Y. (2007). The effects of individuallypersonalized computer-basedinstructional program on solving mathematicsproblems. Computers in HumanBehavior, 23(3), 1195–1210.Ku, H., & Sullivan, H. (2000). Personalizationof mathematics word problems in Taiwan.Educational Technology Researchand Development, 48(3), 49–60.Ku, H., & Sullivan, H. J. (2002). Studentperformance and attitudes using personalizedmathematics instruction.Educational Technology Research andDevelopment, 50(1), 21–34.Lehrer, R. (2003). Developing understanding ofmeasurement. In J. Kilpatrick, W. G. Martin,& D. Schifter (Eds.), A research companion toprinciples and standards for school mathematics(pp. 179–192). Reston, VA: National Councilof Teachers of Mathematics.Lemke, M., Sen, A., Pahlke, E., Partelow, L., Miller,D., Williams, T., & Jocelyn, L. (2004). Internationaloutcomes of learning in mathematicsliteracy and problem solving: PISA 2003 resultsfrom the U.S. perspective (NCES 2005-003).Washington, DC: National Center for EducationStatistics, Institute of Education Sciences,U.S. Department of Education.Lesh, R., Hamilton, E., & Kaput, J. (2007). Foundationsfor the future in mathematics education.Mahwah, NJ: Lawrence Erlbaum Associates.Levy, F., & Murnane, R. (2004). The new divisionof labor: How computers are creatingthe next job market. Princeton, NJ: PrincetonUniversity Press.Ma, L. (1999). Knowing and teaching elementarymathematics: Teachers’ understandingof fundamental mathematics in China andthe United States. Hillsdale, NJ: LawrenceErlbaum Associates.( 83 )

References (continued)Mayer, R. E. (1985). Mathematical ability. In R. J.Sternberg (Ed.), Human abilities: An informationprocessing approach. (pp. 127–150). NewYork: W. H. Freeman.Mayer, R. E. (1987). Educational psychology.Boston: Little, Brown.Mayer, R. E. (1992). Thinking, problem solving,cognition. New York: W. H. Freeman.Mayer, R. E. (1998). Cognitive, metacognitiveand motivational aspects of problem solving.Instructional Science, 26, 49–63.Mayer, R. E. & Hegarty, M. (1996). The processof understanding mathematical problems.In R. J. Sternberg & T. Ben-Zeev (Eds.), Thenature of mathematical thinking. Hillsdale,NJ: Erlbaum.McCloskey, M. (2007). Quantitative literacy anddevelopmental dyscalculias. In D. B. Berch &M. M. Mazzocco (Eds.), Why is math so hardfor some children? The nature and origins ofchildren's mathematical learning difficultiesand disabilities (pp. 415–429). Baltimore:Brookes Publishers.McNeil, N. M., Weinberg, A., Hattikudur, S.,Stephens, A. C., Asquith, P., Knuth, E. J.,& Alibali, M. W. (2010). A is for apple: Mnemonicsymbols hinder the interpretation ofalgebraic expressions. Journal of EducationalPsychology, 102(3), 625-634.Mevarech, Z. R., & Kramarski, B. (2003).The effects of metacognitive trainingvs. worked-out examples on students’mathematical reasoning. British Journalof Educational Psychology, 73(4), 449–471.Montague, M. (1992). The effects of cognitiveand metacognitive strategy instructionon the mathematical problemsolving of middle school students withlearning disabilities. Journal of LearningDisabilities, 25(4), 230–248.Mwangi, W., & Sweller, J. (1998). Learningto solve compare word problems: Theeffect of example format and generatingself-explanations. Cognition andInstruction, 16(2), 173–199.National Academy of Sciences. (2007).Rising above the gathering storm: Energizingand employing America for a brightereconomic future. Washington, DC: NationalAcademies Press.National Council of Teachers of Mathematics.(2000). Principles and standards for schoolmathematics. Reston, VA: Author.National Governors Association Center for BestPractices and Council of Chief State SchoolOfficers (2010). Common core state standardsfor mathematics. Retrieved from http://www.corestandards.org/assets/CCSSI_Math%20Standards.pdfNational Mathematics Advisory Panel (2008).“Foundations for Success: The Final Reportof the National Mathematics AdvisoryPanel.” Washington, DC: U.S. Departmentof Education.National Research Council. (2001). Adding itup: Helping children learn mathematics.J. Kilpatrick, J. Swafford, & B. Findell (Eds.).Washington, DC: National Academies Press,Mathematics Learning Study Committee,Center for Education, Division of Behavioraland Social Sciences and Education.Ng, S. F., & Lee, K. (2009). The model method:Singapore children’s tool for representingand solving algebraic word problems. Journalfor Research in Mathematics Education,40, 282–313.Parker, M. (2004). Reasoning and workingproportionally with percents. MathematicsTeaching in the Middle School, 9(6), 326–330.Pashler, H., Bain, P. M., Bottge, B. A., Graesser,A., Koedinger, K., McDaniel, M., & Metcalfe,J. (2007). Organizing instruction and studyto improve student learning. (NCER 2007-2004). Washington, DC: National Center forEducation Research, Institute of EducationSciences, U.S. Department of Education.Perels, F., Gurtler, T., & Schmitz, B. (2005).Training of self-regulatory and problem-solvingcompetence. Learning andInstruction, 15(2), 123–139.Perie, M., Grigg, W., & Dion, G. (2005). The nation’sreport card: Mathematics 2005 (NCST 2006-453). Washington, DC: Government PrintingOffice, U.S. Department of Education, NationalCenter for Education Statistics.Pólya, G. (1945). How to solve it. Princeton, NJ:Princeton University Press.Porter, A. (1989). A curriculum out of balance:The case of elementary school mathematics.Educational Researcher, 18(5), 9–15.( 84 )

References (continued)Renkl, A. (1997). Learning from worked-outexamples: A study on individual differences.Cognitive Science, 21, 1–29.Reys, B. J., Dingman, S., Olson, T. A., Sutter, A.,Teuscher, D., Chval, K., . . . Kasmer, L. (2006).Executive summary: The intended mathematicscurriculum as represented in state-levelcurriculum standards: Consensus or confusion?Retrieved from http://www.mathcurriculumcenter.org/st_std_exec_sum.pdfRittle-Johnson, B., & Koedinger, K. R. (2005).Designing better learning environments:Knowledge scaffolding supports mathematicalproblem solving. Cognition and Instruction,23, 313–349.Rittle-Johnson, B., & Star, J. R. (2007). Doescomparing solution methods facilitateconceptual and procedural knowledge?An experimental study on learning tosolve equations. Journal of EducationalPsychology, 99(3), 561–574.Rittle-Johnson, B., & Star, J. R. (2009).Compared with what? The effects ofdifferent comparisons on conceptualknowledge and procedural flexibilityfor equation solving. Journal of EducationalPsychology, 101(3), 529–544.Rittle-Johnson, B., Star, J. R., Durkin, K.(2009). The importance of prior knowledgewhen comparing examples: Influenceson conceptual and proceduralknowledge of equation solving. Journalof Educational Psychology, 201, 836–852.Schleppegrell, M. J. (2007). The linguistic challengesof mathematics teaching and learning:A research review. Reading and WritingQuarterly, 23, 139–159.Schoenfeld, A. H. (1988). When good teachingleads to bad results: The disasters of “welltaught”mathematics courses. EducationalPsychologist, 23, 145–166.Schoenfeld, A. H. (1992). Learning to thinkmathematically: Problem solving, metacognition,and sense making in mathematics.In D. Grouws (Ed.), Handbook for researchon mathematics teaching and learning (pp.334–370). New York: Macmillan.Schwartz, D. L., & Martin, T. (2004). Inventing toprepare for learning: The hidden efficiencyof original student production in statisticsinstruction. Cognition and Instruction, 22,129–184.Schwartz, D. L., Martin, T., & Pfaffman, J. (2005).How mathematics propels the developmentof physical knowledge. Journal of Cognitionand Development, 6, 65–88.Selke, D. H., Behr, M. J., & Voelker, A. M.(1991). Using data tables to representand solve multiplicative story problems.Journal for Research in MathematicsEducation, 22(1), 30–38.Siegler, R. S. (2002). Microgenetic studies ofself-explanations. In N. Granott & J. Parziale(Eds.), Microdevelopment: Transitionprocesses in development and learning (pp.31–58). New York: Cambridge University.Siegler, R. S. (2003). Implications of cognitivescience research for mathematics education.In J. Kilpatrick, W. G. Martin, & D. Schifter(Eds.), A research companion to principlesand standards for school mathematics (pp.289–303). Reston, VA: National Council ofTeachers of Mathematics.Silver, E. A., Ghousseini, H., Gosen, D., Charalambous,C., & Strawhun, B. (2005). Movingfrom rhetoric to praxis: Issues faced byteachers in having students consider multiplesolutions for problems in the mathematicsclassroom. Journal of Mathematical Behavior,24, 287–301.Star, J. R., & Rittle-Johnson, B. (2008). Flexibilityin problem solving: The case ofequation solving. Learning and Instruction,18(6), 565–579.Star, J. R., & Rittle-Johnson, B. (2009a). Itpays to compare: An experimental studyon computational estimation. Journal ofExperimental Child Psychology, 102(4),408–426.Star, J. R., & Rittle-Johnson, B. (2009b). Makingalgebra work: Instructional strategiesthat deepen student understanding, withinand between algebraic representations. ERSSpectrum, 27(2), 11–18.Star, J. R. & Seifert, C. (2006). The developmentof flexibility in equation solving.Contemporary Educational Psychology,31, 280–300.Stein, M. K., & Lane, S. (1996). Instructional tasksand the development of student capacity to( 85 )

References (continued)think and reason: An analysis of the relationshipbetween teaching and learning ina reform mathematics project. EducationalResearch and Evaluation, 2, 50–80.Stigler, J. W., & Hiebert, J. (1999). The teachinggap: Best ideas from the world’s teachers forimproving education in the classroom. NewYork: Free Press.Stodolsky, S. S. (1988). The subject matters:Classroom activity in math and social studies.Chicago: University of Chicago Press.Sweller, J., & Cooper, G. A. (1985). The use ofworked examples as a substitute for problemsolving in learning algebra. Cognition andInstruction, 2(1), 59–89.Tajika, H., Nakatsu, N., Nozaki, H., Neumann,E., & Maruno, S. (2007). Effectsof self-explanation as a metacognitivestrategy for solving mathematicalword problems. Japanese PsychologicalResearch, 49(3), 222–233.Terwel, J., Van Oers, B., Van Dijk, I. M. A. W.,& Van den Eeden, P. (2009). Are representationsto be provided or generated inprimary mathematics education? Effectson transfer. Educational Research andEvaluation, 15(1), 25–44.Uhalde, R., Strohl, J., & Simkins, Z. (2006). Americain the global economy: A background paperfor the new commission on the skills of theAmerican workforce. Washington, DC: NationalCenter on Education and the Economy.Verschaffel, L., De Corte, E., Lasure, S.,Van Vaerenbergh, G., Bogaerts, H., &Ratinck, E. (1999). Learning to solvemathematical application problems: Adesign experiment with fifth graders.Mathematical Thinking and Learning,1(3), 195–229.Ward, M., & Sweller, J. (1990). Structuring effectiveworked examples. Cognition and Instruction,7, 1–39.Weiss, I. R., Pasley, J. D., Smith, P. S., Banilower,E. R., & Heck, D. J. (2003). Looking inside theclassroom: A study of K–12 mathematics andscience education in the United States. ChapelHill, NC: Horizon Research, Inc.Woodward, J., Monroe, K., & Baxter, J. (2001).Enhancing student achievement on performanceassessment in mathematics. LearningDisability Quarterly, 24, 33–46.Xin, Y. P., Jitendra, A. K., & Deatline-Buchman,A. (2005). Effects of mathematicalword problem-solving instruction onmiddle school students with learningproblems. The Journal of Special Education,39(3), 181–192.Yetkin, E. (2003). Student difficulties in learningelementary mathematics. Columbus, OH:ERIC Clearinghouse for Science, Mathematicsand Environmental Education. Retrievedfrom http://www.eric.ed.gov( 86 )