Criteria for Identifying Scalable STEM Projects - Massachusetts ...

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsBackgroundA Foundation for the Future: Massachusetts’ Plan for Excellence in STEM Education addresses thecritical need to encourage and support an ever more diverse body of Massachusetts students to befluent in STEM fields, and to pursue STEM education, leading to careers as future STEM professionals.The success of this endeavor, as articulated in the Plan’s Theory of Action, depends on whole-systemtransformation beginning in pre-kindergarten and continuing through post-secondary education.StrategiesThree key strategies guide implementation of the Plan: improve coordination and alignment of themany independent STEM efforts across the state; shape and coalesce a limited number of STEM projectsthat demonstrate essential characteristics to be brought to scale; and shape STEM-related policydecisions that advance the goals of the Plan. Through these strategies Massachusetts will develop newinnovative partnerships, close programmatic gaps, and engage individuals and organizations with thestate’s STEM efforts in a systemic and systematic way to achieve the goals of the Plan.ThemesThree main themes help focus the broad framework of the Plan. These themes highlight the purpose ofeach STEM program, and can be used to identify relationships among the many STEM-orientedprograms in the state. Projects for scale would focus on one or more of these thematic areas.The themes, and their alignment to various elements of the Plan, are identified in the following table:ThemeRaising STEM interest andawareness through partnershipsand community engagementIncreasing student engagementand achievement in STEMthrough classroom programs,enrichment and support, andpersistence in attainment of apostsecondary STEM degree orcareer.Promoting PK-16 educatoreffectiveness throughdevelopment experiencesAlignment toquantitative goals• Student interest(including a focusonunderrepresentedminority students)• Achievement• College preparation• STEM collegedegrees(including a focus ineach goal onunderrepresentedminority students)• Effective PK-16educatorsAlignment toqualitativegoals• Communityengagement• STEMemployersandprofessionals• Academiccoherence• STEMemployersandprofessionals• PK-16educatordevelopment• STEMemployersandprofessionalsAlignment toTheory ofAction elements• Communityengagement• Employers• Industry andeducationprofessionals• Academiccoherence• Employers• Industry andeducationprofessionals• Educatordevelopment• Employers• Industry andeducationprofessionalsAlignment tosubcommitteereport headings• Partnerships• Awareness• Enrichment• Curriculum,instructionandassessment• Educatordevelopment1STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsScale-UpThis document defines the strategy and the criteria for scaling up STEM projects that have thepotential to create more widespread impact on the goals of the STEM Plan.The Operations Board, a subcommittee of the Governor’s STEM Advisory Council, proposes thecriteria in the following sections of this document to inform an assessment of projects for scaleup(“@Scale projects”). These criteria draw upon:• critical gaps and needs as identified through our analyses of STEM themes and state data;• the quantitative goals (Appendix A) defined in the STEM Plan;• the qualitative goals (Appendix B) defined in the STEM Plan;• research-based characteristics for project scale-up;The Operations Board will:• publish this document to inform Regional STEM Network and project leaders of the scaleupcriteria;• prioritize crucial themes based on regional or statewide programmatic and partnershipgaps, and informed by analysis of state data;• call for applications of projects that address prioritized needs and demonstrate readinessfor scale-up;• screen applications based on the criteria of this document, and assess a limited number ofprojects that are determined most ready for scale-up;• recommend @Scale projects to the Governor’s STEM Advisory Council for endorsement.The Governor’s STEM Advisory Council will identify STEM projects to endorse for scale-up;project managers can then seek private funding to support scale-up implementation.The following sections define the criteria for assessing and identifying @Scale projects.2STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsCriteria IQuantitative GoalsThe Massachusetts STEM Plan lists five quantitative goals (see Appendix A for more detail):1. Increase student interest in STEM2. Increase STEM achievement among PreK-12 students3. Increase the percentage of students who demonstrate readiness for college-level study in STEMfields4. Increase the number of students who graduate from a post-secondary institution with a degreein a STEM field5. Increase the number/percentage of PreK-16 STEM classes led by effective educators.1) The program must be aligned with at least one of the Plan’s quantitative goals.2) For each of the quantitative goals identified in Item 1, the program must provide evidence ofachievement, or progress toward achievement, of the goal.3) An evaluation process exists which validates the program’s activities as they relate to thequantitative goals.4) Evaluation data must be provided.3STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsCriteria IIQualitative GoalsThe following four qualitative goals (see Appendix B for more detail) drive the Theory of Action of theMassachusetts STEM Plan:1. Diverse Students and Future STEM Professionals2. Community Engagement3. Academic Coherence4. Educator Development1) The program must be aligned with at least one of the Plan’s qualitative goals.2) Current, accepted educational research must validate the program’s approach and methodology toachieving its stated qualitative goal.3) There must be thoughtful consideration of systemic change, in the context of the qualitative goals,that will occur over time as a result of the scale-up of the program.4STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsCriteria IIIScalabilityAccepted research has shown the following characteristics to be indicators of successful scale-up.1) Users of the program must demonstrate shared ownership of the program. (E.g., educators whohave participated in a Professional Development program, have then applied the learnings to theirpractice.)2) Evidence must exist that the program has caused changes in practice at its present site.3) The program must demonstrate some level of successful scaling since its inception. (e.g., classroom-> classroom)4) The scale-up of the program must have a positive, measurable impact on the Standard (c.f. AppendixA of this document) of each of the goals stated in Criteria I Item 1 of this document.5) On a unit basis, the cost per student, or cost per educator, of the scale-out of the program must bereasonable.6) The program must be packaged for ease of implementation at an adopting site:a. An “implementation guide” must be available for an adopting site.b. Leadership and other resources required for successful implementation at an adopting site mustbe identified.7) In order to be in position to “pass the torch”, the host institution / lead partner must:a. demonstrate the capacity and capability to support adopters (e.g., train the trainer);b. be prepared to act as consultant to leadership of an adopting site;c. identify partners who will sustain the program, if the organization itself is not in the business ofdoing so.8) In order to avoid traps and implementation barriers,a. the skills and knowledge necessary to implement the program must be easily transferrable;b. any infrastructure requirements (e.g., technology) necessary to implement the program must beidentified.5STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsAppendix AState STEM Plan Quantitative Goals and StandardsThe Massachusetts STEM Plan lists five quantitative goals:Goal 1: Increase student interest in STEM.Standard: Increase interest in STEM college majors among college-going MA public schoolgraduates to 35% by 2016 (from 25% in 2009).• Increase interest among the underrepresented gender in fields with a gender-based gapin interest.• Increase interest among underrepresented races/ethnicities in fields with arace/ethnicity-based gap in interest.• Increase interest in fields where there are anticipated gaps in future employment (fromindustry growth and/or from retirement of current employees).• Increase interest in STEM fields at early ages (including preschool and elementary school)to assist in increasing student motivation to attain higher levels of STEM academicachievement/performance.Goal 2: Increase STEM achievement among PreK-12 students.Standard: Increase the percentage of all students scoring Proficient or Advanced on theMCAS mathematics and science & technology/engineering assessments:• Increase the percentage of all 5 th and 8 th grade students scoring Proficient or Advanced onmathematics and science & technology/engineering MCAS assessments by 20 percentagepoints by 2016.• Increase the percentage of all high school students scoring Proficient or Advanced onmathematics and science & technology/engineering MCAS assessments by 10 percentagepoints by 2016.• Reduce the achievement gaps of 5 th grade, 8 th grade, and high school students on themathematics and science & technology/engineering MCAS assessments by 25% between2010 and 2014, and another 25% between 2014 and 2016.6STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsAppendix AGoal 3: Increase the percentage of students who demonstrate readiness for college-levelstudy in STEM fields.Standard: Increase the percentage of MA public high school students who report taking at least 4years of math (from 69% in 2009 [SAT]) and 3 years of lab-based science (from 79% in 2009 [SAT]) to100% in 2016, consistent with MassCORE, as well as increase the percentage of MA public highschool students who report taking advanced mathematics (pre-calculus and above) to 55% (from 44%in 2009 [SAT]) by 2016.• Increase STEM course-taking among the underrepresented gender in courses with a genderbasedgap in participation.• Increase STEM course-taking among underrepresented races/ethnicities in courses with arace/ethnicity-based gap in participation.Goal 4: Increase the number of students who graduate from a post-secondary institution witha degree in a STEM field.Standard: Increase the number of students who complete STEM post-secondary degrees at MApublic and private institutions by 50% from 2008 to 2016.• Increase the number of Bachelor's degrees granted in all STEM majors to all students by 50%by 2016.• Increase the number of Bachelor's degrees granted in all STEM majors to theunderrepresented gender in majors with a gender-based gap in degrees.• Increase the number of Bachelor's degrees granted in all STEM majors to theunderrepresented gender in majors with a gender-based gap in degrees.Goal 5: Increase the number/percentage of STEM classes led by effective educators, fromPreK-16.Standard: TBD`• Future measure of STEM qualifications of Pre-K-16 educators (TBD; likely will vary by level:elementary, secondary, post-secondary)• Future measure of STEM effectiveness of Pre-K-16 educators (TBD: likely will vary by level:elementary, secondary, post-secondary)7STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsAppendix B State STEM Plan Qualitative Goals for TransformationThe following four qualitative goals provide a context for the scope of the change and theincreased capacity required to achieve the intended whole-system transformation.A. Community EngagementEvery Massachusetts community will foster increased student interest in STEM throughprogramming and spreading awareness. To spark and sustain student awareness of,interest in and motivation to pursue advanced STEM education and related careers…• In every community parents, educators, employers, student leaders and STEMprofessionals will be informed and enlisted as advocates to influence, support andsustain student commitment to STEM from Pre-K through post-secondary education.• PreK-16 students will have access to rigorous academic and technical preparation inthe STEM subjects and be encouraged to engage in experiential and applied learningopportunities.• Collaboration is critical. Effective collaboration can enhance existing opportunities andbolster the development of systems at the community level to engage students atvarious points along the STEM pipeline – from preschool to career.B. Academic CoherenceMassachusetts STEM standards, curriculum frameworks, instruction and assessmentswill…• Incorporate a balanced focus on deep content knowledge, mathematical and scientificinquiry and problem solving/design, reflecting post-secondary faculty expectations forcollege and career readiness and employer expectations for STEM careers.• Align vertically across grade levels and horizontally across subject strands to ensurecoherent subject progressions among schools, across districts and through college.• Connect community-based experiential and project-focused learning resources to PreK-12 curricula and/or through collaborative use of STEM related laboratories in thevocational technical schools.8STEM Advisory Council March 11, 2011

STEM Operations BoardCriteria for Identifying Scalable STEM ProjectsAppendix BC. Educator DevelopmentEvery student will learn from highly effective educators in every STEM subject area atevery grade level, PreK-16. Massachusetts educators will…• Possess deep subject matter knowledge that spans grade levels; be skilled in thepedagogy of inquiry and problem solving; and be prepared to incorporate experientialand applied learning that integrates science, technology, engineering and mathematicsinto coherent classroom instruction.• Make effective use of technology as a tool for learning, recognizing its application as anessential resource for every 21 st century STEM profession.• Seek out innovative ways to further improve their understanding of their student’sstrengths and weaknesses, through data analysis and the creation of activeassessments.D. STEM Employers and STEM ProfessionalsEmployers and the community of STEM professionals (from industry and education) canprovide an array of opportunities for experiential learning, both inside and outside theclassroom by...• Participating in educator professional development and communicating theirexpectations for students wishing to pursue a career in their sector. Employers andSTEM professionals will serve as mentors, internship/co-op supervisors, leaders ofcommunity-based after-school and expanded learning time programs and partner withschools to offer new programming and expand existing programming such as thealready state approved STEM programs in vocational technical schools.• Sponsoring university laboratory research and industry-based teacher externships.They will also serve as collaborative partners in high quality professional developmentand pre-service programs.9STEM Advisory Council March 11, 2011