Modern Blanket Toss: Findings and Observations

More documents

Recommendations

Info

2016 FINDINGS Section 4. Discussion Evaluation results indicate that the Modern Blanket Toss program shows significant promise as an effective way to increase the likelihood that participating students would go on to pursue college degrees in STEM areas and ultimately pursue STEM careers. The ability of the Upward Bound local coordinators to facilitate the Modern Blanket Toss program locally was a key indicator of the program’s effectiveness. While the evaluation results are limited by exclusive reliance on the coordinators’ self-reports of their knowledge and capabilities to implement the program at their respective sites, the results nonetheless indicate that site coordinators had, or were able to gain, the skills they needed to implement science practices and project-based learning with their students. On all four scales measuring coordinator knowledge, the coordinators reported increased knowledge from the beginning of the program to the end of the program. In particular, the UAV knowledge scale -- which included items such as how to build a UAV, applications of UAV technology, the safety rules and regulations that apply to flying a UAV, and how to acquire different types of data using UAVs -- increased significantly across all individual items assessed. The GIS knowledge scale, which included items such as how to use GIS mapping software, also increased significantly. However, on several individual items assessed, coordinators did not report knowledge gains even though they initially rated their knowledge on the topic as low. Items for which coordinators responded in this way included selfreported knowledge of how to use GIS mapping software and how to acquire data from databases. While the coordinators either already had, or obtained by the end of the project, the knowledge and/or skills required to implement Modern Blanket Toss content with their students, their initial lack of some key skills may explain why students reported lower content knowledge gains during the school year than they did during the summer program. Further, while all of the communities implemented a community project in the final year of the project, here too the coordinators’ initial lack of some key skills may explain why community projects were not implemented in each of the three program years. In future iterations of the Modern Blanket Toss program, we strongly recommend targeted and specific coordinator training focused on the technology being used (e.g., how to build a UAV), combined with training focused on skills needed to implement community projects, such as how to ask questions using GIS software. Even though the Modern Blanket Toss summer program only reached a third of the Modern Blanket Toss students, it was likely the most essential part of the program as implemented during the three year project period. By student account, the summer component was highly successful. Each summer student reported weekly satisfaction with the program. They reported that the classes held their interest, the presenters were well prepared, the environment was conducive to learning, and that they enjoyed the classes and look forward to more. They liked working with their peers and they liked learning new skills needed to operate the UAVs. More importantly however, students reported greater knowledge gains, greater leadership and communication skill gains, and greater interest in science and STEM careers after the summer program than they did after the school year program. However, the greater reported gains during the summer program may simply be a reflection of the students who attended the summer program component. The average GPA of students who attended the summer program was significantly higher than the control group of Upward Bound students, who did not attend the summer program and whose sites were not involved in the Modern Blanket Toss program. The students who attended the summer program also averaged a greater number of science courses per student before they entered the program than the control group. Unfortunately, the evaluation did not collect any additional data that might further clarify the differences between the groups. 59
2016 FINDINGS The Modern Blanket Toss student outcomes were organized around themes common to informal science education: interest or engagement; self-efficacy (sense of ability); awareness, knowledge or understanding; and behavior changes (National Science Foundation, 2008; National Research Council, 2009; Phillips et al., 2014). In addition, the outcomes were organized around predictors of students’ STEM success, including high school math and science courses taken (Hinojosa et al., 2016) and the grades in those courses. The program also included an outcome for student communication and leadership skills, the development of which was a primary part of the planned program. Many of the program’s expected student outcomes were met, indicating that students who participated in the Modern Blanket Toss program are now more likely to pursue college degrees in STEM areas and ultimately pursue STEM careers. Students reported significant increases in their knowledge of Modern Blanket Toss content areas, such as how to use GIS mapping software, and longer duration of participation in the program led to greater content gains. Students also reported significant gains in their understanding of science practices, such as asking a scientific question from data. Students reported significant increases in their communication skills, such as their ability to talk to a group of strangers. Those target student outcomes that did not achieve a statistically significant change, still nonetheless increased. For example, many of the students already had a positive attitude toward science when they started, so the overall attitude toward science scale score, while shifting positively, did not shift to a statistically significant degree. Similarly, students reported positive increases in leadership skills, but these increases were not statistically significant. Again, students entered the program with relatively high leadership scale scores, so statistically significant results may have been difficult to detect with the small sample size. In future iterations of the Modern Blanket Toss program we would recommend collecting additional outcome data for control group students, such as attitude toward science. This additional data would allow the program to better assess its impacts when pre-data shows positive results and leaves little room for growth. Finally, the student grades and the number of math and science courses completed in high school did not increase during the evaluation period. This is not an unexpected result for several reasons. First the sample size was extremely small and it is difficult to find significant relationships from the data as statistical tests normally require a larger sample size to ensure a representative distribution of the population. Second, we only measured student grades and courses completed over two school years. A longer time period would provide better data. However, we did find that students who participated in Modern Blanket Toss had significantly higher GPAs and science and math grades than control group students. This is telling information. Students who self-selected to participate in the Modern Blanket Toss summer program already were more likely to pursue college degrees in STEM areas and ultimately pursue STEM careers simply based on their higher math and science grades. This is a factor that should be considered in any future evaluation. 60
Page 1 and 2:
BOOK THREE MODERN BLANKET TOSS FIND
Page 3 and 4:
FINDINGS AND ANALYSIS The evaluatio
Page 5 and 6:
CHAPTER 1 Findings Welcome to the f
Page 7 and 8:
2014 FINDINGS EXECUTIVE SUMMARY The
Page 9 and 10: 2014 FINDINGS Sample Eighteen stude
Page 11 and 12: 2014 FINDINGS 2014 Both surveys ask
Page 13 and 14: 2014 FINDINGS Survey respondents ra
Page 15 and 16: 2014 FINDINGS Items can be grouped
Page 17 and 18: 2014 FINDINGS The UAV aircraft kind
Page 19: 2014 FINDINGS I think it should be
Page 22 and 23: 2015 FINDINGS PRE/POST STUDENT SURV
Page 24 and 25: 2015 FINDINGS There was a significa
Page 26 and 27: 2015 FINDINGS Table 14 Content Know
Page 28 and 29: 2015 FINDINGS Items can be grouped
Page 30 and 31: 2015 FINDINGS I liked learning abou
Page 32 and 33: 2015 FINDINGS Nothing it's getting
Page 34 and 35: 2015 FINDINGS Week 12 Concept Count
Page 36 and 37: 2016 FINDINGS The mission of Modern
Page 38 and 39: 2016 FINDINGS INTRODUCTION The Univ
Page 40 and 41: 2016 FINDINGS Analysis: For each co
Page 42 and 43: 2016 FINDINGS and who participated
Page 44 and 45: 2016 FINDINGS LIMITATIONS OF THE EV
Page 46 and 47: 2016 FINDINGS primarily Yup’ik mi
Page 48 and 49: 2016 FINDINGS There was a range of
Page 50 and 51: 2016 FINDINGS Table 17. Average sur
Page 52 and 53: 2016 FINDINGS Students also clearly
Page 54 and 55: 2016 FINDINGS Figure 5. Leadership
Page 56 and 57: 2016 FINDINGS 6) Student academic d
Page 58: 2016 FINDINGS Courses offered by Hi
Page 63 and 64: CHAPTER 2 Observations This chapter
Page 65 and 66: CHALLENGES AND LESSONS LEARNED Teac
Page 67 and 68: SECTION 2 Analysis of Modern Blanke
Page 69: ANALYSIS OF MODERN BLANKET TOSS Wha
show all

Modern Blanket Toss: Findings and Observations

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?