Modern Blanket Toss: Findings and Observations

BOOK THREE
MODERN
BLANKET
TOSS
FINDINGS AND OBSERVATIONS
DR. JOHN MONAHAN
MBT SERIES COMPILED BY DEBRA KOUDA

FORWARD
Findings and Analysis
THE FIRST HALF OF THIS BOOK FOCUSES ON THE EVALUATION FINDINGS FROM EACH YEAR. THE SECOND NOTES THE OBSERVATIONS AND ANALYSIS FROM
THE PRINCIPAL INVESTIGATOR, DR. JOHN MONAHAN AS WELL AS THE CURRICULUM COORDINATOR, ADAM LOW.
Through the leadership classes I have learned that empathy is definitely a big thing […] you need to listen to what […] people are saying and
then decide on an end result by compensating with everybody in a way. And a lot of the time most people don’t realize that when being a leader
you have to listen to the other people. You can’t just make a decision. And what UAV flying is about is you have to have a team. You have to
have a team to fly the UAVs, because there is copilot, pilot, payload operator, spotter. There are so many different roles that not just one person
can do it all. You have to be able to work as a team, and in order for that you need to develop leadership skills.
- MBT Student
PROJECT EVALUATION PLAN
Project evaluation was undertaken by Angela Larson, owner and principal consultant for the Goldstream Group in Fairbanks, Alaska. Larson
has been evaluating educational programs since 1999 and specializes in K-12 science programs, and has also developed educational
projects for the Fairbanks School District and headed the planning and development department of the Fairbanks Native Association.
Larson’s evaluation was based on a logic model developed within the first month of the award, and includes both formative and summative
components using a mixed-methods design to monitor measurable outcomes through both quantitative and qualitative approaches.
Evaluation will focus on providing feedback about the project’s development, implementation, and outcomes to the project staff and
stakeholders to aid in decision-making.
i

FINDINGS AND ANALYSIS
The evaluation addresses three primary questions:
1) How does participation in Modern Blanket Toss
activities impact students’ likelihood to pursue STEM
careers?
2) How does participation in the Modern Blanket Toss
project impact students’ academic performance?
3) What factors contributed to or hindered the
achievement of these two short-term outcomes?
The evaluation uses quasi-experimental design to address questions 1 and
2: students from the four Upward Bound communities participating in the
project will serve as the test group, and students from the remaining six
Upward Bound communities will serve as a control group. The evaluation
will adopt a descriptive design to answer question 3.
To assess students’ satisfaction with and engagement in project activities,
each year the evaluation used
1) a pre- and post- interview with students who attended the summer
component, and
2) a pre- and post- survey with all participating students. Larson conducted
surveys of Upward Bound site coordinators at the beginning of the
project, after the summer component, and at the end of each school
year in order to assess their preparation and knowledge of STEM
content.
In the formative stages, evaluation data was used to inform project
development and measure factors that may contribute to or hinder the
achievement of short-term outcomes. Formative evaluation data will be
analyzed using parametric and nonparametric statistics (e.g., t-tests, chisquares,
analysis of variance [ANOVA]) to measure changes in
understanding and to correlate observed changes with factors -
satisfaction, engagement, prior knowledge and interest, local coordinator
preparation - that may have influenced changes in learning, interest and
self-confidence. Qualitative data was analyzed using content analysis. These
results were used to improve the program materials and activities.
Results that indicate changes in the key indicators of likelihood to pursue a
STEM career, and changes in key academic indicators, will be used for both
formative and summative purposes. Students completed surveys before
and after the summer component as well as in the fall and spring of each
school year, while academic data (see below) was collected each spring.
Results from program participants will be compared to the control group
annually to inform project development, and was compared longitudinally
over the three-year project period to provide summative evaluation
results.
Reports were provided to program staff after major project activities,
annually, and at the end of the three-year project. Formal presentations of
ii

FINDINGS AND ANALYSIS
findings will be given upon request at EPSCoR and/or Upward Bound
meetings and at the concluding workshop.
Because the cohort of students was not initially identified, a baseline of
data was not provided on their levels of STEM interest and proficiency. This
data will come from student records and the aforementioned surveys and
interviews each year. Student progress in these areas was tracked relative
to the baseline.
Larson’s survey results were considered in conjunction with official school
and Upward Bound records of student academic progress and attendance.
Site coordinators had direct access to real-time information to monitor
student progress, and also assist students to complete an Upward Bound
academic plan, which addresses: class selection; goals for grade and GPA
attainment; participation goals; education planning steps; extra-curricular
activity planning and selection; and school participation goals. Attainment
(or non-attainment) of plan goals was another measure of student success,
and was considered alongside classroom and summer session grades,
student information system reviews, attendance patterns, GPA’s and preand
post-testing scores and/or annual assessment results. Results of
Upward Bound “interest inventories” provided data about students’ likely
educational pathways.
iii

CHAPTER 1
Findings
Welcome to the findings from all three years based on the evaluation plan by Angela Larson and Goldstream Group. The
evaluation plan is outlined in the Forward of this book.

SECTION 1
2014 Findings
5

2014 FINDINGS
EXECUTIVE SUMMARY
The first summer session of the Modern Blanket Toss met at
the University of Alaska Fairbanks in June and July of 2014.
The student evaluation data show positive progress towards
the projects objectives.
Student pre/post surveys asked students to self-assess their skills
and attitudes in several areas. The results showed statistically
significant gains in the following areas:
Student interviews demonstrated that the students:
• were enthusiastic about the program
• were confident in what they learned about UAVs
• learned applicable leadership skills
• made positive personal connections
• were inspired to share their knowledge and skills with their communities.
• Content knowledge
• Leadership and communication skills
• Attitude towards science
• STEM career interest
• Engagement in science topics.
Weekly student surveys showed that over the course of the summer
program, students:
• remained satisfied with the program throughout
• increased interest in STEM careers
• decreased their perception that the content of the course was challenging
• increased leadership and communication skills
• increased in ability to work as a team.
6

2014 FINDINGS
INTRODUCTION
As part of the University of Alaska Fairbanks Upward Bound program, students
from four rural communities in Alaska were given the opportunity to participate
in Modern Blanket Toss, a course on UAV flying and applications. In addition to
the Upward Bound goals of encouraging and supporting students in pursuing
higher education, the Modern Blanket Toss program aims to encourage students
to pursue STEM careers by giving them skills in communication, leadership and
cutting edge technology. Community involvement is an important component of
the project.
The specific objectives of the six week course in the summer
of 2014 were for the students to
1) learn how to safely operate small unmanned aircraft systems,
2) understand the basic concepts governing unmanned flight,
3) explore the applications of UAVs,
4) achieve a general understanding and awareness of STEM career fields, and
5) obtain knowledge of UAV applications, operations, and regulations.
PRE/POST STUDENT SURVEY
SURVEY DEVELOPMENT
The Pre/Post Student Survey was designed to measure levels and changes in the
student’s
1) content knowledge and science practices (indicator 1.a);
2) leadership and communication skills (indicator 1.b);
3) attitude towards science (indicator 1.c);
4)STEM career interest (indicator 1.d); and
5) engagement and excitement in project topics (indicator 3.b).
The survey included 5 open-ended questions and 58 Likert-scale items. The presurvey
was administered to students on paper in their first week of the Upward
Bound Academy. The post survey was administered on the last day of classes at
the Academy.
In the classroom the students learned safety rules and checklists. Despite an
unusually rainy and windy June and July, the students practiced flying UAVs in
teams several times each week. They focused their efforts on gathering data for
a map of the UAF frisbee-golf course. Through their teachers, guest speakers
and field trips they were exposed to several individuals working in STEM
careers.
Upward Bound students in the UAV 2014 summer course participated in three
levels of evaluation. First, they completed pre and post surveys about their
attitudes towards science and STEM careers as well as self-assessment of their
science, leadership and communication skills. Second, they completed weekly
surveys on satisfaction and recently covered content. Third, students and
teaching staff were interviewed at the end of the course to share their opinions
and successes.
7

2014 FINDINGS
Sample
Eighteen students completed the pre-survey and seventeen of those students also completed the post-survey. Respondents identified themselves with an ID number on
the surveys and pre and post data was matched for analysis; only matched pairs were included when comparing pre and post values.
Reliability
The items in the survey combine to make 6 constructs or scales. To measure the reliability of our scales, we calculated Crohbach’s alpha coefficient for each scale using
SPSS. Cronbach's alpha is a measure of internal consistency among items included in a scale. 1 A Cronbach’s alpha that is greater than 0.7 is considered evidence that the
items included measure an underlying construct. 2 For all of our scales there was a high rate of internal consistency among the survey items. However, our small sample
size (n=33) may bias the results. 3 Results of the reliability analysis are summarized in Table 1. The survey items for each scale are included in Appendix B.
Table 1 Reliability Analysis for Pre/Post Student Survey
Scale n Items Item Mean Min Max Range Variance
Crohbach’s
Alpha
Content Knowledge 29 14 5.552 3.483 7.586 4.103 2.059 0.963
Science Practices 31 11 2.842 2.548 3.129 0.581 0.035 0.899
Leadership Skills 31 7 3.207 3.065 3.355 0.29 0.008 0.881
Communication Skills 31 12 2.992 2.161 3.774 1.613 0.287 0.937
Attitudes Towards Science 33 4 3.121 2.909 3.333 0.424 0.045 0.818
Attitudes Towards STEM Careers 32 6 3.479 3.313 3.625 0.313 0.018 0.887
8

2014 FINDINGS
Analysis and Results
For each construct, we calculated a scale score for each respondent. The scale score is the average of the response to items included in the scale.
- For “Content Knowledge,” respondents rated their own knowledge from 1 (low level of knowledge) to 10 (high level of knowledge) so the scale score also has a
range of 1 to 10.
- For the “Communication Skills” items, respondents chose “not comfortable at all” (equal to 1), “somewhat comfortable” (equal to 2), “moderately
comfortable” (equal to 3), or “very comfortable” (equal to 4). The scale score is an average between 1 and 4.
- All other items asked respondents to “strongly disagree” (equal to 1), “disagree” (equal to 2), “agree” (equal to 3), or “strongly agree” (equal to 4), so those scales also
range from 1 to 4 with 4 as a more positive score.
- We used an independent
samples t-test to test
whether the mean scale
scores were significantly
different (P < 0.05)
between pre and post
surveys.
All constructs showed
a significantly positive
shift. Students learned
content knowledge,
leadership skills,
communication skills
and science practices.
Their attitudes
towards science and
STEM careers
improved. Results are
shown in Table 2.
Table 2 Paired t-test for Student Pre/Post Survey Constructs
Scale Survey n Mean
Content Knowledge
Science Practice
Leadership Skills
Communication Skills
Attitudes Towards Science
Attitudes Towards STEM Careers
Standard
Deviation
Pre 16 3.82 1.45
Mean
Difference
t df P-Value
Post 16 7.27 1.46 3.45 7.55 15 0
Pre 16 2.68 0.45
Post 16 2.98 0.49 0.3 3.68 15 0.002
Pre 16 3.1 0.45
Post 16 3.33 0.45 0.23 2.7 15 0.016
Pre 16 2.77 0.72
Post 16 3.1 0.63 0.33 2.8 15 0.013
Pre 16 2.94 0.48
Post 16 3.27 0.47 0.33 2.95 15 0.01
Pre 16 3.34 0.4
Post 16 3.55 0.42 0.21 2.27 15 0.039
9

2014 FINDINGS
2014
Both surveys asked students about their previous experience with mapping and
UAVs and if they were excited to learn about such things. Students were given
the option to strongly agree or agree and more students strongly agreed in the
post test. This is shown by the average score for each question. The more
respondents chose “strongly agree,” the closer the average gets to 4. Results are
shown in Table 3.
Table 3 Experience and Excitement to Learn
Skill
Pre-
Average
Post-
Average
Change
I am excited to learn about mapping software. 2.88 3.13 0.25
I am excited to learn about UAV’s. 3.63 3.69 0.06
I have used mapping software before. 1.94 2.44 0.5
I have used UAV’s before. 2.75 3.63 0.88
Table 4 Content Knowledge Items
Skill
Pre-
Average
Post-
Average
Change
How to acquire data from databases 2.56 7.38 4.81
How to use a 3-D printer 1.25 5.69 4.44
How to acquire different types of data using
UAV’s
3 7.19 4.19
Applications of UAV technology 4.31 8.4 4.09
How to use data collected to develop a map/
image that represents reality
2.75 6.56 3.81
How to ask questions using GIS software 2 5.75 3.75
How to fly a UAV 5.5 9.19 3.69
How to use GIS mapping software 1.81 5.06 3.25
The amount of money I can earn from science,
technology, engineering or math career
The safety rules and regulations that apply to
flying a UAV
5.06 7.94 2.88
6.25 9.13 2.88
Principles of flight (e.g., lift, drag, gravity) 5.67 8.5 2.83
The range of science, technology, engineering
and math careers that are available to me
5.06 7.87 2.8
How to build a UAV 2.88 5.56 2.69
The level of man and science I will need for
various science, technology, engineering, and
math careers
5.5 7.81 2.31
It is valuable to know the average responses for individual items within each
construct. Students were asked to rate their knowledge on each topic on a scale
of 1 (low knowledge) to 10 (high knowledge). Predictably, students reported
learning more about how to safely and legally fly a UAV, than how to use a 3-D
printer or ask questions using GIS software. See Table 4.
10

2014 FINDINGS
Survey respondents rated their agreement with statements about their
Leadership Skills. The scale ranges from 1 to 4 where 1 is equal to strongly
disagree and 4 is equal to strongly agree. The largest shift is in respondent’s
ability to “develop, implement, and communicate new ideas to others.” There
was no change in respondent’s ability to “listen carefully to what other people
are saying,” but most students already agreed they could do that in the pre
survey. Results are shown in Table 5.
Table 5 Leadership Skill Items
3) talk with a stranger and
4) talk in a small group of acquaintances at the end of the program.
There was a smaller shift in the students’ ability to
1) talk in a large meeting of acquaintances;
2) talk in a small group of strangers;
3) talk with a stranger; and
Leadership
I am able to develop, implement, and
communicate new ideas to others.
Count
Pre-
Average
Post-
Average
Change
16 3 3.5 0.5
4) talk in a small group of acquaintances probably because most students
already felt comfortable with those types of communication at the beginning
of the course. Results are in Table 6.
I am open to new and diverse perspectives. 16 3.06 3.38 0.31
I am able to work effectively with a diverse
team.
I am able to collaborate and cooperate
effectively with teams.
I am able to leverage strengths of others to
accomplish a common goal.
I am able to set goals and prioritize tasks to
meet goals.
I am able to listen carefully to what other
people are saying.
The respondents showed a clear shift in communication skills. More students
agreed they could
1) talk in a large meeting of acquaintances;
2) talk in a small group of strangers,
15 3 3.31 0.31
16 3.25 3.5 0.25
16 3 3.13 0.13
16 3.13 3.25 0.13
16 3.25 3.25 0
Table 6 Communication Skill Items
Communication
Count
Pre-
Average
Post-
Average
Change
Talk in a large meeting of acquaintances 16 2.56 3.19 0.63
Talk in a small group of strangers. 16 2.25 2.75 0.5
Talk with a stranger. 16 2.06 2.53 0.47
Talk in a small group of acquaintances 16 2.81 3.27 0.45
Present a talk to a group of strangers. 16 2.06 2.44 0.38
Present a talk to a group of acquaintances 16 2.75 3.13 0.38
Talk in a large meeting of friends. 16 3.06 3.44 0.38
Present a talk to a group of friends 16 3.25 3.6 0.35
Talk in a large meeting of strangers 15 2 2.31 0.31
Talk in a small group of friends 15 3.6 3.69 0.09
Talk with an acquaintance. 16 3.6 3.27 0.08
Talk with a friend. 16 3.75 3.81 0.06
11

2014 FINDINGS
Survey respondents rated their agreement with statements about their attitudes
towards science and STEM careers. Three of the STEM career related
statements are negative, so for those items a negative shift in value represents a
positive shift in attitude. See Table 7.
Table 7 Attitudes about STEM Careers and Science Items
Count
Attitudes toward science
Looking at scientific data in different ways helps
me learn.
Pre-
Average
Post-
Average
Change
16 3.06 3.44 0.38
Science is an interesting subject to study. 16 3.13 3.5 0.38
Using a map to study scientific data helps me to
learn.
My scientific investigations are better when I
begin by asking a question.
Attitudes toward STEM careers
I am interested in the way that science,
technology, engineering, or math can be used to
help people.
16 2.75 3.13 0.38
16 2.81 3 0.19
16 3.44 3.75 0.31
Scientists/engineers are really cool people. 16 3.25 3.38 0.13
A career in science, technology, engineering, or
math would be financially rewarding.
A career in science, technology, engineering, or
math would be dull and boring.
15 3.27 3.31 0.05
16 1.56 1.38 -0.19
Scientists/engineers are boring people 16 1.56 1.31 -0.25
Pursuing a degree in a science, technology,
engineering, or math field does not interest me.
16 1.75 1.44 -0.31
WEEKLY STUDENT SURVEYS
At the end of each week, students were given a short satisfaction and content
survey. Students were asked 10 Likert scale questions about their satisfaction
with the Modern Blanket Toss program that week. They were also asked three
open-ended questions about what they learned.
Sample
Each week, between 15 and 17 students completed the satisfaction survey.
Table 8 shows the counts for each week.
Analysis and Results
Table 8 Weekly Counts
Week of
Count
Week 1 13-Jun 17
Week 2 20-Jun 17
Week 3 27-Jun 15
Week 4 4-Jul 15
Week 5 11-Jul 16
Week 6 18-Jul 16
Similar to the previous survey, students were asked how much they agreed with
a series of statements and a numerical value was assigned to the strength of that
agreement; “strongly disagree” equals 1 and “strongly agree” equals 4. The
responses for each item, each week are averaged and compared.
12

2014 FINDINGS
The larger the value, the more and stronger respondents agree. Table 9 reports the average values of the responses
for each week. Satisfaction items remained high for all six weeks. STEM related items shifted in different
directions. Communication and leadership skill items all increased over the 6 weeks.
Table 9 Average of Responses by Week and Item
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Count 17 17 15 15 16 16
The Modern Blanket Toss classes have held my interest this week. 3.35 3.35 3.47 3.27 3.38 3.44
The presenters were well prepared. 3.29 3.35 3.67 3.33 3.38 3.25
The environment (meaning the classroom, field trip location, or other
place where instruction took place) was conducive to learning.
3.41 3.41 3.57 3.27 3.19 3.5
I learned new science and technology concepts this week. 3.06 3.24 3.13 2.8 2.81 3.13
The information was challenging to me. 2.69 2.71 2.8 2.27 2.56 2.38
I enjoyed the classes this week and look forward to more. 3.53 3.59 3.47 3.33 3.5 3.38
I am considering a career in a STEM field. 2.88 3.06 3.2 3.13 3.38 3.31
I feel confident communicating with others about science topics. 3.06 3 3.13 3.27 3.44 3.38
I am becoming more of a leader through this class. 2.65 2.76 3.07 3.07 3.06 3.25
I am comfortable working as part of a team. 3.41 3.53 3.47 3.33 3.69 3.53
13

2014 FINDINGS
Items can be grouped into three categories. General satisfaction items are
shown in Figure 1. Satisfaction remained high (average greater than 3.2) for all six
weeks for all items. STEM Interest items shifted as expected. Students more
strongly agreed that they were considering a STEM career as the weeks went by.
The material covered became less challenging and the concepts were less new
on the weeks when the students were mostly practicing their skills, not learning
new material. See Figure 2.
Figure 3 shows that item responses about communication and leadership skills
generally increased over time.
Figure 1 Satisfaction Items from Weekly
Surveys
Figure 2 Skill and STEM Interests from
Weekly Surveys
Figure 3 Communication and Leadership
Items from Weekly Surveys
14

2014 FINDINGS
Open-Ended Questions
Students wrote similar descriptions about what they learned each week,
suggesting they were similarly paying attention. For all weeks, students liked flying
the UAVs best. Most students didn’t want to change anything about the course,
but a few wanted more flying time.
The following are illustrative:
I learned about UAV's because I didn't know anything about them and that they can
film. I was amazed; I also learned how to control a UAV. (Week 1)
Analysis and Results
Interviews were recorded and transcribed for analysis using Atlas.ti. Following is
a summary of the analysis by primary themes: satisfaction, learning, leadership,
plans after college, communication skills, sharing what they learned with their
community, interest in learning more about UAVs, and different factors that
influenced their learning.
Both students and teaching staff agreed that the summer program was a great
success. The following are examples of student responses.
The leadership cabinet was a very fun experience and very convenient.
I really liked making the video about the 3-D printer and the flying.
Not much, seeing we already learned everything we needed to. I did get more
comfortable with the Frolf course. I also feel like I refined my leadership and
teamwork skills. (Week 5)
Nothing because I have made lots of memories with the people and places.
2014 FINDINGS
This project was amazing, and I hope that we can continue it.
I liked being part of something important, like this program. I
think it's really important.
All agreed that they learned a great deal in the program. Student reported
learning can be divided into 3 categories:
PARTICIPANT INTERVIEWS
Sample
Fourteen students, the teacher and two teaching assistants were interviewed in
the last week of the program to get more detailed information about the
strengths and weaknesses of the course.
1) skills related to getting the UAV in the air and collecting data with it,
2) awareness of technologies they didn’t previously know about, and
3) leadership skills.
All students learned about the importance and function of the flight safety
checklist and the roles necessary for a team to safely and effectively fly a UAV to
collect data.
The following responses to “What did you learn?” are illustrative.
How to fly [a UAV] without panicking that much.
15

2014 FINDINGS
The UAV aircraft kind of like has a brain ... Like it has like a computer, and
then the TX is the controller that controls aircraft, and those two...they kind
of talk to each other. So, they kind of give each other some information,
data, like where to fly and how high to fly. We learned about ... the aircraft,
like what the parts are and how to use those properly and correctly. We did a
lot of experiencing how to fly aircraft...
... a preflight, and then when you’re done flying there’s also a post-flight
checklist. And when you’re doing your preflight checklist... instead of saying
yes or I don’t know, he wanted us to say “check.”
Students learned about types of UAV beyond the ones they used in the course.
They know that sensors and/or cameras can be changed on the UAV depending
on the type of data a particular project needs. They know that 3D printers are a
viable technology now and can be used to create parts for UAVs.
During the leadership portion of the course, students learned four skills that
help make a good leader (listen, learn, love, lead.) They practiced those skills in
leadership cabinets where they solved a problem by following a protocol that
gave everyone a say but also moved the decision process forward. Many
students appreciated that the leadership skills were immediately applicable to
their flight teams and that being a “facilitator” gave them the opportunity to
practice leadership. The following are illustrative.
Yeah. This class made my leadership skills rise a lot. I feel more comfortable
speaking in a bigger crowd than what I used to back at home.
... we were working in groups, and we all worked together, and we took turns
as roles. And I’d have to say [we learned leadership skills], because after the
weeks have gone by of seeing progress in everyone’s ability to work together.
Through the leadership classes I have learned that empathy is definitely a big
thing when it comes to leadership, and you need to listen to what your other
– what the other people are saying and then decide on an end result by
compensating with everybody in a way. And a lot of the time most people
don’t realize that when being a leader you have to listen to the other people.
You can’t just make a decision. And what UAV flying is about is you have to
have a team. You have to have a team to fly the UAVs, because there is
copilot, pilot, payload operator, spotter. There are so many different roles
that not just one person can do it all. You have to be able to work as a team,
and in order for that you need to develop leadership skills, and I’d definitely
say that my leadership skills have improved by being in this program.
Students had a wide range of interests they are considering pursuing in college,
most of them STEM fields. Examples of topics mentioned as possibilities are:
biology, physics, engineering, environmental science, herbology, math, medicine,
nursing, geometry, criminal justice, business, architecture, liberal arts,
biochemistry, and chemistry.
Though many students were interested in STEM fields before attending Upward
Bound, most also thought the Modern Blanket Toss program encouraged them
further to pursue STEM fields in college. The following are illustrative.
You need the science, technology, engineering, and mathematics to work with
UAVs, and UAVs are really cool, so I think this has encouraged a multitude of
people that normally wouldn't choose something 'cause they thought it'd
sound boring.
16

2014 FINDINGS
I’m already really interested in the sciences. I’m not too great at math, but
when I can relate it to physics, I can do the math much more – much better
than just math in general. So, when I – and when it comes to science and
physics, if I relate my mathematics to science and physics, I can understand
it better, so I’d definitely say that I want to do something that involves
science.
Students in general did not answer enthusiastically when asked if they had
learned communication skills in the program, though they did all prepare
presentations for the class. A typical response to the question about
communication skills is:
I don’t know, I’m not that much of a – or I’m not used to being around a lot
of people that I’m not used to, and if I probably do it, I probably won’t be
able to speak in front of them.
I’m able to teach them how to operate one and how to do the payload
checks, the checklist and to know what’s wrong with a UAV so we wouldn’t
have to waste money on a crash.
Pretty well prepared. I don’t know that much, but I know enough... like how
to operate one, the payload checklist, how to find out more for the payload
checklist by experience, and camera lenses, because they get oily, and those
are not in the checklist... And how to teach them a little bit about mapping
and collecting data.
If I had help with others, 'cause it's hard for me to put my thoughts into
words and say it out to others, but like if I had time before telling them.
I've learned a lot of stuff, but I'll probably need help from the other people
that came to explain more.
Both students and teachers agreed there was more to learn. Six weeks was not
much time to cover the technological goals and there was just not enough time
to cover everything that the students were ready to learn. Students became
proficient at data collection, but only a few of them learned about transferring
files to the GINA website and the Frisbee-Golf course data was not compiled
and/or edited into a map before the students went home.
I feel like there’s a lot more that we need to learn in that class. I feel like
we’re missing out, like we didn’t have enough time.
The thing I would like to change is actually put layers of the codes into the
computer like a GIS program software like we did last year and use the data
we collected from the UAVs and put that into software and create a world of
what we collected.
Yeah. What would I change? I’d say nothing. I love the learning process you
guys put us through. It helped out a lot. Rather than just those skills, like
communication skills also and leadership skills. I would probably just throw in
more mapping software time for us. That’s about it.
Two students felt they could share their knowledge with some support from
peers.
17

2014 FINDINGS
I think it should be a little longer here, like six weeks is kind of short to learn
about all this.
Students mentioned the friendships and the people connections as being an
important part of the success of the program.
I thought it was fun because we got to fly in different areas with a lot of
different people we've never met before in the first week, and now we're
laughing and we're getting closer, like friendship close, and, yeah, I've pretty
much liked the past – most – well, all of the six weeks I've been here because
I have so many memories now.
From where I’m from, I call it my nerdy side, but a lot – there’s not a lot of
people that you have that you can talk to about science without them
scoffing, but in a place like this and in the setting that I’m in, whenever I’m
with the UAV class, I can talk about my nerdy side in a way, and I can bring
out the things that I like about science, and it’s much more comfortable for
me to talk about the sciences because it’s what I enjoy doing. And I don’t
exactly have that option at home, but I’d definitely say that this has made it
more comfortable for me to talk about the sciences.
Table 10 Factors that Help in Student Learning
Not very
much
A little
A lot
Doing a presentation 0% 27% 73%
Things you write about 0% 64% 36%
Your own inner motivation to learn 0% 18% 82%
Being an Upward Bound student 0% 9% 91%
Expectations that others have of you 9% 55% 36%
Your friends or peers 9% 27% 64%
Hands-on projects 0% 0% 100%
Your instructors 0% 9% 91%
Technology 0% 27% 73%
At the conclusion of the interview students were asked to rate how different
factors influenced their learning. Results are presented in Table 10. Students most
frequently reported that hands on projects, being an Upward Bound student,
and the instructors were “a lot” of help in learning. Writing things and the
expectations of others were the least helpful to student learning.
18

SECTION 2
2015 Findings
INTRODUCTION
UPWARD BOUND STUDENTS IN THE UAV
COURSE PARTICIPATED IN THREE LEVELS
OF EVALUATION THIS SUMMER. FIRST,
THEY COMPLETED PRE AND POST
SURVEYS ABOUT THEIR ATTITUDES
TOWARDS SCIENCE AND STEM CAREERS
AS WELL AS A SELF-ASSESSMENT OF THEIR
SCIENCE, LEADERSHIP AND
COMMUNICATION SKILLS. SECOND, THEY
SUBMITTED WEEKLY SURVEYS PERTAINING
TO THEIR SATISFACTION LEVEL AND
RECENTLY COVERED CONTENT. THIRD,
STUDENTS AND TEACHING STAFF WERE
INTERVIEWED AT THE END OF THE
COURSE TO SHARE THEIR OPINIONS AND
SUCCESSES.
MBT students flying UAVs in Bethel High School
20

2015 FINDINGS
PRE/POST STUDENT SURVEY
Survey Development
The Pre/Post Student Survey was designed to measure levels and changes in the
student’s
1) content knowledge and science practices (objective 1.a);
2) leadership and communication skills (objective 1.b);
3) attitude towards science (objective 1.c);
Reliability
The items in the survey combine to make 6 constructs or scales. To measure the
reliability of our scales, we calculated Crohbach’s alpha coefficient for each scale
using SPSS. Cronbach's alpha is a measure of internal consistency among items
included in a scale 1 . A Cronbach’s alpha that is greater than 0.7 is considered
evidence that the items included measure an underlying construct 2 . For all of
our scales there was a high rate of internal consistency among the survey items.
However, our small sample size (n=32) may bias the results 3 . Results of the
reliability analysis are summarized below in Table 11.
4) STEM career interest (objective 1.d); and
5) engagement and excitement in project topics (objective 3.b).
The survey included 5 open-ended questions and 58 Likert-scale items. The presurvey
was administered to students on paper in their first week of the Upward
Bound Academy. The post survey was administered on the last day of classes at
the Academy. A copy of the survey is in Appendix A.
Sample
Seventeen students completed the pre-survey and fifteen of those students also
completed the post-survey in 2015. Respondents identified themselves with a
name or an ID number on the surveys so that pre and post data can be
matched by individuals. All responses are valid for use in the reliability tests on
the scales, only matched pairs are included when comparing pre and post values.
21

2015 FINDINGS
Table 11 Reliability Analysis for Pre/Post Student Survey
Scale n Items Item Mean Min Max Range Variance
Cronbach’s
Alpha
Content Knowledge 32 14 4.993 3.313 6.313 3.000 0.792 0.964
Science Practices 26 11 2.783 2.346 3.115 0.769 0.058 0.864
Leadership Skills 27 7 3.201 3.148 3.296 0.148 0.003 0.905
Communication Skills 28 12 2.58 1.929 3.714 1.786 0.250 0.956
Attitudes Towards Science 29 4 3.017 2.793 3.310 0.517 0.056 0.778
Attitudes Towards STEM Careers 30 6 3.239 3.067 3.367 0.300 0.010 0.913
Analysis and Results
For each construct, we calculated a
scale score for each respondent. The
scale score is the average of the
response to items included in the
scale. For “Content Knowledge”,
respondents rated their own
knowledge from 1 (low level of
knowledge) to 10 (high level of
knowledge) so the scale score also
has a range of 1 to 10. For the
“Communication Skills” items,
respondents chose “not comfortable
at all” (equal to 1), “somewhat
comfortable” (equal to 2),
“moderately comfortable” (equal to
3), or “very comfortable” (equal to
4). The scale score is an average
between 1 and 4. All other items
asked respondents to “strongly
disagree” (equal to 1),
“disagree” (equal to 2),
“agree” (equal to 3), or “strongly
agree” (equal to 4), so those scales
also range from 1 to 4 with 4 as a
more positive score. We used a
paired samples t-test to test whether
the mean scale scores were
significantly different (P < 0.05)
between pre and post survey.
22

2015 FINDINGS
There was a significant positive shift in content knowledge during the summer 2015. Science practices and attitudes towards
science also showed a positive shift, but not a significant one. Leadership skills, communication skills and attitudes towards STEM
careers all showed a small, but negative shift. Only the leadership skill shift was statistically significant. Results are shown in Table
12.
Both surveys asked students about their previous experience with mapping and UAVs and if they were excited to learn about such things. The total number of
respondents that agreed to be excited about learning was high (greater than 90%) for both topics on the pre survey. Students were less enthusiastic in the post survey.
Seventy-five percent or more were still excited to learn at the end of the summer. See Table 13.
Table 12 Paired t-tests for Student Pre/Post Survey Constructs
Scale Survey N Mean
Standard
Deviation
Mean
Difference
t df P-Value
Content Knowledge
Science Practice
Leadership Skills
Communication Skills
Attitudes Towards Science
Attitudes Towards STEM Careers
Pre 15 4.35 2.88
Post 15 6.06 1.93 1.70 4.60 14 0.000
Pre 14 2.75 0.42
Post 14 2.87 0.44 0.12 1.37 13 0.195
Pre 15 3.36 0.42
Post 15 3.16 0.36 -0.20 -3.53 14 0.003
Pre 15 2.61 0.80
Post 15 2.56 0.70 -0.05 -0.434 14 0.671
Pre 15 2.97 0.47
Post 15 3.07 0.53 0.09 0.790 14 0.443
Pre 15 3.32 0.43
Post 15 3.23 0.63 -0.089 -0.668 14 0.515
23

2015 FINDINGS
Responses for individual items in each construct can give us more insight. Students were asked to rate their knowledge on each topic on a scale on 1 (low knowledge)
to 10 (high knowledge). Students reported their biggest changes in knowledge about developing a map that represents reality, using GIS mapping software, and asking
questions using GIS software. In the pre-survey, students reported knowing the most about how to fly UAVs, the safety rules that apply to UAVs and the range of STEM
field careers that are available. See Table 14.
Survey respondents rated their agreement with statements about their Leadership Skills. The scale ranges from 1 to 4 where 1 is equal to strongly disagree and 4 is
equal to strongly agree. Shifts were small and all negative. Overall averages above 3 show that students are mostly confident in their leadership skills. See Table 15.
Table 13 Experience and Excitement to Learn
Pre
Post
I am excited to learn
about mapping
software
I am excited to learn
about UAV’s
I have used mapping
software before.
I have used UAV’s
before.
Total
Disagree
Total
Agree
Average
Total
Disagree
Total
Agree
Average
8% 92% 3.23 27% 73% 3
0% 100% 3.57 13% 87% 3.07
71% 29% 2.07 40% 60% 2.67
36% 64% 2.71 27% 73% 2.93
24

2015 FINDINGS
Table 14 Content Knowledge Items
Skill Pre-Average Post-Average Change
How to use data collected to develop a map/image that
represents reality
3.07 6.87 3.80
How to use GIS mapping software 3.15 6.00 2.85
How to ask questions using GIS software 3.07 5.87 2.80
How to acquire data from databases 3.07 5.33 2.26
The communication skill items did not shift in a consistent
direction. More students agreed they could
1)present a talk to a group of strangers;
2) talk in a large meeting of acquaintances;
3) talk in a large meeting of friends and
4)talk in a small group of acquaintances at the end of the
program.
There was little to no shift in student’s ability to
The amount of money I can earn from science, technology,
engineering or math career
The range of science, technology, engineering and math
careers that are available to me
The level of math and science I will need for various
science, technology, engineering, and math careers
4.93 7.07 2.14
4.93 6.73 1.80
5.36 7.13 1.78
1)talk in a large meeting of strangers;
2)talk with a stranger;
3)present a talk to a group of acquaintances and
4)talk with an acquaintance.
The safety rules and regulations that apply to flying a UAV 5.64 7.33 1.69
How to acquire different types of data using UAV’s 3.57 5.13 1.56
Applications of UAV technology 4.36 5.87 1.51
Principles of flight (e.g., lift, drag, gravity) 4.43 567 1.24
How to build a UAV 4.79 5.93 1.15
How to fly a UAV 5.64 6.20 0.56
How to use a 3-D printer 3.50 3.67 0.17
There was a negative shift in student’s ability to
1)talk in a small group of strangers;
2)present a talk to a group of friends;
3)talk with a friend and
4)talk in a small group of friends but most students agreed
from the beginning they already had those skills. Results are
in Table 16.
Survey respondents rated their agreement with statements
about their attitudes towards science and STEM careers.
Three of the STEM career related statements are negative,
so for those items a negative shift in value represents a
positive shift in attitude (See Table 17).
25

2015 FINDINGS
WEEKLY STUDENT SURVEYS
At the end of each week, students were given a
short satisfaction and content survey. Students
were asked 10 Likert scale questions about their
satisfaction with the Modern Blanket Toss program
that week. They were also asked three openended
questions about what they learned.
Sample
Each week, between 16 and 179 students
completed the satisfaction survey. Table 18 shows
the counts for each week.
Analysis and Results
Similar to the previous survey, students were
asked how much they agreed with a series of
statements and a numerical value was assigned to
the strength of that agreement; “strongly disagree”
equals 1 and “strongly agree” equals 4. The
responses for each item, each week are averaged
and compared. The larger the value, the more and
stronger respondents agree.
Table 9 reports the average values of the
responses for each week. Satisfaction items
remained high for all six weeks. STEM related
items shifted in different directions.
Communication and leadership skill items also
shifted in different directions each week.
Table 16 Leadership Skill Items
Leadership
Pre-
Average
Post-
Average
Change
I am able to develop, implement, and communicate new ideas to others. 3.23 3.20 -0.03
I am able to listen carefully to what other people are saying. 3.38 3.27 -0.12
I am able to leverage strengths of others to accomplish a common goal. 3.21 3.07 -0.15
I am able to work effectively with a diverse team. 3.38 3.20 -0.18
I am open to new and diverse perspectives. 3.33 3.13 -0.20
I am able to set goals and prioritize tasks to meet goals. 3.38 3.14 -0.24
I am able to collaborate and cooperate effectively with teams. 3.36 3.07 -0.29
Table 15 Communication Skill Items
Communication Pre-Average Post-Average Change
Present a talk to a group of strangers. 2.00 2.31 0.31
Talk in a large group of acquaintances 2.08 2.33 0.26
Talk in a large meeting of friends. 2.86 3.00 0.14
Talk in a small group of acquaintances. 2.50 2.60 0.10
Talk in a large meeting of strangers 1.77 1.80 0.03
Talk with a stranger. 2.08 2.07 -0.01
Present a talk to a group of acquaintances 2.29 2.27 -0.02
Talk with an acquaintance. 2.64 2.60 -0.04
Talk in a small group of strangers. 2.57 2.47 -0.10
Present a talk to a group of friends 2.92 2.80 -0.12
Talk with a friend. 3.79 3.57 -0.21
Talk in a small group of friends 3.29 3.00 -0.29
26

2015 FINDINGS
Items can be grouped into three categories. General satisfaction items are shown in Figure 1. Satisfaction remained high (average greater than 3.0) for all six weeks for
most items. However, over all enjoyment of the class appears to have decreased in week 6.
Students’ interest in pursuing a STEM field increased steadily each week for the first half of the course. However, weeks 5 and 6 saw a decrease in this area. The material
covered appears to have challenged students through week 5, by week 6 the material became less challenging for the students. Please see Figure 6.
Figure 7 shows that item responses about communication and leadership skills varied though remained positive throughout the course. Overall, students were the most
positive about working as part of a group or team.
Figure 4 Satisfaction Items from Weekly
Surveys
Figure 5 Skill and STEM Interests from
Weekly Surveys
Figure 6 Communication and
Leadership Items from Weekly Surveys
27

2015 FINDINGS
Open-Ended Questions
Students wrote similar descriptions about what they learned each week, suggesting they were paying attention. Throughout most of the course, students liked building,
flying, and learning about the UAVs best. During weeks 5 and 6 students were most interested in the QGIS, GIS and mapping skills. Most students didn’t want to change
anything about the course, but a few preferred more flying time, some hoped for more breaks, and a few others had some suggestions for slowing the pace of the class
Table 17 Attitudes about STEM Careers and Science Items
Table 18 Weekly Survey Counts
Pre-Average Post-Average Change
Week of
Count
My scientific investigations are better when I begin by
asking a question.
Attitudes toward science
2.58 3.00 0.42
Looking at scientific data in different ways helps me learn. 3.21 3.33 0.12
Using a map to study scientific data helps me to learn. 2.92 2.93 0.01
Science is an interesting subject to study. 3.21 3.00 -0.21
Attitudes toward STEM careers
Scientists/engineers are boring people 1.54 1.86 0.32
A career in science, technology, engineering, or math would
be financially rewarding.
A career in science, technology, engineering, or math would
be dull and boring.
Pursuing a degree in a science, technology, engineering, or
math field does not interest me.
I am interested in the way that science, technology,
engineering, or math can be used to help people.
3.21 3.40 0.19
1.50 1.67 0.17
1.77 1.93 0.16
3.36 3.20 -0.16
Scientists/engineers are really cool people. 3.36 3.14 -0.21
Week 1 May 25 17
Week 2 June 1 19
Week 3 June 8 18
Week 4 June 15 18
Week 5 June 22 16
Week 6 July 29 16
or asked for teachers to provide more specific examples
while teaching. Please see the tables and comments below
for more detailed information
What did you like best about the class this week? List one
or two things that stand out to you that you really enjoyed.
The following comments are illustrative of what students
enjoyed the most throughout the class:
I really enjoyed working with my team while putting the
(UAV) design together.
I enjoyed going out to that corner reflector. What I liked
best was going on the computer and making maps.
28

2015 FINDINGS
I liked learning about GPS systems and working with carbon fiber.
I enjoyed learning how to use the
program for drones.
This week I really enjoyed Geocaching!
Geocaching was fun, I got some pretty
cool stuff.
I enjoyed mapping and geo-referencing
with my group. I really enjoyed working
with my community.
I liked learning about GIS skills the best
this week.
Working with the UAV group and also
working on the Prezi.
I liked that we learned to become
better leaders.
Table 19 Averages of Responses by Week and Item
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Survey Count 17 19 18 18 16 16
The Modern Blanket Toss classes have held my interest
this week.
3.44 3.22 3.22 3.22 3.19 3.13
The presenters were well prepared. 3.29 3.33 3.22 3.17 3.25 3.06
The environment (meaning the classroom, field trip
location, or other place where instruction took place) was
conducive to learning.
3.35 3.28 3.35 3.28 3.33 3.06
I learned new science and technology concepts this week. 3.24 3.17 3.18 3.12 3.27 3.13
The information was challenging to me. 2.88 2.76 2.83 2.89 3.00 2.88
I enjoyed the classes this week and look forward to more. 3.35 3.47 3.29 3.22 3.13 2.88
I am considering a career in a STEM field. 2.93 3.00 3.00 3.13 2.86 2.93
I feel confident communicating with others about science
topics.
2.94 2.82 3.00 2.94 2.94 2.75
I am becoming more of a leader through this class. 3.13 2.81 2.94 3.00 3.00 2.88
I am comfortable working as part of a team. 3.53 3.44 3.56 3.44 3.31 3.25
Mine was learning about other communities’ problems, like how Shishmaref is shrinking, and about erosion.
29

2015 FINDINGS
The QGIS lesson, the information was very useful and stimulating.
am looking forward to more.
I liked how we learned more about applying for college and scholarships.
I liked the only time we went to the GIS class. GIS is very fun software
program to use and I can't wait to use it when I go home.
Students also had an opportunity to share what they have learned each week.
The tables below reflect what students have taken away from the class.
What did you learn this week? Please tell us in 3 or 4 sentences.
The following comments are illustrative of what students have learned so far in
the class:
I learned about the GIS. We could become GIS workers. We can get to travel
to different places.
I learned that a quadcopter can't weigh more than 55 pounds and how to
work more with wires and learned how to solder too. I love being in this class.
I learned how to make an essay better. I also learned how obtaining
scholarships is different from everyone. The FAFSA is very important and it
needs to be renewed in January.
I learned how to become a stronger leader. I learned the concept of setting
and completing a goal.
I learned a little about GIS. I learned how to collect data and use it to make
a map. It's all fun.
I learned more about the Geo-referencing in the GIS class. I also learned to
make Prezis with my community and my learning skills are improving.
I learned that we can use the quadcopters to make them fly themselves. I
really liked that. I also learned that they can idle in the air, just sit for as long
as we want.
This week in my class we all flew our first built drones a little bit and we are
going to be building much bigger drones which is very exciting!
I learned what CR and SAR stands for. CR means corner reflector. SAR
Synthetic Aperture Radar. I had fun learning this week.
This week I had fun and learned new things. I learned how to Geo-tag and
what Geo- caching is. We just went Geocaching yesterday and it was fun. I
Lastly, students were asked each week ‘what would you change to make the
class better?’ Most students reported that they did not want to change the class
at all or that they preferred to keep the class the same. Below are a few
examples.
I would not really change anything because it's cool.
30

2015 FINDINGS
Nothing it's getting more exciting every time.
Other students had suggestions for improving the class. Throughout the course,
students listed the need for more breaks during the class. Other students
expressed that they prefer a slower pace for the course. Some students had
suggestions that were more specific. The comments below illustrate.
I want the teachers to be descriptive. I don't know what is in their heads so I
don't always know what they mean.
Have written or digital presentations to explain scientific concepts.
More academically focused time and less “go with the flow" feeling
I would change some of the ways to do teamwork. I hate our teamwork.
More GIS in the UAV group the classes should be integrated over the summer
To interact with GIS some more. I would like to have more classes with GIS
classes.
Week 1
Concept
Count
Percent
(n=17)
UAV (designing, building, flying) 11 64.71%
Map concepts (education, design) 3 17.65%
Technological experience (using the computer and the time lapse
website)
Week 2
Concept
4 23.53%
Count
Percent
(n=17)
UAV (designing, building, flying) 5 27.78%
Learning about the corner reflector 3 16.67%
Soldering 3 16.67%
Technical opportunities (materials, parts, using the computer) 7 38.89%
Week 3
I would not have conflicting leaders, makes for an awkward hierarchy.
Concept
Count
Percent
(n=17)
UAV (designing, building, flying) 10 55.56%
Going off campus/being outside 3 16.67%
Geocaching/Geotagging 6 33.33%
31

2015 FINDINGS
Week 5
Week 8
Concept
Count
Percent
(n=17)
Concept
Count
Percent
(n=17)
UAV (safety, flying) 9 50%
Geo-referencing 8 44.44%
UAV (designing, building, flying) 11 64.71%
Map concepts (education, design) 3 17.65%
Leadership, goal setting, 4L and communication 9 50%
Technological experience (using the computer and the time lapse
website)
4 23.53%
Week 6
Week 7
Concept
Count
Percent
(n=17)
Concept
Count
Percent
(n=17)
UAV (safety, flying) 2 12.5%
GIS/QGIS/Map skills 15 93.75%
Using the GoPro/Practicing photography 9 56.25%
UAV (designing, building, flying) 5 27.78%
Learning about the corner reflector 3 16.67%
Soldering 3 16.67%
Week 4
Technical opportunities (materials, parts, using the computer) 7 38.89%
Concept
Count
Percent
(n=17)
Week 9
UAV (safety, flying) 3 18.75%
Concept
Count
Percent
(n=17)
GIS/QGIS/Map skills 13 81.25%
Geo-referencing 6 37.5%
Prezi 3 18.75%
UAV (designing, building, flying) 10 55.56%
Going off campus/being outside 3 16.67%
Geocaching/Geotagging 6 33.33%
32

2015 FINDINGS
Week 12
Concept
Count
Percent
(n=17)
UAV (safety, flying) 9 50%
Geo-referencing 8 44.44%
Leadership, goal setting, 4L and communication 9 50%
Week 11
Concept
Count
Percent
(n=17)
UAV (safety, flying) 2 12.5%
GIS/QGIS/Map skills 15 93.75%
Using the GoPro/Practicing photography 9 56.25%
Week 10
Concept
Count
Percent
(n=17)
UAV (safety, flying) 3 18.75%
GIS/QGIS/Map skills 13 81.25%
Geo-referencing 6 37.5%
Prezi 3 18.75%
33

SECTION 3
2016 Findings
Executive Summary
The University of Alaska Fairbanks received National Science Foundation Experimental Program to Stimulate Competitive Research (EPSCoR)
Track-3 funding in 2013 to implement Modern Blanket Toss: Engaging Underserved Students in STEM Fields through Unmanned Aerial
Vehicles. Implemented as part of the University of Alaska Fairbanks Upward Bound program, students from five rural communities in Alaska
received the opportunity to take special elective classes during the Upward Bound summer program at the University of Alaska Fairbanks
(UAF), in which they learned unmanned aerial vehicle (UAV) and geographic information systems (GIS) operation, science materials and
methods, and science communication and leadership skills. Those students were expected to return to their home communities where, with
academic-year Upward Bound support, they were to use UAVs as the basis for a local mapping project.
34

2016 FINDINGS
The mission of Modern Blanket Toss was to help these Upward Bound students
develop interest in STEM fields through the use of technology that can directly
improve the well-being of Alaskans. The project’s goal was to demonstrate how
a three-part program of science education built around 1) cutting-edge STEM
technologies (in this case, UAVs), 2) science communication and leadership skills,
and 3) community involvement, can better expose these students to STEM
careers, prepare them for higher education, and enable them to contribute to
their communities.
Seven Upward Bound programs participated in Modern Blanket Toss over the
three year program period: Chefornak, Seward, Bethel, Nikiski, Shishmaref,
Chevak, and Saint Mary’s. These communities vary widely in terms of location,
local economy, population size, poverty levels, racial makeup, and income,
representing a cross-section of rural Alaska. Overall, 28 individual students
attended at least one summer session of the Modern Blanket Toss program – 19
students attended one summer session, six students attended two summer
sessions, and three students attended three summer sessions. Approximately 60
individual students participated in the Modern Blanket Toss activities at their
Upward Bound site.
The purpose of the evaluation was to assess the effectiveness
of the Modern Blanket Toss program in increasing the
likelihood that participating students would go on to pursue
college degrees in STEM areas and ultimately pursue STEM
careers. The evaluation used six indicators to measure the
program’s effectiveness, including 1) Upward Bound local
coordinator capacity to facilitate the program locally; 2)
student satisfaction and engagement with program activities;
3) student knowledge about the Modern Blanket Toss
content; 4) student communication and leadership skills; 5)
student interest in STEM and confidence in STEM abilities;
and 6) student academic data including their overall grade
point average and their math and science course grades
specifically.
Key Findings
At the beginning of the project, Upward Bound local coordinators reported
already feeling they had the knowledge and ability to implement science
practices and project-based learning with their students, yet by the end of the
three year project they reported significant gains in their knowledge about
UAVs and GIS software instrumental in planning and carrying out community
projects.
• Local coordinators made the greatest gains in four areas: how to build a UAV,
applications of UAV technology, the safety rules and regulations that apply to
flying a UAV, and how to acquire different types of data using UAVs.
• Three areas showed no gains, because coordinators reported high levels of
knowledge at the beginning of the program: The range of science, technology,
engineering and math careers that are available to students; the level of math
and science students will need for various science, technology, engineering, and
math careers; and principles of flight (e.g., lift, drag, gravity).
• Coordinators reported a low level of initial knowledge and little growth
related to several important content areas that were potentially important to
community projects: how to ask questions using GIS software; how to use GIS
mapping software; and how to acquire data from databases.
• Students reported consistently high satisfaction with the Modern Blanket
Toss summer program. In particular students noted the following:
• Students were excited about learning new technology.
• Students liked the hands-on aspect of the course, both the experience flying
and the application of mapping.
• Students felt that the connections they made with people was an important
benefit of the program and appreciated working together with like-minded
peers, and cited this as an important aspect of the program.
35

2016 FINDINGS
• Students reported gains in their understanding of the Modern Blanket Toss
content areas and science practices and in their communication skills.
Students also increased their already positive attitudes toward science.
• Student content understanding clearly and significantly (P < 0.05) increased as
a result of participating in the Modern Blanket Toss program. Students
reported the greatest content gains in 1) how to use GIS mapping software,
2) how to acquire different types of data using UAVs, 3) how to use data
collected to develop a map/image that represents
reality, and 4) the safety rules and regulations that
apply to flying a UAV.
• Content gains were greater during the summer
program than during the school-year program.
• Students also reported significant gains (P < 0.05) in
their understanding of science practices. They
reported the greatest gains in four areas: 1) asking a
new scientific question from data; 2) formulating and
refining experimental questions; 3) communicating the
results of data analysis to a broader audience; and 4)
using a map to analyze scientific data.
• Students reported significant increases in their
Students of MBT Bethel.
communication skills (P < 0.05). They reported the
greatest gains in four areas: 1) ability to talk to a group of strangers; 2) ability
to talk with a stranger; 3) ability to talk in a small group of acquaintances; and
4) ability to talk in a large meeting of strangers.
• Many of the students already had a positive attitude toward science when
they started, however, the overall the attitude toward science scale score still
shifted positively. Students reported the greatest changes in two items: 1) my
scientific investigations are better when I begin by asking a question; and 2)
using a map to study scientific data helps me to learn.
• While overall students scale scores for attitude toward STEM careers did not
increase, several individual items did increase, including 1) a career in STEM
would be financially rewarding. In addition, students were less likely to agree
that a career in STEM would be dull and boring and that a degree in STEM
does not interest them. These are both positive outcomes.
• Students who participated in the program two or more school years showed
cumulative gains in understanding of the Modern Blanket Toss content, but
those gains leveled off after two years of participation
for students who participated all three years. Students
who participated in the program two or more years did
not however show corresponding cumulative gains in
their understanding of science practices or
communication skills.
•Students’ grades and the number of math and science
courses completed in high school did not increase
during the evaluation period.
•Students who participated in Modern Blanket Toss had
significantly higher GPAs and science and math grades
than control group students.
•The number of math or science classes taken by
Modern Blanket Toss participants did not change from
pre to post program. However, the sample size was extremely small and it is
difficult to find significant relationships from the data as statistical tests
normally require a larger sample size to ensure a representative distribution of
the population.
• The average math and science grades for Modern Blanket Toss participants
did not change from pre to post program. Again, the sample size was
extremely small and it is difficult to find significant relationships from the data
as statistical tests normally require a larger sample size to ensure a
representative distribution of the population.
36

2016 FINDINGS
INTRODUCTION
The University of Alaska Fairbanks received National Science Foundation
Experimental Program to Stimulate Competitive Research (EPSCoR) Track-3
funding in 2013 to implement Modern Blanket Toss: Engaging Underserved
Students in STEM Fields through Unmanned Aerial Vehicles. Implemented as
part of the University of Alaska Fairbanks Upward Bound program, students
from five rural communities in Alaska received the opportunity to take special
elective classes during the Upward Bound summer program at the University of
Alaska Fairbanks (UAF), in which they learned unmanned aerial vehicle (UAV)
and geographic information systems (GIS) operation, science materials and
methods, and science communication and leadership skills. Those students were
expected to return to their home communities where, with academic-year
Upward Bound support, they were to use UAVs as the basis for a local mapping
project.
The mission of Modern Blanket Toss was to help these Upward Bound students
develop interest in STEM fields through the use of technology that can directly
improve the well-being of Alaskans. The project’s goal was to demonstrate how
a three-part program of science education built around 1) cutting-edge STEM
technologies (in this case, UAVs), 2) science communication and leadership skills,
and 3) community involvement, can better expose these students to STEM
careers, prepare them for higher education, and enable them to contribute to
their communities.
The purpose of the evaluation was to assess the effectiveness of the Modern
Blanket Toss program to increase the likelihood that participating students would
go on to pursue college degrees in STEM areas and ultimately pursue STEM
careers. This report provides the summative evaluation results. The report is
divided into four sections. Section 1 describes the primary methods used by the
evaluation, including student surveys and interviews, academic data, and surveys
of Upward Bound site coordinators. This section also discusses the limitations of
the evaluation methods. Section 2 summarizes the implementation of the
program, including a description of the Upward Bound communities
participating in the program and the number of students participating. Section 3
summarizes the evaluation results by the indicators used to measure the
program’s effectiveness, including 1) Upward Bound local coordinator capacity
to facilitate the program locally; 2) student satisfaction and engagement with
program activities; 3) student knowledge about the Modern Blanket Toss
content; 4) student communication and leadership skills; 5) student interest and
confidence in STEM; and 6) student academic data including their overall grade
point average and their math and science course grades specifically. Section 4
provides a brief discussion of the major findings and recommendations to
address challenges identified in the report.
SECTION 1. EVALUATION METHODS
The purpose of the evaluation was to assess the effectiveness of the Modern
Blanket Toss program in increasing the likelihood that participating students
would go on to pursue college degrees in STEM areas and ultimately pursue
STEM careers. A descriptive approach was used to evaluate indicators related to
the implementation of the program: Upward Bound local coordinator capacity
to facilitate the program locally, and student satisfaction and engagement with
program activities. An outcomes-based approach (Phillips et al., 2014) was used
to evaluate indicators related to the program’s expected outcomes, including:
student knowledge about the Modern Blanket Toss content, student
communication and leadership skills, student interest in STEM and confidence in
STEM abilities, and student academic data including their overall grade point
average and their math and science course grades specifically (Dorsen et al.,
2006; Hinojosa et al., 2016).
To measure the program’s impact on selected indicators, the evaluation used
several data sources: student and coordinator surveys, student interviews,
academic data, observation field notes, and document review. Figure 1 maps
each indicator to the appropriate data collection instrument.
Site Coordinator Survey
37

2016 FINDINGS
alpha is a measure of internal consistency among items included in a scale
(Trochim, 2006). A Cronbach’s alpha that is greater than 0.7 is considered
evidence that the items included measure an underlying construct (George and
Mallery, 2003). For all of our scales there was a high rate of internal consistency
among the survey items. Results of the reliability analysis are summarized in
Table 1 below.
Table 1 Cronbach's alpha results for student pre/post survey
Scale n Items
Item
Mean
Min Max Range Variance Crohbach’s
Alpha
Content Knowledge 217 14 4.78 3.43 6.11 2.68 0.868 0.956
Science Practices 113 11 2.93 2.65 3.16 0.51 0.025 0.913
Figure 1 Modern Blanket Toss data collection instruments mapped to
indicators
DATA COLLECTION METHODS AND ANALYSIS
1. Student Pre/Post Survey
The Student Survey was designed to measure changes in participating students’
1) content knowledge and science practices related to Modern Blanket Toss
content, 2) leadership and communication skills, 3) attitude towards science, 4)
STEM career interest, and 5) engagement and excitement in project topics.
The survey included 58 Likert-scale items. The items in the survey combine to
make 6 constructs or scales. To measure the reliability of our scales, we
calculated Cronbach's alpha coefficient for each scale using SPSS. Cronbach's
Leadership Skills 201 7 3.23 3.15 3.3 0.14 0.002 0.876
Communication Skills 222 12 2.77 2.07 3.7 1.63 0.238 0.936
Attitudes Towards
Science
Attitudes Towards
STEM Careers
172 4 3.13 2.98 3.26 0.29 0.025 0.735
185 6 3.31 3.08 3.4 0.31 0.014 0.849
Note: For the reliability study, all 234 responses were valid.
Administration: The survey was administered to students at the beginning
and end of the summer academies, as well as at the beginning and end of the
school year at the Modern Blanket Toss sites, for a total of 10 administrations.
Many students took the survey several times if they participated in their school
efforts as well as the summer program. Surveys were completed using a paper
survey at the beginning and end of the summer program. Surveys completed at
Upward Bound sites were completed using SurveyMonkey, an online survey
tool.
38

2016 FINDINGS
Analysis: For each construct, we calculated
a scale score for each respondent. The scale
score is the average of the response to items
included in the scale. For “Content
Knowledge,” respondents rated their own
knowledge from 1 (low level of knowledge)
to 10 (high level of knowledge), so the scale
score also has a range of 1 to 10. For the
“Communication Skills” items, respondents
chose “not comfortable at all” (equal to 1),
“somewhat comfortable” (equal to 2),
“moderately comfortable” (equal to 3), or
“very comfortable” (equal to 4). The scale
score is an average between 1 and 4. All other
items asked respondents to “strongly disagree”
(equal to 1), “disagree” (equal to 2),
“agree” (equal to 3), or “strongly agree” (equal
to 4), so those scales also range from 1 to 4
with 4 as a more positive score. We used a paired samples t-test to test
whether the mean scale scores were significantly different (P < 0.05) between
an individual’s first survey and last survey.
Sample: During the three year project there were 234 responses to the
survey from 90 individual students. As the survey was designed to give pre/post
information, only surveys from those students who completed both a pre and a
post-survey were included. In all, 57 students completed the surveys at least two
times; Chefornak had the greatest number of individuals complete the survey
and Shishmaref had the least (Table 2).
Table 2 Count, by school, of individual students who completed two or more surveys.
School
Total
Individuals
May
2014
July
2014
Sept
2014
April
2015
May
2015
Note: In total these students completed 199 valid surveys; some students completed the survey more than two times.
2. Student Weekly Survey
July
2015
Sept
2015
April
2016
May
2016
July
2016 Total
Bethel 11 7 7 5 7 3 2 7 7 2 2 49
Chefornak 19 6 6 12 13 6 6 13 14 4 3 83
Nikiski 13 2 2 4 8 0 0 10 5 1 0 32
Seward 5 0 0 4 4 1 1 0 0 1 1 12
Shishmaref 4 1 1 4 4 1 2 0 0 0 0 13
Chevak and Saint
Mary’s
Table 3 Student weekly survey satisfaction scale Cronbach's alpha results
Scale n Items Item Mean Min Max Range Variance Cronbach’s Alpha
Satisfaction 205 10 3.16 2.74 3.43 0.693 0.047 0.81
5 0 0 0 0 5 5 0 0 0 0 10
Total 57 16 16 29 36 16 16 30 26 8 6 199
The Student Weekly Survey was designed to measure, over the course of the
summer program, changes in participating students’ 1) content knowledge and
science practices related Modern Blanket Toss content and 2) their satisfaction
with the summer program activities, as well as to provide immediate feedback
to course instructors. A copy of the survey is included in Appendix A.
The survey included 10 Likert-scale items that we combined to make a
satisfaction scale. To measure the reliability of the satisfaction scale, we calculated
Cronbach's alpha coefficient for each scale using SPSS. The Cronbach's
alpha for the satisfaction scale was 0.810; a Cronbach’s alpha that is
greater than 0.7 is considered evidence that the items included
measure an underlying construct (George and Mallery, 2003). Results
of the reliability analysis are summarized in Table 3.
39

2016 FINDINGS
Administration: The weekly survey was administered on paper to students
at the end of each week of the summer program (six weeks in years one and
two and three weeks in year three).
Analysis: Survey results were analyzed using descriptive statistics, analysis of
variance (with the year as the explanatory variable and satisfaction scale score
as the response variable),
and content analysis.
Table 4 Count of individuals who
completed weekly summer
satisfaction surveys by week
Week Number
2014
(N=17)
2015
(N=18)
2016
(N=9)
1 17 17 9
2 17 17 8
3 15 18 8
4 15 18 0
Sample: In total, 96
surveys were completed in
year one, 102 in year two,
and 25 in year three. Table 4
illustrates the number of
surveys completed by week
and year.
3. Student Interview
5
6
16
16
16
16
0
0
The purpose of the student
interview was to gather indepth
feedback from
participants about the
Modern Blanket Toss
program and students’ perceptions about the program’s impact on their future
plans for college and career. The interview was divided into three sections: what
students learned; how their participation impacted their future plans; and how
their participation impacted their communication and leadership skills and
whether they will use those skills in the future. Students were offered a $10 i-
tunes gift card as an incentive to participate in the interview.
Administration: Interviews were conducted in person with students and
recorded with students’ permission. Only students with positive parental/
guardian permission were interviewed.
Analysis: Interviews were coded and analyzed using both a priori and
emergent codes using ATLAS.ti software.
Sample: In total, 29 unique students
were interviewed during the course of
the program. Some students were
interviewed more than once (i.e., if
participating in two or more summer
sessions), resulting in 39 interviews.
4. Student Academic Data
Table 5 Count of interviews
by year
The evaluation compared academic data
of Upward Bound students participating
in the Modern Blanket Toss program to
Total Interviews 39
similar Upward Bound students not participating in the Modern Blanket Toss
program. Academic data included GPA, number of science and math courses
taken and grades in those courses, and on-track for graduation. We also looked
at the availability of courses offered by the high schools. Academic data was
provided to the evaluation team by the Upward Bound program.
Analysis: A paired t-test was used to test for difference in the means of GPA,
science grades, and math grades pre and post program. Analysis of variance was
used to test for differences in the mean pre and post GPA, science, and math
grades between the Modern Blanket Toss participants and the Upward Bound
control group students.
Sample: For the purpose of the evaluation, we used academic data for
students who entered the Modern Blanket Toss program in the summer of 2014
Year
Count of
Interviews
2014 (N=17) 14
2015 (N=18) 17
2016 (N=9) 8
40

2016 FINDINGS
and who participated in one or more of the summer programs. For a control
group, we randomly collected data from students enrolled in Upward Bound
sites that did not participate in the Modern Blanket Toss program.
We received data for 16 of the 17 students who started the Modern Blanket
Toss program in the summer of 2014, for six of the nine students who started
the program in the summer of 2015, and for three of the five students who
started the program in the summer of 2016.
Table 7 Count of students with both pre and post academic data
by enrollment year.
Count of Students with Pre and Post
Academic Data
GPA Science Grades Math Grades
Students who started the program in 2014 14 9 13
Table 6 Count of students with academic data by school
year and by the year in which they enrolled in the Modern
Blanket Toss program.
Count of Students with Academic
Data
2013-2014 2014-2015 2015-2016
Students who started the program in 2015 2 2 2
Upward Bound students 15 13 18
Total Count of students 31 24 33
Note: Pre data for students who enrolled in the program in 2014 is from the 2013-2014
school year; pre data for students who enrolled in the program in 2015 is from the
2014-2016 school year; and control group pre-data is from the 2013-2014 school year.
Students who started the program in 2014 16 14 3
Students who started the program in 2015 — 6 2
Students who started the program in 2016 — — 3
Upward Bound Students 15 21
no data
received
Note: For students who enrolled in 2014, data from the 2013-2014 school year is
considered pre-data; for students who enrolled in the program in 2015, data from
the 2014-2015 school year is considered pre-data; and for students who enrolled in
2016, data from the 2015-2016 school year is considered control data. For the
Upward Bound control group students, data from the 2013-2014 school year is
considered pre-data. Only matched pre and post data is included in the analysis .
However, only academic data for which
we have matched pre and post data is
included in the analysis (Table 7).
All of the Modern Blanket Toss students
included in the academic data sample
were in grade 10 or 11 for postprogram
GPA and classes taken,
whereas the Upward Bound students
were primarily in grades 10 or 11, but a
third were also in grades 9 and 12.
Items printed using the 3-D printer.
41

2016 FINDINGS
Table 8 Count of students by grade level during the 2014-2015
school year (the first school year of the program)
Grade in
2014-2015
5. Site Coordinator Survey
2014 MBT Students Control
Female Male Total Female Male Total
9 0 0 0 2 1 3
10 5 5 10 1 5 6
11 5 4 9 7 2 9
12 0 0 0 1 3 4
Total 10 9 19 11 11 22
The purpose of the Site Coordinator Survey was to determine how
comfortable and capable the site coordinators felt in presenting and engaging in
Modern Blanket Toss activities. The survey looked at two areas related to
coordinators’ capacity to facilitate the Modern Blanket Toss program locally: their
technical skills (such as ability to use GIS software or fly a UAV) and their ability
of help students use data to ask a scientific questions. A copy of the survey is in
Appendix A.
The Coordinator Survey included 29 Likert-scale items. The items in the survey
combine to make four constructs or scales. To measure the reliability of the
scales, we calculated Cronbach's alpha coefficient for each scale using SPSS
(Trochim, 2006; George and Mallery, 2003). Results of the reliability analysis are
summarized in Table 9.
Administration: The Coordinator Survey was administered to coordinators
annually using SurveyMonkey. The Coordinator Survey was administered three
times: once in the fall of 2014, once in the fall of 2015, and once in the spring of
2016.
Analysis: We computed a ratio for all numerical responses to calculate scale
responses and used analysis of variance to test for a difference in mean scale
score by program year with program year as the explanatory variable and mean
scale score as the response variable.
Sample: Four coordinators completed the survey all three times; three
additional coordinators responded in 2014, but their responses are not included
in the analyzed data because they had only one data point.
Table 9. Cronbach’s alpha results for coordinator survey scales
Scale n Items Item Mean Min Max Range Variance Crohbach’s Alpha
Career Knowledge 16 3 0.74 0.68 0.78 0.094 0.003 0.842
GIS 15 4 0.56 0.36 0.83 0.47 0.045 0.829
Science Practices 15 12 0.79 0.47 0.9 0.43 0.018 0.911
UAV 8 8 0.68 0.55 0.9 0.35 0.013 0.928
42

2016 FINDINGS
LIMITATIONS OF THE EVALUATION
The results of this evaluation are limited by several factors. First, we rely greatly
on self- reported data. Self-reported data is limited by the fact that it rarely can
be independently verified. Further, self-reported data contain several potential
sources of bias that should be noted as limitations: 1) selective memory
(remembering or not remembering experiences or events that occurred at
some point in the past); 2) telescoping (recalling events that occurred at one
time as if they occurred at another time); 3) attribution (the act of attributing
positive events and outcomes to one's own agency but attributing negative
events and outcomes to external forces); and 4) exaggeration (the act of
representing outcomes or embellishing events as more significant than is actually
suggested from other data). Also, much of the work of this project takes place
during the school year at the remote sites. It is very difficult to assess what
information and support was available to students at the different sites, and to
what degree the implementation of these supports differed. Additionally, our
sample size is extremely small; it is difficult to describe statistically significant
relationships from the data, as statistical tests normally require a larger sample
size to ensure a representative distribution of the population. Lastly, the impact
of the project on student’s career choices will not be observable until months
or years after the end of data collection (University of Southern California,
2016).
Section 2. Implementation
Modern Blanket Toss participants were drawn from the ranks of UAF Upward
Bound students. Upward Bound is a secondary education program within the
UAF Division of General Studies which works with low-income, high-risk high
school students who are identified as the first in their families to attend college.
Modern Blanket Toss, aligned with the Upward Bound program, had two distinct
components: summer programs held at UAF, and school-year implementation at
participating students’ schools.
SUMMER PROGRAMMING
First, a subset of students from the participating Upward Bound sites
participated in a six-week summer program held on the UAF campus. All
participants took two core Upward Bound courses (e.g., language arts,
mathematics, laboratory science, foreign languages, technology skills, study skills,
service learning, and financial literacy) as well as Modern Blanket Toss courses
focused on UAVs, geospatial cognition, and associated science and technology
and communication and science leadership training.
The first summer program in 2014 focused on flight safety, UAV flying skills with
both simulators and quadcopters, and collecting data with the UAVs. Students
were given glimpse on the last few days of how the collected photos and videos
could be linked to maps on the computer.
In the summer of 2015, there were two afternoon sections devoted to MBT.
One section focused on flying and building bigger, stronger UAVs to be used for
data collection and the other section focused on GIS skills to use and integrate
the collected data. Students in the build section learned about design by
calculating loads and lifts as well as hands on wiring, soldering, sanding and
cutting. Both groups worked together for the last week or so to fly UAVs,
collect photographic data and then merge the data into a map.
43

2016 FINDINGS
The final summer ambitiously aimed to both teach students a new software,
Agisoft, for using their data, as well as facilitate a practical application project that
involved making a map of specific lands at Chena Hot Springs. The Fairbanks
segment of the course was shortened to three and a half weeks. In the last two
weeks of the 2016 summer program, Upward Bound, including Modern Blanket
Toss students, participated in a science and culture field trip that involved
camping on the Big Island of Hawaii. In addition, Modern Blanket Toss students
met with the Upward Bound students of the University of Hawaii Hilo and
assisted students in developing the initial safety and skills for initial UAV flight.
SCHOOL YEAR IMPLEMENTATION
Second, Upward Bound local coordinators conducted after-school sessions with
students focused on implementation of community-based UAV exercises and
projects. In addition to the primary Upward Bound goal of coordinators serving
as academic support for college bound students, the coordinators were also
given materials and support to help the students with UAV projects. The
modules in the 2014-2015 year focused primarily on building UAVs at the
schools. A new flight school style curriculum was introduced in the fall of 2015
to encourage
Table 10 Local project implemented by
students in Modern Blanket Toss
communities
Site
Chefornak
Seward
Bethel
Nikiski
Local Project
Improving local maps
Spring flood prediction
Mapping dangerous ice on winter routes
Bluff erosion monitoring
students to earn
flying skill and
controlled
independence with
the UAVs. In the
final year, students
in each community
were encouraged
to work on a local
project; Table 10
describes project
topics by site.
PARTICIPATING
COMMUNITIES
AND
STUDENTS
Seven Upward
Bound programs
participated in
Modern Blanket
Toss over the
three year
program period:
Chefornak,
Seward, Bethel,
Nikiski,
Shishmaref,
Chevak, and Saint
Mary’s. These
communities vary
widely in terms of
location, local
economy,
population size,
poverty levels,
Table 11 Count of students participating in
Modern Blanket Toss summer program by
Upward Bound program community.
Community
Summer
2014
Summer
2015
Summer
2016
Chefornak 6 5 4
Seward 0 1 1
Bethel 8 2 2
Nikiski 2 0 1
Shishmaref 1 1 0
Chevak 0 3 0
Saint Mary’s 0 3 0
Total 17 15 8
Note: Overall, 28 individual students attended the summer Modern
Blanket Toss summer program; 19 students attended 1 session, 6
students attended two sessions and three students attended three
sessions.
racial makeup, and income, representing a cross-section of rural Alaska. Bethel
(population 6,080) is a primarily Yup’ik community off the road system which
serves as a hub and service community for dozens of Alaska Native
communities in the Yukon-Kuskokwim Delta. Shishmaref (population 563), also
located off the road system, is a traditional Inupiat mixed-subsistence community,
in which residents combine cash incomes with living off the land or sea. Nikiski
(population 4,493) and Seward (population 2,693) are primarily white
communities located on the Kenai Peninsula a few hours’ drive from Anchorage,
the state’s largest city. Nikiski’s economy focuses on oil and gas while the
economy of Seward centers on tourism. Chefornak (population 418) is a
44

2016 FINDINGS
primarily Yup’ik mixed-subsistence community located in the Yukon-
Kuskokwim Delta. Chevak (population 938) is a Cup’ik mixed-subsistence
community located east of Hooper Bay. Saint Mary’s (population 507) is also
a primarily Yup’ik mixed-subsistence community located 450 air miles west of
Anchorage (Alaska Community and Regional Affairs, 2016). Table 11 details
participation by site for each year.
Section 3. Findings
This section of the report summarizes the outcomes of the program based
on the six indicators chosen to measure the program’s effectiveness: 1)
Upward Bound local coordinator capacity to facilitate the program locally; 2)
student satisfaction and engagement with program activities; 3) student
knowledge about the Modern Blanket Toss content; 4) student
communication and leadership skills; 5) student interest and confidence in
STEM; and 6) student academic data including their overall grade point
average and their math and science course grades specifically (Dorsen et al.,
2006; Hinojosa et al., 2016).
1) Upward Bound local coordinator capacity to facilitate the program
locally
The evaluation looked at four areas related to coordinators’ capacity to facilitate
the Modern Blanket Toss program locally: coordinator knowledge of STEM
careers, their knowledge of and ability to use GIS software, their knowledge of
and ability to use UAVs, and their knowledge of and ability to implement science
practices with their students. Coordinator self-reported capacity at the
beginning of the project is shown in Figure 2.
As the project began, the Upward Bound coordinators were least confident in
their knowledge of and ability to use GIS software and in their knowledge and
ability to use UAVs and most confident in their knowledge and ability to
implement science practices with their students. By the end of the program
Upward Bound coordinators increased their confidence in all areas: careers, GIS,
Figure 2. Site coordinators self-reported confidence related to Modern Blanket Toss technical
skills at the beginning of the project
UAV, and science practices and significantly (P < 0.05) increased their
confidence related to GIS and UAV topics.
Table 12 Modern Blanket Toss Upward Bound
coordinator survey scale scores by program year
Program Year 2014 (N=8) 2015 (N=4) 2016 (N=4)
Careers 0.61 0.88 0.88
GIS 0.44 0.66 0.66
UAV 0.53 0.8 0.81
Science Practices 0.73 0.87 0.84
Note: Bold indicates a significant increase in the average scale score
(P < 0.05).
45

2016 FINDINGS
Table 13 Average Modern Blanket Toss Upward Bound coordinator survey scale items by year.
The local coordinators made the greatest gains in four areas: 1) how to build a
UAV, 2) applications of UAV technology, 3) the safety rules and regulations that
apply to flying a UAV, and 4) how to acquire different types of data using UAVs
(Table 12). Three areas showed no gains, because the average response was
already 8 out of 10 or higher at the beginning of the grant. These three areas
were: understanding of the range of science, technology, engineering and math
careers that are available to students; the level of math and science students will
need for various science, technology, engineering, and math careers; and
2014 2015 2016
How to build a UAV 4 8.5 8.5 4.5
Applications of UAV technology 5 8.5 9 4
The safety rules and regulations that apply to flying a UAV 5.5 7.5 8.75 3.25
How to acquire different types of data using UAVs 4 7.25 7 3
How to ask questions using GIS software 2.5 5 5.25 2.75
How to use data collected to develop a map/image that represents reality 4 6 6.75 2.75
The amount of money students can earn from a science, technology, engineering or math career 5.87 8.75 8 2.25
How to fly a UAV 5.5 6.75 7.5 2
How to use GIS mapping software 4.25 5 6 1.75
Note: Average is the average Likert scale response on a scale of 1 to 10 where 1=”no knowledge” and 10 = “a lot of knowledge.”
Total
Change
principles of flight (e.g.,
lift, drag, gravity).
Several areas showed
little growth over the
life of the grant: how to
ask questions using GIS
software, how to use
GIS mapping software,
and how to acquire
data from databases.
These three
knowledge areas were
important for planning
and carrying out
community projects.
The results of openended
responses
echoed the results in
Table 13. Openresponse
questions
indicated that teachers
felt comfortable from
the beginning with
project-based learning
and the science
practices required by the project. Most reported having previous experience
with project-based learning. Upward Bound coordinators who were teachers
specifically named using project-based learning in teaching many topics, including
rockets, robotics, finance, statistics, math, and environmental science. One
teacher summed it up like this:
The range of science, technology, engineering and math careers that are available to students 8.25 8.5 9.25 1
The level of math and science students will need for various science, technology, engineering, and math careers 8.25 9 9 0.75
Principles of flight (e.g., lift, drag, gravity) 8.25 8.25 8.5 0.25
How to acquire data from databases 6.5 7.5 6.5 0
How to use a 3-D printer 3.25 3.75 3.25 0
I have been using project-based learning throughout my teaching
career. It provides a way for students to understand the
application of their knowledge.
46

2016 FINDINGS
There was a range of initial experience with mapping software reported by
coordinators at the beginning of the project, and only one coordinator reported
increased experience by the end of the project. Another coordinator felt they
were just beginning to understand the mapping of the UAV data and could do it
with some assistance.
All coordinators claimed to be novices in all things UAV at the beginning of the
project. By the end, all coordinators reported having more confidence in their
own understanding of the technology; all coordinators also reported gaining
understanding of the educational application of UAV technology, but some
reported still struggling with educational application despite gains. As one
teacher put it,
science and technology
concepts this week” and
“the information was
challenging to me.”
However, responses to
most individual survey
items, as with scale
scores, remained
consistent.
Students were excited
about being introduced
Table 14 Modern Blanket Toss summer
program average satisfaction scale scores
by program year
Program Year Count Mean Satisfaction Score
2014 93 3.2
2015 90 3.1
2016 22 3.2
UAVs are difficult to use for educational
purposes. Students want to fly the UAVs, but
have difficulty with some of the advanced
concepts for designing and building.
Another coordinator was able to use the UAV as a theme
for several kinds of lessons.
We've had lessons on the history of flight,
drones, some basics of flight like lift and drag
and some battery and electricity principles.
2) Student satisfaction and engagement with
program activities
Students reported high satisfaction during all three
summer programs and there were not statistically
significant differences between the years (P < 0.05). Table
14 illustrates this outcome.
Over the project’s three years, there was an increase in
student satisfaction with the summer program as
measured by several survey items, including “I learned new
Table 15 Average responses to satisfaction survey scale items by program year.
2014 2015 2016 Total Change
The Modern Blanket Toss classes have held my interest this week. 3.38 3.28 3.28 4.5
The presenter were well prepared. 3.38 3.22 3.39 4
The environment (meaning the classroom, field trip location, or other place
where instruction took place) was conducive to learning.
3.39 3.28 3.32 3.25
I learned new science and technology concepts this week. 3.03 3.19 3.44 3
The information was challenging to me. 2.57 2.87 3.48 2.75
I enjoyed the classes this week and look forward to more. 3.47 3.22 3.2 2.75
I am considering a career in a STEM field. 3.16 2.98 3.01 2.25
I feel confident communicating with others about science topics. 3.21 2.9 2.92 2
I am becoming more of a leader through this class. 2.98 2.96 3.06 1.75
I am comfortable working as part of a team. 3.49 3.42 3.44 1
Note: Average is the average Likert scale score achieved for each satisfaction area on a Likert scale with four choices:
Strongly Disagree (1), Disagree (2), Agree (3), and Strongly Agree (4). The higher the average the greater the
respondents' satisfaction.
47

2016 FINDINGS
to the UAV technology that was brand new to many of them. They also liked
the hands-on aspect of the course, both the experience flying and the
application of mapping. Students felt that the connections they made with
people were an important benefit of the program.
The exposure to the technology both amazed and surprised students. The
following interview excerpts illustrate this outcome:
I don’t know how to say it, but if I had to in one word I would say
amazing, because it’s an experience like no other. [...] It was so
new to me and diverse. I’ve never worked with anything like this,
so, yeah, that’s why it was so amazing. [2014:10:01]
My favorite part was building the drones, the huge one and the
Lego ones. Then after that, flying them after building them,
because it was exciting how – what we're make progress on.
[2015:15:19]
Students appreciated working together with like-minded peers, and cited this as
an important aspect of the program.
I thought it was a lot of fun. Yeah. And just being around other
people that are genuinely interested in what's going on. [...] It's
nice to be around people who are doing the same thing and
having fun with it. [2015:2:22]
It's kind of been the window to science for me. [2016:7:10]
I didn't even know about UAVs at all before this Modern Blanket
Toss. [2016:8:9]
Most students, regardless of which summer they attended, said the best part of
the summer program was flying the UAVs. There was a range of confidence
levels, and many students were proud of themselves
for learning the skills they needed to operate the
UAVs.
I was nervous, but then I was excited. So
[the UAV instructor] would ask, "Who
wants to go first?" And I'd be right there
saying, "Oh, I want to go first." [2016:9:4]
I got to fly the drones [laughter] and I got
to help other people from other villages.
[2015:13:5]
I've never known there was drones and all
that, and then UAV class helped me like
how to fly it and how to control it
[2014:1:5]
Table 16 Comparison of student content
knowledge and science practices scale scores
from first survey to last survey.
Scale
n
Pre-
All
I thought it was fun because we got to fly in different areas with
a lotta different people we've never met before in the first week,
and now we're laughing and we're getting closer, like friendship
close, and, yeah, I've pretty much liked the past – most – well, all
of the six weeks I've been here because I have so many memories
now. [2014:1:1]
Post-
All
Note: Bold indicates a statistically significant (P < 0.05) shift. The
difference between the scale scores of first and last responses for
individuals were compared by school location; no construct showed
significant difference by location.
n
Pre-
Summer
Post-
Summer
Content Knowledge 57 3.13 5.39 28 3.04 6.34
Science Practices 57 2.7 2.89 28 2.66 3.02
3) Student knowledge about the
Modern Blanket Toss content
Participating students significantly increased
their knowledge about the Modern Blanket
Toss content (P < 0.05) and the scientific
practices used in the program (P < 0.05). The
table below describes the change in student
content knowledge and science practices scale
scores from the first survey they completed
(either during the summer program or during
the school year) to the last survey they
completed (either during the summer
program or during the school year), see Table
16.
48

2016 FINDINGS
Table 17. Average survey responses to content items from first to last
survey.
First
Survey
Note: Average is the average of Likert scales responses from 1 to 10.
Last
Survey
Change
How to use GIS mapping software 1.53 4.37 2.84
The safety rules and regulations that apply to
flying a UAV
How to acquire different types of data using
UAVs
How to use data collected to develop a map/
image that represents reality
The amount of money students can earn from a science,
technology, engineering or math career
4.21 6.95 2.74
2.28 4.96 2.68
2.51 5.14 2.63
4.23 6.75 2.53
How to ask questions using GIS software 2 4.4 2.4
How to build a UAV 3.02 5.34 2.32
Applications of UAV technology 3.21 5.49 2.28
How to fly a UAV 3.61 5.7 2.09
How to acquire data from databases 2.46 4.51 2.05
The range of science, technology, engineering and math
careers that are available to students
4.14 6.18 2.04
Principles of flight (e.g., lift, drag, gravity) 4.11 5.81 1.7
The level of math and science students will need for various
science, technology, engineering, and math careers
4.35 6.02 1.67
How to use a 3-D printer 2.25 3.74 1.49
Science content included principles of flight (e.g., lift, drag, gravity), safety rules
and regulations that apply to flying a UAV, and applications of UAV technology.
Students reported the greatest content gains in 1) how to use GIS mapping
software, 2) how to acquire different types of data using UAVs, 3) how to use
data collected to develop a map/image that represents reality, and 4) the
safety rules and regulations that apply to flying a UAV. These results are
illustrated in Table 17.
Science practices included topics such as
asking scientific questions, comparing
data to other students’ data, and
explaining patterns in scientific data.
Students reported the greatest gains in
understanding of science practices in four
areas: 1) asking a new scientific question
from data, 2) formulating and refining
experimental questions, 3)
communicating the results of data
analysis to a broader audience, and 4)
using a map to analyze scientific data.
These results are illustrated in Table 18.
In interviews, students reported learning
about UAVs and mapping, especially
speaking confidently about data
collection and learning new software. For
example:
I have learned a lot. I know how
to, like I have learned the basics
of mission planning. I’ve learned
a lot with computers that I don’t
know, like, right now I'm making
a map, a 50 photos map.
[2016:3:6]
Table 18. Average survey response
last survey.
Science Practice Area
I am capable of using a map to analyz
scientific data.
I am capable of asking a new scientific
from data that I have collected.
I can communicate the results of my d
analysis to a broader audience.
I can formulate and refine experimen
questions.
I can explain patterns in scientific data.
I can analyze scientific data in many different w
I can select tools to gather data.
I am capable of deciding what data are to be ga
answer a question.
I can compare my data to other students’ data.
I can study data with the help of maps.
I can make and use a model to test a design.
Note: Average is the average of Likert scales res
49

2016 FINDINGS
I learned how to operate the mapping with QGIS and Arc
something. [2016:6:3]
Yeah, because I'd rather work with computers, just to write the
same, so I learned a lot about GIS and Agisoft. [2016:8:6]
s to content items from first to
e
question
ata
al
First
Survey
Last
Survey
Change
2.47 2.82 0.35
2.79 3.05 0.27
2.74 3 0.26
2.66 2.91 0.25
2.42 2.64 0.22
ys. 2.56 2.75 0.19
thered to
2.75 2.93 0.18
2.75 2.91 0.16
So, I was able to fly it multiple times,
and I was able to fly it semiautomatically
and help map out
Chena Hot Springs, the land there,
and I'm learning how to map.
[2016:7:3]
Students also spoke about how their UAV/
mapping knowledge could be applied either in
their communities or in their futures.
I want to try map for like some trails
to other villages, or trails to places
like people go on thing with.
[2016:8:13]
Because we can collect data with
UAVs, the problem, and we can solve
them with some, like, what’s eroding
and what’s not. We could sense heat
sensors or we could find people with
them too, with heat signatures
[2016:3:9]
going to want them and the people will buy them. [2016:7:15]
People say that they herd their sheep with the drones now
[2016:1:7]
I can use them if I graduate. I can use them for like
environmental things to observe around and how I can use it
without damaging it [2016:8:16]
For those students who remained in the program for two or more years, their
content knowledge continued to increase, but seemed to level off after the sixth
time they were surveyed, which correlates with attending two summer
programs and participating during one school year, or participating during two
school years and attending one summer program. However, knowledge of
science practices did not continue to increase with ongoing participation in the
program and stayed relatively flat for individuals over survey dissemination
occurrences (Figure 3).
2.96 3.11 0.14
2.9 3.02 0.12
2.7 2.79 0.09
ponses from 1 to 4.
If I wanted to have a career in UAVs I
could definitely use them. For
example, Bernie Karl at Chena Hot
Springs, he's interested in having UAVs
there, and since UAVs are a growing
industry then more businesses are
Figure 3. Content knowledge and science practice scale scores for
individual students over time
50

2016 FINDINGS
Students also clearly made
greater gains in science
content knowledge during the
summer program than during
the school year (Figure 4).
Students did report significant increases in their communication skills. They
reported the greatest gains in four areas: 1) ability to talk to a group of
strangers; 2) ability to talk with a stranger; 3) ability to talk in a small group of
acquaintances; and 4) ability to talk in a large meeting of strangers. These results
are shown in Table 20.
Figure 4. Average content knowledge scale scores by
survey date. July dates represent summer post-surveys;
April dates represent school year post-surveys; May
and September dates represent pre-surveys.
Table 20 Average survey responses to communication items from
first to last survey.
Communication Areas
First
Survey
Last
Survey
Change
Present a talk to a group of strangers. 2 2.49 0.49
Talk in a large meeting of strangers 1.82 2.23 0.41
4) Student communication and leadership skills
There were positive differences in self-reported leadership and communication
skills; however, only communication skills showed a statistically significant (P <
0.05) shift and then only when all students, both those that participated in the
summer course
Table 19 Comparison of student leadership and
communication scale scores from first survey to
last survey.
Scale
n
Pre-
All
Post-
All
Note: Bold indicates a statistically significant (P < 0.05) shift.
Note: The difference between the scale scores of first and last
responses for individuals were compared by school location; no
construct showed significant difference by location.
n
Pre-
Summer
Post-
Summer
Leadership Skills 57 3.17 3.19 28 3.15 3.26
Communication
Skills
57 2.65 2.82 28 2.63 2.88
and those just in
the school
course, are
included (this is
because the
smaller n of the
summer
students makes
a similar size
shift less likely to
be significant).
The results are
shown in Table
19.
Present a talk to a group of acquaintances 2.44 2.79 0.35
Talk in a small group of acquaintances. 2.63 2.91 0.28
Talk with a stranger. 2.11 2.37 0.26
Talk in a large group of acquaintances 2.39 2.61 0.22
Talk in a large meeting of friends. 2.88 3.07 0.19
Talk in a small group of strangers. 2.35 2.54 0.19
Talk with an acquaintance. 2.79 2.91 0.12
Present a talk to a group of friends 3.09 3.12 0.04
Talk with a friend. 3.75 3.58 -0.18
Talk in a small group of friends 3.54 3.29 -0.25
Note: Average is the average of Likert scales responses from 1 to 4.
Students did not report significant increases in their leadership skills, but did
report large gains in two areas: 1) ability to set and prioritize goals, and 2) ability
to work effectively with a diverse group of people.
51

2016 FINDINGS
When asked about what they learned about communicating science, students
reported they did not like the idea of speaking to a group, but realized that
knowing the content helps.
Table 21 Average survey responses to leadership items from first
to last survey.
Leadership Areas
I am able to set goals and prioritize tasks to
meet goals.
I am able to work effectively with a diverse
team.
I am able to leverage strengths of others to
accomplish a common goal.
I am able to develop, implement, and
communicate new ideas to others.
First
Survey
Note: Average is the average of Likert scales responses from 1 to 4.
Last
Survey
You really have to know what kind of information you’re talking
about if you’re going to be the one teaching it to other people.
[2016:2:1]
Not very well, but like not – I'm good on it. [2016:9:14]
Change
3.14 3.25 0.11
3.16 3.26 0.1
3.09 3.14 0.05
3.16 3.18 0.02
I am open to new and diverse perspectives. 3.11 3.12 0.02
I am able to collaborate and cooperate effectively with
teams.
I am able to listen carefully to what other people are
saying.
3.23 3.19 -0.04
3.31 3.2 -0.11
I feel all prepared like have a 50/50 chance and I feel like I will
know the words if I am going to present [2016:10:13]
Students related communication and leadership skills to teamwork.
We talk to each other, shared knowledge to each other, shared
ideas. I mean gave ideas and one know we were cooperating a lot
when we're about to fly the drones, and we were helping each
other during QGIS and I just talked for those who the students
who were behind. [2016:10:22]
That we communicated with each other. We got to know each
other. We became friends, all of us did. [2016:3:27]
Through the leadership classes I have learned that empathy is
definitely a big thing […] you need to listen to what […] people
are saying and then decide on an end result by compensating with
everybody in a way. And a lot of the time most people don’t
realize that when being a leader you have to listen to the other
people. You can’t just make a decision. And what UAV flying is
about is you have to have a team. You have to have a team to fly
the UAVs, because there is copilot, pilot, payload operator,
spotter. There are so
many different roles
that not just one
person can do it all.
You have to be able to
work as a team, and in
order for that you
need to develop
leadership skills.
[2014:9:16]
52

2016 FINDINGS
Figure 5. Leadership and communication scale scores for
individual students over time.
Leadership and
communication
scale scores
remained relatively
flat for students
over time (Figure 5).
Leadership and
communication
skills scale scores
increased during
the summer
program and
decreased during
the school year
program (Figure 6).
5) Student interest in STEM
The attitude towards science scale score shifted positively, but only significantly
for those that participated in the summer course. There was not a significant
shift in the attitude toward STEM careers scale. The results are shown in Table
22.
Table 22 Comparison of student interest in STEM scale scores
from first survey to last survey.
Scale n Pre-All Post-All n Pre-Summer Post-Summer
Attitudes Toward
Science
Attitudes Toward
STEM Careers
57 2.93 2.99 28 2.91 3.14
57 3.19 3.17 28 3.34 3.35
Note: Bold indicates a statistically significant (P < 0.05) shift.
Note: The difference between the scale scores of first and last responses for individuals
were compared by school location; no construct showed significant difference by location
Overall the attitude toward science scale score shifted positively. Students
reported the greatest changes in two items: 1) “my scientific investigations are
better when I begin by asking a question,” and 2) “using a map to study scientific
data help me to learn.” Table 23 details these results.
Figure 6. Average leadership and communication skills scale scores by survey date. July
dates represent summer post-surveys; April dates represent school year post-surveys; May
and September dates represent pre-surveys.
While overall students’ scale scores for attitude toward STEM careers did not
increase, several individual items did increase, including 1) a career in STEM
would be financially rewarding. In addition, students were less likely to agree that
a career in STEM would be dull and boring and that a degree in STEM does not
interest them. These are both positive outcomes.
We may not have seen a large shift in the student interest in STEM scale scores
because many of the students were already interested in STEM before they
began participating in the Modern Blanket Toss program. Many students in the
53

2016 FINDINGS
final year of the program reported that they were already interested in STEM
and continue to be.
Table 23 Average survey responses to attitude toward
science items from first to last survey.
Attitude Toward Science
My scientific investigations are better when I
begin by asking a question.
Using a map to study scientific data helps me to
learn.
Looking at scientific data in different ways helps
me learn.
First
Survey
Last
Survey
Note: Average is the average of Likert scales responses from 1 to 4.
Change
2.8 2.93 0.13
2.75 2.88 0.13
3.14 3.2 0.06
Science is an interesting subject to study. 3.2 3.17 -0.03
I mean I'm already pretty interested in science. Maybe I wasn’t
really interested in technology and I’ve learned a lot about
technology actually, like not just the drones, but the
programming and the software we use, like map things out. It's
kind of a little bit tedious, but it’s – yeah, it’s interesting to
know, like how that kind of stuff works. [2016:1:12]
It's important, because you learn a lot of stuff. You get to like –
how do you say it? You get to be a part of like a group that like
wants to learn and like do stuff, and be able to like know what
they want to be when like later on in the year, and it's really
helpful [2016:9:7]
There was one exception; this student cites shifting from not self-identifying as
interested in STEM to “opening my mind more to it” because of the
opportunities afforded by involvement in Modern Blanket Toss:
Table 24 Average survey responses to attitude toward STEM
careers items from first to last survey.
Attitudes toward STEM careers
A career in science, technology, engineering,
or math would be financially rewarding.
A career in science, technology, engineering,
or math would be dull and boring.
Pursuing a degree in a science, technology,
engineering, or math field does not interest
me.
Note: Average is the average of Likert scales responses from 1 to 4.
I never really thought of
myself as a kind of techy
person, and now I have the
opportunity to – I don't
know, open my mind more
to it, and I am. [2016:7:4]
Attitude toward science and
STEM careers scale scores
increased more during the
summer program than during the
school year (Figure 7).
First
Survey
Last
Survey
Change
3.13 3.32 0.19
3.19 3.12 -0.07
2.96 2.89 -0.07
Scientists/engineers are really cool people. 3.25 3.19 -0.06
Scientists/engineers are boring people 3.34 3.316 -0.02
I am interested in the way that science, technology,
engineering, or math can be used to help people.
3.38 3.24 -0.14
Figure 7. Average attitude toward science and
toward STEM careers scale scores by survey
date. July dates represent summer post-surveys;
April dates represent school year post-surveys;
May and September dates represent pre-surveys
54

2016 FINDINGS
6) Student academic data
Grade Point Average
GPAs of Modern Blanket Toss students were higher in than the Upward Bound
students, but not significantly (P = 0.281). However, the shift from pre-program
to post-program was not significant for either group (Table 25).
Table 25 Average pre and post-program student GPA for Modern
Blanket Toss participants and Upward Bound control group
students program in the summer of 2014.
GPA Pre-
Program
GPA Post-
Program
Modern Blanket Toss (n=16) 3.07 3.09 0.764
Upward Bound Control (n=15) 2.73 2.84 0.151
Sig.
Number of Math Courses and grades
The number of math courses completed by Modern
Blanket Toss students and Upward Bound control group
students was not significantly different either at preprogram
or post-program (Table 26).
Table 26 Average number of math courses completed pre- and post-program for
Modern Blanket Toss participants and Upward Bound control group students
Count of Math
Courses
Pre-Program
Average Courses
Per Student
Count of Math
Courses
Post-Program
Average Courses
Per Student
Modern Blanket Toss (n=17) 30 1.76 25 1.47
Upward Bound Control (n=18) 32 1.78 28 1.56
The average math pre-program grade for Modern Blanket Toss participants was
significantly higher than that of the control group students (P = 0.021). However,
the shift from pre-program to post-program was not significant for either group
(Table 27).
Table 27 Average pre and post-program average math grades for
Modern Blanket Toss participants and Upward Bound control
group students
Math Pre-
Program
Math Post-
Program
Modern Blanket Toss (n=17) 2.87 2.83 0.765
Upward Bound Control (n=18) 2.07 2.56 0.238
Sig.
55

2016 FINDINGS
Number of Science Courses and grades
Table 28 Average number of science courses completed pre and postprogram
for Modern Blanket Toss participants and Upward Bound control
group students
The number of science courses completed by Modern Blanket Toss
students and Upward Bound control group students was not
significantly different either at pre-program or post-program.
Pre-Program
Post-Program
Count of Science
Courses
Average Courses
Per Student
Count of Science
Courses
Average Courses
Per Student
Modern Blanket Toss (n=11) 30 2.72 20 1.82
Upward Bound Control (n=13) 28 2.15 26 2
The average science pre-program grade for Modern Blanket Toss
participants was significantly higher than that of the control group
students (P = 0.044). However, the shift from pre-program to postprogram
was not significant for either group.
Table 29 Average pre and post-program average science grades for
Modern Blanket Toss participants and Upward Bound control group
students
Science Pre-Program Science Post-Program Sig.
Modern Blanket Toss (n=11) 3.18 3.18 1
Upward Bound Control (n=13) 2.17 2.22 0.773
56

2016 FINDINGS
Courses offered by High Schools
Availability of science classes could influence how students enrolled. Course catalogs for the site high schools show that all schools offer a range of math and science
courses. When they do not have the resources locally, they provide distance education opportunities to students. However, it should be noted that courses in these
school districts are not offered regularly, and in some cases, low initial course enrollment may result in a course being cancelled that semester.
Table 30 Math courses offered in Modern Blanket Toss targeted
school districts
Algebra I 1 1
Geometry 1 1
Advanced Algebra 1
Algebra II 1
Pre-Calculus/
Trigonometry
Nikiski Seward Chefornak Shishmaref Bethel
1 1
Calculus 1 1
Statistics 1
Data Analysis 1
Note: Courses in these school districts are not offered regularly, and in some cases, low
initial course enrollment may result in a course being cancelled that semester.
1
1 All available,
through
various
1
delivery
methods -
Information
1
direct
not available
classroom,
1
video
teleconferen
ce, online
Distance
courses
courses
available
Table 30 Math courses offered in Modern Blanket Toss targeted
school districts
Earth Science
Biology 1 1
Physical Science 1 1
Chemistry 1 1
Nikiski Seward Chefornak Shishmaref Bethel
Physics 1 1
AP Science 1
Science Electives 1 1
Information
1
not available 1
Note: Courses in these school districts are not offered regularly, and in some cases, low
initial course enrollment may result in a course being cancelled that semester.
1
1
Distance
courses
available
All available,
through
various
delivery
methods -
direct
classroom,
video
teleconferen
ce, online
courses
57

2016 FINDINGS
Section 4. Discussion
Evaluation results indicate that the Modern Blanket Toss program shows significant promise as an effective way to increase the likelihood that participating students
would go on to pursue college degrees in STEM areas and ultimately pursue STEM careers.
The ability of the Upward Bound local coordinators to facilitate the Modern Blanket Toss program locally was a key indicator of the program’s effectiveness. While the
evaluation results are limited by exclusive reliance on the coordinators’ self-reports of their knowledge and capabilities to implement the program at their respective
sites, the results nonetheless indicate that site coordinators had, or were able to gain, the skills they needed to implement science practices and project-based learning
with their students. On all four scales measuring coordinator knowledge, the coordinators reported increased knowledge from the beginning of the program to the end
of the program. In particular, the UAV knowledge scale -- which included items such as how to build a UAV, applications of UAV technology, the safety rules and
regulations that apply to flying a UAV, and how to acquire different types of data using UAVs -- increased significantly across all individual items assessed. The GIS
knowledge scale, which included items such as how to use GIS mapping software, also increased significantly. However, on several individual items assessed, coordinators
did not report knowledge gains even though they initially rated their knowledge on the topic as low. Items for which coordinators responded in this way included selfreported
knowledge of how to use GIS mapping software and how to acquire data from databases.
While the coordinators either already had, or obtained by the end of the project, the knowledge and/or skills required to implement Modern Blanket Toss content with
their students, their initial lack of some key skills may explain why students reported lower content knowledge gains during the school year than they did during the
summer program. Further, while all of the communities implemented a community project in the final year of the project, here too the coordinators’ initial lack of some
key skills may explain why community projects were not implemented in each of the three program years. In future iterations of the Modern Blanket Toss program, we
strongly recommend targeted and specific coordinator training focused on the technology being used (e.g., how to build a UAV), combined with training focused on
skills needed to implement community projects, such as how to ask questions using GIS software.
Even though the Modern Blanket Toss summer program only reached a third of the Modern Blanket Toss students, it was likely the most essential part of the program as
implemented during the three year project period. By student account, the summer component was highly successful. Each summer student reported weekly satisfaction
with the program. They reported that the classes held their interest, the presenters were well prepared, the environment was conducive to learning, and that they
enjoyed the classes and look forward to more. They liked working with their peers and they liked learning new skills needed to operate the UAVs. More importantly
however, students reported greater knowledge gains, greater leadership and communication skill gains, and greater interest in science and STEM careers after the
summer program than they did after the school year program.
However, the greater reported gains during the summer program may simply be a reflection of the students who attended the summer program component. The
average GPA of students who attended the summer program was significantly higher than the control group of Upward Bound students, who did not attend the
summer program and whose sites were not involved in the Modern Blanket Toss program. The students who attended the summer program also averaged a greater
number of science courses per student before they entered the program than the control group. Unfortunately, the evaluation did not collect any additional data that
might further clarify the differences between the groups.
59

2016 FINDINGS
The Modern Blanket Toss student outcomes were organized around themes
common to informal science education: interest or engagement; self-efficacy
(sense of ability); awareness, knowledge or understanding; and behavior changes
(National Science Foundation, 2008; National Research Council, 2009; Phillips et
al., 2014). In addition, the outcomes were organized around predictors of
students’ STEM success, including high school math and science courses taken
(Hinojosa et al., 2016) and the grades in those courses. The program also
included an outcome for student communication and leadership skills, the
development of which was a primary part of the planned program.
Many of the program’s expected student outcomes were met, indicating that
students who participated in the Modern Blanket Toss program are now more
likely to pursue college degrees in STEM areas and ultimately pursue STEM
careers. Students reported significant increases in their knowledge of Modern
Blanket Toss content areas, such as how to use GIS mapping software, and
longer duration of participation in the program led to greater content gains.
Students also reported significant gains in their understanding of science
practices, such as asking a scientific question from data. Students reported
significant increases in their communication skills, such as their ability to talk to a
group of strangers.
Those target student outcomes that did not achieve a statistically significant
change, still nonetheless increased. For example, many of the students already
had a positive attitude toward science when they started, so the overall attitude
toward science scale score, while shifting positively, did not shift to a statistically
significant degree. Similarly, students reported positive increases in leadership
skills, but these increases were not statistically significant. Again, students entered
the program with relatively high leadership scale scores, so statistically significant
results may have been difficult to detect with the small sample size. In future
iterations of the Modern Blanket Toss program we would recommend collecting
additional outcome data for control group students, such as attitude toward
science. This additional data would allow the program to better assess its
impacts when pre-data shows positive results and leaves little room for growth.
Finally, the student grades and the number of math and science courses
completed in high school did not increase during the evaluation period. This is
not an unexpected result for several reasons. First the sample size was
extremely small and it is difficult to find significant relationships from the data as
statistical tests normally require a larger sample size to ensure a representative
distribution of the population. Second, we only measured student grades and
courses completed over two school years. A longer time period would provide
better data.
However, we did find that students who participated
in Modern Blanket Toss had significantly higher GPAs
and science and math grades than control group
students. This is telling information. Students who
self-selected to participate in the Modern Blanket Toss
summer program already were more likely to pursue
college degrees in STEM areas and ultimately pursue
STEM careers simply based on their higher math and
science grades. This is a factor that should be
considered in any future evaluation.
60

2016 FINDINGS
Citations
Dorsen, Jennifer, Behtany Carlson, and Leslie Goodyear. 2006. Connecting
informal STEM experiences to career choices: Identifying the Pathway. A
literature review produced by the ITEST Learning Resource Center. Retrieved
from: http://stelar.edc.org/sites/stelar.edc.org/files/itestliteraturereview06.pdf.
Friedman, A. (Ed.). March 12, 2008. Framework for Evaluating Impacts of Informal
Science Education Projects [On-line]. (Available at: http://insci.org/resources/
Eval_Framework.pdf).
George, D. and Mallery, P. 2003. SPSS for Windows step by step: A simple guide
and reference. 11.0 update. 4th edition. Boston: Allyn and Bacon.
National Research Council. 2009. Learning Science in Informal Environments:
People, Places, and Pursuits. Washington, DC: The National Academies Press. doi:
10.17226/12190.
Phillips, T. B., Ferguson, M., Minarchek, M., Porticella, N., and Bonney, R. 2014.
User’s Guide for Evaluating Learning Outcomes in Citizen Science. Ithaca, NY:
Cornell Lab of Ornithology.
Trochim, William. 2006. Research Methods Knowledge Base. Web Center for
Social Research Methods. Retrieved from: http://
www.socialresearchmethods.net/.
University of Southern California. 2016. Research Guides. Organizing your social
sciences research paper: Limitations of the study. Retrieved from: http://
libguides.usc.edu/c.php?g=235034&p=1561758
61

CHAPTER 2
Observations
This chapter covers the challenges and lessons learned during the Modern Blanket Toss project. Following that, there is an
analysis of the program from the perspective of those involved.

SECTION 1
Challenges and Lessons Learned
Curriculum
In the first year, we learned about the importance of students building their UAVs because when students crashed a purchased UAV, there
was no sense of being able to fix it. A license model worked well for students to earn the right to fly, but it did not address what they should
do with the license.
63

CHALLENGES AND LESSONS LEARNED
Teacher Support
During the school year, teachers need to be able to rely on an
online curriculum for students who might fall behind to “get up to
speed” with the rest of the group.
These teachers also need to sign off on student progress on the
hands-on portion of the flight school curricula in the form of a
rubric.
Teachers need to be concerned with outreach into the community
to find individuals who are willing to present to the students and
possibly pitch ideas for projects.
Teachers need to set up ceremonies each quarter where students
are celebrated for their accomplishments within their local
communities and as part of a network of other schools doing similar
work.
We realized that teachers need instruction on how to do this kind of
community organization and outreach.
It is also crucial that teachers are comfortable with the growth
mindset and the engineering design process where every
experience is a learning opportunity.
Lastly, if teachers are to monitor their students, they need to be
familiar using the chosen Learning Management System (LMS).
When this comfortability exists, technologies have the power to be
can be the vehicle whereby students make a meaningful difference
in their community. Many STEM professionals cite that inspiration
for their career choice came from the ability for them to contribute
and give back to their communities.
Summer Sessions
It is easier to control the student experience during the summer
sessions. Therefore those times should be spent with students
involved in community projects that follow the successful model
that we observed in summer of 2016.
64

CHALLENGES AND LESSONS LEARNED
Community Projects
The MBT project revealed many challenges in the way that we help
students utilize technology to address issues within their
communities.
The coordinators who work with the students need training in the
areas of the growth mindset, project management, and how to be
part of a support network.
Most communities in MBT were not successful in achieving their
project goals in part due to an absence of an engaged community
member.
During the 2016 summer session, we worked with stakeholders in
the Fairbanks community to find issues and or projects that are
achievable by the students using their UAV and GIS skills. We
found positive results working with Bernie Karl, owner of the Chena
Hot Springs Resort. Once the project was first vetted by MBT staff
to assure that the task was reasonable, Mr. Karl had the opportunity
to meet with the students. He expressed that he was in need of
data to support a significant business decision. From that first
opening conversation, the students connected with Mr. Karl and
were excited to make something happen.
With this motivation, it was like dealing with an entirely different
group of learners. It was easy to guide them into asking the
questions and setting up the resources. At that point, it was easy
for us to ensure their success.
We noticed this in the communities as well. Those students that had
a person from their community who revealed a need for the data
were able to move much further on their projects than those that
were internally managing themselves.
65

SECTION 2
Analysis of Modern Blanket Toss
How We Can Use MBT to Assist in Curriculum Development
Modern Blanket Toss has considerably informed the curriculum design for future grant initiatives. There are four
components that make up a robust curriculum to both educate and empower youth on their way to becoming STEM
professionals.
66

ANALYSIS OF MODERN BLANKET TOSS
First, we have a
thorough online
student curriculum
focused on the
proper use of the
technology. The
knowledge transfer
happens through
online assessments
and associated realworld,
hands-on
projects.
Second, we provide an appropriately designed
technology kit for the lessons.
Third, to find compelling student projects, we provide
a roadmap to teachers on how to engage STEM
professionals as well as the broader community.
Fourth, to prepare and support teachers in their role
as a guide to students, we provide a comprehensive
teacher training program that includes online and
face-to-face components.
Each of these elements is important and learned as
part of MBT.
What Worked Really Well
EQUIPMENT
The K'NEX copters served their purpose for their
designed mission, showing students how simple UAVs
function, and then applying that knowledge to more
complex UAVs. The Phantoms by DJI accomplished
the purpose of reliably collecting aerial footage and
ease of flight for the students.
The Hexacopters worked well but could have been
more efficient if the frame and parts could have
purchased, and the UAV put together in a clearly
established fashion. As it was, we learned about interference by putting the
compass too close to the battery. A valid lesson, but not when it leads to
a costly crash.
TRAINING
The most effective method for motivating students to work collaboratively
using the GIS technology and UAVs was in orchestrating a service learning
project for them to participate in that was attainable. If the representative
organization was engaged in connecting with the students about the
underlying issue, students were very motivated to employ the technology
and present meaningful data. When the students received recognition for
their work, they usually beamed with pride and spoke highly of the
experience.
67

ANALYSIS OF MODERN BLANKET TOSS
What We need to Address in Future Teacher Trainings
• Many components are essential to a successful teacher training:
• A segment on how to create a Growth Mindset environment when everyone is doing something different. This unit can involve
brain research, testimonials from successful STEM professionals, ultimately setting the tone for how STEM learning occurs.
• Course and program administration in online and real-world synchronous environments. Instruction includes how to monitor
student progress, utilize rubrics, and create meaningful ceremonies to celebrate achievements.
• Explanation and demonstration of items provided in the technology kits. There is an assumption that students may surpass the
teacher in knowledge and skill level of the various technology tools. However, the teacher should focus on and be concerned
about how to assess student proficiency, accessing outside expert help when needed.
• Culminate in a unit on how to present and search for community projects by having organized STEM community resources.
68