zmWmQs

Analyzing completion rates in the First French xMOOC
Matthieu Cisel
We sought to assess whether the participation in the
peer evaluation process was associated with achievement
within this certificate. Out of the 615 participants,
555 (90.2 %) took part in the peer evaluation process at
least once. Once a learner had started evaluating an assignment,
he evaluated the four assignments that he was
supposed to evaluate in 93.7 % of the cases. Taking part in
the peer evaluation process was strongly associated with
achievement within the advanced certificate (χ 2 = 146, df
= 2, p-value < 0.001) (Figure 2.B). Those who evaluated
assignments were much more likely to get the advanced
certificate. Indeed, 78.0 % of participants who took part
at least once in the evaluation process, 8.0 % of those
who did not, obtained the advanced certificate. Among
the 555 assessors who took part in the evaluation process,
only 45.6 % gave some feedback on the assignment.
Giving feedback was slightly associated with achievement
(χ 2 = 22, df = 2, p-value < 0.001), but the association was
weaker. The completion rate was higher among those who
gave feedback at least once (87.0 %) than among those
who did not (70.5 %).
Discussion
The course completion rate was higher than those usually
observed in MOOCs (Breslow et al., 2013, Jordan 2013,
Kisilcec et al., 2013). On the one hand, this may be due to
the course design and on the other hand, to the type of
participants who followed the course. MOOCs were little
known in France when Introduction to Project Management
was launched (Cisel & Bruillard, 2013). The participants
of the first session of this course may therefore be considered
as early-adopters, probably more motivated than
most MOOC participants. The MOOC design doubtlessly
influenced its completion rate. The workload necessary
to obtain a certificate was significantly lower for the basic
certificate of completion than for the advanced one.
Moreover, the course duration was shorter than most
MOOCs since it lasted only five weeks. It was therefore
easier to complete the course for learners who had little
spare time. It is hard to compare MOOC completion rates
among courses given the diversity of workloads that they
involve. The highest completion rate reported so far was
Functional Programming Principles in Scala, where around
20% learners completed the course (Miller & Odersky,
2013). Taken alone, the completion rate of the advanced
certificate of Introduction to Project Management is more
in line with the general results reported (Breslow et al.,
2013, Jordan 2013, Kisilcec et al., 2013)
This course enrolled more male than female learners.
This result is in line with results reported from other
courses (Huhn, 2013, Kizilcec et al., 2013), sex ratio depending
highly upon the topic of the course. Before being
turned into a MOOC, this course was an online module
taught in the context of lifelong learning courses. As a
consequence, it is not surprising that this course attracted
more full-time workers than students. Around a quarter of
the participants came from French-speaking, low HDI African
countries. In comparison, the proportion of learners
from low HDI countries in the three Computer Science
courses described by Kizilcec et al. (2013) was below 3 %.
This proportion is doubtlessly dependant on the teaching
language; it would be interesting to compare course demographics
from MOOCs in English, Spanish and French,
since these three languages are used worldwide as official
languages in a variety of countries. It is obvious that
the participants did not consist of a random sample of the
general population, especially for low HDI countries, given
the required technological infrastructure, literacy level
and language comprehension.
Completion rates were associated mostly with personal
aims, which depended on employment status, geographical
origin, experience in project management and time
constraints. Further research is needed to understand
why participants from developing countries and job seekers
targeted higher levels of completion than others. Time
constraints must also be taken into account in the interpretation
of personal aims, since the situation is fundamentally
different for a person looking for a job and for a
full-time employee. A deeper analysis of the diverse motivations
and constraints of the different groups of learners
is needed to better understand both their objectives and
achievements.
Understanding drop-out rates and, generally speaking,
the gap between learners’ objectives and achievements is
a major issue for MOOC pedagogy. The achievement gap
observed in low HDI countries is probably caused by various
cultural and technical issues such as low broadband,
for instance, or language issues. As a reminder, French
is not the native language of many African learners and
could be an obstacle. Kizilcec reported that in the Probabilistic
Graphical Models MOOC (Graduate level course),
completion rates were four times smaller in low HDI
countries compared to very high HDI countries, reaching
only 2 %. Further research is needed to understand the
causes of the gap between objectives and achievements
in those countries.
Identification of indicators associated with course completion
is a major issue in order to tailor and adapt eventual
interventions (Arnold & Pistilli, 2012). For this reason,
we searched for simple indicators, such as participation
in forums and in the peer evaluation process. Regarding
the use of forums, learners were more active than what
was reported in some other MOOCs (Randy, 2013). For
example in the Circuits and Electronics (xMOOC) course
held on the edX platform, only 3 % of all learners interacted
with others. In many courses, this proportion is lower
than 5 %. This is probably due to the humble size of Introduction
to Project Management compared to courses like
6.002x, which made it easier for the MOOC staff to interact
with learners on the different aspects of the course.
Getting quick feedback from the MOOC team may have
encouraged participants to post more frequently on the
Research Track | 29