zmWmQs

Dropout Prediction in MOOCs using Learner Activity Features
Sherif Halawa, Daniel Greene, John Mitchell
Absent Mode (M2) Predictor
For most students, absences of several days at a time are
not uncommon. As the absence lengthens, however, the
probability that the student may not continue to persist
in the course increases. The job of this predictor is to redflag
a student once he or she has been absent for a certain
number of days, called the “absence threshold”.
To determine the optimum threshold, we studied the
variation of accuracy with threshold. The threshold was
varied from 0 to 3 “course units”, where a course unit is defined
as the time period between the release of two units
of course content. For most courses, a course unit is 1
week long. The variation of specificity and recall with the
threshold is presented in Figure 3.
Results
Figure 3. Variation of specificity and recall with the absence
threshold At very low thresholds, S is very low and R is very high
because almost every student has an absence at least as long as
the threshold. As the threshold is increased, S improves and R
deteriorates. We identified the point at 2 course units (14 days
for a typical course) as a convenient threshold, where R and S are
both above 0.75, and have selected this value to be the threshold
of our M2 predictor.
Specificity and Recall
First, we evaluate our predictor’s specificity and recall
observed over 10 test courses different from the 6 training-set
courses. Table 3 shows the best, median, and worst
observed recall and specificity figures.
Table 3. Best, median, and worst specificity and recall for various
predictor components
Individual feature predictors
assnperformance
video-skip assn-skip lag
M1
predictor
M2
predictor
Integrated
predictor
Specificity
Best 1.0 0.86 0.96 0.96 0.85 0.93 0.68
Median 1.0 0.82 0.92 0.84 0.72 0.80 0.58
Worst 0.96 0.40 0.73 0.47 0.36 0.70 0.29
Recall
Best 0.008 0.58 0.38 0.43 0.67 0.98 0.99
Median 0.006 0.30 0.25 0.17 0.48 0.93 0.93
Worst 0.00 0.23 0.10 0.14 0.41 0.77 0.91
In order to develop an understanding of what the
strengths and weaknesses of our predictor are, we need
to provide some interpretation of the numbers in Table 3.
The ‘assn-performance’ (assessment performance)
Feature
This feature generally has high precision and specificity.
Over 95% of students it flags (students with average assessment
scores below 0.5) eventually dropout. However,
its recall was observed to be generally very low compared
to the other features. For the majority of MOOC quizzes,
mean scores are in the range of 70% - 85%. Even though
some students occasionally score below 50% on certain
quizzes, there are very few students whose average quiz
scores are below 50%. This could be attributed to the deliberate
easiness for which MOOC assessments are designed,
or due to MOOCs’ self-selective nature (students
who believe that the course will be too difficult refrain
from enrolling or refrain from attempting assessments).
Research Track | 63