Forging new pathways of research and innovation in open and distance learning

Learning Analytics in Distance Education: A Systematic Literature Review
Selcan Kilis, Yasemin Gülbahar
Tempelaar, Rienties, &
Giesbers (2015)
Tobarra, Robles-
Gómez, Ros,
Hernández, &
Caminero (2014)
Van Leeuwen,
Janssen, Erkens, &
Brekelmans (2015)
Volk, Kellner, &
Wohlhart (2015)
Xie, Yu, & Bradshaw
(2014)
Yasmin (2013)
Undergraduate
students
Graduate
students
Undergraduate
students
Secondary
education
Undergraduate
students
Graduate
students
Case study Student log data,
surveys
Statistics tracking, inferential statistics
(correlation), prediction modelling
Quantitative Student log data Semantic analysis,
network analysis
Experimental
study
Student log data,
diagnosis,
interventions
Descriptive statistics, inferential statistics
Quantitative Student log data Inferential statistics (logistic regression)
Mixed method
Student log data,
discussion
Social network analysis, inferential statistics
(correlation, multiple regression, pairedsamples
t-test), qualitative analysis of
discussion transcripts
Quantitative Student log data Data mining approach (classification tree),
descriptive statistics
As can be seen in Table 2, the studies selected in relation with learning analytics in distance
education were published between 2012 and 2016, inclusive. The majority of studies (56%,
n = 14) were published in 2015, followed by 2016 (28%, n = 7). Two studies were published in
2014, whereas there was only one in both 2012 and 2013. When reviewing the literature, it is
apparent that studies commenced in this area in 2011, but they are mostly review or
theoretical studies. Empirical studies started later and gained popularity especially within
2015. The majority of the research (80%, n = 20) was performed at the university-level of
education, i.e., with undergraduate students. Four research studies were conducted with
graduate students and one was undertaken with secondary education students. Distribution
analysis of research studies based on applied methodology shows that 52% (n = 13) were
purely quantitative, 28% (n = 7) were case studies, 8% (n = 2) were mixed-method, 4% (n = 1)
was purely qualitative, 4% (n = 1) had an experimental design, and 4% (n = 1) employed
design-based research. According to the data sources used, most (88%, n = 22) used student
log data stored in the background of e-learning module, while three (12%) did not use a
student log. Finally, in regard to data analysis, more than half (60%, n = 15) applied learning
analytics methods such as sentiment analysis, social network analysis, neural networks, etc., as
well as data-mining techniques like classification tree, and random forest technique. Of those
research that applied learning analytics methods, 32% (n = 8) applied only statistical analysis
methods, whereas 8% (n = 2) used qualitative analysis.
Overall, of the 25 empirical research studies that were reviewed, based on content analysis, it
was seen that most were published in last two years (i.e. as from 2015). Many focused on
university-level education, especially at the undergraduate level. Research design variants
included mixed-method, case study, experimental design, and design-based research, etc. It
can be inferred that there is a variation in the applied methodology. The most frequent data
source used was, as expected, the log data stored in the background of user applications.
Researchers mostly analysed data using learning analytics techniques such as network
analysis. The emerging point in the data analysis is the use of statistical methods, specifically
structural equation method and multiple regression techniques. It is clear that there is a
variation in criteria including applied methodology, data sources and data analysis techniques.
314 Reaching from the roots – 9 th EDEN Research Workshop Proceedings, 2016, Oldenburg
ISBN 978-615-5511-12-7