Volume Two - Academic Conferences

Proceedings
of the
10 th European Conference
on e-Learning
Brighton Business School
University of Brighton
UK
10-11 November 2011
Volume Two
Edited by
Sue Greener and Asher Rospigliosi
University of Brighton
UK

Copyright The Authors, 2011. All Rights Reserved.
No reproduction, copy or transmission may be made without written permission from the individual authors.
Papers have been double-blind peer reviewed before final submission to the conference. Initially, paper
abstracts were read and selected by the conference panel for submission as possible papers for the
conference.
Many thanks to the reviewers who helped ensure the quality of the full papers.
These Conference Proceeding have been submitted to the Thomson ISI for indexing.
Further copies of this book can be purchased from
http://www.academic-bookshop.com
ISBN: 978-1-908272-22-5 Book
Published by Academic Publishing Limited
Reading
UK
44-118-972-4148
www.academic-publishing.org

Contents
Paper Title Author(s) Page
No.
Preface xii
Biographies of Conference Chairs, Programme
Chair, Keynote Speaker and Mini-track Chairs
Biographies of Contributing authors xv
Revisiting the Personal Transferable Skills
Debate - an eLearning Pedagogical Perspective
Survey of Teachers’ use of Computer/Internet in
Secondary Schools in South West Nigeria
Issues and Challenges in Implementing
eLearning Projects in Higher Education: The
Case of Jordan
The use of Open Educational Resources in Intra-
Organisational eLearning and Continuing
Education
Constructing a Survey Instrument for Assessing
Characteristics of Effective Online Teachers
i
Samuel Adu Gyamfi, Lene Tolstrup
Sorenson and Thomas Ryberg
Babatunde Alabi Alege and Stephen
Olufemi Afolabi
Hussein Al-Yaseen, Saheer Al-Jaghoub
and Nidal Al-Salhi
Antonios Andreatos
xiii
Jonathan Barkand 34
When Agents Make Suggestions About Readings Orlando Belo 41
Some Reflections on the Evaluation of Virtual
Learning Environments
Designing A New Curriculum: Finding The Right
Blend
Critical Success Factors for the Adoption of
eLearning in the Kingdom of Saudi Arabia
Educational Institutions
Challenges in Developing e-Submission Policy
and Practice
Enhancement of e-Testing Possibilities With the
Elements of Interactivity Reflecting the Students’
Attitude to Electronic Testing
e-Assessment Using Digital Pens – a Pilot Study
to Improve Feedback and Assessment Processes
Digital Educational Resources Repositories in
Lower and Middle Education in Portugal: Quality
Criteria in the International Context
Nabil Ben Abdallah and Françoise Poyet 48
Andrea Benn 56
Latefa Bin Fryan and Lampros Stergioulas 63
Alice Bird 73
Martin Cápay, Martin Magdin and
Miroslava Mesárošová
Tim Cappelli 91
Cornélia Castro, Sérgio André Ferreira and
António Andrade
eLearning: Roles in Distance Tertiary Education Ivana Cechova, Dana Zerzanova and Jana
Berankova
Independent Learning in Need or in Crisis?
Independent Learning Under the new Four-Year
Undergraduate Curriculum in Hong Kong
The Development and Application of a web
Based Metacognitive Mapping Tool
An Exploratory Comparative Study of Distance-
Learning Programmes
The Optimal Teaching Style Based on Variability
of Study Materials
Yin Ha Vivian Chan, Delian Dawn Gaskell,
Mei Ah Tan and Lip Yan Felix Chao
1
8
16
23
82
100
109
117
Serdar Çiftci and Mehmet Akif Ocak 124
Marija Cubric, Karen Clark and Mariana
Lilley
Blanka Czeczotková, Kateřina
Kostolányová and Jana Šarmanová
134
145

Paper Title Author(s) Page
No.
Changing Academics, Changing Curriculum: How
Technology Enhanced Curriculum Design can
Deliver Strategic Change
Web Conferencing for us, by us and About us –
the Leeds Met Elluminate User Group
Tools for Evaluating Students’ Work in an
Interactive (Open) Virtual Space: Case Study of
an eLearning Course in an International Network
of Universities
Putting Things in Context - Designing Social
Media for Education
ii
Christine Davies 152
Mark de Groot, Gill Harrison and Rob Shaw 156
Jana Dlouhá, Martin Zahradník, Jiří Dlouhý
and Andrew Barton
Jon Dron, Terry Anderson and George
Siemens
Experimental Assessment of Virtual Students Michaela Drozdová, Ondřej Takács and
Jana Šarmanová
Priming for Modules: A Case Study Evaluation of
‘Pre-Workshop’ Online Resources for an
Executive MBA Course
Computer-Mediated Reading and its Impact on
Learners’ Reading Comprehension Skills
Do you see What I see? - Understanding the
Challenges of Colour-Blindness in Online
Learning
Researching in the Open: How a Networked
Learning Instance can Challenge Ethical
Decision-Making
Making Constraints and Decisions Explicit to
Support Project-Based Collaborative Learning
A Strategy for the Inductive Generation of
Learning Objects in Low-Tech Contexts
Cognitive Communication 2.0 in the Classroom –
Resonance of an Experience in Higher Education
To What Extent Does a Digital Audio Feedback
Strategy Support Large Cohorts?
Messages of Support: Using Mobile Technologies
to Support the Transition of Students on
Articulation Routes From Higher National Level to
Degree
Blended Learning at the Alpen-Adria-Universität
Klagenfurt
Evaluating the use of Social Networking Sites as
a Tool for Knowledge Sharing for Developing
Higher Education in Developing Countries: An
Exploratory Study of Egypt and Iraq
The Relationship Between Mindful Learning
Processes and Course Outcomes in Web-Based
Learning
Researching and Sharing – Business School
Students Creating a Wiki Glossary
Glenn Duckworth
Francisco Perlas Dumanig, Maya Khemlani
David and Rodney Jubilado
Colin Egan, Amanda Jefferies, Edmund
Dipple and David Smith
166
177
186
195
203
210
Antonella Esposito 218
Gert Faustmann 225
Ana Mª Fernández-Pampillón, Elena
Domínguez, José Mª Lahoz, Dolores
Romero, Isabel de Armas, Susana Palmaz
and Jorge Arús
Sérgio André Ferreira, Cornélia Castro and
António Andrade
235
246
Rachel Fitzgerald 256
Julia Fotheringham and Emily Alder
266
Gabriele Frankl and Sofie Bitter 274
Elaine Garcia, Ibrahim Elbeltagi, Sawasn
Al-Husseini and Ahmed Abdelkader
284
Danny Glick and Roni Aviram 295
Andrea Gorra and Ollie Jones 303

Paper Title Author(s) Page
No.
A Qualitative Evaluation of Academic Staff’s
Perceptions of Second Life as a Teaching Tool
Introducing and Using Electronic Voting Systems
in a Large Scale Project With Undergraduate
Students: Reflecting on the Challenges and
Successes
A Methodology for Incorporating Usability and
Accessibility Evaluations in Higher Education
The Virtual Learning Environment - Directions for
Development in Secondary Education
Mutlimodal Teaching Through ICT Education: An
e-Twinning Program as a Case Study of
Intercultural Exchange
Effectiveness and Learners’ Evaluation of
Combining Audio and Written Online Formative
Feedback for Language Learning
iii
Rose Heaney and Megan Anne Arroll 311
Amanda Jefferies 319
Anne Jelfs and Chetz Colwell 326
John Jessel 332
Paraskevi Kanari and Georgios Potamias
340
Rosario Kane-Iturrioz 345
Model of eLearning Project Evaluation Jana Kapounova, Jana Sarmanova and
Marketa Dvorackova
Bridging the Gap – From Teacher to eTeacher Andrea Kelz 363
Open Courses: The Next big Thing in eLearning? Kaido Kikkas, Mart Laanpere and Hans
Põldoja
Using a Social Networking Environment to
Facilitate Transition Into Higher Education
Evaluation of the Quality of Learning Scenarios
and Their Suitability to Particular Learners’
Profiles
Models of eLearning: The Development of a
Learner-Directed Adaptive eLearning System
Can eLearning Enhance Practice-Based Design
Courses?
Sophisticated Usability Evaluation of Digital
Libraries
Social Networks, eLearning and Internet Safety:
Analysing the Stories of Students
Learning Management Versus Classroom
Management in Technology-Supported Blended
Learning
How to Represent a Frog That can be Dissected
in a Virtual World
Learning by Wandering: Towards a Framework
for Transformative eLearning
Online Student Engagement: Unfulfilled Promises
or Promises Unfulfilled?
355
370
John Knight and Rebecca Rochon 377
Eugenijus Kurilovas, Inga Zilinskiene and
Natalija Ignatova
380
Stella Lee, Trevor Barker, and Vive Kumar 390
Jake Leith, Joanna Zara and Malcolm
McInnes
399
Stephanie Linek and 408
Birgy Lorenz, Kaido Kikkas and Mart
Laanpere
415
Arno Louw 423
Robert Lucas 434
Marie Martin and Michaela Noakes 442
Linda Martin, Gary Spolander, Imran Ali
and Beulah Maas
Personalized e-Feedback and ICT Maria-Jesus Martinez-Argüelles , Josep-
Maria Batalla-Busquets, Patricia Noguera-
Guerra and Ernest Pons-Fanals
449
456

Paper Title Author(s) Page
No.
Evaluation of Multimedia Tools and e-Feedback
in Virtual Learning Environments
iv
Maria-Jesús Martínez-Argüelles, Marc
Badia-Miro, Carolina Hintzmann and Dolors
Plana-Erta
Cyberbullying: A Workplace Virus David Mathew 473
Learning in Smart Environments – From Here to
There
Using Courseware for More Than Courses: You
May Already Hold the Lease on a Versatile Virtual
Meeting Space
An Analysis of Collaborative Learning as a
Prevalent Instructional Strategy of South Africa
Government eLearning Practices
Ideas for Using Critical Incidents in Oral
Debriefing From a Business Strategy Simulation
Game
Volume Two
eNOSHA and Moodle – the Integration of two
eLearning Systems
CASE Learning to Structure and Analyze a Legal
Decision
A Framework for Decision Support for Learning
Management Systems
Learning for Life - Building Blocks to Holistic
Education
Student's Characteristics for Note Taking Activity
in a Fully Online Course
Freeing Education Within and Beyond Academic
Development
Why Recording Lectures Requires a new
Approach
465
Peter Mikulecky 479
Karen Hughes Miller and Linda Leake 485
Peter Mkhize, Magda Huisman and Sam
Lubbe
492
Jonathan Moizer and Jonathan Lean 502
Peter Mozelius, Isuru Balasooriya and
Enosha Hettiarachchi
509
Antoinette Muntjewerff 517
Phelim Murnion and Markus Helfert 526
Shekhar Murthy and Devi Murthy 535
Minoru Nakayama, Kouichi Mutsuura and
Hiroh Yamamoto
550
Chrissi Nerantzi 558
Paul Newbury, Phil Watten, Patrick Holroyd
and Clare Hardman
eSubmission – UK Policies, Practice and Support Barbara Newland, Lindsay Martin and Andy
Ramsden
Harnessing the Internet for Authentic Learning:
Towards a new Higher Education Paradigm for
the 21st Century
Motivational Predictors of Academics’ Electronic:
Publishing in Nigerian Colleges of Education
Psycho-Social Predictors of Students With
Disabilities’ eLearning: Usage at the Federal
College of Education (Special), Nigeria
An Integrated Environment for Providing Learning
Style Information in a Unified Manner
Using Lifeworld-led Multimedia to Enhance
Learning
567
578
Abel Nyamapfene 586
Maruff Akinwale Oladejo and Adelua
Olajide Olawole
Adelua Olajide Olawole and Maruff
Akinwale Oladejo
Fatemeh Orooji, Fattaneh Taghiyareh and
Zahra Rahimi
Andy Pulman, Kathleen Galvin, Maggie
Hutchings, Les Todres, Anne Quinney,
Caroline Ellis-Hill and Peter Atkins
593
601
609
620

The Project Mobile Game Based Learning Thomas Putz 628
Using the Common Cartridge Profile to Enhance
Learning Content Interoperability
The Design and Development of an eLearning
System Based on Social Networking
Kansei Design Model for eLearning: A
Preliminary Finding
Changing Teacher Beliefs Through ICT:
Comparing a Blended and Online Teacher
Training Program
Moodle and Affective Computing: Knowing who´s
on the Other Side
Using Google Applications to Facilitate an
Effective Students’ Collaboration in the Teaching
of Informatics to Students of Secondary
Education
Training Methods and Tools: Could eLearning be
a Viable Solution to Solve SMEs Training
Problems?
Using Blended Learning to Develop Critical
Reading Skills
A Mobile aid Tool for Crafting Active Learning
Experiences
King-Sized eLearning - how Effective can an
Online Approach be for Large Module Groups?
v
Ricardo Queirós and José Paulo Leal
637
Andrik Rampun and Trevor Barker 646
Fauziah Redzuan, Anitawati Mohd
Lokman, Zulaiha Ali Othman and Salha
Abdullah
Bart Rienties, Simon Lygo-Baker, Natasa
Brouwer and Danielle Townsend
Manuel Rodrigues, Florentino Fdez-
Riverola and Paulo Novais
658
670
678
Eleni Rossiou and Erasmia Papadopoulou 686
Andrée Roy 697
Zuzana Šaffková 705
Ahmed Salem 716
Marie Sams, Mary Crossan and Kate
Mottram
Designing Effective Online Group Discussions Rowena Santiago, Amy Leh, and Minoru
Nakayama
The Game and the Alternating Roles of
Learner/Teacher as Facilitators of the Learning
Process in Organizations
Implementing and Evaluating Problem-Based
Virtual Learning Scenarios
The Evolution of eLearning Platform TESYS User
Preferences During the Training Processes
Teachers’ Skills set for Personal Learning
Environments
Bridging the Feedback Divide Utilising Inclusive
Technologies
Post-Academic Masters Course in Management
of Transfusion Medicine: Why the Difference in
Access to the eLearning Between Countries?
Engagement With Students in ‘Middle Ground’: A
Flexible Learning Environment Allowing
Simultaneous Access to Social Networking Sites
and Formal Academic Space
The Learning Management System as a Social
Mediator: A Story With a Happy Ending
Can the Medium Extend the Message? Using
Technology to Support and Enhance Feedback
Practices
724
731
Vitor Santos and Luis Amaral 739
Maggi Savin-Baden, Cathy Tombs and
Katherine Wimpenny
Adriana Schiopoiu Burlea, Amelia Badica
and Carmen Radu
Zaffar Ahmed Shaikh and Shakeel Ahmed
Khoja
746
754
762
Angela Shapiro and Aidan Johnston 770
Cees Th. Smit SibingaI 776
Anne Smith and Sonya Campbell 780
Dina Soeiro, António Dias de Figueiredo
and Joaquim Armando Gomes Ferreira
788
Mekala Soosay 794

Implementation and Analysis of an Online,
Student Centred Learning Environment to
Support Personalised Study
The Danger of the Downward Spiral: Teachers
and Digital Literacy
PeerWise - The Marmite of Veterinary Student
Learning
iSELF: An Internet-Tool for Self-Evaluation and
Learner Feedback
Using a Learning Management System for
Executing Role Play Simulations
The Effects of Self-Directed Learning Readiness
on Learning Motivation in Web 2.0 Environments
Usage Cases: A Useful way to Improve
Effectiveness of eLearning web Based Platforms
The Virtual Path to Academic Transition: Enabling
International Students to Begin Their Transition to
University Study Before They Arrive
Identifying and Locating Frames of Reference to
Inform the Design of Virtual Worlds in Higher
Education
Reaction Lecture: Text Messaging to Increase
Student Engagement in Large-Scale Lectures
A Holistic Approach to Instructional Design for
Blended Learning Environments
vi
Iain Stewart, William McKee and Kevin
Porteous
802
Caroline Stockman and Fred Truyen 811
Amanda Sykes, Paul Denny and Lesley
Nicolson
820
Nicolet Theunissen and Hester Stubbé 831
Tone Vold 841
Chien-hwa Wang and Cheng-ping Chen 846
Cristina Wanzeller and Orlando Belo 854
Julie Watson 862
Katherine Wimpenny, Maggi Savin-Baden,
Matt Mawer, Nicole Steils and Gemma
Tombs
Koos Winnips, Joost Heutink, and Hans
Beldhuis
870
878
Li Zhong Zhang 886
PhD Papers 895
Evaluating the Impact of an Arabic Version of an
Adaptive Learning System Based on the Felder-
Silverman’s Learning Style Instrument
Negotiating Doctoral Practices and Academic
Identities Through the Adoption and Use of Social
and Participative Media
Enabling Disruptive Technologies for Higher
Education Learning and Teaching
Exploring the Potential of a Mobile Computer lab
in a Developmental Context: The Teacher’s
Perspective
Collaborative eLearning in a Developing Country:
A University Case Study in Uganda
Applying the Multimedia Learning Theory in the
Primary School: An Experimental Study About
Learning Settings Using Digital Science Contents
Designing a U-Learning Course Platform for the
Identified Teacher Training Needs
Nahla Aljojo, Carl Adams, Huda Saifuddin
and Zainab Alsehaimi
897
Andy Coverdale 909
Michael Flavin 917
Fortunate Gunzo and Lorenzo Dalvit 925
Evelyn Kigozi Kahiigi, Henrik Hansson,
Mats Danielson, F.F Tusubira and Mikko
Vesisenaho
Fabio Serenelli, Enrico Ruggeri, Andrea
Mangiatordi and Paolo Ferri
932
943
Nazime Tuncay and Hüseyin Uzunboylu, 953
Work In Progress Papers 971
Someone to Talk to – Using Automated
Characters to Support Simulated Learning
Activities
Liz Falconer and Manuel Frutos-Perez 973

Extreme Scaffolding in the Teaching and
Learning of Programming Languages
Benefits and Barriers: Applying eLearning in the
Context of Organisational Change to Improve the
Learning Experience for Mature, Part-Time
Students
Posters with Papers
Investigating Student Engagement With an
Electronically Delivered Simulation of
Professional Practice
Instrumental Distance Learning in Higher Music
Education
Reflections on Academic Blogging as a Vehicle
for Professional Development
A Framework for Understanding Online Learning
Communities
Trust in Distributed Personal Learning
Environments: The Case Study of LePress PLE
Breaking Down Barriers: Development of a Wiki
Based Module to Enhance the International
Learning Experience
eLearning in German Higher Education:
Technology Implementation as a Challenge for
Organizational Change
vii
Dan-Adrian German 978
Simon McGinnes
982
Olivia Billingham 989
Karin Levinsen, Rikke Orngreen, Mie Buhl,
Marianne Løkke Jakobsen and Jesper
Andersen 1
Peps Mccrea
Sónia Sousa, David Ribeiro Lamas, José
Braga de Vasconcelos and Ilya Shmorgun 1
Sónia Sousa, David Ribeiro Lamas and
Vladimir Tomberg
Karen Strickland, Liz Adamson, Carolyn
Blight and Wendy McInally
Novita Yulianti, Michael Lund and Georg
Müller-Christ
993
997
1000
1006
1012
1015

Conference Committee
Conference Executive
Sue Greener, Brighton Business School, University of Brighton, UK
Asher Rospigliosi, Brighton Business School, University of Brighton, UK
Andrea Benn, Brighton Business School, University of Brighton, UK
Laurence Olver, Brighton Business School, University of Brighton, UK
Stephanos Avakian, Brighton Business School, University of Brighton, UK
Barbara Newland, Centre for Learning and Teaching, University of Brighton, UK
Steve Kilgallon, School Of Environment and Technology, University of Brighton, UK
Avril Loveless, School of Education, University of Brighton, UK
Lyn Pemberton, School of Computing, Engineering and Maths., University of Brighton, UK
Committee members
The conference programme committee consists of key people in the e-learning community around the world.
The following people have confirmed their participation:
Ariffin Abdul Mutalib (Universiti Utara Malaysia, Malaysia); Siti aishah Abdullah (University Technology Mara,
Kelantan, Malaysia); Tofan Cezarina Adina (Spiru Haret University, Romania); Wilfried Admiraal (Universiteit
van Amsterdam, Netherlands); Shafqat Ali (University of Western Sydney, Australia); Abdallah Al-Zoubi
(Princess Sumaya University for Technology, Jordan); Margarida Amaral (University of Porto, Portugal);
Antonios Andreatos (Hellenic Air Force Academy, Greece); Alla Anohina (Riga Technical University, Latvia);
Jane Ardus (Stevenson College,Edinburgh, UK); Mohamed Arteimi (7th of April University, Tripoli, Libya);
William Ashraf (University of Sussex, UK); Bunyamin Atici (Firat University, Turkey); Anders Avdic (Orebro
University, Sweden); Simon Bachelor (Gamos, Reading, UK); Joan Ballantine (University of Ulster, UK);
Trevor Barker (University of Hertfordshire, UK); Josep-Maria Batalla (Universitat Oberta de Catalunya,
Spain); Orlando Belo (University of Minho Campus de Gualtar, Portugal); David Benito (Public University of
Navarre, Pamplona, Spain); Yongmei Bentley (University of Luton, UK); Daniel Biella (University of Duisburg-
Essen, Germany); Radu Bilba (George Bacovia University,, Romania); Eric Bodger (University of
Winchester, UK,); Stephen Bowman (Ravensbourne College of Design and Communication, UK); Willem-
Paul Brinkman (Delft University of Technology, Netherlands); Ann Brown (CASS Business School, London,
UK); Mark Brown (Massey University,Palmerston North, New Zealand); Giuseppe Cannavina (University of
Sheffield, UK); Sven Carlsson (School of Economics and Management, Lund University, Sweden); James
Carr (University of Newcastle, UK); Maggie Carson (Edinburgh University, UK); Antonio Cartelli (University of
Cassino,, Italy); Rommert Casimir (Tilburg University, The Netherlands); Ivana Cechova (University of
Defence, Czech Republic,); Maria Celentano (University of Lecce, Italy); Athina Chatzigavriil (LSE, London,);
Satyadhyan Chickerur (M.S.Ramaiah Institute of Technology, Bagalore, India); Burhan China (PDSA,
Somalia); Barbara Class (University of Geneva, Switzerland); Lynn Clouder (Coventry University, UK);
Thomas Connolly (University of West of Scotland, UK); Ken Currie (Edinburgh University, UK); Valentina
Dagiene (Institute of Mathematics and Informatics, Vilnius, Lithuania); Mark De Groot (Leeds Metropolitian
University, UK); Antonio De Nicola (ENEA, Italy); Carmen De Pablos Heredero (Rey juan Carlos University,
Spain); Rajiv Dharaskar (GH Raisoni College of Engineering, Nagpur, India); Vicenzo Di Lecce (Politecnico
di Bari, Italy); Martina Doolan (University of Hertfordshire, UK); Christopher Douce (Institute of Educational
Technology, Walton Hall, UK); Yanqing Duan (University of Luton, UK); Jane Eberle (Emporia State
University, USA); Colin Egan (University of Hertfordshire, Hatfield, UK); Bulent Gursel Emiroglu (Eskisehir
Yolu Baglica Mevkii, Turkey); Chew Esyin (University of Glamorgan, UK,); Ariwa Ezendu (London
Metropolitan University, Uk); Bekim Fetaji (South East European University, Tetovo, Macedonia); Andrea
Floros (Ionian University, Greece); Duncan Folley (Leeds Metropolitian University, England); Katie Goeman
(Free University of Brussels (VUB), Belgium); Colin Gray (Edinburgh Napier University, Scotland); Susan
Greener (University of Brighton, UK); David Guralnick (Kaleidoscope Learning, New York, USA); Richard
Hall (De Monfort University, Leicester, UK); Patricia Harvey (Greenwich University, London, UK); Thanos
Hatziapostolou (International faculty of the university of sheffield, Greece); Rose Heaney (University of East
London, UK,); Alan Hilliard (University of Hertfordshire, Hatfield, UK); Uwe Hoppe (Bildungswerk der
Sächsischen Wirtschaft gGmbH, Germany); Md. Fokhray Hossain (Daffodil International University,
Bangladesh); Stefan Hrastinski (Uppsala University, Sweden); BALDE IDIATOU (NOBLE GROUP
ORGANISED SOLUTIONS, GUINEA); Antonin Jancarik (Faculty of education, Charles University, Czech
Republic); Amanda Jefferies (University of Hertfordshire, Hatfield, UK); Runa Jesmin (Global Heart Forum,
UK); Aidan Johnston (Glasgow Caledonian University, UK); Paul Jones (University of Glamorgan, UK);
Geraldine Jones (University of Bath, UK,); Jowati Juhary (National Defence University of Malaysia,
viii

Malaysia); Tuomo Kakkonen (University of eastern Finland, Finland); Michail Kalogiannakis (School of
Pegadogical and Technicological Education, ASPETE, Crete); Clifton Kandler (University of Greenwich,
UK,); Jana Kapounova (University of Ostrava , Czech Republic); Andrea Kelz (University of Applied Sciences
Burgenland,Campus Pinkafeld, Austria); Saba Khalil (Virtual University of Pakistan, Lahore, Pakistan);
Jasna Kuljis (Brunel University, UK); Sunaina Kumar (Indira Gandhi National Open University, New Delhi,
INDIA); Swapna Kumar (University of Florida, USA); venkata Durga kumar (Sunway University College,
Malaysia,); Blair Kuntz (University of Toronto, Canada,); Eugenijus Kurilovas (Vilnius Gediminas technical
university / institute of mathmatics and informatics of Vinius University, Lithuania); Eleni Kyza (Cyprus
University of Technology, Lemesos, Cyprus); Maria Lambrou (University of the Aegean Business School,
Greece); Andy Lapham (Thames Valley University, UK); Mona Laroussi (Institut National des Sciences
Appliquées et de la Technologie, Tnis and Lille, Tunisia); Deepak Laxmi Narasimha (University of Malaya,
Malaysia); Fotis Lazarinis (Applied Informatics in Management and Finance, Greece); Denise Leahy (Trinity
College, Dublin, Ireland); Kate Lennon (Glasgow Caledonian University, UK); Mariana Lilley (University of
Hertfordshire, UK); Jorgen Lindh (Jonkoping International Business School, Sweden); Gi-Zen Liu (National
Cheng Kung University, Taiwan); Ying Liu (Cambridge University, UK); Jenny Lorimer (University of
Hertfortshire, UK); Sam Lubbe (University of South Africa, South Africa); Nick Lund (Manchester
Metropolitan University, England,); Alejandra Magana (Purdue University, United States of America, United
States of America); Adnan Mahmood (University of Jinan, P.R.China,); Francis Maietta (Real Thinking
Company, UK); Christina Mainka (Heidelberg University , Germany); Chittaranjan Mandal (School of IT,IIT
Kharagpur, India); Augostino Marengo (University of Bari, Italy); Maria J Martinez-Arguelles (Universitat
Oberta de Catalunya, Spain); Sephanos Mavromoustakos (Cyprus College, Cyprus); Erika Mechlova
(University of Ostrava, Czech Republic); Cherifa Mehadji (University of Strasbourg, France); Rosina Merry
(the school of Education The University of Waikatio, New Zealand); Linda Joy Mesh (Universita degli Studi di
Siena, Italy); Jaroslava Mikulecka (University of Hradec Kralove, Czech Republic); Peter Mikulecky
(University of Hradec Kralove, Czech Republic); Mike Mimirinis (Middlesex University, London, UK); Julia
Mingullon (Universitat oberta de catalunya, Spain); Ali Moeini (University of Tehran, Iran); Johann Moller
(University of South Africa (UNISA), South Africa,); Peter Monthienvichienchai (Insitute of Education,
London, UK); Pam Moule (University of the West of England, Bristol, UK); Radouane Mrabet (ENSIA,
Morocco); Minoru Nakayama (Tokoyo Institute of Technology, Japan); Julian Newman (Glasgow Caledonian
University , UK); Chetsada Noknoi (Thaksin University, Songkhla, Thailand); Abel Nyamapfene (University of
Exeter, United Kingdom); Sinead O’Neill (Waterford Institute of Technology, Ireland); Kamila Olsevicova
(Univeristy of Hradec Kralove, Czech Republic); Rikke Orngreen (Aarhus University, Denmark); Jalil
Othman, (University of Malaya ,Malaysia); Kutluk Ozguven (Dogus University, Turkey); Ecaterina Pacurar
Giacomini (Louis Pasteur University, FRANCE); Alessandro Pagano (University of Bari, Italy); Vasileios
Paliktzoglou (University of eastern Finland, Finland); Stefanie Panke (University of Ulm, Germany); George
Papadopoulos (University of Cyprus, Cyprus); Iraklis Paraskakis (South East European Research Centre
(SEERC) Research Centre of the University of Sheffiled, Thessaloniki, Greece); Vivien Paraskevi (TECFA -
FPSE, Educational Technology Unit, University of Geneva, Switzerland, Switzerland); Angie Parker (Anthem
College Online, USA,); Paul Peachey (University of Glamorgan, Treforest, UK); Arna Peretz (Ben Gurion
Univeristy of the Negev, Omer, Israel); Christine Perry (University of the West of England, Bristol, UK);
Donatella Persico (Istituto Tecnologie Didattiochje-Consiglio Nazionale Ricerche, Genova, Italy); Pit
Pichappan (Annamalai University, India); Selwyn Piramuthu (University of Florida, Gainesville, USA); Michel
Plaisent (University of Quebec in Montreal, Canada); Lubomir Popelinsky (Masaryk University, Czech
Republic); Andy Pulman (Bournemouth University, UK); Muhammad Abdul Qadir (Mohammad Ali Jinnah
University, Islamabad, Pakistan); Ricardo Queirós (ESEIG/KMILT & CRACS/INESC, Portugal); Susannah
Quinsee (City University, London, UK); Abdul Rafay (Asia Pacific University College of Technology &
Innovation, Malaysia); Liana Razmerita (Copenhagen Business School, Denmark); Christopher Reade
(Kingston University, UK); Hugo Ribeiro (University of Porto, Portugal,); Vivien Rolfe (De Monfort University,
Leicester, UK); Asher Rospigliosis (University of Brighton, UK); Florin Salajan (North Dakota State University
, Canada); David Sammon (Univesity College Cork, Ireland); Gustavo Santos (University of Porto, Portugal);
Venkat Sastry (Defence College of Management and Technology, Cranfield University, UK); Guy Saward
(University of Hertfordshire, UK,); Brian Sayer (University of London, UK); Jeanne Schreurs (Hasselt
University, Diepenbeek, belgium); Jane Secker (London School of Economics, UK); Angela Shapiro
(Glasgow Caledonian University, Scotland); Aileen Sibbald (Napier University, Scotland, UK); Petia Sice
(University of Northumbria, Newcastle-upon-Tyne, UK); Gurmeet Singh (The University of The South Pacific,
Suva , Fiji, Fiji); Cees Th. Smit Sibinga (Academic insitute for the international development of transfusion
medicine, The Neverlands); Alisdair Smithies (Manchester Medical School, UK); Keith Smyth (Napier
University, Edinburgh, UK); Bent Soelberg (Copenhagen Business School, Denmark); Or Kan Soh
(University Tunku Abdul Rahman (UTAR), Malaysia); Yeong-Tae Song (Towson University, Maryland, USA);
Michael Sonntag (FIM, Johannes Kepler University, Linz, Austria); Rumen Stainov (University of Applied
Sciences, Fulda, Germany); John Stav (Sor-Trondelag University College, Norway); Roxana Taddei
(Université Clermont Ferrand 2, Montpellier, France); Yana Tainsh (University of Greenwich,, UK); Heiman
Tali (The Open University, Israel); Bénédicte Talon (Université du Littoral, France); Marian Theron (False
ix

Bay College, Tokai, South Africa); John Thompson (Buffalo State College, USA); Claudine Toffolon
(Université du Mans - IUT de Laval, France); Eulalia Torras-Virgili (Open University of catalonia, Spain);
Kathryn Trinder (Glasgow Caledonian University, UK); Christopher Turner (University of Winchester , UK);
Karin Tweddell Levinsen (Danish University of Education, Denmark); Aimilia Tzanavari (University of Nicosia,
Cyprus); Huseyin Uzunboylu (Near East University, CYPRUS); Linda Van Ryneveld (Tshwane University of
Technology, Pretoria, South Africa); Carlos Vaz de Carvalho (Porto Polytechnic, Portugal); Andreas Veglis
(Aristotle University of Thessaloniki, Greece); Bruno Warin (Université du Littoral, Calais, France); Fahad
Waseem (University of Northumbria, Middlesbrough, UK); Garry Watkins (University of Central Lancashire,
UK); Anne Wheeler (Aston University, UK); Steve Wheeler (Faculty of Education, University of Plymouth,
UK); Nicola Whitton (Manchester Metropolitan University, UK); Roy Williams (University of Portsmouth, UK);
Shirley Williams (University of Reading, UK); Rowena Yeats (University of Birmingham, UK); Panagiotis
Zaharias (University of the Aegean, Greece); Mingming Zhou (Nanyang Technological University,
Singapore); Chris Zielinski (External relations and Governing Bodies, World Health Organization, Geneva,
Switzerland); Anna Zoakou (Ellinogermaniki Agogi, Greece);
x

Preface
These Proceedings represent the work of contributors to the 10th European Conference on e-Learning,
ECEL 2011, hosted this year by Brighton Business School, University of Brighton, UK. The Conference Chair
is Sue Greener, and the Programme Chair is Asher Rospigliosi, both from Brighton Business School, UK.
The conference will open with a keynote address by Don Clark, former CEO of EPIC software and winner of
the ‘Outstanding Achievement in e-learning Award’. Expect to be enlivened by his challenge: “Don't lecture
me!, ..and why technology is only scalable solution”. Also on the first day we have Anne Boddington from the
University of Brighton speaking on the topic of “Designing Education and Reshaping Learning”. The second
day will be opened by Professor Gráinne Conole, Director of the Beyond Distance Research Alliance at the
University of Leicester in the UK with a talk entitled “Trajectories of learning - new approaches and
directions".
With an initial submission of 239 abstracts, after the double blind, peer review process there are 108
academic papers, 7 Phd Papers, 3 Work in Progress papers and 3 non academic papers in these
Conference Proceedings. These papers reflect the truly global nature of research in the area with
contributions from Australia, Austria, Belgium, Canada, China, Czech Republic, Denmark, Estonia, France,
Germany, Greece, India, Iran, Ireland, Israel, Italy, Japan, Jordan, Lithuania, Malaysia, Nigeria, North
Cyprus, Norway, Pakistan, Portugal, Romania, Saudi Arabia, Singapore, Slovakia, South Africa, Spain,
Sweden, Taiwan, The Netherlands, Turkey, United Kingdom and the United States.
A selection of papers – those agreed by a panel of reviewers and the editor will be published in a special
conference edition of the EJEL (Electronic Journal of e-Learning www.ejel.org ).
We know you will enjoy and be stimulated by this conference. We hope you will also revel in the uniqueness
of Brighton.
Sue Greener and Asher Rospigliosi
November 2011
xi

Biographies of Conference Chairs, Programme Chairs and
Keynote Speakers
Conference Chairs
Dr Sue Greener is a University teacher: HRM, Business Context, Research Methods
and Learning & Development and has received a Teaching Excellence award from the
University of Brighton and is Programme Leader for the Foundation Degree in
Business. Sue is also the Course Director: online final year undergraduate course with
students in diverse world regions, her researcher interests are focused on e-learning
strategy, teacher development and reflective learning. Sue is the co-founder of the
Business e-Learning Research Group and a member of the CROME research group
on employment issues at Brighton Business School. Her Doctoral research focused
on exploring students’ readiness for online learning. Sue holds a BA, MBA, EdD, FHEA and is a Chartered
Fellow of CIPD.
Programme Chair
Asher Rospigliosi lecturers on e-commerce, management information systems, IS
strategy, public sector IS and digital marketing at the University of Brighton. His
research interests extend to e-learning and innovation in SMEs. Asher is a cofounder
of the Business e-Learning Research Group and a member of the CROME
research group on employment issues at Brighton Business School.
Keynote Speakers
Gráinne Conole is Professor of e-Learning at the Open University, with research
interests in the use, integration and evaluation of Information and Communication
Technologies and e-learning and impact on organisational change. She was
previously chair of educational innovation at Southampton University and before that
Director of the Institute for Learning and Research Technology at the University of
Bristol. She has extensive research, development and project management
experience across the educational and technical domains She serves on and chairs a
number of national and international advisory boards, steering groups, committees
and international conference programmes.
Donald Clarke was CEO and one of the original founders of Epic Group plc, which
established itself as the leading company in the UK e-learning market, floated on the
Stock Market in 1996 and sold in 2005. Describing himself as ‘free from the tyranny of
employment’, he is now a board member of Ufi LearnDirect (Government agency
delivered e-learning to 2.8 million learners), Caspian Learning (learning games tool
provider), LearningPool (content provider), Brighton Arts Festival, and a school
governor. Donald has won many awards for the design and implementation of elearning,
notably the ‘Outstanding Achievement in e-learning Award’.
Anne Boddington is Dean of the Faculty of Arts at the University of Brighton.
Educated and qualified as an Architect and subsequently as a Cultural Geographer,
she currently leads a Faculty of .3,900 students and a portfolio that includes the Visual
and Performing Arts, Architecture, Design, Media Studies, Literature, Languages and
Humanities. She is also Co-director of the ADM HEA Subject Centre, which we are
proud to host at the University. Anne is a Fellow of the Royal Society of Arts (RSA)
and an affiliate member of the Royal Institute of British Architects (RIBA) and is an
elected member of the executive for the Council for Higher Education in Art & Design
(CHEAD). Her initial research interests were rooted in the design and development of
the urban and cultural landscape and identity but have expanded to include the strategic design and
development of learning and research space and its relationships to pedagogic practice and to educational
strategies and governance.
xii

Mini Track Chairs
Dr Antonios Andreatos is a Professor at the Computer Engineering Division of the
Hellenic Air Force Academy. He received the Diploma in Electrical Engineering from
the Univ. of Patras, the M.S. in Computer Engineering from the Univ. of
Massachusetts, the M.Ed in Adult Learning from the Hellenic Open Univ. and the
Ph.D. in Computer Engineering from the National Technical Univ. of Athens.
Research interests include e-Assessment, Active Learning methods, ICT & Web 2.0
& Open Resources in Education, Didactics of Computer Engineering, etc. He has
published over 60 papers in journals and conference proceedings and a book. He is
also involved in the scientific committees of many conferences in his fields of interest.
Orlando Belo is an associate professor in the Department of Informatics at Minho
University, Portugal. His main research topics are related with data warehouse design,
implementation and tuning, ETL services, database preferences, and distributed
multidimensional structures processing. During the last few years he was involved with
several projects in the decision support systems area designing and implementing
computational platforms for specific applications like fraud detection and control in
telecommunication systems, data quality evaluation, and ETL systems for industrial
data warehousing systems. More recently, he was developing some research work
establishing OLAP usage profiles and optimizing OLAP selection methods, applying some of the techniques
studied on these areas over typical e-learning scenarios.
Dr Colin Egan is a senior lecturer in the School of Computer Science, at the University
of Hertfordshire. Colin has had an interest in accessibility and accessibility issues for a
number of years and has presented his work to a range of International Conferences.
Dr Amanda Jefferies is a Reader in Technology Enhanced
Learning at the University of Hertfordshire. She is passionate about
promoting a positive learning experience for students in HE through careful choice of
technologies, to enhance their engagement with studies. She is well-known for
innovative research into understanding the student experience through using student
constructed reflective video and audio diaries, a technique she refined through her JISCsupported
‘Learner Journeys’ STROLL project during 2007-2009. She has presented
her research to international audiences across the UK, Europe and in North America.
Dr Eugenijus Kurilovas is a Research Scientist in Vilnius University Institute of
Mathematics and Informatics, an Associate Professor in Vilnius Gediminas Technical
University, and a Head of International Networks Department in the Centre of
Information Technologies in Education of the Ministry of Education and Science of
Lithuania. He has published over 50 scientific papers, 2 books chapters and 2
monographs, participated in many large scale international scientific projects. He is a
member of 12 scientific committees of the international journals and conferences. His
papers presented on ECEL-2008, 2009 and 2010 have been selected for publication in
Electronic Journal of e-Learning.
Dr Barbara Newland is a National Teaching Fellow with over 17 years experience of
leading educational developments in Higher Education. She is a Principal Lecturer in the
Centre for Learning and Teaching at the University of Brighton. Previously, Barbara was
Blended Learning Co-ordinator at Glasgow Caledonian University, Manager of the
Educational Development Services at Bournemouth University and the Learning
Technology Team at Durham University.
Dr Keith Smyth is a Senior Teaching Fellow and Senior Lecturer in Higher Education at
Edinburgh Napier University, where he leads the MSc Blended and Online Education
(www.napier.ac.uk/ed/boe). Keith is interested in how simple uses of technology can
enhance learning, and where blended and online approaches can underpin community
and work-based learning, broader skills development, and more inclusive educational
practices.
xiii

Biographies of contributing authors (in alphabetical
order)
Samuel Adu Gyamfi is currently a PhD Fellow at the centre for Communication, Media and Information
technologies (CMI) at Aalborg University, Copenhagen. He is currently researching on developing the
Personal Transferable Skills (PTS) of the university graduate through E-learning.
Imran Ali is a Senior Learning technologist at Coventry University. He is passionate about the potential of
technology in education. His background is in multimedia production and education. He currently supports
staff across the Faculty of Health and Life Sciences in making effective use of technology for teaching,
learning, assessment and research purposes.
Nahla Aljojo worked as an IT manager in the economics and administration department at King Abdul Aziz
University in Saudi Arabia. She has a master’s degree in computer systems and Information Technology
from the Washington International University (USA) and is also a Ph.D. doctoral student at the School of
Computing at the University of Portsmouth (UK).
Hussein Al-Yaseen holds a PhD degree in Information Technology from the University of Brunel, UK. His
areas of interest are in information systems evaluation, project management and the implementation of
information systems in developing countries. Hussein is currently an associate professor at the Management
Information Systems Department at Al-Ahliyya Amman University, Jordan.
António Andrade PhD in Technologies and Information Systems MSc in Information and Management
Director of the MSc in Information and Documentation.
Roni Aviram is Chair of the Center for Futurism in Education, Department of Education, Ben-Gurion
University. Aviram is interested in the impact of ICT on education and society, and in structuring theoretical
and practical change processes in education. He has led R&D projects dedicated to designing virtual
lifelong learning environments for enhancing human development and flourishing.
Jonathan Barkand has a B.S. in Technology Education and an M.S. in Multimedia Technology from
California University of Pennsylvania. He is currently completing his dissertation work at Duquesne
University for a doctorate in Instructional Technology. His area of focus is online education and its effects on
teachers and students.
Trevor Barker is Principal Lecturer and University Fellow in Teaching and Learning in the Department of
Computer Science at the University of Hertfordshire. He obtained a PhD for research into developing and
evaluating adaptive multimedia computer systems in education. His recent research relates to the design
and evaluation of adaptive educational systems and the affordances of virtual worlds such as Second Life
for study and work.
Josep Batalla has a Degree and a Phd in Economics from the University of Barcelona. He is a Professor of
Economics and Business Studies at the Universitat Oberta de Catalunya (UOC), where he teaches courses
in the field of applied economics. Resident scholar at the Internet Interdisciplinary Institute
Olivia Billingham is a research fellow in the e-learning development unit at the University of the West of
England. Her research interests include student perceptions and experiences of technology enhanced
learning.
Latefa Bin Fryan (BSc, MSc) is a Researcher in the School of Information Systems, Computing and
Mathematics at Brunel University, UK, where she is pursuing her PhD in the area of e-Learning systems.
She received M.Sc. (2009-2010) in Information Systems Management from Brunel University, UK. She
has published papers in several international conferences.
Sonya Campbell As senior service manager within Library and Information Services, Sonya supports the
delivery of flexible learning environments and the development of services designed to maximise learner
productivity and success. Current research includes the opportunities social media may bring to enable
student engagement, learning and collaboration. Teaching themes are corporate strategy, effective
management and customer service.
xiv

Cornélia Castro is a PhD student in Educational Sciences: Educational Computing Master in Quality
Control: Environment . Graduation in Pharmaceutical Sciences High School Chemistry and Physics teacher
. Research interests: ICT integration in the classroom, Web 2.0 tools in Education, Open Educational
Resources, Learning Objects, Repositories and Social Media in Education
Martin Cápay works as a professor assistant at the Department of Informatics. He deals with the theory of
teaching informatics subjects, mainly programming. He participates in the projects aimed at the usage of new
competencies in teaching and also in the projects dealing with learning in virtual environment using elearning
courses.
Tim Cappelli is an experienced Project Manager who has successfully managed and contributed to a range
of Technology Enhanced Learning and change management projects. Tim has developed a rich
understanding of the application of informal and collaborative learning through the use of social networking
tools that reinforce the links between individual and organisational development.
Ivana Cechova graduated from the Faculty of Arts at Masaryk University, Brno, with specialisations in
pedagogy, English and Russian language and literature. She has worked as Head of Research and Deputy
Head of the Language Department at the Faculty of Economics and Management. Currently she is a senior
lecturer at the Language Training Centre of the University of Defence.
Vivian Chan Yin Ha is a Senior Instructor of the Independent Learning Centre at the Chinese University of
Hong Kong. Her research interests include language teaching, learning autonomy, modern and
contemporary Chinese literature, and gender studies.
Serdar Ciftci is a doctoral candidate studying in Department of Educational Technology, Institute of
Educational Studies of Gazi University. He is also an expert working in Directorate of Computer Center of
Gazi University. His researcher interests are focused on e-learning, metacognition, web-based learning
strategies.
Chetz Colwell is a Project Officer in the Learning and Teaching Development team of the Institute of
Educational Technology at the Open University. Her role is to support the university in making online
teaching materials accessible to disabled students by conducting technical accessibility testing, and
conducting student evaluations.
David Comiskey lectures in Architectural Technology at the University of Ulster. He is a Fellow of the Higher
Education Academy and his research interests include the use of video and screencasting technology to
improve the student learning and feedback experience. He was recently funded by the Higher Education
Academy to continue researching this area.
Mary Crossan has worked at Coventry University for six years and has industry experience of over 16
years. Her experience has been mainly in the financial area but with a high emphasis on managing people
and resources for project delivery. Research interests are in the areas of improving student experience and
work-based learning approaches.
Marija Cubric is a Principal Lecturer at the UH Business School, where she teaches information systems,
and project management related subjects. She is also a member of the UH Learning and Teaching Institute
that promotes the use of blended learning practices across the University. Her research interests include
educational technologies, and their influence on learning and teaching processes.
Maya Khemlani David (Faculty of Languages and Lingusitics, University of Malaya) received the Linguapax
Award in 2007. She is an Honorary Fellow of the Chartered Institute of Linguists, United Kingdom and an
Honarary Member of the Foundation of Endangered Languages. Her areas of specialisation are crosscultural
communication, discourse analysis and language maintenance and language shift.
Christine Davies is a Senior Lecturer in Technology-Enhanced Learning at the University of Glamorgan.
Originally a Biology teacher, she has taught in schools and FE, and continues to teach on OU science
courses and on Wales-based post-graduate education courses. She has also worked for JISC RSC Wales
as an e-Learning Advisor.
Mark de Groot works for the Centre for Teaching and Learning team at Leeds Metropolitan University. He
has more than 15 years’ experience initiating and supporting learning technology related staff development
activities across and beyond the University.
xv

Jiří Dlouhý is a specialist on cybernetics and works at the Environmental Center of the Charles University as
head of the Environmental Education department. He is also a member of executive board of the European
Environmental Bureau, chairman of the Society for Sustainable Living; and a founding member of the
International Society of Information Specialists
Elena Dominguez-Romero holds a Master’s degree in Innovative Teaching in Tertiary Education (Huelva,
2008). She has participated in several Research Projects concerning Innovative Teaching (Universidad de
Huelva: 2005, 2006a, 2006b; UCM: 2009, 2010). The results of this ECEL paper are derived from her
participation in a current project funded by UCM (268-2010 UCM).
Jon Dron is a member of the Technology Enhanced Knowledge Research Institute (TEKRI) at Athabasca
University, Canada, an Honorary Faculty Fellow in the Faculty of Education & Sport, University of Brighton,
UK, and a National Teaching Fellow of the UK Higher Education Academy.
Michaela Drozdova graduated in 2009 with a Masters degree in Secondary Teaching Mathematics and
Informatics approbation. In September 2009 she began teaching high school Teleinformatics, while
commencing doctoral studies in the Department of Information and Communication Technology in Education
at the University of Ostrava.
Glenn Duckworth currently works at the University of the West of England in the role of e-Learning
Development Officer. He is employed by the Faculty of Business and Law and is a member of the e-Learning
Development Unit. He previously worked at the University of Huddersfield as Senior Lecturer, Researcher
and Chief Psychology Technical Officer.
Ibrahim Elbeltagi is a lecturer in information and knowledge management at the School of Management,
University of Plymouth. His publications are related to electronic commerce, adoption of ICT, information
systems in developing countries, social networking and knowledge management. He has more than 30
journal and conferences papers published in national and international journals and conferences.
Antonella Esposito. Elearning practitioner since 1996, Antonella led the CTU (E-learning center) of the
University of Milan for seven years. Currently she is a PhD candidate in the E-learning program, Open
University of Catalonia and is completing a dissertation on digital scholarship for the MRes in Educational
and Social Research, Institute of Education, University of London.
Liz Falconer completed her MPhil at Manchester Polytechnic in 1992 and her PhD at Salford University in
2001. In 2001 she moved to the University of Bath as Director of the Centre for Distance Education. She is
now Professor of Technology Enhanced Learning at the University of the West of England.
Gert Faustmann studied Information Technology at the Technical University Berlin. From 1992 to 2001 he
was a software developer (Siemens AG), researcher (Fraunhofer Institute for Software and Systems
Engineering) and consultant (debis - later T-Systems). He is now professor and course director for the
division of computer science at Berlin School of Economics and Law.
Sérgio André Ferreira has a Degree in Geography, specialization in Education Sciences. Master in
Sciences of the Education, specialization in Education Computer Sciences. PhD student in Sciences of the
Education, specialization in Education Computer Sciences (Grad Student). Research interests in Technology
Enhanced Learning Environments. Geography teacher and Teacher Trainer in the areas of ICT and
Geography.
Rachel Fitzgerald, MSc is a Senior Lecturer (Information Science) at Northampton Business School,
University of Northampton and a PhD student at the University of Lancaster. A former learning technologist,
she holds CMALT certification and a Teaching Fellowship Award from the University of Northampton.
Current research interests include Networked Learning and Distance Education Innovation.
Michael Flavin is a research student at the Open University’s Institute for Educational Technology. He
previously gained a PhD in nineteenth-century literature (Kent, 1999). He has taught at King’s College
London and the Open University, and is currently Head of Learning and Teaching at the IFS School of
Finance, a degree-awarding business school in London.
Gabriele Frankl is head of the eLearning Service department at the Alpen-Adria-Universität Klagenfurt since
2008. Her main research interests are eLearning and Blended Learning, Knowledge Management and winwin
constellations as well as self-organisation processes. She has successfully implemented eLearning and
Knowledge Management systems in the production industry.
xvi

Manuel Frutos-Perez is the Leader of the E-learning Development Unit at the University of the West of
England, Bristol, UK. He has worked for several universities in the UK, Spain and Austria. He has also
written and contributed to numerous textbooks and e-learning interactive materials.
Elaine Garcia is undertaking a part time PhD at the University of Plymouth focusing on the use of blogs
within education. This is alongside employment as Head of Operations and Resources at Plymouth College
of Art. Research interests include: Social Networking Sites, Social Media, Blogs, Social Learning and
Knowledge Management.
Danny Glick is an international e-learning consultant and an expert in the implementation of large-scale
computer-based solutions. He has worked as a consultant for both public and private organizations. Glick is
a research fellow at the Center for Futurism in Education at Ben-Gurion University, where he conducts
research on mindful learning processes in Web-based learning environments.
Andrea Gorra is a Senior Lecturer in the Business School at Leeds Metropolitan University (UK). Prior to
this she has worked as a researcher in the area of learning technologies. Further research interests include
grounded theory methodology, and the use of social software and mobile learning devices for assessment,
learning and teaching.
Rose Heaney is a learning technology advisor and Teaching Fellow at the University of East London with
responsibility for facilitating the use of learning technologies in the schools of Health, Sport & Bioscience and
Psychology. One of her research interests is the use of Second Life® in clinical education as well as for
more general purposes.
Carolina Hintzmann has a Degree in Economics from the Universitat de Barcelona, Spain. She is a teacher
in the Economics and Business Sciences Department, Universitat Oberta de Catalunya (UOC), Barcelona,
Spain
Karen Hughes Miller, PhD, is Director for Graduate Medical Education (GME) Curriculum Design,
Evaluation and Research, University of Louisville School of Medicine. Her work in adult education includes
teacher education, military education, and currently medical education. She is interested in curriculum design
and analysis and education research, and enjoys the challenges of teaching advanced content to very bright
adult learners.
Anne Jelfs is Manager of the Learning and Teaching Development team of the Institute of Educational
Technology at the Open University. She manages a team that conducts evaluations of OU materials
including usability testing. She also leads on a university-wide initiative to embed accessibility within the
university's production and delivery systems.
John Jessel is head of the PhD Programme in the Department of Educational Studies at Goldsmiths,
University of London. His research activities focus on the social and cognitive processes that underlie
learning and development both inside and outside the school setting and within the context of the use of
digital technologies.
Aidan Johnston, BSc. (Hons), PGDip, AHEA, is a Blended Learning Advisor with many years’ experience
designing and embedding video and audio resources to enhance the teaching and learning experience of the
learner. He has collaborated on projects involving the JISC, BBC Information & Archives, The National
Science Foundation, The Higher Education Academy and REAP.
Rosario Kane-Iturrioz is a Senior Lecturer in Spanish at Coventry University and also a member of the
Chartered Institute of Linguists and the Higher Education Academy (UK). Her research interests are the
integration of Information and Communication Technology (ICT) into language learning and assessment and
the use of VLEs as an effective learning tool.
Jana Kapounova is an associate professor at the Department of ICT, University of Ostrava in the Czech
Republic. She teaches subjects as Educational Technology, ICT in Education, eLearning. She guarantees
studies of ICT in Education in bachelor, master and Ph.D. degrees. Her research field is eLearning and
evaluation of its quality.
Shakeel Khoja is a Commonwealth Academic Fellow. He received his Ph.D. from the University of
Southampton, UK, in 2001, and is working as a Professor at IBA, Karachi. Shakeel’s research interests
xvii

include Learning Technologies, Web Technologies, and Internet programming. He has a professional career
of over 15 years and has fifty research publications to his credit.
Kaido Kikkas is an Associate Professor at Estonian Information Technology College and an Associate
Professor of Social and Free Software at Tallinn University. His main focus of research and teaching has
been a wide range of ethical and social issues of IT (information society, licensing, hacker culture etc).
John Kleeman is the Founder and Chairman of Questionmark. John wrote the first version of the
Questionmark assessment software system and founded Questionmark in 1988 to market, develop and
support it. John has been heavily involved in assessment software development for over 20 years and has
also participated in several standards initiatives including IMS QTI.
John Knight started his career teaching EAP at universities in Turkey and Qatar, but developed an interest
in computers and education in the late 90s and has worked in the field ever since. He now works in the
Learning Development Unit, Bucks New University, where he has a particular interest in technologyenhanced
learning and teaching.
Katerina Kostolanyova has worked in the Faculty of Education, Institute of Information and Communication
Technologies in Ostrava since 1999. She specializes in eLearning technology. Her further professional
growth focuses on students’ learning styles in the e-Learning environment. She is an author and co-author of
almost thirty professional articles and ten e-contents.
Vivek Kumar is an Associate Professor in the School of Computing and Information Systems at Athabasca
University researching in online learning technologies. Vivek's research centres around Technology-
Enhanced Teaching, Learning, and Research that extends to mixed-initiative human-computer interaction,
causal modelling, model tracing, automated instructional design, lifelong learning, cognitive modelling of selfregulated
and co-regulated learning, semantics of online learning interactions, and competency modelling in
portfolios.
Eugenijus Kurilovas is Research Scientist in Vilnius University Institute of Mathematics and Informatics and
Associate Professor in Vilnius Gediminas Technical University. He is a member of 12 scientific committees,
has published more than 50 scientific papers, and participated in more than 10 EU-funded projects. His
papers for ECEL 2008, 2009 and 2010 were published in Electronic Journal of e-Learning.
Linda Leake, M. Ed., serves as a computer specialist, University of Louisville Blackboard technical
support/trainer and instructional designer for the Delphi Center for Teaching and Learning. She received
both her Bachelor of Science in Business Administration degree in 1994 and her Master of Education in
Instructional Technology in 2003 from the University of Louisville.
Jake Leith leads a Business and Professional Practice in Fashion & Textiles and Design & Craft at the
University of Brighton, and has over 20 years’ experience in successfully running his own design practice
and 13 years as a teacher-practitioner. Leith is currently President Elect of the Chartered Society of
Designers.
Birgy Lorenz is an eSafety trainer in Estonia. Her activities include being part of developing National
Curricula ICT syllabus, writing articles about e-safety, project management in TurvaLan project what has
recently been awarded by: Microsoft (2009): Innovative Teacher Award and European Schoolnet (2010): 1 st
eLearning Award, in ‘Internet Safety’ category.
Beulah Maas is a lecturer in social and health care leadership and management in the Faculty of Health and
Life Sciences at Coventry University. She is an occupational psychologist and her key areas of teaching and
research are developing and managing people and social entrepreneurship.
Linda Martin is a senior lecturer and qualified social worker working in the Faculty of Health and Life
Sciences at Coventry University. She teaches leadership and management in health and social care,
including programmes which are entirely on line and has a particular interest in developing approaches to
student engagement.
Marie Martin is an education consultant in Northern Ireland specialising in e-learning, and is adjunct faculty
of Duquesne University, Pittsburgh, USA, where she earned her doctorate in education in 2007. She has
presented internationally and authored several publications on aspects of e-learning. In 2010 she was
Rooney International Visiting Scholar in Robert Morris University, Pittsburgh.
xviii

David Mathew works at the Centre for Learning Excellence at the University of Bedfordshire. In addition to
his work as a Learning Technologist, working cross-faculty on online learning programmes, he co-edits The
Journal of Pedagogic Development and teaches writing/publishing. He is also interested in psychoanalysis
and creative writing, and published a novel in 2011.
Peps Mccrea is a Senior Lecturer in Initial Teacher Education at the University of Brighton. He is interested
in pedagogy and the role of the teacher in our increasingly digital age.
Peter Mikulecky is a professor of Managerial Informatics at the Faculty of Informatics and Management at
the University of Hradec Kralove, Czech Republic, since 1993. He has been the head of the Department of
Information Technologies since 1994; recently he acts also as Director for Research and Director of
Postgraduate Studies.
Jonathan Moizer is a lecturer in business operations and strategy. His research interests include the use of
computerised simulation for both decision-making and learning. He has particular interest in the use of
system dynamics modeling as a simulation approach. In recent years Jonathan has organized three national
workshops on using simulation games in teaching and learning for the Higher Education Academy.
Kate Mottram works at Coventry University as a Graduate Teaching Assistant. Her research interests
include general management and the theories surrounding quality in business organisational settings along
with management in engineering.
Peter Mozelius is working as a researcher and IT-Pedagogue at the Department of Computer and Systems
Sciences, Stockholm University, Sweden. His research interests are in the fields of ICT4D, e-learning,
software engineering and software visualization. During the last six years he has been working in aid projects
in developing countries in southern Asia.
Antoinette Muntjewerff is an Assistant Professor in General Legal Theory at the University of Amsterdam.
Studied Educational Science and Law. PhD research involved theoretical and empirical studies into legal
case solving and evaluation of an instructional environment for learning to solve legal cases PROSA. Her
research is on modelling legal knowledge and legal reasoning for developing electronic materials for learning
the law.
Phelim Murnion is a Senior Lecturer in Information Systems in the School of Business at Galway-Mayo
Institute of Technology; lecturing on business information systems, data warehousing, and knowledge
management. His research interests include Data Warehousing, Business Intelligence and Educational
Technology. Phelim is a member of the Business Informatics (research) Group at Dublin City University.
Chrissi Nerantzi is an Academic Developer at the University of Salford. She is involved in the development
and delivery of the Postgraduate Certificate in Academic Practice (PGCAP) and CPD across the university.
Chrissi’s current professional interests are social media for learning, creative teaching and learning in HE,
online Problem-Based Learning, Open Educational Practice and mobile learning.
Michaela Noakes is a Doctoral candidate and Research Assistant in Instructional Technology in the School
of Education at Duquesne University. She received her MS-ISM, MBA and M. A. from Duquesne. She also
is an Adjunct Faculty member at Duquesne University, Point Park University and Butler County Community
College. She was recently inducted into Pi Lambda Theta, the International Honor Society for Educators.
Abel Nyamapfene received his Ph.D. in Computing from the University of Surrey, Guildford, UK in 2006,
and his MSc. in Communication Engineering and B.Sc. (Hon) in Electrical Engineering from the University of
Zimbabwe in 1990 and 1997 respectively. He is currently the Assistant Director for Education (Engineering)
at the University of Exeter, UK.
Maruff Akinwale Oladejo is a faculty in the Department of Educational Foundations, Federal College of
Education (Special), Oyo State, Nigeria. He is a member of the Editorial Advisory Board, JournalsBank
Publishing Inc. (UK, Pakistan, India and West Africa).
Fatemeh Orooji is a Ph.D. Candidate in the Department of Computer Engineering, Software Engineering
group, at the University of Tehran, Iran. She is interested in web- based educational systems (WBES) and
technology-enhanced learning. Her research activities focuses on Collaborative and Networked Learning,
Educational Data Mining, Adaptive and Personalized Learning.
xix

Kanari Paraskevi is a five year experienced school teacher of French Language in Greece with a master’s
degree in French Literature and is a PhD student in the same field. She has carried out several cultural
programs promoting education through art and theatrical research. She speaks four languages and has
published various articles concerning French Literature.
Lyn Pemberton is Reader in Human Computer Interaction at the University of Brighton. Lyn has been
involved in numerous learning technology projects, mostly concerned with aspects of communication, writing
and language learning. Her most recent projects have involved interactive television for learning, augmented
reality and in particular mobile language learning.
Katie Piatt has been working in the field of Learning Technologies for 10 years, with experience in
development, implementation and evaluation. Her research focuses on methods of student engagement.
Current projects include investigating the integration of social media with traditional environments and the
introduction of game-based learning principles as part of the ALT special interest group.
Andy Pulman works for the School of Health and Social Care (HSC) at Bournemouth University, where
he manages and creates web resources and e-learning initiatives. He began studying for a part-time PhD in
2008, concerning the role of Web 2.0 and Mobile technology to support people living with chronic illness and
enhance quality of life.
Thomas Putz is responsible for the initiation, the application and the coordination of national and
international projects and acts as the contact point with funding organizations in the evolaris. He has been
involved as coordinator and as specialist in eLearning and mLearning in many projects funded under Adapt,
Grundtvig, Socrates, FP6, eContent+ and the LLP.
Ricardo Queirós is an assistant professor at ESEIG/IPP which is responsible for Programming Languages
courses. He is a Ph.D. student in the Faculty of Sciences of the University of Porto. His scientific activity is
mostly related with e-Learning standards and Interoperability. He is member of CRACS/ INESC-Porto and a
founding member of KMILT research group.
Zahra Rahimi is a Master of Science Student in the Philosophy of Science group, at Amir Kabir University of
Technology. She received a Bachelor degree in Computer Engineering- Software, from University of Tehran.
She is interested in interdisciplinary topics, subjects such as use of software in education and philosophy of
intellectual intelligence.
Yambu Andrik Rampun recently completed his M.Sc Software Engineering (Department of Computer
Science) at the University of Hertfordshire. He previously received his Diploma in Computer Science and
B.Sc (Hons) degree Software Engineering from University Technology Mara (Department of Computer
Science and Mathematics). He is interested in designing, development and evaluation of e-learning
applications as well as human computer interactions.
Fauziah Redzuan is a junior lecturer at the University of Technology MARA, Malaysia. She holds a Master
of Science in Information Technology from the University of Science Malaysia in the area of multimedia
learning. Currently she is pursuing her study in the area of emotion and online learning using the Kansei
Engineering technique.
Bart Rienties is lecturer at the Centre for Educational and Academic Development at University of Surrey.
His primary research interests are focussed on computer-supported collaborative learning and the role of the
teacher to design effective blended and online learning courses.
Manuel Rodrigues teaches Computer Science at Secondary School Martins Sarmento, Portugal, where he
is responsible for the IT infrastructure of the school. He has a Computer Science degree and an MsC, both
from Minho University, Portugal. Currently he’s a PhD Student at Vigo University, Spain, and working on
recommendation and personalization systems for E-learning platforms such as Moodle.
Andrée Roy is an engineer and professor of Management Information Systems at the Université de
Moncton. Ms Roy is also an e-learning consultant and the owner of a business specialising in the field of elearning
called Web Training Solutions. Previously, Ms Roy worked as an industrial engineer and in the
information technology and economic development sectors.
Zuzana Šaffková is a senior lecturer at the Department of English, Technical University of Liberec, Czech
Republic, where she specializes in the teaching of academic reading and writing to teacher trainees.
xx

Research interests include the investigation of the effectiveness of e-learning in developing student
academic capability, especially critical reading competence.
Ahmed Salem is an Assistant Professor at the faculty of Engineering, King Abdul Aziz University at KAU. He
graduated from CMU, PA, USA and he is interested in Active Learning, E-Learning and Excellence in
engineering teaching. His contributions include a Design Helping Mechanism for Active Learning Courses
that aid in designing learning experiences.
Rowena Santiago is a tenured professor (emeritus) in Instructional Technology and founding/former director
of the Teaching Resource Center, California State University, San Bernardino. Her research activities
include learner characteristics in e-learning (US and Japan), the use of faculty learning communities for
faculty development, and the use of student response systems for active learning.
Vitor Santos is an invited Professor at Trás os Montes e Alto Douro University (UTAD), Minho University
(UM) and Lusofona University (ULHT), teaching Information Systems, Compilers, and Artificial Intelligence
courses in Computer Science and Informatics Engineering Degrees. Before that, he was the Microsoft
Portugal Academic Computer Science Program Manager for 8 years.
Marie Sams has worked in higher education and the NHS in a number of roles over the past fifteen years.
This has included areas predominantly in marketing management and project management. Her research
interests are in team creativity and peer learning.
Maggi Savin-Baden is Professor of Higher Education Research at Coventry University, UK, and is director
of the Learning Innovation Research Group. She has been researching learning for over 20 years and has
just completed her 11th book. In her spare time she runs and tries out extreme sports
Adriana Schiopoiu Burlea is Professor PhD at the University of Craiova and visiting professor at
universities in France and Poland. She is Romanian ambassador for AGRH. She has written more than 20
books and 130 articles in the field of Management. She is a member of the editorial boards of many
prestigious journals and conferences.
Fabio Serenelli is about to conclude the PhD course in "Information Communication Technology applied to
knowledge society and learning processes" at the Milano Bicocca University where he collaborates as e-
Learning Manager and Instructional Designer at the CPM (Multimedia Production Center). He adopts an
interdisciplinary approach and his research interests are related to instructional formats for multimedia
learning, infographics and User Experience Design, One to One Computing, PLE - Personal Learning
Environment.
Angela Shapiro MSc, Dip Ad Ed, TQFE has been a Lecturer at Glasgow Caledonian University for ten
years. She teaches across all academic schools applying an advanced academic literacies (AAL) approach.
This includes workshops across the university within the context of particular programmes, as well as
distance and blended learning support to students.
Cees Smit Sibinga is professor of International Development of Transfusion Medicine, University of
Groningen. He founded the Academic Institute for International Development of Transfusion Medicine
(IDTM), focusing on higher education in developing countries. Objective - to support leadership development.
The first post-graduate course designed and implemented is a Masters in Management of Transfusion
Medicine (MMTM).
Dina Soeiro. Besides being a trainer of college teachers on E-learning, Dina supports and studies the use of
Moodle at the College of Education of the Polytechnic Institute of Coimbra where she teaches. She is
finalizing her PhD on ICT in Education at the University of Coimbra and she is a researcher at the CISUC.
Mekala Soosay is a university teaching fellow and Technology-Enhanced Learning (TEL) team member with
a passion for preparing and disseminating creative methods for assessment, learning and teaching. She
also trains colleagues in applying TEL for their teaching practice. She has worked on a number of JISC and
EU-funded projects, the current ones being PC3 and EuroPlot.
Sonia Sousa holds a PhD in Education from Sheffield Hallam University, UK and an honors degree in
Communication Engineering from Universidade Fernando Pessoa, Portugal. She is currently researching the
influence of trust in online communities. R&D work includes Michigan State University's MIND Lab, USA,
Universidade Fernando Pessoa Multimedia Research Center, Portugal, as well as Cape Verde and
Mozambique.
xxi

Iain Stewart has worked at Glasgow Caledonian University for more years than he cares to remember. He is
primarily interested in the ways technology can be used to improve the learning experience of students. His
other activities include working to improve FE to HE articulation and critical testing of the elasticity of
deadlines.
Caroline Stockman works at the Catholic University of Leuven as a lecturing assistant for the course of
'Online Publishing', a key subject within the Master's degree of Cultural Studies. Her time is divided between
Belgium and the UK, where she trains teaching staff in secondary and higher education in the use of
educational technology.
Amanda Sykes (PhD, DipAcPrac, FHEA) works with students of Science, Engineering and Veterinary
Medicine at The University of Glasgow to enhance their learning. She is particularly interested in using
emerging technologies to shift the focus from didactic lecturing towards scholarship that enables students to
understand their own learning process as well as academic content.
Fattaneh Taghiyareh is an Assistant Professor of Computer Engineering- Software & Information
Technology, at the University of Tehran, where she has served since 2001. She received a Ph.D. in
Computer Engineering from the Tokyo Institute of Technology in 2000. Her research interests include
Human-Centered Computing applied to Learning Management Systems and based on Multi-Agent Systems.
Nicolet Theunissen is senior research scientist ‘Training Innovations’ at TNO, The Netherlands. Her
research is dedicated to the development and testing of training solutions for professionals. These learning
solutions imply both innovative learning technology and didactics for adult learning. Formerly, she worked at
Utrecht University, Leiden University and at the Netherlands Institute of Health Services Research.
Vladimir Tomberg works as a researcher in the Centre for Educational Technology, Tallinn University,
Estonia. His main research topics are interoperability of online assessment tools and course coordination
within blog-based Personal Learning Environments. He has stayed recently as a visiting researcher in Simon
Fraser University at Vancouver, Canada and in RWTH at Aachen, Germany.
Panos Vlachopoulos is Lecturer in the Centre for Learning Innovation and Professional Practice at Aston
University. He leads the Postgraduate Certificate in Professional Practice in Higher Education and advises
Aston University staff on the development of fully online distance learning programmes. His research
interests include the online facilitation of student-centred learning and reflective practice.
Chien-hwa Wang is a professor in the Department of Graphic Arts and Communications at National Taiwan
Normal University. His recent research emphasis is Web 2.0 applications in Education. He also directed a
government supported digital archiving project to preserve Taiwanese puppetry master Tien-lu Li’s historic
works and collections .
Cristina Wanzeller is professor at the Informatics Department, Technology and Management School, Viseu
Polytechnic Institute (IPV), Portugal. Her teaching activities focus on Information Systems and Databases
areas. She is a member of the IPV’s research centre CI&DETS. Her main scientific areas of research are
Web Usage Mining, Case-Based Reasoning, e-Learning and Information Systems.
Julie Watson is Principal Teaching Fellow in eLearning in Modern Languages at the University of
Southampton. She manages eLanguages which produces and licenses the EAP and Study Skills Toolkits.
She also led the development of the Prepare for Success website, works on projects involving emergent
technologies, and develops online courses and resources.
Phil Watten is from the University of Sussex, has been researching and developing learning-based
technology and production methods since 2005. Using the Sussex-based Media Technology Lab, he has
produced studio-based and portable production systems and is now working on fully automated approaches.
Katherine Wimpenny, PhD, MSc, DipCOT, Cert Ed, is a Research Fellow at Coventry University within the
Learning Innovation Research Group. She is involved in researching the impact of virtual worlds on teaching
and learning. She has recently gained funding to conduct two qualitative research syntheses, enjoys
supervising PhD students and professional background is occupational therapy.
Koos Winnips is advisor in ICT and Education, with a particular interest in digital pedagogies: sorting out
learning arrangements with the help of the electronic learning environment. He has worked at the University
of Twente in the Netherlands and Glasgow Caledonian University as researcher, lecturer and advisor of ICT
in Education.
xxii

Novita Yulianti is a PhD student at the Faculty of Business Studies and Economics, University of Bremen,
Germany. She received a Master of Science degree on International Technology Transfer Management. Her
current research interests include change management, organizational management, information and
technology management, innovation management.
Li-Zhang Zhang received BA from Huazhong University of Science and Technology, China, MSc from the
University of London, UK, and PhD from the University of Sydney, Australia. He currently teaches
International Business and Economics at La Trobe University, Australia. His research interests include
Asian-Australian Business and Trade, E-Commerce, Theory and Practice in English-Chinese Translation,
and Online Education and Training.
xxiii

eNOSHA and Moodle – the Integration of two eLearning
Systems
Peter Mozelius 1 , Isuru Balasooriya 2 and Enosha Hettiarachchi 2
1
Stockholm University, Department of Computer and Systems Sciences,
Sweden
2
University of Colombo School of Computing, Sri Lanka
mozelius@dsv.su.se
irb@ucsc.cmb.ac.lk
eno@ucsc.cmb.ac.lk
Abstract: eNOSHA is an open source Learning Object Repository (LOR) developed at the University of Colombo
School of Computing (UCSC) in a collaboration between UCSC in Sri Lanka, and two universities from Sweden
during 2009 and 2010. eNOSHA is a system where content developers and instructional designers can store and
reuse learning objects on 4 different aggregation levels. The system was built based on a need analysis at UCSC
in late 2008 and was taken into use at the eLearning Centre at UCSC in early 2010. The system has so far been
successful and supported the organisation and reuse of elearning content at the UCSC eLearning Centre (eLC).
However, there still exist several reasons for further improvements when it comes to usability and userfriendliness.
Moodle is one of the most popular open-source Course Management Systems (CMS) and has been
used in the daily work at UCSC during the last 5 years. Moodle is an effective system for building courses and
structure course material but features for storing, retrieving and version handling of learning objects is still under
construction in the Moodle community. Persons working with course development need a LOR as well as a CMS
in their daily work but to be forced to multiple logins and switching between systems is not good usability or userfriendly.
This paper is about the integration between the eNOSHA system and the Moodle system and how it best
should be done. Should the eNOSHA system be connected and integrated as a Moodle module in collaboration
with the Moodle developing community or is it a better idea to build a module in eNOSHA that handles the
communication with Moodle? From a developer’s perspective, the building of a Moodle module to handle the
connection to the eNOSHA system would be a fast and convenient alternative since the Moodle module template
provided by moodle.org could be used as a skeleton for an integration of the additional functionality. However,
the Moodle community did not like the idea of integrating the eNOSHA LOR as an additional Moodle module and
the main reason is that they have other plans for storage of learning objects in the version 2.0 of Moodle that is
expected to be released in September 2010. After some more communication with the Moodle community we
decided to choose the other alternative and construct the integration as a part of the eNOSHA system. The first
testing of the system integration at the UCSC has so far given us positive feedback and this extension will be
included in the coming version 1.6 of the eNOSHA Learning Object Repository.
Keywords: learning object repository, system integration, eNOSHA, Moodle, content management system, open
source
1. Introduction
The history of elearning and distance education in Sri Lanka started with the establishment of
University of Colombo School of Computing in September 2002 This new department was started to
meet the challenges of the 21th century. During the last years the traditional face to face courses in
Computer Science and Software Engineering has been transformed into the net based and interactive
elearning distance programme eBIT (Mozelius and Hettiarachchi, 2010). The elearning Center at
UCSC creates and publishes a large amount of learning material for its internal and external degree
programs. The eNOSHA learning object repository was built to address the need for a consistent and
stable repository to store and retrieve all that content. But since Moodle was already an established
learning management system in the institute, the need to combine Moodle with eNOSHA arose.
Before the integration between Moodle and eNOSHA, both systems worked individually. Users who
are in need of learning objects or resources have to search the required content from the eNOSHA
system and download them to their own local machine which is later uploaded to the Moodle system.
Therefore they have to maintain two separate logins to access both system and this was not possible
through a single system. Addition to that they also have to keep a separate place in their own local
machine to store the content. When the first version of eNOSHA was released some of the
universities also mentioned the usefulness of combing the LOR with the Moodle CMS. All these
reasons were taken into consideration and the development of the integration was started. The
eNOSHA-Moodle system integration should enable Moodle users, especially teachers in courses to
import learning objects from available eNOSHA instances locally or through the web. A refined
509

Peter Mozelius et al.
searching and advanced searching facilities will help the user to better locate and reuse content that
are already available. Users can import content either as resources or as SCORM packages, after
searching and previewing the learning objects. The content will be imported with the title that is
assigned to it in the eNOSHA system, which they have the freedom to change. After filling the
required additional information about the learning object users can add it to the actual course they are
working with.
1.1 Aim of paper
The aim of this paper is to describe and analyse the integration between the eNOSHA Learning
Object Repository and the Moodle Content Management System.
2. Method
During the development phase user tests have been conducted to improve how the integration of the
systems from the users perspective. Unit-tests were also done to test the quality of the code and the
technical functionality. The systems integration was carried out as a common project between UCSC
in Sri Lanka and the Stockholm University (SU) in Sweden in the Sida-funded NeLC project. The
method used for the project is action research (Cohen et al, 2010) and a focus group with staff from
both universities has been consulted to optimize the design. As a method for the software
development we have used Scrum (Schwaber and Beedle, 2001). Scrum is an efficient agile method
for rapid system development in small groups of 5 to 10 persons.
3. The eNOSHA system
eNOSHA is a free and open source LOR developed at the University of Colombo School of
Computing in a collaboration between UCSC in Sri Lanka, and two universities from Sweden during
2009 and 2010. A LOR can be defined as a storage and search system for digital learning objects
with support for sharing and reusing the digital assets. The more the amount of learning objects
grows, the need for quality LORs to sift the information increases as well. LORs can be categorized
into:
Content repositories: All learning objects are stored on accessible servers
Linking repositories: Portals with links to content provided by others
Hybrid repositories: A combination of 1 and 2. (McGreal, 2008)
eNOSHA is, until version 1.6, a Type 3 LOR where content developers and instructional designers
can store and reuse learning objects on 4 different aggregation levels. In the UCSC customized
version of the system learning objects are divided into 4 granularity levels:
Atom : Basic level for digital assets like text, images, and sound files
Collection of atoms : A combination of atoms, like XHTML documents with JavaScript or Flash
applications
Course module: A section/module/part of a course containing atoms and collections of atoms
Full course : A full course with all the course modules for the specific course included
At other universities or organizations the granularity levels can be specified in other ways depending
on actual curricula and they can be redefined in the eNOSHA admin module. (Mozelius and
Hettiarachchi, 2010)
A LOR needs a metadata set as the base for searching after stored learning objects. Metadata can be
defined as data about data and the metadata set in the eNOSHA system is based on the Learning
Object Metadata model (LOM) standard. Since the LOM standard has many metadata fields in
common with the SCORM content packaging standard. At UCSC it was a natural choice to choose
LOM when all existing courses in the UCSC programmes exclusively use digital content following the
SCORM standard. (Hatakka and Mozelius, 2009) In Moodle SCORM is used as a course format as
well as an activity. SCORM packages in Moodle can send information about students’ activities and
test scores. (Cole and Foster, 2008)
The eNOSHA system consists of several modules such as upload, search, user management,
administration, help, error handling and statistics. Most important parts of a LOR are the search and
upload modules.
510

Peter Mozelius et al.
The Search module is divided into two parts, Search and Advanced Search. In the basic search users
can search for resources by selecting a specific category or by typing in a word or a phrase. In
advanced search users can search for all fields in the metadata set. Results can be filtered based on
authors, keywords, category, title or description.
The Upload module consist of functionalities to upload content based on four aggregation levels -
atoms, collection of atoms, course modules and full courses.
File upload : The user can upload material after filling in some metadata fields (mandatory and
optional) This function was enhanced with user templates and bulk uploading. A file upload in
eNOSHA is shown below in Figure1.
Figure 1: File Upload in eNOSHA
3.1 Some features of the system
Search (Simple and advanced): A simple search was implemented for fast access to frequently
used standard material. The advanced search was implemented to enable a more detailed search
for resources. If a search returns in a huge amount of learning objects the result set can later be
filtered on various criteria.
Administration: The administration module is broken down in to 2 separate modules, system
administration and user administration.
Commenting: Users are able to comment on the learning objects uploaded to the system.
User Templates: When a user uploads several files with almost the same metadata they can
create a user template and use it to upload several objects.
Bulk Upload: Upload of similar files with metadata shared in a template. Only the unique file
specific metadata has to be filled in.
Help: A module for glossary and tutorials on how to use the system, showing sample data
uploads and search along with other administrative tasks.
511

Peter Mozelius et al.
Error Handling: A module for storing the source of errors and details including originating module
and error messages
Statistics: A module for statistic reporting of the system, including the most rated or downloaded
materials, active users, most searched keywords, most uploaded category etc.
The eNOSHA version described above will with its 7 core modules be distributed and made public
under a GNU/GPL licence as version 1.5 of the eNOSHA LOR
Figure 2: eNOSHA ver 1.5 core modules
4. The Moodle system
Moodle is an open source CMS that has been designed in a process with an open dialogue between
teachers and developers, where different models for learning design and pedagogy has been
discussed (Berggren et al, 2005) The name has 2 meaning, a verb for a lazy meandering through the
system with creativity and insight, and an acronym for Modular Object-Oriented Dynamic Learning
Environment. In the early development phase of the CMS the main architect was Martin Dougiamas
at University of Perth in Australia and his main idea was to construct a CMS as an educational
process and not as an engineering process. Today Dougiamas works full time with the Moodle
system together with a developing community with members from all over the world. (Cole and Foster,
2008)
512

Peter Mozelius et al.
For the moment, (August 2010), there are 53160 active sites using Moodle that have registered from
215 different countries. (Moodle.org, 2010)
The Moodle Tracker, is the development communities database for recording and managing bugs,
improvements and feature requests for Moodle. This is the forum for suggesting new features and
modules in the Moodle system and since 2006 there has been discussion on how to implement an eportfolio
for sharing resources in the CMS and currently it is the Mahara e-portfolio that is the
candidate for integration in the Moodle system. This integration is sometimes called Mahoodle and
the added features to Moodle are, except from the e-portfolio, blogging tools and functionality for
creating online learner communities. (Mahara.org, 2010)
Figure 3: A UCSC course in Moodle
5. The integration
Using eNOSHA as a separate standalone repository for adding and reusing resources in the daily
work with course development in Moodle has been criticized by content developers and instructional
designers at the UCSC eLearning Centre. In the rapid and rational process of course and content
development at the eLearning Centre the procedures with dual login and shifting between the
systems workspaces was an extra burden that interfered in their daily work. Since eNOSHA is built
as a standalone repository and there are a lot of reusable components already uploaded and ready
for production use. Some kind of linking of the systems proved to be the ideal solution and an userfriendly
integration has been on the wishlist of the eNOSHA focus group during the last year.
An integration of the systems could be implemented in two ways:
The intersystem communication could be built as a Moodle module
The intersystem communication could be built as a eNOSHA module
In a developer’s perspective, the first alternative would be very convenient and facilitated by the
template provided by moodle.org, NEWMODULE. It is practically a skeleton for a module in which the
developers can insert the new extended functionality. However, the Moodle community did not like the
idea of integrating the eNOSHA LOR as an additional Moodle module and the main reason is that
they had other plans for storage of learning objects in the version 2.0 of Moodle that was released in
late 2010. After some further communication with the Moodle community we understood that there
513

Peter Mozelius et al.
were no chances of hIsuru Baving the eNOSHA system as a Moodle module even if there were no
explicit criticism of the eNOSHA functionality. We decided then to choose the second alternative and
construct the system integration as a part of the eNOSHA system.
In Moodle 2.0 it is possible to link to several external repositories but there is still no dedicated and
tailor made learning object repository integrated in the system. Questions about the earlier promises
about built-in repository in Moodle 2.0 have been asked in the Moodle discussion fora. No clear
answers have been given from the Moodle community so far (July 2011), but there are requests from
Moodle users and for organizations practicing large scale elearning some kind of repository is a must.
(Using Moodle, 2011) The earlier promises about a repository in Moodle 2.x has been grounded on
the extension and integration of the Mahara e-portfolio. Moodle users are hoping that this feature will
be implemented in the Moodle ver 2.2 that should be released in the end of 2011. But despite the
earlier discussions there is still no guarantee that this functionality will be developed. (Mahara
Community, 2011)
Not all universities and organizations have yet upgraded to Moodle 2.0 and all the three universities
that are involved in the further development are still using Moodle 1.9. Stockholm University and the
UCSC in Sri Lanka are planning to upgrade during late 2011 or early 2012 but no exact decision is yet
taken. The eNOSHA integration has been working well in Moodle 1.9 as well as in Moodle 2.0. During
the autumn of 2011 the integration with Moodle ver 2.0 will be further tested and fine tuned tested at
the Department of Computer and System Sciences at Stockholm University in Sweden when
eNOSHA ver 1.6 will be revised and updated to ver 1.7.
Since both the systems are built with a XAMP architecture the eNOSHA Moodle module was also
built on the NEWMODULE but customized according to eNOSHA repository structure. The XAMP
acronym should be interpreted as:
X for any platform
A for the Apache web server
M for the MySQL database
P for the PHP programming language
As an example of XAMP the eNOSHA LOR system is tested on the Windows and Linux platforms. In
addition to the main PHP programming language, JavaScript and Ajax have been used on the client
side. All data and metadata in eNOSHA are stored in a MySQL relational database and the LOR
system has a localization scheme based on language files.
In a focus group analysis 3 basic needs were identified for the new eNOSHA Moodle module:
Login/Authentication
Search for learning objects
Import of learning objects
With these new functionalities handled outside the Moodle core a user logged into the Moodle system
will be able to search for and import learning objects stored in the eNOSHA LOR. This new
functionality is built by using remote procedure calls, where the eNOSHA repository acts as a server,
and the eNOSHA Moodle module as the requesting client. eNOSHA’s Moodle module connects to the
eNOSHA repository via XML-RPC linking, where the requests and results are XML based. XML-RPC
consists of a set of implementations that allow software running in different environments to make
remote procedure calls over the Internet.
The remote procedure calls are using the HTTP protocol for the transport and XML in the encoding.
XML-RPC is designed to be as simple as possible, but can also transmit and process more complex
data structures. (XML-RPC, 2010) Parameters of user actions/selections are passed on and executed
in the remote repository and then transported back to be displayed in Moodle. A PHP script in the
eNOSHA repository handles the function calls and returns the results.
514

Figure 4: Searching in eNOSHA in Moodle
Peter Mozelius et al.
Figure 5: Inter-system communication by remote procedure calls
A main consideration in the integration of the 2 systems was the flexibility of multiple repositories. A
user can select between external eNOSHA installations, specified by the administrator. This requires
the user to have an account in the remote repositories. Once the user have logged on to a selected
repository, they can search for learning objects of interest and preview them, and if desired, import
them to the Moodle system. Once logged in, the same repository can be accessed without re-logging
in whilst the same Moodle user is still logged in and users will always have the option of logging out of
a remote repository to login to another.
Almost all the resource types stored in eNOSHA will be classified as resources to a Moodle course,
except for SCORM packages. They will be considered as SCORM/AICC Packages in Moodle terms.
The user will automatically be redirected according to the file type they have chosen to import.
6. Discussion and conclusions
If both the described integration alternatives had been possible we would have chosen to develop the
intersystem connection as a Moodle module. But life is sometimes about to optimize Plan B when
Plan A no longer is an option. We think that we have constructed a working solution that would
facilitate the daily work with digital content for instructional designers and content developers at the
eLearning Centre at UCS in Sri Lanka. To have the possibility to work with multiple systems in a
515

Peter Mozelius et al.
single login will speed up the development process for the UCSC staff as well as increase the general
usability of the eNOSHA system. Since both the systems’ metadata sets are based on the LOM
standard importing and exporting learning objects between the systems seems to work well but the
graphical user interface in the integrated features might need amendments.
The described eNOSHA-Moodle integration has been tested since July 2010 at the UCSC in Sri
Lanka and so far without any discovered problems or major bugs. As the next step we like to
evaluate the eNOSHA system online in the Finnish TUP tool (TUP Online, 2010). eNOSHA was
installed in May 2011 at the Department of Computer Science at the University of Eastern Finland
(UEF) in Joensuu, Finland. At this department the Moodle system has been used since many years
as the main platform for elearning. Since computer scientists in Joensuu have a long tradition of
usability testing we think that the collaboration with the Science Park in Joensuu will be a great
opportunity for further improvement of the new eNOSHA module as well as the older existing core
modules in the system. During the 2011 autumn semester the eNOSHA system will be used as the
main repository in the VisSCoS programme for distance education in Computer Science (Suhonen
and Sutinen, 2007) at the UEF in Joensuu. To get a more complete evaluation of the search
functionality in the Moodle system as well as the general usability of the system’s core functionality,
the collaboration with Joensuu and The University of Eastern Finland offers a great opportunity for
further improvement.
7. Future work
The eNOSHA- Moodle integration has so far only been tested in detail by the UCSC in Sri Lanka and
the UEF in Finland. It would be of great importance to install and test the system at more universities
and educational organizations. Moodle is just one of the many Content Management Systems and
Virtual Learning Environments used in current elearning. If eNOSHA in the future should be a global
LOR alternative there is a need for integration with other similar systems like Sakai and Blackboard.
References
Berggren, A, Burgos, D et al. (2005) "Practical and Pedagogical Issues for Teacher Adoption of IMS Learning
Design Standards in Moodle LMS", Journal of Interactive Media in Education, 2005(02)
Cohen, L, Lawrence M, and Morrison K. (2010) “Research Methods in Education”, Routledge, England,
Cole, J, and Foster, H. (2008) " Using Moodle: teaching with the popular open source course management
system”, 2nd Edition, 2008. O'Reilly Media, Inc.
Hatakka, M., and Mozelius, P. (2009) "A model for a Learning Object Repository Metadata Set – A Case Study at
UCSC, Sri Lanka”, eASIA, Colombo, Sri Lanka 2009.
McGreal, R. (2008) "A Typology of Learning Object Repositories", Springer Berlin Heidelberg, 2008, pp 5-28.
Mahara Community (2011) “Mahara/Moodle integration - Mahara repository for Moodle 2.x”
http://mahara.org/interaction/forum/topic.php?id=3835 (retrieved 31/07/2011)
Mahara.org: (2010) “Open Source e-portfolio” http://mahara.org/ (retrieved 16/08/2010)
Moodle.org: (2010) “Registred sites” http://moodle.org/sites/ (retrieved 16/08/2010)
Mozelius, P., and Heitiarachchi, E. (2010) "eNOSHA, a Free, Open and Flexible Learning Object Repository”,
eIndia, Hyderabad, India 2010.
Schwaber, K., and Beedle, M. (2001) “Agile software development with Scrum” Prentice Hall PTR Upper Saddle
River, NJ, USA, 2001.
Suhonen J., and Sutinen E. (2007) “Learning Computer Science over the Web: The VisSCoS Odyssey” Idea
Group Inc, 2007
TUP Online: (2010) http://cs.joensuu.fi/~tup/ (retrieved 22/11/2010)
Using Moodle (2011) “The Mahara repository plugin” http://moodle.org/mod/forum/discuss.php?d=156003
(retrieved 31/07/2011)
XML-RPC Home Page: http://www.xmlrpc.com/ (retrieved 22/11/2010)
516

CASE Learning to Structure and Analyze a Legal Decision
Antoinette Muntjewerff
University of Amsterdam, Faculty of Law, Amsterdam, The Netherlands
muntjewerff@uva.nl
Abstract: Legal practitioners and legal scientists need to have knowledge of the general rules that apply in the
legal system. This involves both knowledge of the legislation and knowledge of the decisions by judges that
function as general rules of law. Law students preparing themselves for the legal profession need to acquire
these kinds of knowledge. A student has to have knowledge about where to look for decisions, understand the
structure of decisions and learn to determine what makes a decision relevant to the body of applicable rules in
the legal system. Legal education primarily aims at acquiring insight in the legal sources, their history and
background. This basic knowledge is of great importance; legal problem solving is hardly possible without an
understanding of the legal knowledge. To illustrate the use of this knowledge in practice, teachers work through
decisions as examples. However, it is difficult, if not impossible, to learn by explanation or by imitation alone. A
more effective way to obtain expertise is by actually performing the task, i.e. students should do the exercises,
while the teacher provides feedback on their solutions. For effective learning, also the solution process should be
monitored and provided with feedback. Furthermore it is desirable for students to be able to ask for help at any
time during the process. They should also be able to practice over and over again. An ideal situation would have
a teacher available for every student, monitoring the student while practicing and providing support where and
whenever necessary. However, this being not practically feasible, the second best option is to offer the student
electronic support. CASE (Case Analysis and Structuring Environment) is an environment where a law student
can practice with finding decisions, with structuring its text and with analysing the decision in order to be able to
determine in what way it adds to the body of applicable rules in the legal system. CASE is developed using a
principled and structured design approach. A short description of this approach is followed by an analysis of the
learning task, the difficulties law students experience and the remedies proposed on the basis of both the task
analysis and the stated difficulties. This is followed by a description of architecture, functionality, platform and
implementation of CASE and a description of a session with CASE and future work.
Keywords: instructional design, coaching systems, legal problem solving
1. Introduction
The law that applies in a legal system such as the Dutch legal system consists of general rules that
are determined or acknowledged by authoritative bodies. The two most important authoritative bodies
within the Dutch legal system are the legislator and the judge. While it is obvious the legislator
determines rules that apply in general, this is more complicated with judges. A judge has to decide in
individual cases, she has to construct a legal solution based on the facts of the case and the
applicable legal rules. In the majority of cases that come before the court, a judge formulates a
decision that applies only to the case at hand. These decisions do not add to the body of applicable
rules in the legal system. However, in cases where a judge first has to construct an applicable rule,
before being able to decide the case on the basis of this rule, we have a different type of decision.
The rule constructed by the judge to decide the case, may add to the body of applicable rules in the
legal system. Legal practitioners and legal scientists need to have knowledge of the general rules that
apply in the legal system. This involves both knowledge of the legislation and knowledge of the
decisions by judges that function as general rules of law. Law students preparing themselves for the
legal profession also need to acquire knowledge about the role of decisions by judges in the legal
system, and they need to understand the two categories of decisions by judges. A student has to
have knowledge about where to look for decisions of the second category, understand the structure of
decisions and learn to determine what makes a decision relevant to the body of applicable rules in the
legal system. Legal education primarily aims at acquiring insight in the legal sources, their history and
background. This basic knowledge is of great importance; legal problem solving is hardly possible
without an understanding of the legal knowledge. To illustrate the use of this knowledge in practice,
teachers work through decisions as examples. However, it is difficult, if not impossible, to learn by
explanation or by imitation alone. A more effective way to obtain expertise (skill) is by actually
performing the task, i.e. students should do the exercises, while the teacher provides feedback on
their solutions. Not only feedback on the solution provided by students is important.
For effective learning, also the solution process should be monitored and provided with feedback.
Furthermore it is desirable for students to be able to ask for help at any time during the process. They
should also be able to practice over and over again. An ideal situation would have a teacher available
517

Antoinette Muntjewerff
for every student, monitoring the student while practicing and providing support where and whenever
necessary. However, this being not practically feasible, the second best option is to offer the student
electronic support. Using a computer program as the instructional medium has a number of
advantages. It may offer individualized instruction and practice combined with immediate support and
feedback. It can have the capacity to adapt to the individual student’s performance and, last but not
least, may support the management of information.
CASE (Case Analysis and Structuring Environment) is an environment where a law student can
practice with finding decisions, with structuring its text and with analysing the decision in order to be
able to determine in what way it adds to the body of applicable rules in the legal system.
These functionalities are implemented in two integrated modules in CASE:
A module to compile and store decisions
In essence a database containing a selection of decisions used in legal education. The law student
can do a search (key word and/or full text) for a specific decision or a set of decisions. Decisions can
be added to the database and key words can be indicated for each decision by the teacher. This
module can be used separately or in combination with the second module.
A module to structure and analyse decisions
In essence an instructional environment for learning to structure and analyze a decision to determine
how it adds to the body of applicable rules in the legal system. This module builds on the first module.
It presents the student the text of a selected decision together with a framework containing the main
elements in a decision text (as, for instance, the different parties and their roles in the various stages
of their procedures before the different courts). It allows the student to fill the framework with the
relevant parts from the text of the decision. The activities of the student are monitored and compared
to a model where deviations are diagnosed to be able to present the student with a hint or a
remediation.
CASE is developed using the principled and structured design approach as described in the HYPATIA
project (Muntjewerff 2002a, Muntjewerff 2002b). A short description of this approach is followed by an
analysis of the learning task, the difficulties law students experience and the remedies proposed on
the basis of both the task analysis and the stated difficulties. This is followed by a description of
architecture, functionality, platform and implementation of CASE and a description of a session with
CASE and future work.
2. Principled and structured design approach
The HYPATIA project (Muntjewerff 2002a, Muntjewerff 2002b) aims at designing and developing new
additional electronic materials for law students to learn the law. Law students experience difficulties in
acquiring legal knowledge and in using this knowledge. These problems are acknowledged by law
teachers. However, there is no material available to help students to overcome these difficulties.
HYPATIA aims to fill this gap developing electronic to offer individualized instruction and practice by
adapting to the individual student's performance combined with immediate support and feedback.
Electronic tools may also support the management of information and present different
representations and visualizations of legal knowledge and legal tasks. The principled and structured
design approach guides the development process in such a way that difficulties and mistakes
encountered during the design process may be accounted for. The design process involves two
interrelated research streams: basic research and applied research.
Basic research is concerned with developing well-founded models of legal knowledge and skills to be
learned by law students, examining the difficulties they have with acquiring legal knowledge and legal
skills and finding remedies to enhance effective and efficient learning of the required knowledge and
skills. In the applied research part, basic research findings are used to construct computer supported
models of legal knowledge and legal reasoning to diagnose and remedy the specific difficulties of law
students in learning the law. Instructional design decisions are made on the basis of a global theory
on learning and instruction. In this way the design process will result in a coherent and consistent
instructional model. It finally indicates that electronic materials are evaluated extensively
518

Antoinette Muntjewerff
(developmental testing and field testing). The design approach was founded and used successfully in
the construction of an instructional environment for learning to solve legal cases: PROSA.
The approach taken in PROSA is reusable for a variety of applications for learning the law. The legal
case solving research within HYPATIA has been realized and reported in detail (see, for instance,
Muntjewerff 2000, Muntjewerff and Groothuismink 1999, Muntjewerff 2002c).
3. Analysis
What is structuring and analyzing a decision? In order to answer this question and to design an
environment to support law students in finding, reading, structuring and analyzing decisions to
indicate and understand the legal meaning of a decision, it is necessary to analyze the task.
The HYPATIA design approach starts with (re)constructing explicit models of legal knowledge and
legal reasoning. In this (re) construction process, two components are distinguished. (1) A theoretical
component of exploration, conceptualization and specification of legal knowledge and legal reasoning
resulting in explicit models of legal knowledge and legal reasoning. Two perspectives are taken within
this approach: a legal perspective and a knowledge engineering perspective.
From the legal perspective different legal sources, being legal empirical research, legal educational
practice, legal dogmatic and legal theoretical research, are examined to specify the required models.
The knowledge engineering perspective is used to construct models at a high level of explicitness as
they have to be executed by a computer (see, for example, Haan den and Sartor 1999, Breuker and
Van de Velde 1994, Valente 1995). This explicitness of models is exactly what is needed in
instruction. (2) An empirical component where empirical studies are carried out to acquire insight in
the way legal practitioners and legal scientists handle legal knowledge in general and in carrying out
specific legal tasks. In this case, law students are studied to see how they handle and use legal
knowledge to perform a specific legal task and what difficulties they experience.
The legal sources that were examined to model the task of reading and comprehending decisions all
describe a series of steps to be taken by the student when reading a decision to determine the legal
significance (Bos 2003, Eemeren van et.al. 1996, Franken et.al. 1991, Henket and Hoven van den
1999, Jansen 1999, Scholten 1974, Soeteman and Wolthuis 2003, Stolwijk and Bosch 2002).
However, merely instructing a method does not work for novices (see for details Muntjewerff 2000).
This is partly due to the fact that instructing a method is a problem in itself, as it is difficult to
communicate a method, because this requires the translation of actions into words.
A method is in fact empty; explaining content is much more “substantial” and therefore easier. The
somewhat paradoxical situation is that novices have to learn to determine the legal meaning by
determining the legal meaning. Law students especially have difficulties with determining what the
decision adds to the body of applicable rules in the legal system. Based on findings in research in
legal problem solving it is stated that the difficulties are first of all caused by insufficient mastery of, or
insight in, the subject matter. Secondly, especially for novices, methods, often as a side effect,
emerge from (novice) problem solving, instead of being the driving force. The subject matter appears
to be the major source for finding or trying (a) solution (steps). On closer inspection, a decision is a
legal solution for a specific problem situation constructed on the basis of abstract legal rules.
Structuring and analyzing a decision is in fact the task of reconstructing the problem situation
(consisting of a reconstruction of both the facts and the legal question), tracing the abstract legal rules
that were applied and specifying the legal solution consisting of the argument structure and the
conclusion.
Reading and understanding a decision is not a trivial activity. Observations with first year law students
reading decisions showed that they experience difficulties with seeing through the composition of the
decision, with reconstructing the argument structure and with determining the legal significance of the
decision.
These difficulties are first of all caused by the fact that a decision is an incomplete reproduction of
what happened. Next to that the text of the decision contains many references, both explicit and
implicit, to regulations, other decisions and concepts. The fact that a decision has a stratified structure
which is also not supported by recognizable clues or elements in the text does not help either.
519

Antoinette Muntjewerff
All of this means that the student has to reconstruct the process and the product which involves
keeping track of intermediate results. To support the student in performing these tasks, the following
remedies are proposed. Present the student a structure to help her to reconstruct the decision,
support the management of information and engage the student in structuring and analyzing the
decision by having her actually carry out these tasks. This is realized by presenting the student with
both the full text of the decision and a framework which visualizes the elements in a decision
necessary to reconstruct the decision in order to determine the legal significance of the decision.
There are no applications available that support law students in structuring and analyzing a decision
suiting the Continental legal system. For the Anglo-American legal system, the CATO application is
available (Aleven 1997). In CATO the student is trained to construct arguments with cases.
4. Architecture and modular design
The aim of the CASE project is to realize an environment in which law students are supported in
structuring and analyzing a decision. This means that both the decision at hand has to be presented
to the student, as well as the framework for analysis. The student must be able to select text
fragments from the decision and paste these within the correct cell in the relevant table in the
framework. Since finding cases is also part of the training of law students search facilities have to be
available in the environment. The functionality of searching for a decision is implemented in the first
module. The functionality of structuring and analyzing a decision is implemented in the second
module. Other basic requirements are maintenance and re-use. It should be possible to make
changes to the system and its content without much costs and efforts. Errors in system and content
should be easily traceable and correctable. It must be possible to add and delete content without
causing problems elsewhere in the system. Transparency of the architecture and tools are therefore
design goals, as it may facilitate maintenance.
The system has functions for adding decisions, adding key words to decisions and preparing
decisions for analysis. System functionalities are attributed to a user on the basis of her status:
administrator, editor, teacher or student. The database module holds the decisions and allows for
search and retrieval of cases and allows teachers to prepare cases for use in the analysis module.
Students can use the database module to locate cases on the basis of key words and/or full text
search to find specific decisions. When the student wants to structure and analyze a decision she can
select one of the reported decisions. This decision and the analyzing framework are then made
available to the student. The student can start structuring the decision by selecting text fragments in
the decision and pasting these in the correct part of the frame.
5. Platform and implementation
CASE is implemented using a web-based server-side application model. The user interacts with the
system using a standard web browser, such as Netscape Navigator, Apple Safari or MS Internet
Explorer. CASE is developed using Open Source Software, MySQL (4.0.14) and PHP (4.3.2) and
JavaScript. The MySQL database backend contains a number of tables, the most prominent ones
being a text fragment table, a solution table and a table storing the student's activities. CASE’s
primary component is the server-side application implemented in PHP (4.3.2). This application
handles form processing, storage and retrieval of information from the various tables in the database
and generating the HTML pages that are output to the user.
A small number of simple functions are implemented using client-side JavaScripts. CASE offers
extensive support for administrative-, editing-, browsing-, tracking- and educational tasks. Using the
same portal, administrators can add, remove and change users and cases; editors can add keywords
to cases and prepare the solution framework of a case for use; teachers can use the interface to track
the results of students, previewing the solution framework and for browsing and searching the
database; and students can browse and search the database, and test their analysis skills.
The search engine allows for both Boolean keyword- and free text search in combination with
metadata fields such as: date, name, court etc. The principal concept in CASE is that a precedent can
be seen as an ordered set of text fragments, each of which can be labeled according to their place in
the solution template. The student can select a text fragment and place in a specific position within
the solution framework. Text fragments can be as short as a single sentence, but more often, they are
as long as a paragraph. The text fragments are stored in a database along with metadata such as a
reference to their position in the solution. Although a text fragment as described is the basic building
520

Antoinette Muntjewerff
block, these fragments can have one or more sub-fragments (such as single words) which can also
be selected by the student. For instance, the text fragment
“Op het beroep van Ronald G, geboren te Amsterdam op 6 aug. 1954, wonende te
Amsterdam, req. van cassatie tegen een bij verstek gewezen arrest van het Hof te
Amsterdam van 12 dec. 1977, waarbij in hoger beroep een vonnis van de Rb.”,
contains the sub-fragment 'Ronald G', the accused. In some cases the student needs to select the
whole sentence, and in others only the sub fragment. The solution framework consists of a number of
tables, such as parties, facts, claim and the argument structure before the Supreme Court. Each table
is two dimensional and contains a small number of cells, e.g. facts as presented by the initiator, and
facts presented by the opponent.
Each cell in the solution, therefore, can be designated by three coordinates: table, row and column.
These coordinates are used to mark the proper location of text fragments within the solution
framework.
They allow the student's solution to be tested against the solution defined by the teacher; the cell in
which the student places the fragment has to match the metadata reference of the text fragment. In
the case of an incorrect placement of a fragment, its position relative to the correct place is also
known. This allows for standardized responses to common errors. For instance, when a student puts
the initiator’s name in the opponent’s cell, the following response can be generated on the basis of
this mixing up of the parties in the dispute: "This indeed is one of the parties in the dispute, but
unfortunately it is not the opponent.”. To get a basic idea of the functionality of the system we now
describe a session with CASE.
6. A session with CASE
As mentioned above, CASE distinguishes four types of user: administrators, editors, teachers and
students. User rights are distributed in an incremental fashion in CASE, this means that a teacher has
access to both student- and teaching facilities; an editor has access to editing-, teaching- and student
facilities; and the administrator user has rights to do everything the other users can, plus adding,
removing and changing users, and removing cases from the database. This section describes a
typical process from preparation to analysis of a case.
After login, the editor is presented with a menu containing multiple options. Since the editor recently
came upon a decision relevant for law students, she decides to add it to the CASE database. The
editor’s menu gives access to the add decision screen.
Here she fills in a few facts about the decision (name, publication date, court etc.) and with copy-
paste actions she adds the text of the decision to the database. Next, she visits the metadata editor
(see figure 1).
The metadata editor interface is used to add or change metadata of a decision and, more importantly,
to add new keywords, or remove existing ones. After completing this procedure, the decision can be
searched for using the search interface.
The next step is the preparation of the decision for use. The prepare tool offers an interface that
mimics the regular structuring and analysis interface: the editor needs to place pieces of text in the
correct position within the solution framework (see figure 2).
Where the regular interface checks whether the correct text is in the correct position by consulting the
database, the prepare tool writes the action of the editor to the database. The editor in a sense
teaches the solution of the case at hand. Note that the editor does not have to add feedback to the
database. Feedback is provided to the student in a case-independent way. When the teacher only
wants part of the text fragment to be part of the solution, the editor can simply mark these smaller
parts. This results in a text fragment with color coded sub fragments that can be placed in the solution
table (e.g. Mr Jean-Gustave Funke in figure 3). After the editor has finished the above steps, the
decision is ready for use by both teachers and students.
521

Figure 1: Metadata editor
Antoinette Muntjewerff
The teacher is not allowed to change the information or the solution framework of a decision.
However, he can add students to the CASE user database, and preview the correct answers (the
prepared solution framework) for each decision. More importantly, the teacher has access to a
student tracking facility to analyze student behavior.
This way the teacher can determine whether a student came to his or her end-result by simply trying
every option, or by purposefully placing fragments in the solution framework.
Students can search the decision database using the search interface (see figure 3). This interface
allows for metadata search – i.e. on publication date, publication place, court type, court location – but
also supports Boolean keyword search and Boolean full text search. The student can also browse
through all decisions in the database. The search result page offers support for associative search
because key words and other attributes of the cases found are shown. The student can click on any
of these to start a search on this attribute. Thus, for example, searching on all decisions with the same
keyword of one of the decisions that were found by the original search is done by simply clicking on
that keyword in the results page. From the same page, the student can print a decision or open it for
structuring and analysis.
522

Figure 2: The prepare tool
Figure 3: Search the database
Antoinette Muntjewerff
523

Antoinette Muntjewerff
The structuring and analysis interface, shown in figure 4, is divided into three frames. The left frame
shows all text fragments of the decision at hand. The top right frame contains the tables of the
solution framework. The bottom right frame provides feedback to the student’s actions. A text
fragment is placed in a cell of the solution table by first selecting the cell, and then selecting the
fragment to fill this cell. Once placed, the application will check the combination of cell and fragment
and provide a feedback message from the database in the feedback frame. Text fragments can be
removed from a cell by clicking the ‘x’-button in the table. Once the student has placed all correct
fragments in a specific table, she is notified of this through the feedback frame.
Figure 4: Structuring and analyzing a decision
7. Summary and future work
Learning the law involves reading, structuring and analyzing decisions to be able to indicate the legal
significance of the decision. Law students experience difficulties especially with determining what the
decision adds to the body of applicable rules in the legal system. Within the current curriculum there is
not enough time to read and analyze decisions in the presence of a teacher who may provide
immediate feedback. Law students are also not presented with models that may guide them in the
process of reading and analyzing decisions. In learning the law it is essential to know how to structure
and analyze a decision.
CASE was designed to present the law student with an instructional environment in which she is able
to analyze a decision in such a way that the structure is made explicit and the legal meaning can be
extracted. CASE is implemented as a web-based server-side application model using open source
software. CASE is easy to maintain and re-use and can be made available in different languages.
Future work involves testing the effectiveness of CASE. The claim that law students are supported by
CASE in structuring and analyzing a decision in such a way that they are able to grasp the legal
significance of the decision should be tested. The claim that it is easy to add a decision, to add key
words and to prepare a decision for use should also be tested.
Acknowledgements
The CASE project was partially funded by the Association of Universities in the Netherlands (VSNU).
References
Aleven, V. (1997) Teaching Case-Based Argumentation Through a Model and Examples, Pittsburgh: University
of Pittsburg.
Ashley, K.D. and Brüninghaus, S. (2009) “Automatically Classifying Case Texts and Predicting Outcomes”,
Artificial Intelligence and Law Vol. 17, No. 2, pp. 125-165.
524

Antoinette Muntjewerff
Breuker, J.A. and Van de Velde, W. (1994) CommonKADS library for expertise modeling. Reusable problem
solving components. Amsterdam: IOS Press.
Valente, A. (1995) Legal Knowledge Engineering. Amsterdam: IOS Press.
Bos, A.M. (2003) “Jurisprudentieleidraad”, Encyclopedie der Rechtsgeleerdheid Vaardigheden. Amsterdam:
Universiteit van Amsterdam.
Eemeren van, F.H., Feteris, E.T., Grootendorst, R., Haaften van, T., Harder den, W, Kloosterhuis, H. and Plug,
J. (1996) Argumenteren voor juristen. Oefenboek. Grondingen: Wolters Noordhoff. (Argumentation for Legal
practitioners)
Ericsson, K.A., Charness, N., Feltovic, P.J. and Hoffman, R.R. (Eds.) (2006) The Cambridge Handbook of
Expertise and Expert Performance. New York: Cambridge University Press.
Franken, H. (1991) Practicum Methoden en Technieken II. Casusoplossen. Bundel ter dienste van het Practicum
Methoden en Technieken van de faculteit der rechtsgeleerdheid van de Rijksuniversiteit te Leiden. Arnhem:
Gouda Quint. (Solving Legal Cases)
Haan den, N. and Sartor, G. (1999) “Model-based Legal Knowledge Engineering” Gaines, B.R. (Ed.).
International Journal of Human-Computer Studies. Special Issue Model-based Legal Knowledge
Engineering. Academic Press, pp. 1037 - 1042.
Hage, J.C. (2009) “Statische en dynamische rechtsfeiten.“, Ars Aequi: juridisch studentenblad, 58, 91-98.
Henket, M. and P. van den Hoven (1999) Juridische Vaardigheden in argumentatief verband. Groningen: Wolters
Noordhoff. (Legal Skills)
Jansen, L. (1999) Nieuwe introductie in het recht. Utrecht: LEMMA. (Introduction to Law)
Keukens, W.T.M. and Nieuwenhuijzen van den M.C.A. (2008) Raad & Daad. Over de rechtsvormende taak van
de Hoge Raad. Ars Aequi Libri, Nijmegen. (The Task of Judges)
Lubbers, A.O. (2007) Belastingarresten lezen en analyseren. SDU. (Reading and Analyzing Tax decisions)
Muntjewerff, A.J. and Groothuismink, J. (1999b) “Supporting the Learning of Legal Case Solving using a
Computer Program as the Instructional Environment”, Proceedings CATE’99 Computers and Advanced
Technology in Education. Philadelphia, USA: IASTED.
Muntjewerff, A.J. (2000) An Instructional Environment for Learning to Solve Legal Cases. PROSA. Amsterdam:
University of Amsterdam.
Muntjewerff, A.J. (2002a) HYPATIA research program. Models of Legal Knowledge and Legal Reasoning in
Electronic Materials for Learning the Law. Amsterdam: University of Amsterdam, ISBN 90-806969-1-9.
Muntjewerff, A.J. (2002b) “Principled and Structured Design of Electronic Materials for Learning the Law”, Bench-
Capon, T.J.M., Daskalopulu, A. and Winkels, R.G.F. (eds.), Legal Knowledge and Information Systems.
Jurix 2002: The Fifteenth Annual Conference. Amsterdam: IOS Press, 2002, pp. 133-142.
Muntjewerff, A.J. (2002c), “Evaluating the Instructional Environment for Learning to Solve Legal Cases PROSA.”,
Torrellas, Gustavo A. and Uskov, Vladimir (eds.), Proceedings of the IASTED International Conference
Computers and Advanced Technology in Education. Anaheim Calgary Zurich: ACTA Press, pp. 374 - 379.
Scholten, P. (1974) Algemeen Deel. Zwolle: W.E.J. Tjeenk Willink. (General Part.On Legal Decisionmaking).
Soeteman, A. and Wolthuis, A.J. (2003) Syllabus Juridische argumentatie 2003. Amsterdam: VU. (Legal
Argumentation)
Stolwijk, S.A.M. and Bosch, M. (2002) “Het bestuderen van arresten in strafzaken”, Arresten
strafrecht/strafprocesrecht met annotaties ten behoeve van het onderwijs verzameld. Deventer: Kluwer.
(Studying Legal Decisions in Criminal Cases)
525

A Framework for Decision Support for Learning
Management Systems
Phelim Murnion 1 and Markus Helfert 2
1
School of Business, Galway-Mayo Institute of Technology, Galway, Ireland
2
School of Computing, Dublin City University, Dublin, Ireland
phelim.murnion@gmit.ie
markus.helfert@computing.dcu.ie
Abstract: Learning Management Systems (LMS) provide a valuable platform for e-learning that offer great
flexibility. However, compared to traditional learning environments they are challenging and complex for decisionmakers,
both teachers and learners. At the same time, LMS environments offer opportunities for analysis by
storing large quantities of data, such as web log files and data about students and content, which are not
generally available in the traditional environment.Motivated by approaches in other domains, such as ecommerce
and clinical management, in this article we propose to relatethe complex decision environment with
the possibilities of using large quantities of data.In this paper we review relevant literature on educational data
mining (EDM) and combining that with a standard data mining methodology we propose a conceptual framework
that appropriately relates the methods of data mining to the settings of teaching and learning in a LMS
environment. In contrast to other frameworks, our conceptual framework enables EDM research to be more
integrated with the task domain. In our framework, teaching and learning activities and the decisions required to
control those activities are addressed by relating the following three elements: pedagogy; learning activities; and
decision-making. The significance of our work is that the framework enables us to compare between different
research studies as well as provide practical guidelines for developing EDM solutions. The framework also
provides a number of further directions for researchers which follow naturally from a decision-centric perspective
and from the full implementation of the contextual phases of the data mining life cycle
Keywords: educational data mining, learning management systems, decision support, programme evaluation
1. Introduction
New technologies for learning are being developed and introduced at a rapid rate. Perhaps the most
commonof these is Learning Management Systems(LMS), also known as Course Management
Systems (CMS) orVirtual Learning Environments (VLE).These systems offer a variety of tools to
enable educators to distribute information to students, produce content material, prepare
assignments, engage in discussions, and to enable collaborative learning with forums, chats, file
storage areas, and news services. LMSs such as Moodle and Blackboard have recently become
ubiquitous in tertiary/higher education, with (citing US statistics) almost 100% of institutions using
LMS technology (Green 2010), and 60% using an approved campus-wide LMS(Pam Arroway 2010).
At the same time, teaching and learning in a LMS environment presents new challenges and
opportunities. The teacher loses some of the advantages of the traditional learning environment and
while the learner gains more freedom to make their own decisions. As a result, the decision-making
environment for both teachers and learners becomes more complex. However, LMS environments
also store large quantities of data, such as web log files and data about students and content, which
are generally not available in the traditional environment. This combination of a complex decisionmaking
environment and large quantities of raw data presents problems but also opportunities for
practitioners (teachers) and researchers. Moreover, researchers in e-learning, particularly in the area
of programme evaluation, have identified the need for more research based on the data in the LMS
(Janossy 2008) to supplement traditional survey-based and experimental methods.
In other problem domains, such as e-commerce and clinical management, this combination of a
complex decision environment and large quantities of data has been addressed by information
systems researchers using data mining and other analytical approaches. Motivated by this
observation, we focus on ‘Data Mining’, which involves the automatic extraction of implicit and
interesting patterns from large data collections (Klosgen 2002). The field of e-Learning, with the large
amounts of usage data automatically stored in the LMS web log files and the difficulty of making
decisions in online learning, is well suited to data mining(Zaïane 2001). This approach, which has
become established as “educational data mining” or EDM (Castro, Vellido et al. 2007), has been
applied to a variety of problems in LMS environments. In data mining, it is an accepted principle that
an understanding of the context or setting in which data mining is deployed is important (Shearer
2000), (Lavrac 2004), (Hofmann and Tierney 2009). In the specific field of EDM several researchers
526

Phelim Murnion and Markus Helfert
have identified the need for a better integration with the teaching and learning context (Gaudioso and
Talavera 2006), (Romero, ventura et al. 2008).
Theobjective of this paper is to address decision support for LMSsby describing a conceptual
framework that enables the integrationof data mining methods with the context or settings of teaching
and learning in a LMS environment. The main contribution is to address a limitation in existing EDM
research with a framework which can guide future research and practice. The remainder of the paper
is structured as follows. In section two, we develop a description of the problem of integrating EDM
with the teaching and learning context.Based on that description, we propose a set of categories for
analysingEDM research. Using these categories we present in section three a detailed analysis of the
EDM literature. Drawing from this analysis, in section four we propose and describe a conceptual
framework for an improved integration of EDM research with the learning context followed by a
discussion and comment on the proposed framework.
2. Related work
In order to develop an appropriate basis for the analysis of the limitations in EDM research,in this
section we review existingwork, first by describing the EDM research field in general and then by
examining relevant related work in data mining methodologies. The general perspective provides us
with an overview of the problem of integrating EDM research with the teaching and learning context.
The work on data mining methodologies allows us to convert that overview into a more detailed
model, which provides the basis for the analysis in section three.
2.1 Educational Data Mining
Data Mining (also known as knowledge discovery from data) is a well-established approach for
extracting patterns from large quantities of data (Berry and Linoff 2004), using a variety of statistical,
machine-learning and other data-mining algorithms, in order to explore and understand the
phenomena underneath the data or to support decision making (Peng, Kou et al. 2008). It has been
applied successfully to a number of scientific and commercial domains (Lavrac 2004). Due to its
origins in data analysis and machine learning, data mining research has tended towards a technical
orientation, focussing on techniques/tasks such as: data clustering, classification, association rule
mining and sequential analysis (Peng, Kou et al. 2008). Educational data mining (EDM) is the
application of the data mining approach to the different types of educational data (Romero and
Ventura 2010), (Baker and Yacef 2009).It has been recognized that the field of eLearning in a LMS
environment is well suited to the data mining approach (Zaïane 2001), due to the two features of large
amounts of raw data(stored in the LMS web log files) and the complexity of the decision making
problems (Peng, Kou et al. 2008).This EDM research approach(Castro, Vellido et al. 2007)has
included investigations of a wide variety of situations in the LMS environment including: evaluating
learner activity (Zaïane 2001), (Muehlenbrock 2005), (Pahl 2006); providing support to educators
(Gaudioso and Talavera 2006) and recommending student actions(Sacin, Agapito et al. 2009). The
above-mentioned technical orientation is reflected in EDM as well, where papers have tended to focus
on the application of these techniques to educational data.
Despite the strongly technical orientation of EDM research there have been a number of
recommendationsfor more work on integrating data mining with the context of teaching and learning.
An early paper in the field describes the concept of “integrated web usage mining” (Zaïane 2001), in
which a data mining system would be integrated into the eLearning system. A review paper in 2007
describes a model in which “data mining in educational systems is an iterative cycle of hypothesis
formation, testing, and refinement. Mined knowledge should enter the loop of the system and guide,
facilitate and enhance learning as a whole. Not only turning data into knowledge, but also filtering
mined knowledge for decision making” (Romero and Ventura 2007). Our paper emphasises this view.
However, the impact of these recommendations on the research work is not clear. Frequently EDM
research has been focussed on the techniques of data mining with little or no reference to the details
of the learning context (Zaïane 2001), (Etchells, Nebot et al. 2006). In contrast some researchers
have based their EDM interventions on a comprehensive model of the learning context using
established pedagogic theory on (for example) learner-content interaction (Pahl 2006), and group
work(Perera, Kay et al. 2009). Of particular significance is the work of Elena Gaudioso and colleagues
(Talavera and Gaudioso 2004),(Gaudioso and Talavera 2006) on collaborative learning, which
describes not only a detailed educational context but a purpose and role for the results of the EDM
527

Phelim Murnion and Markus Helfert
intervention within that context. On this basis we postulate a spectrum of EDM research in terms of
integration with the educational context from studies with little or no integration with the context to
studies which are highly integrated. However, the level of integration is unlikely to be a simple scalar
variable. Any analysis of an ‘integration’ construct within the EDM literature requires a more concrete
version of this construct. Since the relationship of data mining to the domain context is a feature of the
data mining methodology adopted, an examination of data mining methodologies is a necessary next
step.
2.2 Data Mining methodologies
Data mining arose from a number of related computational and statistical approaches, which form a
set of data mining methodologies. The methodologies can have an emphasis on technical aspects
rather than the processes and the relationship to the task domain(Peng, Kou et al. 2008). However,in
order to structure the methodologies,research and practice in data mining has expanded in
perspective to include steps along a data mining process or data mining model (figure 1). This model
can be described as the ‘technical perspective’ on data mining and has been commonly used in EDM
research(Luo 2001), (García E. 2007).
Figure 1: Data Mining Model: Technical perspective
As data mining matured as a discipline and as data mining applications were implemented in a wider
variety of problem domains the technical steps were subsumed into a more comprehensive
methodology known as the data mining cycle (e.g. figure 2).A number of data mining
methodologies/cycles have been developed but the process of developing these methodologies has
exhibited two common features(Hofmann and Tierney 2009): the replacement of a sequence of steps
with a cyclical, iterative process, and a greater focus on the connections between the data mining
process and the underlying problem context.The most widely accepted model, known as the CRISP-
DM (CRoss Industry Standard Process for Data Mining) Cycle(Shearer 2000),exhibits both features
(figure 2).
Figure 2: CRISP-DM cycle (Shearer 2000)
528

Phelim Murnion and Markus Helfert
The first phase, Problem definition, is the start of the cycle. Based on an understanding of the task
domain problem, a data mining problem or hypothesis can be derived. The next four phases (data
exploration, data preparation, modelling, and evaluation) constitute the technical data mining
steps,resulting in a model, knowledge or information which can be deployed. Finally in the
Deployment phase the results of the data mining are used to solve the originally defined problem. For
a complete implementation of the cycle Deployment and Problem definition must be appropriately
connected, as the results of the deployment should be measured and fed back into problem definition
for the next iteration of the cycle (Berry and Linoff 2004). On this basis, the extent of integration with
the task domain consists of: 1. A definition of the problem to be solved; 2. A description of the
deployment method(s); and 3. The proper inter-relationship between problem definition and
deployment and the underling task domain.
2.3 Categorising EDM research
On the basis of the above definitions we propose a maturity model for categorising EDM research into
stages; representing development along an ‘integration’ axis (figure 3).
Figure 3: Integration with the educational context
The stages are incremental and hierarchical, each stage assuming the completion of the previous
stage and representing a higher level of maturity. An EDM intervention at the Problem stage includes
a definition of the problem, but no clear description of the deployment phase. In the Solution stage,
both problem definition and deployment are described. In the Integrated stage both problem definition
and deployment are appropriately related in a model based on the task domain. On this basis, EDM
research can be analysed by categorising each study into one of the three stages.
3. Analysis ofEDM research
The maturity model described in figure 3 provides a set of criteria for analysing the EDM literature.
The model forms the basis for a detailed analysis of research related to EDM. In order to select a
sufficient number of papers, we definerelevantpapers, that they are recent; relate to the correct
learning environment; and address a relevant educational research task. Recency is determined by
selecting from the latest review paper. Environment and task are classifications defined in several of
the existing review papers(Castro, Vellido et al. 2007), (Romero and Ventura 2010).
The selection of the relevant subset of literature follows a threestep process:
1. Identify all review papers in the field:
[1] (Castro, Vellido et al. 2007), [2] (Romero and Ventura 2007), [3] (Baker and Yacef 2009), and [4]
(Romero and Ventura 2010)
2. Select the most recent review, which also has the largest number of papers (300), which is [4]
above.
3. The selected review classifies references according to types of educational environment.
According to the focus of this paper we can dichotomously divide the types into:
Learning Management Systems, which is the focus of this study, (29 papers) and,
All other types (Traditional education, Web-based Education, Intelligent Tutoring Systems;
Adaptive Educational Systems, Tests/Questionnaires, and Texts/Contents)
529

Phelim Murnion and Markus Helfert
4. Using another typology popular in the field, we can categorise papers by educational task(Baker
and Yacef 2009), or educational objective(Romero and Ventura 2010)again using a binary division
consisting of:
Supporting teaching and learning (which is the focus of this study), versus
Supporting theory advancement (student models, domain content models, etc.)
Based on these steps, the twenty-nine papers per criterion 3.a. are sampled to select papers
according to criterion 4.a. This will focus the review on papers relevant to this study: those which
examine support for teaching and learning in a LMS environment.
Figure 4: EDM research; integration with the educational context
Out of the full sample of papers (per criterion 3.a), some are excluded because they do not meet
criterion 4.a or for other practical reasons. Papers excluded are listed in Appendix A
The analysis in figure 4 shows that EDM research is not highly developed in terms of integration with
the educational context. At each stage in the development of the integration (vertical axis) there are
fewer papers. The most notable gap is in the integration stage. Furthermore there is no clear
development over time. On the one hand, all the papers in the Integration stage are in the later years.
On the other, most of the papers in the Problem stage are also in the later years (5 for 2008).
4. Proposed conceptual framework
As our analysis in section three shows, there is a significant gap in EDM research;in the form of an
inadequate model of the task domain of teaching and learning. In order to address this gap, we
propose a conceptual frameworkthat enables EDM research to be more integrated with the task
domain. The basis for that framework is the related work on the data mining cycle, as outlined in
section 4.1 and the refinement of that model for the special case of the task domain, which is teaching
and learning, as outlined in 4.2. Therefore, we propose an educational data mining cycle (figure 5)
which incorporates a conceptual linkage (dashed triangle) to the task domain of teaching and
learning.
530

Phelim Murnion and Markus Helfert
Figure 5: Framework: An educational Data Mining cycle
4.1 Data Mining and the framework
The CRISP-DM data mining model provides a generic template for the application of data mining.
From the data mining model discussed in section twowe can identify two key elements for our
framework:Problem definition, and Deployment. In the data mining model these two phases enable
the integration of the data mining process with the task domain. Any conceptual model must
incorporate these elements, but in a way that is specific to the domain of teaching and learning but
generic across that domain.
The review of the relevant research in section threeprovides evidence to support the importance of
the elements of the data mining cycle identified above.In addition, the description of an integration
maturity model in figure 3 and the analysis of the literature presented in figure 4 suggest that the main
gap in existing research is in the integration of the two elements with each other and with the context
of the task domain. On that basis the elements of the conceptual framework should at least include:
Table 1: Framework elements
Element
Problem definition
Deployment
Task Domain
However,a further step in the construction of the model is to refine the description of the elements in
table one by reference to the special characteristics of teaching and learning.For each of the three
elements above, the model shouldspecify a corresponding definition which is widely applicable across
the domain of teaching and learning. That requires a brief examination of the educational task
domain.
4.2 Educational decision-making and the framework
Existing best practice (as described in the analysis of the literature in section three) presents a basis
for any improved conceptual framework.Of the EDM studies in that analysis (see figure 4), three
papers fall into the most mature stage: Integration.
All three studies provide a context, described in terms of learning theory, which informs, constrains
and relates the data mining phases: problem definition and deployment. The first study (Jovanović,
Duval et al. 2007), describes the context as ‘learning object context’: the interaction between learning
content, student and activities; with the emphasis on the learning content. The second (Wang 2008),
describes the context as the inter-relationship between student, content, and domain knowledge with
the emphasis on the student. The third (Perera, Kay et al. 2009), describes the context as
collaborative learning (incorporating group work theory) with the emphasis on activities. Clearly, each
531

Phelim Murnion and Markus Helfert
study defines the context differently and as a result each has a unique definition of problem definition
and deployment.
However, a feature common to all three studies is their operational orientation. An educational system
clearly is a set of operations (in which content, learner, and task interact) resulting in outcomes. But
decision support approaches like data mining are not designed to support operations. Another
element is required which relates operations to outcomes. This concept is addressed in another EDM
study, (Gaudioso and Talavera 2006), already mentioned in the related work. The context of the study
is collaborative learning;which is described using an appropriate theoretical context. Based on that
context the operations (the problems to be solved) are described in terms of the creation of virtual
communities. The critical difference between this and other EDM studies is the next element. The
authors describe a separate process called adaptive collaborative support. (ACS): the mechanism for
ensuring that the virtual communities operate in a manner which meets the goals of collaborative
learning theory, thus relating operations to outcomes. The data mining intervention is addressed, not
directly at the operations of the collaborative groups, but to provide decision support for the ACS
function.
Motivated by this example, we suggest that a general model for our framework can be extrapolated
from this particular study. Three elements are necessary in an educational system: an underlying
pedagogy (theory); a set of teaching and learning activities (operations) and a related set of control
decisions.It is towards the controls decisions that EDM should be directed. This perspective of
educational decision making and control allows us to extend the model described in table 1 by
providing a definition for each element, resulting in table 2.
Table 2: Framework elements and definitions
Element Definition
Task Domain Learning Theory
Problem definition Teaching & Learning Activities
Deployment Control Decisions
Learning theory constrains the kind of learning and teaching activities that should occur and also
provides the goals/outcomes which decide what control decisions to make. Learning and teaching
activities generate the data which EDM approaches can turn into knowledge for decision-making
(Romero and Ventura 2007).This control systems view has been examined in the educational
literature from a number of different perspectives; learner control (Williams 2001), learning
cybernetics (Liber 2003) and soft systems methods (Warwick 2008).
This specification can be combined with the standard data mining cycle to provide a
specificframework for this task domain, the Educational Data Mining Cycle in figure 5.Using this
model, the deployment phase for EDM consists of providing knowledge fordecision making
processes. The decision making processes are part of systems which control the teaching and
learning activities identified in problem definition.
5. Discussion and concluding remarks
The proposed framework makes a number of contributions. Firstly, the framework addresses the
thesis raised at the start of this paper; providing a way to integratethe methods of data mining withthe
context of teaching and learning in a LMS environment. Secondly, it provides a bridge between EDM
approaches and general educational theories. This enables EDM research to increase the impact on
and relationship with other areas of educational technology research, a problem identified in an earlier
review paper(Baker and Yacef 2009).The framework also provides a reference model for constructing
further EDM interventions (based on the dashed triangle in the framework diagram, figure 5)that EDM
designers can use to support the implementation of the data mining cycle in education.Finally the
framework provides further directions for researchers that follow from a decision-centric and control
systems perspective. For example, EDM research has tended to focus on gathering knowledge to
direct attention to a problem. In other problem domains, decision support tools are considered more
useful when deployed at later steps in decision making such as when devising or evaluating
solutions(March and Hevner 2007).
532

Phelim Murnion and Markus Helfert
A limitation of the framework is that it is a perspective, a way of examining EDM research, which
requires further validation. In further research we aim to apply this framework within a live learning
situation. This will provide further insight into the validity of the framework and its utility in enabling
decision support in a LMS environment.
6. Appendix A
Table 3: Papers excluded from literature review (section three)
References
Paper Notes
(Baruque, Longo et al. 2007) Portuguese (Brasil)
(Castro, Vellido et al. 2005) Spanish
(Liu and Shih 2007) Does not meet criterion 4a
(Matsuda, N. et al. 2007) Wrong category (ITS)
Baker, R. and K. Yacef (2009). "The State of Educational Data Mining in 2009: A Review and Future Visions."
Journal of Educational Data Mining1(1): 3-17.
Baruque, C. B., C. J. Longo, et al. (2007). Analysing users' access logs in Moodle to improve e learning.
Proceedings of the 2007 Euro American conference on Telematics and information systems. Faro, Portugal,
ACM.
Berry, M. J. A. and G. S. Linoff (2004). Data Mining Techniques : For Marketing, Sales, and Customer
Relationship Management. Hoboken, NJ, USA, John Wiley & Sons, Incorporated.
Castro, F., A. Vellido, et al. (2005). Detecting atypical student behaviour on an e-learning system. Proc. Simposio
Nacional de Tecnologıas de la Informacion y las Comunicaciones en la Educacion, Granada, Spain.
Castro, F., A. Vellido, et al. (2007). Applying Data Mining Techniques to e-Learning Problems. Evolution of
Teaching and Learning Paradigms in Intelligent Environment. J. Kacprzyk, Springer Berlin Heidelberg. 62:
183-221.
Etchells, T. A., A. Nebot, et al. (2006). Learning what is important: Feature selection and rule extraction in a
virtual course. European Symposium on Artificial Neural Networks, Bruseles, Belgium.
García E., R. C., Ventura S., Calders T. (2007). Drawbacks and solutions of applying association rule mining in
learning management systems. International Workshop on Applying Data Mining in e-Learning 2007.
Gaudioso, E. and L. Talavera (2006). Data mining to support tutoring in virtual learning communities: experiences
and challenges. Data Mining in E-Learning. C. R. M. a. S. VENTURA. Cordoba, WITPress.
Green, K. C. (2010). The 2010 Campus Computing Survey. Campus Computing Survey.
Hofmann, M. and B. Tierney (2009). An Enhanced Data Mining Life Cycle. IEEE Symposium on Computational
Intelligence and Data Mining, CIDM 2009, Nashville, TN, USA, IEEE.
Janossy, J. H., T. (2008). Proposed Model for Evaluating C/LMS Faculty Usage in Higher Education Institutions.
Society for Information Technology & Teacher Education International Conference, Las Vegas, Nevada,
USA.
Jovanović, J., E. Duval, et al. (2007). LOCO-Analyst: A Tool for Raising Teachers’ Awareness in Online Learning
Environments. Creating New Learning Experiences on a Global Scale, Springer Berlin / Heidelberg. 4753:
112-126.
Klosgen, W., & Zytkow, J. (2002). Handbook of data mining and knowledge discovery. New York, Oxford
University Press.
Lavrac, N., Motoda, H., Fawcett, T., Holte, R., Langley, P. & Adriaans, P., (2004). "Lessons learned from data
mining applications and collaborative problem solving." Machine Learning57(1-2): 13-34.
Liber, O. (2003). "Cybernetics, eLearning and the education system." International Journal of Learning
Technology1(1): 127-140.
Liu, F.-j. and B.-j. Shih (2007). Learning Activity-Based E-Learning Material Recommendation System.
Multimedia Workshops, 2007. ISMW '07. Ninth IEEE International Symposium on.
Luo, O. R. Z. a. J. (2001). Towards Evaluating Learners’ Behaviour in a Web-Based Distance Learning
Environment. Proc. IEEE International Conference on Advanced
Learning Technologies (ICALT 2001),, Madison, WI,USA.
March, S. T. and A. R. Hevner (2007). "Integrated decision support systems: A data warehousing perspective."
Decision Support Systems43: 1031-1043.
Matsuda, C. N., et al. (2007). Predicting students performance with SimStudent that learns cognitive skills from
observation. International conference on Artificial Intelligence in Education, Amsterdan, Netherlands.
Muehlenbrock, M. (2005). Automatic Action Analysis in an Interactive Learning Environment. Workshop on
Usage Analysis in Learning Systems at AIED-2005, Amsterdam.
Pahl, C. (2006). Data Mining for the Analysis of Content Interaction in Web-based Learning and Training
Systems. Data Mining in E-Learning, WIT Press: 41-56.
Pam Arroway, E. D., Guangning Xu, Dan Updegrove (2010). EDUCAUSE Core Data Service Fiscal Year 2009
Summary Report. EDUCAUSE Core Data Service.
533

Phelim Murnion and Markus Helfert
Peng, Y., G. Kou, et al. (2008). "A Descriptive Framework for the field Of Data Mining And Knowledge
Discovery." International Journal of Information Technology & Decision Making07(04).
Perera, D., J. Kay, et al. (2009). "Clustering and Sequential Pattern Mining of Online Collaborative Learning
Data." IEEE Trans. on Knowl. and Data Eng.21(6): 759-772.
Romero, C. and S. Ventura (2007). "Educational data mining: A survey from 1995 to 2005." Expert Systems with
Applications(33): 135-146.
Romero, C. and S. Ventura (2010). "Educational Data Mining: A Review of the State of the Art." IEEE
TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND
REVIEWS40(6): 601-618.
Romero, C., S. ventura, et al. (2008). Data mining algorithms to classify students. Int. Conf. Educ. Data Mining,
Montreal, Canada.
Sacin, C. V., J. B. Agapito, et al. (2009). Recommendation in Higher Education Using Data Mining Techniques.
2nd International Conference on Educational Data Mining (EDM'09), Cordoba,Spain.
Shearer, C. (2000). "The CRISP-DM model: The new blueprint for data mining." Journal of Data
Warehousing5(4).
Talavera, L. and E. Gaudioso (2004). Mining Student Data To Characterize Similar Behavior Groups In
Unstructured Collaboration Spaces Workshop Artif. Intell. CSCL. Valencia, Spain.
Wang, F.-H. (2008). "Content Recommendation Based on Education-Contextualized Browsing Events for Web-
Based Personalized Learning." Educational Technology & Society11(4): 94-112.
Warwick, J. (2008). "A Case Study Using Soft Systems Methodology in the Evolution of a Mathematics Module."
TMME5(2&3): 269-290.
Williams, M. D. (2001). Learner-Control and Instructional Technologies Handbook of Research for Educational
Communications and Technology. D. H. Jonassen, AECT.
Zaïane, O., & Luo, J (2001). Web usage mining for a better web-based learning environment. Proceedings of
conference on advanced technology for education, Proceedings of conference on advanced technology for
education.
534

Learning for Life - Building Blocks to Holistic Education
Shekhar Murthy 1 and Devi Murthy 2
1 IIC Academy, Visakhapatnam, India
2 Johnson Grammar School, Hyderabad, India
shekhar@iictechnologies.com
devi_murthy@hotmail.com
Abstract: Robert Byrne philosophically stated, “The purpose of life is a life of purpose.” The paradox of modern
educational approaches lies in the acquisition of the means to life, rather than a life with meaning. It’s no wonder
that despite material successes, modern society finds itself in restless disquiet. While mental happiness has been
found to correlate with materialistic and spiritual happiness, studies also indicate that materialistic well-being
alone may not lead to spiritual happiness (Bhattacharya 2010). The learning principles and their pedagogical
connotations have been extensively researched in literature, reported, and applied to suit learning styles and
methods. Bloom’s taxonomy (Bierly, Kessler, and Christensen 2000) and Kirkpatrick’s model (Bates 2004) have
been used with reasonable success in building and evaluating learning strategies. Yet, while the focus has
remained anchored on doing things right, at times, one is left reflecting, “Are we doing the right things?” The
existence of a skills-gap between an academically produced intake and industry expectations has become a
cliché. Lack of adequate professional and life skills are impacting harmony in teams, creating mistrust in
intentions, affecting work-life balance, and above all, shaking ethical foundations in society. The moot question to
address is, “does our educational system build good human beings?” Has an overdose of material pursuits
diverted us from the wisdom of good living and holistic fulfillment? This paper addresses lifelong learning within a
holistic framework. The authors propose specific lifelong learning constructs ranging from pre-school education to
retirement from active work-life. Beyond the normal precincts of workplace competencies, the authors explore
life-skills that are needed to make societal pursuits worthwhile. Through an internet-based survey, the authors
identify key life-skill-gaps that grapple with and divert a modern life from purposeful living, and suggest methods
by which such learning content could be embedded into educational systems. The paper favors community-linked
group-based learning and suggests methods to adapt current learning approaches and technologies to build
holistic life-competencies for the millennial generation (for purposes of this paper persons’ born after 1980).
Keywords: lifelong learning, learning approaches, skill-gaps, life-skills, wisdom-gap
1. Introduction
“Wisdom is not a product of schooling, but the lifelong attempt to acquire it.” — Einstein
Education is the cornerstone of the knowledge-based society. More so, when the quandaries of
hectic-paced life impact on harmony in society and the competitive pressures aggravate healththreatening
stresses. Life satisfaction index in fact, has been found to positively correlate with the
educational development index (Bhattacharya 2010).
Maslow’s model of his identified “hierarchy of needs” introduced the concept of “B-values” that of
truth, justice, order, goodness, beauty, simplicity and unity (Maslow 1971). Most of the emphasis in
current educational approaches seems to equip learners to meet the physiological and safety needs;
with perhaps a sprinkling of belongingness and esteem needs. The layers that deal with the need to
understand, self-actualization and transcendence remain relatively untouched (Martin and Joomis
2007).
In terms of Blooms taxonomy, the affective domain is increasingly receiving attention in recent times.
This is evidenced from work on the “measurement options of positive psychology” (Lopez and Snyder
2003); use of “emotional intelligence” (Goleman 1995) instruments in hiring and training and recently,
self-development efforts through “Mojo” (Goldsmith 2010).
In a metaphysical sense, the universe is an expansive university which compels us to obtain an
integrated view of life. The universe may not confer degrees; but through life’s challenging personal
experiences, builds in us a strong will-power, enhances our ability to discriminate between right and
wrong, and live life with courage, self-confidence and sacrifice. Modern educational systems can act
as natural extensions of the universe to foster harmony amongst the “living kingdom,’ and
environment, and fill our lives with an ambition that surpasses personal achievements for societal
good. Knowledge, skill and attitudes required for professional excellence and materialistic growth
could be developed on the substratum of positive human virtues. Holistic education must therefore
involve a “harmonious blend of the outer-world and the inner-world” (Bhatta 2007). The ancient Hindu
535

Shekhar Murthy and Devi Murthy
wisdom declared holistic education through a three dimensional framework as shown in Figure 1
where, region-1 belongs to the realm of mind and body; region-2 embraces the heart; and region-3
addresses the soul. The role of heart in the education process reflects on cultural bearings, values,
feelings, and spiritual practices (Young 2003).
In recent times, efforts towards Integral Theory (Wilber 2006) on education, embracing spirituality and
“new ways knowing” into education (edited by Awbrey, Dana, Miller, Robinson, Ryan, and Scott 2006)
and many other studies using integrative approaches have been explored, tried, and reported. These
along with progressive pedagogies such as, constructivism (von Glasersfeld 1989), learning by doing
(Dewey 1916), transformative education (Mezirow 1991), lifelong learning (Fischer 1998), multiple
intelligences (Gardner 1983), and ethical classroom among many others have been purposefully
collaged to build effective learning ecologies.
The holistic framework in Figure 1 provides direction to an educational approach. However,
operationalizing a learning structure around it requires well-defined learning outcomes and
performance measures. Holistic education manifests through building of appropriate capabilities,
skills, behaviors, and values consistently demonstrated by the learner, both as an individual, and as a
team member (Hare 2006).
Ethics - Discrimination
Feelings - Justice
2
3
1
Knowledge & Skill
Building; Maslow’s
hygiene factors
Wisdom Selfactualization
Higher
purpose of life
Figure 1: Holistic education (Upanishad, Sacred Hindu Scripture)
The competencies for lifelong and holistic learning have been discussed in numerous papers, notably
in the faculty handbook (Duncan-Hewitt, Leise and Hall 2005), secular spiritual education (Bigger
2008), and holistic education (Arguelles, McCraty and Rees 2003). Figure 2 maps ‘the educational
grid’ where the components of holistic lifelong education viz.: academic and technical competencies;
behavioral and team skills; creative skills; professional skills; emotion management; physical and
mental well-being; cultivating values; 21 st century skills, and spiritual well-being have been captured. It
is by no means suggested here that the entire ambit of competencies and skills fall under the purview
of formal education settings; instead, what indeed is intended is to emphasize that formal education
can become an enabler to develop these competency-set, within, and outside of educational
campuses.
The authors also believe that a significant paradigm shift must occur in modern education focusing on
the ‘joy of living’ as its ultimate outcome. Only a joyous-being can trigger subsidiary outcomes such as
positivity in thought, endearing creativity, richness in thought-word-deed, abundance in generosity,
productivity at work, effective team player, passionate learners and workers, and excellence in
536

Shekhar Murthy and Devi Murthy
application of knowledge, skills, and competencies for the universal good. True education ought to be
transformative.
21 st Century Skills Physical and mental well-being Values and beliefs
Global Awareness
Information and media literacy
Environmental responsibility
Living in uncertainty
Social networking
Business, economic and
financial literacy
Body awareness and hygiene
Yoga and physical exercises
Healthy dietary habits
Healthy lifestyle
Meditation
Positive thoughts
Identifying values
Expanding identity
Aligning with social values
Exploring beliefs
Valuing nature
Recognizing personal potential
Emotion Management Spiritual well-being Creative Skills
Feeling loved
Being joyful - Laugh
Being grateful
Coping - Accepting loss
Feeling secure
Appreciating others
Acknowledging others
Responding to success
Responding to failure
Being courageous
Accepting help
Being humble
Self-esteem
Empowerment
Equanimity at all times
Ethical
Visionary
Playful
Gratitude
Moral courage
Being trustworthy
Hearing inner voice
Purposeful excellence
Right against wrong for societal
good
Grandeur
Insight
Muse
Curiosity
Imagine
Dream
Visualize
Create
Inspire
Appreciation of aesthetics
Professional Skills Academic Competencies Behavioral Skills
Prioritization and time
management
Accepting change
Planning and Project
management
Ownership
Quality orientation
Excellence & continuous
improvement
Respecting deadlines
Client orientation
Communication skills
Commitment
Branding skills
Resource management
Negotiation skills
Conflict management
Learn & apply knowledge
Learn & applying skills
Explore Creatively
Rigor in experimentation
Promote curiosity
Build innovative spirit
Orient towards research
Learning to learn
Deal with ambiguity
Recognize interrelationships
Explore interdependencies
Promote environmental
protection
Promote reading and open
discussions
Adhere to safety and
professional standards
Problem solving
Reflection on ideas
Integrity
Collaborative spirit
Community service
Accepting divergent views
Empathy
Accepting feedback
Leadership in action
Team above self
Belief in self
Accepting diversity
Maintain positive relationships
Building consensus
Respect for societal norms
Flexibility
Social maturity
Sharing - Caring
Cooperation
Figure 2: The educational grid – competencies and skills for holistic lifelong education
The need for transformative education from a purely materialistic perspective to higher echelons of life
is no longer a choice. The World Health Organization projects depression as major factor for disability,
after cardiovascular disease. The perceived decline of moral values in society and the increasing
need for environmental ethics (Light and Rolston 2003), points to an urgent need to reinforce ethical
education, and for society to transparently demonstrate moral values in everyday lives.
2. The problem statement
The underlying theme and propositions covered in this paper are shown in Figure 3. The results from
an internet survey conducted by the authors in May 2011 (described below) have been used as the
basis for the development of the proposed holistic learning framework. The paper attempts to link the
537

Shekhar Murthy and Devi Murthy
educational value chain from “where we are” to “where we could be.” Specific learning constructs
have been proposed that encompass lifelong learning framework. The authors also identify current
skill-gaps in workforce and suggest ways to bridge them by leveraging on the strengths of Millennial.
Holistic
Education
Current
educational
system
Where are we?
Where should we be?
Proposed
Framework
Main focus
Opinions on positive and negative
outcomes
Probable areas for improvement
Refocus on education to “Joy of
Living”
Holistic framework with strategies
and methods
Leverage on strengths of
millennials and meet aspirations of
future
Figure 3: Theme and outline of the paper
3. Current educational system – where are we?
The focus of current educational system caters to the hygiene factors of life (region-1 of Figure 1). A
quick survey of reported studies and blogs reiterates that the focus of our educational system is
oriented mainly towards attainment of academic knowledge with somewhat limited technical skills as
permitted within available resources. The performance reflected by ‘end-of-the-course grades’
becomes sacrosanct to success and often becomes a filter for separating the good from ‘not so good’
students. Achievement of “high-academic-grade” seems to automatically translate as preference in
employment with reasonable assurance of economic security. In this context, Krishnamurti said that
success-oriented education is also fear-oriented education (quoted by McAuley 2008).
The conventional approaches to education employ the paradigm of ‘social acceptability’ and
‘individual conditioning’ as the norm. These apparently impact on the ability of individuals to think
independently. Foucault (1985) observes “there are times in life when the question of knowing if one
can think differently than one thinks, and perceive differently than one sees, is absolutely necessary if
one is to go on looking and reflecting at all.” A human being in fact, exercises two instruments: that of
knowledge, which enables acquisition of technical skills and that of intelligence, which enables
observation and discovering the self (Krishnamurti 1974).
Over emphasis on university degrees and academic qualifications rather than on the attainment of
rational, emotional and spiritual intelligence, can lead to a discordant selfish society where personal
interests override social gains. In organizational context, these factors can affect harmony and growth
at the workplace. Not surprisingly therefore, there is a global crying out on the skills-gap between
academically produced intake and industry needs. The global theme on the skills-gap converges
across geographical divides and cultures, and increasing proportion of university graduates are found
lacking in technical, professional and behavioral skills as numerous studies and surveys consistently
show: Employability skills (Juhdi, Jauhariah and Yunus 2007); Graduate employability (Archer and
Davison 2008); Skills-gap survey (The Higher Education Forum 2010); The Skills Gap (America’s
Edge 2010); Bridging the Skills Gap (Galagan 2010); Across the Great Divide (Bridgeland, Milano and
Rosenblum 2011).
Figure 4 summarizes the surveys referred above. It depicts the current focus of education vis-à-vis the
skills-gap reported by employers of educated workforce. The directional arrow in the figure
emphasizes the relative importance of attributes listed.
538

Shekhar Murthy and Devi Murthy
Figure 4: Current focus of education and resultant skills-gap
In the context of educational focus and skills-gap, regions 2 and 3 of Figure 1 must not be seen as
watertight segments, but instead, viewed as a composite whole of an integrated holistic education. It
should be recognized that future talent development requires both formal and informal approaches to
learning. In this context, lifelong learning (Bremner 2010, Bennet 2008), just-in-time-learning,
eLearning (Taylor 2010), blended-learning, using adult learning methods (Collins 2004) become a
significant enabler to building the requisite talent pipeline for future workplaces. The future learning
ecology needs suitable adaptation to the strengths of Millennials, who would be our future learners,
which includes, 24x7 social connectivity, technology savvy, media agnostic, multitasking, and
informally flexible life- styles (Prensky 2001). Prensky candidly observes, “the single biggest problem
facing education today is that our current teachers, speak an outdated language (that of the pre-digital
age) and are struggling to teach a population that speaks an entirely new language.
In terms of information-centered versus transformation-centered education, and learner-centric versus
teacher-centric, the learning facilitation grids shown in Figure 5 clarifies relative paradigm positions.
The framework for holistic lifelong learning proposed in this paper seeks to achieve “purpose with
growth” using blended learning models.
4. Survey on goals and methods of education
A short internet-based survey was conducted by the authors in May 2011 to seek the respondents’
opinion and feedback on the current goals and methods of education and what they should be in
future. The broad aspects covered in the survey are shown in Table 1.
The respondents were contacted through Internet, LinkedIn and Facebook, mostly from the authors’
professional groups comprising executives, teachers, trainers and students. One hundred and eight
persons took the survey of which six had partially completed their responses, and were therefore
discarded from the resultant analysis.
539

Shekhar Murthy and Devi Murthy
Figure 5: Learning facilitation grid
Table 1: Survey on goals and methods of education
Opinions sought in the Survey Range of choices
What should be the goals of education? Dissemination of knowledge
(Respondents were asked to rank from most Building skills for societal growth
important to least important)
Economic empowerment
Preparing researchers for knowledge and technology
advancement
Unleashing true potential of self
Self-realization and spiritual growth
Preserving environment and social order
What should be the emphasis of education? Learning to be (Interpretative skills)
(Respondents were asked to rank from most Learning to know (Cognitive skills)
important to least important)
Learning to do (Resolute skills)
Learning to live together (Relational skills)
What aspects (of those listed alongside) does Think differently
our current educational system encourage Question everything
learners to do?
Find their passions and follow them
(Respondents were asked to rank from Explore technology and knowledge
always to hardly ever)
Work together and share experiences
Empathize and help community building
Be creative and inquisitive
What beliefs (from those listed alongside) Good grades guarantees success in life
does our educational system continually Inspire learners to follow their passions
reinforce?
Unfold learner's potential
(Respondents were asked to rank from most Help learners to discover purpose of life and what they wish
agree to least agree)
to be
Create authentic relationships with people and environment
What amongst the choices alongside does Skill-based competence
our educational system promote?
Social-based competence
(Respondents were asked to rank from Self-esteem and well-being
highest evidence to least evidence)
Healthy positive attitude
Cultural diversity
Confidence to face uncertainty
Exploration and investigation
Experimentation
Imagination
Rigor and discipline
540

What amongst the choices alongside should
our educational system support?
(Respondents were asked to rank from most
significantly to least significantly)
Shekhar Murthy and Devi Murthy
Opinions sought in the Survey Range of choices
What amongst the strategies listed alongside
does our educational system use?
(Respondents were asked to rank from
strongly agree to strongly disagree)
Connect life experiences with learning
Involve learners as active participants
Design programs to help participants reach their goals.
Show learners how learning will benefit them
Provide opportunities for sharing of experiences, questions,
and exercises with peers
Accommodate different learning styles by offering a variety
of training methods
Reinforce learning through timely feedback and reflection
Use input from peers to identify learning needs
Keep a personal portfolio and record learning events
Seek feedback on performance from others
Teach others
Keep a to-learn list
Always have something to read
Participate in research
Actively participate in local and national organizations
Actively participate in local multi-disciplinary conferences
Audit performance and improve performance through selfcorrection
The respondents were asked to rank the effectiveness of current educational system from amongst five
choices in the Likert scale (average being the center choice)
Each of the respondents also made one specific suggestion to improve the educational system
Seven of the survey questions were of ranking type, where respondents were forced to exercise onechoice
over the other. The questionnaire was so designed to obtain positive dispersion among tight
and similar choices. In most cases, all attributes indicated as alternatives in the survey questions must
actually be seen as desirable constituents of lifelong holistic education yet, ranking among these
attributes has been sought only to identify key priority components in education. Therefore, readers
are advised that whilst reading the survey indicators, even those ranked as low focus areas, does not
in any way mean that they are insignificant. In fact, they still remain very essential and significant
constituents of a transformative educational system, albeit a bit lower in priority from the respondents’
point of view.
Seventy percent of the sample polled opined that the current educational system is only averagely
effective. Twenty-one percent gave a below-average effectiveness rating. This only reinforces our
view that skills-gap and near-absence of life-skills orientation in our educational system is a hard
reality which needs urgent intervention. Among the significant changes that respondents sought in
educational system are: introduction of experiential, skill-based learning with orientation towards
collaborative work; social values; spiritual foundations to the purpose of life; and above all, build an
attitude of learning to learn with joy and fun. Figure 6 shows the ‘word-scatter’ of major changes that
respondents would like to see in our educational system.
The survey respondents considered dissemination of knowledge and unleashing the true potential of
self as the most significant goals of education. The other preferences that respondents gave for goals
of education are shown in Figure 7.
The respondents suggested that the emphasis of education must be more on practice using the
learning by doing pedagogy. The priority for selecting suitable strategies for the development of
learning components as obtained from the survey is shown in Figure 8.
Specific operational components that can be used to build effective and robust educational systems
were also studied through the survey. Learning methods, such as teaching others and obtaining
inputs from peers to refine learning needs, found more preference amongst the respondents. Self-
541

Shekhar Murthy and Devi Murthy
correction through audited performance and maintaining a personal portfolio to record learning events
received lower importance. The priority preference exercised by respondents for choosing operational
components of learning strategy is shown in Figure 9.
Figure 6: Goals of education
Self-
realization
and
spiritual
growth
Preserving
environment
and social
order
Knowledge
for
research to
advance
technology
and
science
Direction of Priority
Skills for
societal
growth
Figure 7: Priorities to accomplish the goals of education
Learning to be
Learning to know
Direction of Priority
Economic
Empowerment
Learning to live together
Unleashing
true
potential of
self
Dissemination
of knowledge
Learning to do
Figure 8: Strategies to select appropriate learning components
The authors believe that it would be unfair if some of the positive aspects of current educational
system are not brought to the fore. The survey therefore analyzed the strengths of our educational
system which provide broad relative indications than absolute truths.
Our current educational system overemphasizes achievement of good grades and scores, whereas,
discovering the true purpose of life, and help learners to know what they really want to be gets
relatively little attention. Overemphasis on grades breeds competition, which often leads to fear of
accomplishment and consequential stresses that adversely affects well-being (Kohn 1999). Creating
authentic relationships with people and environment also receives little emphasis. The survey reveals
that our educational system reinforces certain beliefs over others, which from the standpoint of holistic
lifelong learning are very significant. Figure 10 shows the preferences of respondents with regard to
beliefs reinforced by current educational approaches.
What kind of attributes does our educational system nurture? The respondents felt that our
educational approaches being more knowledge-driven are biased towards exploring technology and
knowledge. Their responses suggested that the system does not encourage students to think
differently and creatively. Learning is conditioned to rigid structures and syllabi. Neither, is the
542

Shekhar Murthy and Devi Murthy
emphasis of education towards community building and empathy is high. Figure 11 shows the survey
results with regard to the attributes that our educational system nurtures and encourages.
Figure 9: Operational components to build learning strategies
Help learners to
discover purpose
of life and what
they want to be
Create authentic
relationships with
people and
environment
Figure 10: Beliefs of current educational system
Think
differently
Be creative
and
inquisitive
Empathize
and help
community
building
Direction of Emphasis
Inspire learners
to follow their
passions
Direction of Emphasis
Find their
passions
and
follow
them
Figure 11: Emphasis of current educational system
Question
everything
Unfold learner's
potential
Work
together
and share
experiences
Good grades
guarantees
success in life
Explore
technology
and
knowledge
The other facets of our educational system appear to favor skill-based competencies delivered with
rigor and discipline than helping students to build confidence to face uncertainties in life. The
respondents also underscored the point that social-based competencies and allowing imagination to
flow do not find much emphasis in our academic curriculum and learning methods. Figure 12 shows
the attributes that current educational approaches support.
In terms of the desirable attributes that educational approaches should actively pursue, respondents
opined that learning should be connected with life experiences and built around an atmosphere of
sharing and collaboration. Blended learning has been favored. Figure 13 shows the preferences of
respondents with regard to desirable attributes in educational systems.
543

Shekhar Murthy and Devi Murthy
Figure 12: Attributes that current educational system support
Direction of Emphasis
Involve
learners as
active
participant
s
Reinforce
learning
through
timely
feedback
and
reflection
Show
learners
how
learning
will
benefit
them
Design
programs
to help
participants
reach their
goals.
Accommodate
different
learning styles
by offering a
variety of
training
methods
Provide
opportunities
for sharing of
experiences,
questions,
and
exercises
with peers
Connect life
experiences
with learning
Figure 13: Desirable attributes in educational system
Superimposing regions 1 to 3 of Figure 1 which defined the holistic lifelong education model with the
indicators obtained from the survey, a holistic learning grid can be constructed that maps the learning
intent, strategy and attributes as shown in Figure 14.
5. Proposed holistic lifelong learning framework
Learning is ideally a lifelong continuum. Viewing learning interventions at kindergarten, school,
university, workplace, and post-retirement life as one holistic transformation is the key to human
capital development and increased life satisfaction.
The authors present a holistic lifelong learning framework that comprehensively captures the stages,
and strategies, in the learning value-chain. Figure 15 depicts the framework which offers seamless
evolution of learning precepts covering the paradigms, Nurture – Sculpture – Structure – Culture –
Departure.
At the preschool level, learning must revolve around nurturing. A child’s growth must be filled with
wonderment! Learning pedagogy must promote WOW, Inquiry, Discovery and Exploratory precepts. A
child must be encouraged to grow accepting its uniqueness of existence without undue societal
judgmental overlap.
544

Experiential
Collaborative
Share experiences
Social-Values
Preserving environment and social order
Community building
Cultural diversity
Self-esteem and well-being
Healthy positive attitude
Reflective thought
Teach others
Create authentic relationships with people
Learning with nature
L i t li t th
Shekhar Murthy and Devi Murthy
Transformative
Discover purpose of life
Inspire learners to follow their passions
Confidence to face uncertainty in life
Think differently
Imagination
Connect life experiences with learning
Unleashing true potential of self
Creative
2
Spiritual growth
Self-realization
3
Learning to know
Learning to do
Learning from mistakes
• Participate in conferences and
workshops
• Feedback from peers
• Exploration and experimentation
• Action-driven learning
• Accommodate different learning
styles
• Goal-driven learning
• Question everything
• Economic Empowerment
• Application-Oriented
• Project-driven
• Research-oriented
• Dissemination of knowledge
Holistic Lifelong Learning Grid
Outcomes
Joy in learning
L i t b
Figure 14: Holistic lifelong learning grid
At the school level, a child’s growth needs to be molded and sculpted. The pedagogical focus should
encourage creativity, amalgamating this with curiosity and a questioning analytical temperament.
Opportunities must be provided to reflect upon the how’s and why’s, and learning must be deeply
rooted into practice and experience.
1
• Willingness to learn to learn
• Reflecting on learning process
• Deriving societal good from learning
• Apply learning to protect environment
• Active engagement with experience
• Ethical conduct with honest intent
• Realize the true self
• Create happiness in society
• Create and share wealth in society
• Physical and emotional well-being
At University level, teenager adolescents need guided structuring. The pedagogical shift must be
woven around collaborative team-based activities, encompassing opportunities to research on a wide
array of subjects. Participants should be able to appreciate and visualize a unified underlying theme
amongst nature’s diverse manifestations. Learners should be able to compare and contrast, to obtain
an integrative outlook reinforced with experiential learning through project based activities.
At work, an adult’s behavior must truly reflect matured tolerance, constructive balanced views thriving
in a global cultural landscape. Learning at work should be based upon adult-to-adult interactions,
where diverse viewpoints are heard, debated, analyzed and innovative “out-of-the-box thinking”
approaches must be promoted. At the workplace, collaboration with large teams in an invigorating
entrepreneurial spirit holds the key.
545

Shekhar Murthy and Devi Murthy
Stages Focus Drivers for Holistic Lifelong Learning
Nurture WIDE W:Wow I:Inquiry D:Discovery E:Exploratory
Sculpture CARE C:Creative A:Analytical R:Reflective E:Experiential
Structure TRIP T:Team-based R:Research I:Integrative P:Projects
Culture DICE D: Diverse I: Innovative C:Collaborative E:Entrepreneurial
Departure CESS C:Consultative E:Ethical S: Social S: Security
Figure 15: Holistic Lifelong Learning Framework
Post-Work life is the stage of departure, where learning should become more contemplative and
largely society-oriented. Learning at this phase of life should ideally be linked with consultative and
advisory roles, spun around ethical, social and security considerations.
Each stage of this naturally evolving progression must focus on how to learn and the use of right tools
in assimilating, internalizing, impacting thought, behaviors and outcomes. Each of these stages has a
key focus, indicated with an acronym: WIDE-CARE-TRIP-DICE-CESS translating into 20 cardinal
learning drivers presented in the framework.
6. Millennial centric learning
Learning ecology must absorb societal changes and needs of the future. The methods and processes
that sustained educational systems in the past need not automatically translate as success factors in
the future. In this context, the holistic learning framework elucidated in this paper must be aligned to
leverage the strengths of the Millennial.
The Millennial generation breeds in a networked world with real time connectivity. Cellular phones and
a laptop have become a necessity. The Millennial generation is more exposed to their world through
Facebook, LinkedIn, Twitter and YouTube, than face-to-face. Their connections to the outer world are
without much constraint. To the Millennial, life is a rush; multi-tasking; on the move; and restlessly
hectic (Shekhar 2011). Their feelings are expressed in smileys, and communications encapsulated in
Three-Letter-Acronyms.
546

Shekhar Murthy and Devi Murthy
To the Millennial an instructor-led, chalk-and-talk pedagogy may spell boredom. Group-driven tasks
over social networks, wikis and blogs suits them better. They would prefer flexibility in learning
curriculum that empowers learners to become an integral part of educational system from design to
delivery. Learning must necessarily be fun for them. The salient strengths of Millennial which should
be factored in learning design, methods and delivery are presented in Figure 16.
Figure 16: Strengths of millennial generation
Learning styles that could effectively leverage the strengths of the Millennial and engage them
purposefully have been depicted in Figure 17. Holistic lifelong learning approaches must make efforts
to use these techniques in curriculum design and delivery.
Figure 17: Learning styles for millennial
547

7. Conclusions
Shekhar Murthy and Devi Murthy
Our current educational system requires a holistic transplant that addresses the core purpose of
human life. Whilst the current thrust of education focuses largely on knowledge advancements and
building workplace competencies, it does not encourage learners to explore their potentials to the
fullest extent, nor build adequate life skills to manage and transform their lives to create happy
societies. The survey analysis presented in this paper provides broad indicators of deficient attributes
in current educational approaches.
The knowledge economy of today demands creative and innovative solutions to the challenges faced
by society. The global citizens of the future require much more environmental and cultural sensitivity
than ever before. Societal growth can no longer be measured by material prosperity alone; it needs to
factor ethical values, harmony and happiness as a composite currency of growth. Value-based holistic
education is the need of the day.
The authors recommend holistic lifelong learning and creating an environment that fosters life’s
context as the essential driver for talent building. The holistic lifelong learning framework presented in
this paper provides for a seamless transition of learning methods from pre-school education to
retirement with clearly defined pedagogical drivers. Sathya Sai Baba, the spiritual leader of India often
said “true education is not taught, but caught.”
Are we ready to change our learning ecology to meet the challenges of the Millennial? That’s the
moot question.
References
America's Edge. (2011) "The Skills Gap: Reversing Washington’s Lack of Skilled Workers through early
Learning," America's Edge Report, Washington.
Archer, W., and Davison, J. (2008) Graduate Employability: the Views of the Employers, London, the Council for
Industry and Higher Education (CIHE).
Arguelles, L., McCarty, R., and Rees, R. A. (2003) “The Heart in Holistic Education,” ENCOUNTER: Education
for Meaning and Social Justice, Vol.16, No.3.
Awbrey, S. M, Dana, D., Miller, V. W., Robinson, P., Ryan, M. M. and Scott, D. K. (Eds.) (2006) Integrative
Learning and Action: A call to Wholeness, New York: Peter Lang.
Bates, R. (2003) "A Critical Analysis of Evaluation Practice: The Kirkpatrick Model and the Principle of
Beneficence," Evaluation and Program Planning, Vol. 27, pp. 341–347.
Bennet, A., and Bennet, D. (2008), “Moving from knowledge to wisdom, from ordinary consciousness to
extraordinary consciousness”, in VINE, Vol. 38, No. 1, 2008.
Bhatta, C. P. (2007) “Holistic Personality Development through Education: Ancient Indian Cultural Experiences,”
Paper read at the International Cultural Research Network and University of Strathclyde conference on
Exploring Cultural Perspectives in Education, Glasgow, Scotland.
Bhattacharya, S. (2010) “Relationship between Three Indices of Happiness Material, Mental and Spiritual,”
Journal of Human Values, Vol. 16, No. 1, pp. 87-125.
Bierly III, P.E., Kessler, E.H., Christensen, E.W. (2000) "Organizational Learning, Knowledge and Wisdom,"
Journal of Organizational Change Management, Vol. 13, No. 6, pp. 595-618.
Bigger, S. (2008) “Educational Futures,” Spiritual Education e-journal of the British Education Studies
Association, Vol.1, No. 1.
Bremner, V. (2010) “Learning for Life,” Strathclyde University, Glasgow
Bridgeland, J., Milano, J., and Rosenblum, E. (2011) “Across The Great Divide: Perspectives of CEOs and
College Presidents on America’s Higher Education and Skills Gap,” Civic Enterprises Corporate Voices for
Working Families in Association with: Institute for a Competitive Workforce, U.S. Chamber of Commerce.
Collins, J. (2004) “Education Techniques for Lifelong Learning,” Principles of Adult Learning, RadioGraphics, Vol.
24, No. 5.
Dewey, J. (1916) Democracy and Education: An Introduction to the Philosophy of Education, The Free Press:
New York NY.
Duncan-Hewitt, W., Leise, C., and Hall, A. (2005) Affective Domain, Module in 2nd edition of the Faculty
Guidebook, Pacific Crest, Lisle, IL.
Fischer, G. (1998) Making Learning a Part of Life-Beyond the 'Gift-Wrapping' Approach of Technology, In Alheit,
P. and Kammler, E. (Eds.) Lifelong Learning and Its Impact on Social and Regional Development, Donat
Verlag, Bremen, pp. 435-462.
Foucault, M. (1985) The Use of Pleasure, Trans. R. Hurley. New York: Vintage
Galagan, P. (2010) "Bridging the Skills Gap," The Public Manager, www.thepublicmanager.org.
Gardner, H. (1983) Frames of Mind, New York: Basic Books.
Goldsmith, M. (2010) Mojo: How to Get It, How to Keep It, How to Get It Back if You Lose It, Hyperion Books.
Goleman, D. (1995). Emotional intelligence: Why it can matter more than IQ, New York: Bantam.
548

Shekhar Murthy and Devi Murthy
Hare, J. R. (2006). “Towards an Understanding of Holistic Education in the Middle Years of Education,” Journal of
Research in Education, Vol. 5, pp. 301–22.
Kohn, A. (1999) “From Degrading to De-Grading,” High School Magazine, Vol. 38–43.
Juhdi, N., Jauhariah, A., and Shaharudin. (2007) “Study on Employability Skills of University Graduates,” The
Business Wallpaper, Vol.2, No.1.
Krishnamurti, J. (1974) On Education, Pondicherry, India: All India Press.
Light, A., and Rolston, H. (2003) Environmental Ethics – An Anthology, Blackwell Publishers.
Lopez, S. J., and Snyder, C. R. (2003) Positive Psychological Assessment: A Handbook of Models and
Measures, Washington, DC: American Psychological Association.
Martin, D. and Joomis, K. (2007) Building Teachers: A Constructivist Approach to Introducing Education,
Belmont, CA: Wadsworth, pp. 72–75.
Maslow, A. H. (1971) The Farther Reaches of Human Nature, Esalen Books, Viking Press.
McAuley, C. E. (2008) “Krishnamurti and Education: Application of Liberation,” Communication Studies, Sonoma
State University.
Mezirow, J. (1991) Transformative Dimensions of Adult Learning. San Francisco: Jossey-Bass
Prensky, M. (2001). “Digital Natives, Digital Immigrants - A new way to look at ourselves and our kids,” From on
the Horizon, NCB University Press, Vol. 9 No. 5, http://www.marcprensky.com/
Shekhar, M. (2011) “Millennial-Centric Learning: Revisiting Education with a Changed Mindset,” Paper read at
the National Management Convention, ASBM, Bhubaneswar, India.
Taylor, E. (2010) “Perspectives on the Future of Learning,” Tony Sheehan Ashridge Business School,
http://www.ashridge.org.uk
The Higher Education Forum supported by 1SOS, and Westat (2010) “Skills Gap Survey,” Survey for the Indian
Banking, Financial Services, and Insurance Sector, India.
Von Glasersfeld, E. (1989) Constructivism in Education, In Husen, T. and Postlewaite, N. (Eds.) International
Encyclopedia of Education, Oxford: Pergamon Press, pp. 162-163.
Wilber, K. (2006) Integral spirituality, Boston, MA: Shambhala Press.
Young, L. (2003) The Book of the Heart, New York: Doubleday.
549

Student's Characteristics for Note Taking Activity in a Fully
Online Course
Minoru Nakayama 1 , Kouichi Mutsuura 2 and Hiroh Yamamoto 1
1
Human System Science / CRADLE, Tokyo Institute of Technology, Tokyo,
Japan
2
Faculty of Economics and Graduate School of Engineering, Shinshu
University, Matsumoto, Japan
nakayama@cradle.titech.ac.jp
Abstract: Since student's notes are a reflection of the progress of their education, analysis of notes taken can be
used to track the learning process of students who participate in fully online courses. This paper examines the
relationship between the performance of note-takers and the characteristics of these students, including their
personalities, learning experiences, information literacy and note-taking skills. A fully online course was conducted
for undergraduate students in Economics. Participants were asked to study each course module and present their
notes to the lecturer every week. The student’s learning performance was then measured using online tests,
weekly confirmation tests with a proctor, and a final exam. The total number of valid participants in the courses was
54. Three factors of note-taking skills were extracted, according to the survey. They are (1) Recognizing note taking
functions, (2) Methodology of utilizing notes and (3) Presentation of notes. The first factor score correlates with
mean scores of confirmation tests, while the third factor score correlates with online test scores. Therefore, the
three factors are responsible for different aspects of learning. The sum of the assessment scores for note-taking
correlates with mean scores of online tests (r=0.51), with confirmation tests scores (r=0.54) and with the scores of
the final exam (r=0.46). The effectiveness of the contents of the notes for exam scores was also measured.
Keywords: note taking, fully online learning, student's characteristics, learning experience, correlation analysis
1. Introduction
The online learning environment is expanding throughout the world, in particular at universities which
offer courses internationally. Though the establishment of an online learning environment requires a
great deal of resources, a cost-benefit assessment of the effectiveness and usefulness of online
courses has been made, and discussed previously. It has often been suggested that many participants
have difficulty with continuing their education online, and the evaluation of the learning process has
been restricted to the online tests and access logs in the case of online courses up until now
(Nakayama et al. 2009).
When instruction is provided face-to-face, students' evaluations of their attitudes and learning
processes can be readily observed, however. However, since most learning processes require the
transfer and summarization of knowledge using notes, note-taking has become a primary method of
evaluating conventional learning evaluation. As student's notes are a reflection of the progress of their
education, analysis of notes taken can be used to track the learning process (Kiewra 1985, 1989;
Kiewra et al. 1995; Kobayashi 2005). In previous studies, students who participated in fully online
courses were asked to take notes in order to evaluate their learning process (Nakayama et al. 2010,
2011).
This paper examines the relationship between the performance of note-takers and the characteristics of
these students, including their personalities, learning experiences, information literacy and note-taking
skills. The following topics are addressed:
To evaluate student's note-taking skills in a university level learning environment which uses ICT
(information communication technology) as a part of the course, a set of questionnaires were
developed to measure their skills. Also, some factors were extracted from the students’ self
assessments.
The relationship between the assessment of the contents of notes taken in a fully on-line learning
environment and student’s characteristics were measured to extract factors used in note-taking
activities.
550

2. Method
2.1 Fully on-line learning course
Minoru Nakayama et al.
The notes taken by students were evaluated during an Information Systems Network course which was
a fully online course given to undergraduate students in Economics at a Japanese national university.
The online learning course consisted of modules with lecture videos, slides and online tests.
Participants were asked to study one module every week, and take notes from the online learning
materials without any printed materials and face-to-face sessions being provided. The learning pace
was set by the lecturer so that students needed to complete one module per week. For each session,
students were encouraged to take an on-line test, which was a function of the learning management
system (LMS). Also, weekly confirmation tests with a proctor were conducted to monitor their progress.
Both the final scores of online tests and the weekly confirmation test scores for every module were
recorded and used in determining final grades (Nakayama et al. 2010).
Though students were not obligated to take notes, these were a necessary requirement for answering
questions in online tests. It was also necessary to review notes before joining the weekly confirmation
test sessions in the classroom. The students took notes of the lectures given and the slides presented.
Instructions and guidance were not provided to the participants, in order to maintain the impartiality of
the survey.
2.2 Note-taking assessment
At the weekly test sessions, participants were required to present their notebooks. The lecturer
reviewed and assessed these notes during the test, then returned them to students afterwards.
The contents of the slides presented by the lecturers can be used as a comparison standard for
student’s notes of each module, and can be used to evaluate these notes.
The contents of students' notes were evaluated using a 5-point scale (0-4), 4:Good, 3:Fair, 2:Poor,
1:Delayed, 0:Not presented (Nakayama et al. 2010). If a student reproduced the same information in his
or her notebook, note-taking was rated as ``Fair''. ``Fair'' note-taking is the through reproduction of
information given. If any information was omitted, a “Poor” rating was given. In a sense, ``Poor''
note-takers failed to completely reproduce the information transmitted completely. When students
wrote down additional information from the lecture, the note-taking was rated as ``Good''. The ``Good''
note-takers included those who integrated the knowledge given in the lecture with relevant prior
knowledge (Mayer et al. 1990), as some of this is are related to each other, and some of the knowledge
given in the lecture is related to relevant prior knowledge. At this point, several kinds of constructivistic
learning activities were occurring (Tynajälä 1999; Tam 2000).
The total number of valid participants in the courses was 54.
2.3 Characteristics of students
In this study, student's characteristics were measured using three constructs. These constructs were:
personality (Goldberg 1999; IPIP 2004), information literacy (Fujii 2007) and learning experience
(Nakayama et al. 2007). Personality: A very popular psychological characteristic of humans is
personality. To observe the personalities of students, the International Personality Item Pool (IPIP)
inventory (IPIP 2004) was used. This five factor personality model was proposed by Goldberg
(Goldberg 1999), and provides five component scores: ``Extroversion'', ``Agreeableness'',
``Conscientiousness'', ``Neuroticism'' and ``Openness to Experience''.
Information literacy: Information literacy inventories were defined and developed by Fujii (2007) to
measure information literacy. The survey consists of 32 question items, and 8 factors emerge from
these questions as follows: interest and motivation, fundamental operational ability, information
collecting ability, mathematical thinking (reasoning) ability, information control ability, applied
operational ability, attitude, and knowledge and understanding. These 8 factors can be summarized as
two secondary factors: operational skills and attitudes toward information literacy (Nakayama et al.
2008). The validity of applying this metric to university students has been confirmed (Fujii 2007).
551

Minoru Nakayama et al.
Learning experience: The authors have measured students' on-line learning experiences using a
construct with a 10-item Likert-type questionnaire. Three factors have been extracted in previous
studies as follows: Factor 1 (F1): overall evaluation of the e-learning experience, Factor 2 (F2): learning
habits, and Factor 3 (F3): learning strategies (Nakayama et al. 2007).
2.4 Survey of notes taking behaviour
The note-taking skills of students and their behaviour may reflect not only note-taking performance but
also learning achievement (Nye et al. 1984). Therefore, the construct for note-taking behaviour may be
a key to the evaluation of the learning process. Note-taking skills are sometimes discussed, but have
not been identified as major techniques for studying at Japanese universities, however. To observe
students’ note-taking abilities, attitudes and techniques, 20 original inventories have been developed by
the authors. The inventories were created using question items from Cornell style notes (Pennsylvania
State Univ. 2010) and items from other previous studies. The inventories are displayed in Table 1.
Further details will be discussed in the results section. All constructs were surveyed at the beginning of
course.
3. Results
3.1 Note-taking assessment
The percentages of note assessment levels across the weeks of the course are summarized in Figure
1. According to the figure, percentages for “Fair” are almost always higher than for the other
assessment levels through course period. The percentages rates of note takers rated “Good” are very
low. This suggests that students are unable to create “Good” notes as the course progresses. Also, the
percentages of “Poor” assessment levels are almost always the lowest. These tendencies have been
confirmed in a previous report of a fully online course (Nakayama et al. 2010). All assessments are
summed up as ratings for all weeks of the course, and note assessment scores are calculated for each
participant. The scores indicate a kind of note-taking performance ability (Nakayama et al. 2010,
Nakayama et al. 2011). For the following analysis, all participants were divided into two groups: high
assessment scores and low assessment scores. The high and low groups consist of 30 and 24 students
respectively.
Figure 1: Grade percentages of note-taking assessments
3.2 Note-taking behavioural factors
Valid responses were summarized from 54 participants. To extract note-taking factors for this survey,
factor analysis was conducted using Promax rotation. Table 1 shows the factor loading matrix and the
correlation coefficients across three factor axes, and the contribution ratio of each factor while ignoring
the other factors is also illustrated. As a result, three factors were extracted and each factor was given a
label such as F1: Recognizing note taking functions, F2: Methodology of utilizing notes, F3:
Presentation of notes. These factors emphasize three aspects of note-taking. According to the
552

Minoru Nakayama et al.
correlation coefficients, all factor axes correlate with each other; therefore the three factors are related
to each other thought they are identified separately.
Table 1: Questionnaire for note-taking skills and factor loading
No. Question item F1 F2 F3
1 I understand the syllabus summary of this course 0.63 -0.09 0.13
2 NT during sessions to understand the course contents 0.79 0.11 -0.06
3 NT during sessions to clarify the contents 0.87 -0.11 0.02
4 NT during sessions to review the contents later 0.52 0.18 -0.24
5 NT is for understanding the whole course not only the session topics 0.82 0.02 -0.01
6 NT consists of what teacher presented and talked about 0.79 0.09 0.05
7 I think about the meaning and importance of words during NT 0.53 0.29 0.21
8 I use a colored pen or marker to highlight important points 0.56 -0.04 0.17
9 I use notes taken to review the contents of a session in advance of a test 0.20 0.55 -0.13
10 I use NT to review the notes taken after the session 0.33 0.58 -0.14
11 I use NT to revise the notes taken after the session -0.08 0.89 -0.01
12 I use NT to write some additional information in the notes taken -0.10 0.92 0.04
13 I think about relationships between contents of the notes taken 0.18 0.77 0.05
14 Notes of surveyed contents are added to notes taken -0.02 0.71 0.17
15 I have an original writing format for NT -0.02 0.62 0.24
16 Notes are taken so that other participants can understand the contents 0.12 0.03 0.72
17 Notes are taken so that even non-participants can understand the contents 0.08 -0.27 0.88
18 Classmates are considered when notes are taken 0.05 0.12 0.71
19 I have NT skills -0.17 0.23 0.67
20 My NT techniques have improved -0.03 0.14 0.73
F1: Recognizing note taking functions 1.00
F2: Methodology of utilizing notes 0.47 1.00
F3: Presentation of notes 0.31 0.39 1.00
Contribution ratio 0.27 0.24 0.21
NT: Note-taking
The factor scores between the two groups of note taking assessments are summarized and compared
in Figure 2. For the first factor, "Recognizing note taking functions", the score of the high group reaches
4 out of 5 points, as they know the functions of taking notes well, while the score of the low group stays
in the middle of the scale. There is a significant difference between the two groups (t(52)=3.9, p

Minoru Nakayama et al.
3.3 Effectiveness of student's characteristics
Various factors of student's characteristics may affect note-taking performance. Some relationships
between them have been confirmed in the results for a blended learning environment (Nakayama et al.
2011). In this study, the effectiveness of note-taking in a fully online learning environment is
investigated.
Figure 3: Factor scores of personalities between two note-taking assessment groups
Figure 4: Factor scores of information literacy between two note-taking assessment groups
First, mean factor scores of personality are compared between high and low note-taking assessment
groups. The results are illustrated in Figure 3. There are significant differences in scores for
"Agreeableness" and "Conscientiousness" (p

Minoru Nakayama et al.
Figure 5: Factor scores of learning experiences between two note-taking assessment groups
3.4 Effectiveness of note-taking
To determine the effectiveness of note-taking during an online course, mean rates of weekly test scores
across each week are summarized in Figure 6. There are some differences in rates between levels of
note-taking assessments such as during the fourth and fifth weeks. In some weeks, the level of
note-taking assessment affects the test scores, but this effect may be depend on the content of the
course.
To confirm the detailed relationship between student's learning performance and scores related to
note-taking performance, a correlation analysis is conducted and the correlation coefficients are
summarized in Table 2. Significant coefficients are displayed in bold font.
The sum of assessment scores for note-taking correlates with mean scores of online tests (r=0.51),
weekly confirmation test scores (r=0.54) and with final exam scores (r=0.46). The effectiveness of the
contents of the notes for exam scores was measured.
Three factors of note-taking skills, (NT-F1) Recognizing note taking functions, (NT-F2) Methodology of
utilizing notes and (NT-F3) Presentation of notes are examined to determine their correlational
relationships with student's characteristics.
Figure 6: Test score rate change with note-taking assessment
555

Minoru Nakayama et al.
Table 2: Correlation coefficients between note-taking skills and student’s characteristics
Note score NT-F1 NT-F2 NT-F3
Online test 0.51 0.18 0.25 0.30
Weekly test 0.54 0.33 0.24 0.01
Final exam 0.46 0.21 0.23 -0.06
IPIP-F1 -0.09 0.19 0.05 0.27
IPIP-F2 0.22 0.45 0.29 0.33
IPIP-F3 0.13 0.39 0.48 0.52
IPIP-F4 -0.03 0.22 0.16 0.29
IPIP-F5 -0.05 0.27 0.35 0.21
IL-Skills 0.18 0.55 0.47 0.47
IL-Attitude 0.14 0.46 0.54 0.55
Experience-F1 0.23 0.29 0.28 0.42
Experience-F2 0.16 0.51 0.55 0.22
Experience-F3 0.25 0.44 0.42 0.39
For personality, these three factors of note-taking skills strongly correlate with "Agreeableness" and
"Conscientiousness" as Figure 3 confirms. Additionally, “Neuroticism” correlates with the NT-F3 score,
and “Openness to Experience” correlates with both NT-F1 and NT-F2.
For information literacy scale scores, all three factors of note-taking skills strongly correlate with both
secondary factors of information literacy, skills and attitude.
For learning experience, both NT-F1 and NT-F2 correlate with factor scores of learning experience.
These results suggest that the note-taking skills affect student's learning performance while their
characteristics are related to their note-taking skills.
If a detailed causal relationship can be determined, the development of effective learning support may
be possible. This will be a subject of our further study.
4. Summary
This paper determines the relationship between the performance of note-takers and characteristics of
students (personalities, learning experiences, information literacy and note-taking skills) in a fully online
learning environment. Three factors of note-taking skills have been extracted, according to the survey.
They are (1) Recognizing note taking functions, (2) Methodology of utilizing notes and (3) Presentation
of notes. Three factor scores were calculated.
Student's notes were assessed by the lecturer, and two note-taking performance groups were created
using the sums of the note-taking assessment scores. There are significant difference in factor scores
for note-taking, personality, information literacy and learning experience between high and low level
note-taking performance groups.
Additionally, the sum of assessment scores for note-taking correlates with mean scores of online tests
(r=0.51), the weekly confirmation test scores (r=0.54) and with final exam scores (r=0.46). The
effectiveness of the contents of the notes for exam scores was measured. The three factor scores also
correlate significantly with the scores of personalities, information literacy, and learning experiences.
These results have again confirmed that note taking skills reflect certain aspects of the learning process
during online learning.
Acknowledgments
This research was partially supported by the Japan Society for the Promotion of Science (JSPS),
Grant-in-Aid for Scientific Research (B-22300281: 2010-2012).
556

References
Minoru Nakayama et al.
Kiewra Kenneth A. (1985) “Students' Note-Taking Behaviors and the Efficacy of Providing the Instructor's Notes for
Review”, Contemporary Educational Psychology, 10, pp.378-386
Kiewra Kenneth A. (1989) “A Review of Note-Taking: The Encoding-Storage Paradigm and Beyond”, Educational
Psychology Review, 1(2), pp.147-172
Kiewra Kenneth A., Benton Stephen L, Kim Sung-Il, Risch Nancy, Maribeth Christensen (1995) “Effects of
Note-Taking Format and Study Technique on Recall and Relational Performance”, Contemporary
Educational Psychology, 20, pp.172-187
Keiichi Kobayashi (2005) “What limits the encoding effect of note-taking? A meta-analytic examination”,
Contemporary Educational Psychology, 30, pp.242-262
Mayer Richard E, Roxana Moreno, Michelle Boire, and Shannon Vagge (1990) “Maximizing Constructivist
Learning From Multimedia Communications by Minizimizing Cogmitive Load”, Journal of Edcational
Psychology, 91(4), pp.638-643
Nakayama Minoru, Hiroh Yamamoto, Rowena Santiago (2007) “The Impact of Learner Characteristics on Learning
Performance in Hybrid Courses among Japanese Students”, The Electronic Journal of e-Learning, 5(3), pp.
195-206
Nakayama Minoru, Hiroh Yamamoto, Rowena Santiago (2008) “Impact of Information Literacy and Learner
Characteristics on Learning Behavior of Japanese Students in Online Courses”, International Journal of Case
Method Research & Application, Vol. XX. No. 4. pp. 403-415
Nakayama Minoru, Hidetomo Kanazawa, Hiroo Yamamoto (2009) “Detecting Incomplete Learners in a Blended
Learning Environment among Japanese University Students”, International Journal of Emerging Technology
in Learning (iJET), Vol. 4, Issue. 1, pp. 47-51
Nakayama Minoru, Kouichi Mutsuura, Hiroh Yamamoto (2010) “Assessing Student Transitions during Blended
Learning Activities”, Proceedings of 9th ECEL, pp.394-400
Nakayama Minoru, Kouichi Mutsuura, Hiroh Yamamoto (2011) “A feasibility study of learning assessment using
sudent’s notes in an online learning environment”, Proceedings of DICTAP2011, in press.
Nye Pauline A., Crooks Terence J., Powley Melanie, Gail Tripp (1984) “Student note-taking related to university
examination performance”, Higher Education 13, pp.85-97
Tam Maureen (2000) “Constructivism, Instructional Design, and Technology: Implications for Transforming
Distance Learning”, Educational Technology & Society, Vol.3(2), pp.50-60
Päivi Tynajälä (1999) Towards expert knowledge? A comparison between a constructivist and a traditional learning
environment in the university, International Journal of Educational Research, 31, pp.357-442
Pennsylvania State University, Learning Centre, “Lecture Note Taking”,
http://www.ulc.psu.edu/studyskills/note_taking.html, Accessed on Aug. 30th, 2010.
Yoshihisa Fujii (2007) “Development of a Scale to Evaluate the Information Literacy Level of Young People
-Comparison of Junior High School Students in Japan and Northern Europe”, Japan Journal of Educational
Technology, Vol. 30(4), pp.387-395
557

Freeing Education Within and Beyond Academic
Development
Chrissi Nerantzi
University of Salford, UK
c.nerantzi@salford.ac.uk
Abstract: What can Academic Development (AD) and other professional areas and disciplines learn from freerange
farming? Open Educational Resources (OER) and Open Educational Practice (OEP) are mushrooming
and Massive Open Online Courses (MOOC) are here already. The idea of moving away from battery-type
Academic Development Activities and silo modules and programmes towards open cross-institutional
approaches in line with OEP are explored within this paper based on a recent small-scale, fully-online study. This
brought together academics and other professionals who support learning, from different disciplines and
professional areas who are studying towards a Postgraduate Certificate (PgCert) in Teaching and Learning in
HE/Academic Practice during a facilitated open Problem-Based Learning (PBL) task around assessment and
feedback using freely available social media. The study aimed to explore if and how online PBL can be used
within PgCert provisions to provide opportunities to connect, communicate and collaborate in a community of
practice beyond institutional walls. The phenomenographic methodology underpinned this research. Participants’
experiences in this open Academic Development activity were captured through individual remote interviews, a
series of questionnaires and reflective accounts. Findings indicate that open online PBL has the potential to
enable learners and educators to break out of academic and virtual silos. It also widens meaningful collaborative
learning within Academic Development in multi-disciplinary and multi-institutional groups –something participants
in this study commented extremely positively about. This provides evidence that freeing AD is the way forward to
share available resources, and establish more organic and healthy learning communities beyond the module,
programme and institutional level. Recommendations are made to Academic Developers and other tutors on how
to bring learners from different programmes, institutions and countries together online using social media to
create the conditions and the environment for a meaningful, rich and fruitful exchange and enable collaborative
formal and informal learning.
Keywords: open educational practice, academic development, social media, problem-based learning,
phenomenography
1. Introduction
This paper explores if, and how, freely available Web2.0 technologies can be used effectively within
Academic Development provision and other professional areas and Disciplines to create an open
networked learning environment. This type of environment can enable learning beyond the institution,
in the spirit of open education, bridging formal and informal learning. It also provides enhanced and
extended opportunities for connectedness and peer learning.
A small-scale, fully online PBL trial with PgCert participants from seven Higher Education Institutions
around the UK has been carried out to test this hypothesis. The trial was conducted over a period of 3
months. Ten individuals participated in total working in two groups of five including the facilitators.
This paper describes this trial. Findings are shared and recommendations are made for other
practitioners on how open online Academic Development activities could be used. Accredited
undergraduate and postgraduate provision is considered, linking institutions, to create and maintain
more open learning communities to share expertise, resources and most importantly bringing together
learners and tutors from around the world.
The background, research methods used, results and findings of this study are discussed and
recommendations are made for future implementations within and beyond Academic Development
activities.
2. Background
Problem Based Learning (PBL) is an active, collaborative student-centred teaching and learning
approach (Savin-Baden 2003; Hmelo-Silver et al. 2009). Boud (1985, 13) stated that “the principle
idea behind problem-based learning is that the starting point for learning should be a problem, a query
or a puzzle that the learner wishes to solve”. Real-life open-ended scenarios, triggers or problems are
used to engage small groups of students in meaning-making (Torp and Sage 2002) and the coconstruction
of knowledge using a PBL model. Beyond subject-specific knowledge creation and
construction, PBL also enables the development of more generic and transferable skills and
558

Chrissi Nerantzi
introduces students to research. PBL is also seen as a self-directed learning approach. A tutor
facilitates and assists learners in becoming more autonomous and discovering new thinking and
knowledge through collaborative and networked learning.
PBL was introduced in the 1960s in Medical Education (Barrows & Tamblyn 1980) and has spread
since then to a large number of different disciplines and institutions. Today it is used within
undergraduate and postgraduate provision around the world.
Web2.0 technologies and pedagogies such as networked learning (Jones & Steeples 2002) and
connectivism (Siemens 2004) are re-shaping the way educators learn, deliver and support learning
(Kear 2011). Today, a variety of technologies are used in face-to-face, blended and online provisions
with differing numbers of learners. These enable a more participatory and rich way to connect,
interact, learn and co-create with others when and how it suits them best. Increasingly we see
learners choosing digital tools and platforms, as well as digital devices for their formal and informal
learning needs.
While learning online is flexible, it can also be extremely challenging. Network-directed learning
(Siemens 2011) plays an increasingly important role and acts as an enabler of social learning. Social
media are frequently used to connect learners outside formal learning situations, and networks of
different kinds are used by learners to engage in a variety of collaborative learning activities and
connect with peers and experts around the globe. Could this open networked-collaborative learning
model also be useful for formal programmes to enrich the experience? Would PBL be an effective
vehicle to implement such online collaborative and networked learning activities (as stated by
Donnelly 2009) due to its structure and process?
The spirit of openness and sharing knowledge came to life with the Open Educational Movement.
OpenCourseWare (OCW), Open Educational Resources (OER) and courses on a wide range of
scales are offered already. Wiley (2006) states that “a shift towards ‘openness’ in academic practice is
not only a positive trend, but a necessary one in order to ensure transparency, collaboration and
continued innovation.” (online). Ten years have passed since MIT made their first OCW available and
other institutions have followed. Many have joined the OCW Consortium. The interest in OER has
since grown and there are now a number of OER repositories available to educators and learners
worldwide, as well as opportunities to engage in open access courses of small, large and extra large
scale.
However, currently there is limited evidence of OEP within Academic Development. A few OER
projects for Academic Practice have been funded and produced by various HE institutions, and been
made available to the wider community. Academic Development units play a vital role in modelling
innovative practice and enable academics and other professionals who support learning to immerse
into new ways of teaching, learning and thinking aiming to transform their practice and triggering a
shift in their beliefs (Mezirow 1997). Engaging in the design and delivery of open access collaborative
courses is something that is under-represented at the moment in an Academic Development context.
PBL is also not widely used in this context (Barrett 2010). There is even less evidence of blended and
fully online PBL in the same area (Donnelly 2005) and no evidence has been found of a more global,
open and online or blended collaborative inter-institutional PBL application within Academic
Development or elsewhere.
Research was carried out into whether Web2.0 technologies could be used effectively for online PBL
within Academic Development in the context of an open-access and collaborative learning experience
–specifically within PgCert provisions. A small-scale UK trial was carried out to learn together online.
This is described in detail in the following section.
3. Research methods
In the spirit of networked learning, an online PBL trial with participants from England and Scotland
was conducted from September to November 2010. It was based on the model of Computermediated
collaborative problem-based learning (CMCPBL) (Savin-Baden 2003) itself based on CSILE
(Scardamalia and Bereiter 1994) in which small groups worked together, synchronously and
asynchronously, to co-construct new knowledge through the application of online PBL.
559

Chrissi Nerantzi
The aim was to explore if PBL successes in other identified subjects could be replicated within
Academic Development but this time fully online, and specifically within the Postgraduate Certificate
(PgCert) in Academic Practice or similar programmes. The author acted as the trial organiser but did
not participate in the trial which was set up as a naturalistic study.
In order to study the variations of lived experiences, the research methodology was based on
phenomenography (Marton, 1994). This enables one to “describe qualitative varieties in people’s
experience of phenomena” (Dortins, 2002, p. 207), and “focuses on student perspectives” (Boustedt,
2008, p. 28). Phenomenography was also used as the main data collection and analysis method.
Patterns that emerge through the limited variations of experiences are captured in categories of
descriptions. Within this paper the category of description linked to ‘Online cross-institutional
collaborative learning’ is presented. Individual remote and in-depth interviews were conducted using
the web-conferencing tools Elluminate and Skype. Reflections on the experience of participants in the
trial were externalised through a series of open-ended questions. The interviews were recorded using
Elluminate and MP3 Skype Recorder, transcribed manually and collated into a Microsoft Excel
spreadsheet where data were filtered and analysed based on common themes that came up during
the interviews. Through this process the categories of descriptions emerged. Additional data were
collected through reflective accounts and initial and final surveys.
Eight new academics and two academic developers participated. Two multi-disciplinary, multiinstitutional
groups were formed each with four participants and one academic developer assigned to
each group to act as the PBL facilitator. The total number of participants is in line with Cousin’s (2009)
recommendation of ten as an optimum number of participants in phenomenographic studies. Virzi
(1992) also explored usability problems in application development and recommends that issues can
be identified by groups of four to five individuals.
Freely available Web2.0 technologies, such as a Wordpress group blog, Pbworks collaborative wikis
and the Skype web-based conference tool were utilised during the trial. The trial included an initial
stage to enable all participants to familiarise themselves with the technologies used and learning
online. A socialisation stage with tutors and peers followed. During this participants had also the
opportunity to explore the basic concepts around PBL and engage in a conversation about these. The
main PBL task followed which stretched over 5 weeks and was conducted in two groups. Both groups
were given the same scenario. This included issues around assessment and feedback. At the end
each group shared their findings with the other group and received feedback from peers and their
tutor.
Media-rich self-study materials were made available throughout the trial to help participants
understand the technology used and the concepts of PBL. Participants were also given access to
resources specifically linked to the PBL task to enable them to focus on the collaborative activity.
This was a small-scale study. All participants were volunteers, and busy professionals with limited
time available. None of the participants had experienced learning and/or teaching online before and
only a small number were familiar with PBL. There were issues with the technologies used due to
participants’ unfamiliarity. These were the main limitations of this study.
4. Results
Despite challenges and difficulties, the two PBL groups worked collaboratively and successfully
completed the set task. Peer-to-peer and tutor feedback was provided at the end of the trial.
Below are presented the results linked to ‘Online cross-institutional collaborative learning’. These
highlight how participants experienced online PBL in collaboration with colleagues from other
institutions.
4.1 Online cross-institutional collaborative learning
4.1.1 Multi-institutional collaboration
Participants and facilitators found working with colleagues from other institutions both a novelty and
beneficial. Many cited it as the main reason they had decided to participate in this trial. In recent
years, MOOCs have been very popular; attracting thousands of participants through providing
560

Chrissi Nerantzi
individuals from across the globe the opportunity to connect with others and form learning networks
(Downes, 2010) beyond institutional boundaries or identities.
One participant confirmed that
"Communicating with people from other institutions was one of the best aspects of the
trial, it was good to exchange ideas with people from other institutions [...] it was novel
and exciting – this aspect kept me going on the trial really!"
Whilst another stated
“The good side was that I got to meet people which I wouldn’t have done working with
people in different institutions. It is always nice to meet new people and work in different
ways, so it is a good experience.”
4.1.2 Multi-disciplinary groups
Overall, participants were positive about working in online multi-disciplinary groups.
“It was very positive. Especially because we all came from different backgrounds.
Enriching my experience a lot. Because, I was a scientist and I looked at the problem in a
very scientific way. Divided it in my head and categorised it. And they were more global
and social and personal. I didn’t think very much on the personal aspect, aspect, as I told
you, first of all, I was very sort of puzzled by the scenario and I felt, because I didn’t see
the problem to solve. And they saw the problem more globally and they had that insight
that I didn’t perhaps have.”
One participant, however, noted that
“it was really frustrating dealing with people who came from different disciplines. [One
participant] came from such a natural science background and there was a lot
misunderstanding there in terms of how things are done because of the different culture
of disciplines and I know that this is a problem on the PgCert course that we got here
generally, in term of introducing natural sciences to educational research is not always a
happy experience and particularly get people understand what reflection is and how to
articulate it.”
In the ongoing debate around multi- and mono-disciplinary academic development, McLean (2009)
highlights the importance of multidisciplinary conversations; they have the potential to enrich the
exchange of ideas and co-construction of knowledge. Conversely, Healey and Jenkins (2003) favour
a discipline-focused academic development approach which might explain the difficulties and the
frustrations expressed only by the above participant.
4.1.3 Community
In the words of one participant’s blog entry, participants missed the “real human contact–eye-to-eye,
smile, feeling the other’s real presence”. There was “the sense of writing into a black hole”, and of
missing the feeling of being part of a community. This was upheld by another participant, who noted
that “I would have liked to come away feeling it was more of a community being created”.
Socialisation activities were available at the start of the trial, to enable participants and facilitators to
get to know each other, but these were not fully explored.
Donnelly (2010) notes that online interaction appears more successful when there is an interpersonal
and social dimension which can lead to enhanced participation, motivation and learning in an online
environment. This is in line with the findings of this trial and with Wenger et al. (2011, 10) who
recognise that “The formation of a community creates a social space in which participants can
discover and further a learning partnership related to a common domain.”
4.1.4 Group size
On the matter of group size, one participant mentioned:
“Actually we lost one person and that might have been a blessing actually. Just in terms
3 people are easier to organise than 4.”
561

Chrissi Nerantzi
agreeing with Novak (1989) and Donnelly (2009) who note that smaller groups make online
communication and collaboration more effective and active.
Another participant, disagreed, stating
“We had only three of us, and this small number may have limited the scope of my
learning experience.”
4.1.5 Rules
As also recognised by Shea (1994), participants highlighted the importance of establishing ground
rules when working with others online.
One participant, for example, stated
“The basic manners and etiquette must be clearly communicated at the beginning; For
instance at the beginning I was apologising to cut other's writing, but I later found out that
it was taken for granted. I wished that we had a discussion on those very basic ethics
and manners working online within our team.”
There was also a reluctance to proceed without all responses from other group members, as one
participant noted
“I think we also lacked leadership. I think to be fair, all three of us, are used to be leaders
in our own setting, we were very keen, I think, initially, not to tread on each other’s toes
too much.”
4.1.6 Facilitation
This was the theme on which participants commented most extensively. In the anonymous final
survey, one participant stated that
“The chief thing that the trial highlighted for me was the importance of the facilitator to the
success of the project. It is a lot more work doing things this way, and the facilitator
needs to be pretty “hands on" in the absence of face-to-face meetings between group
members.”
Reflecting on their roles and performance, both facilitators agreed an imperative need to improve
facilitation; to offer the support and guidance required to participants during online PBL activities with
the intent of enhancing engagement and learning. Both also agreed that they learned a lot and now
have a better understanding of what does and does not work in online collaborative PBL. For
example, one stated that
“There is a lot I learnt from the whole process even I was disappointed with myself how I
facilitated. I don’t think I did a good job. [...] I have to admit, it didn’t go as well as I
wanted it to.”
while a participant mentioned
“I felt a bit like, I was not knowing which direction I was taking and a bit sort of always in
a doubtful sort of perspective, whether I’m actually reading the right material, whether I’m
going to the right things, whether I’m following all the right stuff that I’m needing. a little
bit in the dark [...].”
The above observation is echoed in a number of responses from participants who also felt
disorientated and unsure, and were seeking informed support in what they were supposed to be
doing. The hands-off approach adopted by the new PBL facilitators in this trial is in line with Savin-
Baden’s (2003, 50) observations that “facilitators new to problem-based learning often feel that it is
better to say less – or even nothing – so that the students feel that they are taking the lead in the
learning.”.
4.1.7 Technologies
Some participants, it was noted, felt confused, frustrated and irritated –being unsure how, and on
what criteria, the collaborative tools had been chosen and how they would be used. This frustration of
online participants towards technology is echoed by Hara & Kling (1999).
562

One participant commented
Chrissi Nerantzi
I was curious about the choice of tools. Were they what facilitators felt comfortable with?
I am happy online. I forget how daunting people find the technology. [...] Oh!, it is really
complicated. [...] How to buy in? To give them the initial knowledge-base. People are
selective of which platforms they use.
The use of two different platforms, “rather than having an integrated environment” (participant) for the
trial, added to the confusion. This is highlighted by the participant who noted that “navigating through
the blog, using the wiki as well, it became more frustrating as it progressed”. This is affirmed by
another participant, who decries “the irritating platforms I found the set-up very cumbersome”, and by
facilitators’ comments.
Leinonen et al (2009) documented similar experiences, finding it a challenge to deliver an open course
at the University of Art and Design Helsinki. They maintained that “The communication tools used in the
course — blogs and wiki — were found by most participants rather confusing and sometime frustrating”
(online). The complexity added to learning through the use of multiple tools and environments used for
online courses is also noted by Levy (2011).
5. Discussion
The findings of this trial strongly support the notion that participants enrolled on institutional PgCerts
value the opportunity to work with colleagues from other institutions. Many of them participated in the
trial for this reason, and found that this more open approach enabled them to make new connections.
Wenger et al. (2011) discuss the value of learning in social networks and communities of practice
and, both during and after the trial, participants recognised the value and potential of online
collaborative learning in this open and networked format. Many current PgCert programmes already
enable individuals from different disciplines in the same institutions to come together, creating wider
communication, collaboration, multidisciplinary learning and knowledge co-construction beyond
academic and discipline-specific silos.
The opportunity, and perhaps the need, now exists to broaden this scope, and create more open
online collaborative learning opportunities for PgCert participants beyond institutional boundaries.
This trial has provided evidence that these can encourage a culture of openness, sharing and
exchange and be beneficial for the institutions as well as those individuals involved. Widely and freely
available social media can be used to enable and facilitate a more open educational offering within
accredited and non-accredited Academic Development provision. This model can provide the space
to be more explorative, creative and outwards facing. It can help develop enhanced networking, team
and collaboration skills. It also immerses staff involved in teaching or supporting learning in HE into
alternative more open and fruitful delivery approaches which have the potential to be transformative
for their own practice and provide food for thought about potential learning partnerships.
Põldoja (2010, 2) highlights that “learning is a social process and open content is not the only way to
change the educational system towards openness. In addition to open content we need open learning
environments and teaching practices”. In the last few years such environments and courses have
been created (Põldoja 2010) as well as Massive Open Online Courses (MOOCs) a name given by the
participants of the Connectivism and Connective Knowledge Course 2008 (Siemens 2008) who were
around 2,200 (Downes 2010).
Responses by facilitators also indicated clearly that a multi-institutional approach is welcome and that
there is a place for PBL within PgCerts to facilitate such open activities, especially if linked to
assessment. Assessment should enable participants to build new knowledge and develop their
contextualised problem analysis and problem solving skills through collaborative learning (Birenbaum
and Dochy, 1996). Using PBL for delivery and assessment constructively aligned with the intended
learning outcomes (Biggs 1999) has the potential to make PBL more effective because students “will
learn what they think they will be assessed on” (Biggs, 2004, 3).
To make online PBL more effective in the context of open and collaborative education, it will be
important to design and plan such activities thoroughly before implementation. Participants should be
able to personalise the technologies they are using, and be provided with a collaborative platform and
framework which is well supported and facilitated, has a clear focus and in which activities are
scaffold (Juwah 2002) as well as enable peer support and learning. These activities should enable
563

Chrissi Nerantzi
familiarisation with the technology and PBL, and lay the foundations for learning partnerships, as well
as a learning community in which collaboration and learning can take place and strengthen selfdirected
and network-directed learning.
6. Conclusions
The overall aim of this research project was to introduce and evaluate an online PBL approach within
Academic Development that would connect participants from different institutions. A small-scale PBL
trial was conducted using social media.
Findings linked to the category ‘Online cross-institutional collaborative learning’ show that there is the
need to open our programmes and create opportunities for collaboration beyond module, programme
silos and institutions. Participants in this study valued the opportunity to connect and learn with
colleagues from other institutions and felt that this was an enriching learning experience despite the
difficulties they were confronted with.
Open learning is currently still uncommon within Academic Development. It is recommended that
module and programme teams explore options for freeing their programmes of studies and working
together with other institutions to promote a more open educational model based on network-directed
learning using social media and enable learners to choose the digital tools they would like to use.
Such a cross-institutional design and delivery model would also be beneficial for other professional
areas and disciplines.
Overall, there are many benefits from such initiatives for learners, educators and institutions beyond
the positive effect it has on learning and engagement such as
Using existing resources and expertise more effectively through sharing and exchange with other
institutions.
Utilising freely available social media tools and technologies, accessible to or owned by learners,
enabling enhanced connectivity, thereby increasing buy-in.
Adapting and creating resources collaboratively, preferable as OER and sharing with other
learning communities.
Developing and delivering sessions, modules and programmes in collaboration and partnership,
thus enriching institutional offers.
Providing learners the opportunity to connect with other learners beyond module and programme
level and become active members of more open learning communities.
Using opportunities for collaboration and shared pedagogical and subject-specific research and
scholarly activities to raise standards of teaching and create good relationships among
institutions, transforming competitiveness into cooperation –aiming for a common good.
Open cross-institutional learning makes learners feel in charge and responsible for their learning.
Learners become more explorative and their appetite for learning increases. At the heart of open
learning is the opportunity to connect with others and build bridges and networks for collaborative
learning and knowledge co-construction with short-and long-term benefits as Wenger et al. (2011, 12)
note “Being more interconnected often increases the sense of community, and a desire to learn about
a shared concern often motivates people to seek connections.”
Acknowledgments
The author would like to thank Dr. Nicola Whitton for her valuable comments and suggestions during
the writing process and also for her encouragement.
References
Barrett, T. (2010) “The problem-based learning process as finding and being in flow” in Innovation in Education
and Teaching International, Vol. 47, No. 2, May 2010, Routledge, pp. 165-174.
Barrows, H. S. and Tamblyn, R. M. (1980) Problem-based Learning. An Approach to Medical Education.
Springer, New York.
Biggs, J. B. (1999) Teaching for Quality Learning at University, SRHE and Open University Press, Buckingham.
Biggs, J. B. (2004) “Aligning teaching for constructing learning, HEA”, available at
http://www.heacademy.ac.uk/assets/York/documents/resources/resourcedatabase/id477_aligning_teaching
_for_constructing_learning.pdf [accessed 1 May 2011]
564

Chrissi Nerantzi
Birenbaum, M. and Dochy, F. (eds.) (1996) Alternatives in assessment of achievements, learning processes and
prior knowledge. Kluwer Academic Publishers, Boston.
Boud, D. (1985) Problem-based learning in education for the professions, Higher Education Research and
Development Society of Australasia, Syndney.
Boustedt, J. (2008) “A Methodology for Exploring Students’ Experiences and Interaction with Large-scale Software
through Role-play and Phenomography”, ICER 08, September 6-7, Sydney Australia, available at
http://portal.acm.org/citation.cfm?id=1404524 [accessed 13 May 2010]
Chernobilsky, E., Nagarajan, A. and Hmelo-Silver, C. E. (2005) “Problem-Based Learning Online: Multiple
Perspectives on Collaborative Knowledge Construction”, CSCL '05: Proceedings of the 2005 conference on
Computer support for collaborative learning, pp. 53-62, available at http://www.eduprojects.eu/euclides/elibrary/Chernobilsky.pdf
[accessed 16 May 2011].
Cousin, G. (2009) Researching Learning in Higher Education, An Introduction to Contemporary Methods and
Approaches, Routledge, Oxon.
Donnelly, R. (2005) “Using Technology to support project and problem-based learning”, Barrett, T., Mac
Labhainn, I & Fallon, H. (eds.) Handbook of Enquiry and Problem-based Learning: Irish Case Studies and
International Perspectives, National University of Ireland, Galway, AISHE Readings, Number 2, pp. 157-
177, available at http://www.nuigalway.ie/celt/pblbook/contents.htm [accessed 6 May 2011]
Donnelly, R. (2009) “The nexus of problem-based learning and learning technology: Does it enable transformative
practice?”, European Journal of Open, Distance and E-Learning, 2009/II, available at
http://www.eurodl.org/?p=current&article=371 [accessed 19 May 2011]
Donnelly, R. (2010) “Harmonizing technology with interaction in blended problem-based learning”, Computers &
Education 54, pp. 350-359.
Dortins, E. (2002) “Reflections on phenomenographic process: Interview, transcription and analysis: Interview,
transcription and analysis”, Quality Conversations, Proceedings of the 25 th HERDSA Annual Conference,
Perth, Western Australia, 7-10 July 2002, in pp. 207-213.
Downes, S. (2010) “New Technology Supporting Informal Learning”, Journal of Emerging Technologies in Web
Intelligence, 2(1), pp. 27–33.
Hara, N. and Kling, R. (1999) “Students’ Frustrations with a Web-based Distance Education Course”, First
Monday, 4 (12), available at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/issue/view/113
[accessed 24 June 2010]
Healey, M. and Jenkins, A. (2003) “Discipline-based educational development”, H. Eggins & R. Macdonald (Eds.)
The scholarship of academic development (pp. 47–57). Buckingham: SRHE/Open University Press.
Hmelo-Silver, C. E., Derry, S. J., Bitterman, A. and Hatrak, N. (2009) “Targeting Transfer in a STELLAR PBL
Course for Pre-service Teachers”, The Interdisciplinary Journal of Problem-based Learning, Volume 3, No.
2, Fall, pp. 24-42.
Jones, C. and Steeples, C. (2002) “Perspectives and Issues in Networked Learning”, Steeples, C and Jones C.
(eds.) Networked Learning: Perspectives and Issues, London: Springer, pp. 1-14.
Juwah, C. (2002) “Using communication and information technologies to support problem-based learning”, available
at
http://www.heacademy.ac.uk/assets/York/documents/resources/resourcedatabase/id449_using_it_to_supp
ort_problem-based_learning.pdf [accessed 5 May 2011]
Kear, K. (2011) Online and social networking communities. A best practice guide for educators, open & flexible
learning series, Routledge, Oxon.
Leinonen, T., Vadén, T. and Suoranta, J. (2009) “Learning in and with an open wiki project: Wikiversity’s potential
in global capacity building”’, First Monday, 14(2), available at
http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/art icle/view/2252/2093 [accessed 30 May 2011]
Levy, D. (2011) “Lessons Learnt from participating in a Connectivist Massive Open Online Course (MOOC)”,
presented at the annual Chase conference for instructional technologies research, February 17, 2011, The
Open University, Raanana, Israel, available at
https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B5ppG_u6D8hFOWE2YTY1NWM
tNzM3Ny00MGQ0LWE2NzMtYzMzYjM0NmZkZjc5&hl=en&authkey=CMOUnb8C [accessed 1 June 2011]
Marton, F. (1994) “Phenomenography as a Research Approach”, Husen, T. and Postlethwaite, N. (2 nd ed) The
International Encyclopedia of Education, Vol. 8, Pergamon, pp. 4424-4429, available
athttp://www.ped.gu.se/biorn/phgraph/civil/main/1res.appr.html [accessed 7 May 2011].
McLean, M. (2009) “Critical Mass”, Ramsden, P. (ed.) Teachers as learners – the development of academic staff,
HEA: Academy Exchange 8, August 2009, pp. 24-26.
Mezirow, J. (1997) Transformative Learning: Theory to Practice, in: Transformative Learning in Action: Insights from
Practice. New Directions for Adult and Continuing Education. no. 74, edited by P. Cranton, Jossey-Bass, San
Francisco, CA, pp. 5–12.
Novak, J. (1989) “Helping student learn how to learn: a view from a teacher-researcher”, Third Congress of
Research and Teaching in Science and Mathematics, Santiago de Compostela, Spain, Sept. Reviewed in PS
News 69.
Põldoja, H. (2010) “EduFeedr: following and supporting learners in open blog-based courses”,Open ED 2010
Proceedings. Barcelona: UOC, OU, BYU. available at http://hdl.handle.net/10609/4861 [accessed 30 May
2011]
Savin-Baden, M. (2003) Facilitating Problem-Based Learning, Illuminating Perspective, SRHE and Open University
Press, Maidenhead.
565

Chrissi Nerantzi
Scardamalia, M. and Bereiter, C. (1994) “Computer support for knowledge-building communities”, The Journal of
Learning Sciences, 3(3), pp. 256-283.
Shea, V. (1994) Netiquette, Albion Books, San Francisco.
Siemens, G. (2004) “Connectivism: A Learning Theory of the Digital Age”, available at
http://www.elearningspace.org/Articles/connectivism.htm [accessed 6 May 11]
Siemens, G. (2008) “MOOC or Mega-Connectivism Course”, available at
http://ltc.umanitoba.ca/connectivism/?p=53 [accessed 30 May 2011]
Siemens, G. (2011) “Moving beyond self-directed learning: Network-directed learning”, 1 May, available at
http://www.connectivism.ca/?p=307 [accessed 17 May 2011]
Torp, L. and Sage, S. (2002) Problems as possibilities: Problem-based learning for K-16 education (2nd ed.)
Association for Supervision and Curriculum Development, Alexandria, VA.
Virzi, R. (1992) “Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough?”, Human
Factors, 34(4), pp. 457-468. Available at
http://coursesite.uhcl.edu/hsh/PeresSc/Classes/PSYC5911www/Refining%20the%20test%20phase.pdf
[accessed 20 May 2011]
Wenger, E., Trayer, B. and de Laat, M. (2011) Promoting and assessing value creation in communities and
networks: a conceptual framework, Rapport 18, Ruud de Moor Centrum, Open Universiteit, available at
http://www.social-learning-strategies.com/documents/Wenger_Trayner_DeLaat_Value_creation.pdf
[accessed 3 June 2011]
Wiley, D. (2006) “Open Source, Openness, and Higher Education”, innovate, Oct/Nov, Volumne 3, issue 1,
available at
http://www.innovateonline.info/pdf/vol3_issue1/Open_Source,_Openness,_and_Higher_Education.pdf
[accessed 19 May 2011]
566

Why Recording Lectures Requires a new Approach
Paul Newbury, Phil Watten, Patrick Holroyd and Clare Hardman
University of Sussex, Brighton, UK
P.Newbury@sussex.ac.uk
P.L.Watten@sussex.ac.uk
P.Holroyde@sussex.ac.uk
C.L.M.Hardman@sussex.ac.uk
Abstract: It is now commonplace for Universities to record lectures with video cameras. Indeed there are several
off-the-shelf systems, which Universities can purchase to provide this type of functionality, e.g. Echo360, Panopto
etc. There are also several distribution outlets available, such as iTunesU and YouTube EDU, which Universities
can use to distribute this recorded media to students. However, the capture of standard lecture material with
these systems can only provide partial support to learning. Material recorded in this way can be engaging for
students who attended the original lecture, but has less efficacy for students who are seeing the material for the
first time. To be truly effective learning mechanisms in their own right, these new recording systems need to
address two key issues. Firstly, current lecture material is overwhelmingly designed for the live lecture theatre
audience. Consideration is rarely given to how these materials will support learning when viewed as stand-alone
learning resources. Secondly, as lecture theatres are rarely designed for video capture, the off-the-shelf
recording systems are often severely limited by the environment, equipment and resources available. Lighting
and camera position are key considerations that have a big impact on the quality of the captured material, but are
generally restricted by the environment required for the live audience. This paper reviews these two key issues
and presents both a framework for the production of teaching material targeted at video capture, and the bespoke
recording system developed for online learning in the School of Informatics at the University of Sussex.
Additionally the paper covers analysis of download rates, qualitative staff and student feedback and lecture
attendance and shows that using this framework has a significant effect on the student interaction with recorded
material. Other types of online support such as providing copies of lecture slides are also discussed and a
tangible improvement in engagement over these techniques is shown.
Keywords: digital video, video streaming, multimedia, podcast, eLearning
1. Introduction
The majority of Universities now use some form of digital capture to record at least part of their lecture
provision and the recording of audio and slides (often referred to as podcasts, screencasts or
enhanced podcasts) has been around for several years. However, the use of mainstream video is a
fairly recent innovation (Copley 2007), with the vast majority of current higher education material
being produced from a single camera placed at the back of the lecture theatre with the possible
addition of a separate capture of the slides. There are several off-the-shelf systems that Universities
can use for this type of capture such as Echo360 (Echo360 2011), Panopto (Panopto 2011) etc. As
might be expected several studies have shown that this is generally advantageous to the students
(Woo 2008, Bradley 2009), however this type of material is often best used for students revisiting the
lecture (after attending) to refine their notes or for revision (Bongey 2006). There are several key
issues with this type of media when using it as a learning resource in its own right, for example when
the student has missed the original lecture or the material is being used in a distance learning context:
The material is generally designed for the live lecture, with little consideration to the constraints of
the recording or delivery mechanism used. For example the screen/board is often hard to read on
the recording. Text and diagrams are often too small to see when reproduced on video
The recorded material lacks focus, the presenter, audience and screen are all included in the
capture
Lighting is often an issue. It is hard to light both the presenter and the screen in a reasonable
manner
These problems (as can be seen in Figure 1) often combine to make the recorded lectures of limited
use as a teaching medium, and can undermine the additional learning that might be considered
implicit with this type of support material (Woo 2008, Bradley 2009).
The issue of the presentation being difficult to read is often addressed by the use of a screen capture
of the presentation slides provided along with the video recording (see figure 2), however this in itself
does not completely address the issue and can cause two additional problems. Firstly, it can reduce
the students’ view of the presenter’s interaction with the slides. The viewer has a choice of reading
567

Paul Newbury et al.
the slides or watching the presenter, unlike the live lecture where students can see both the presenter
and read the part of the presentation being discussed at the same time. Secondly, it can encourage
the creator of the teaching material to ignore issues of display resolution as they know that the slides
will be reproduced in full, which inherently leads to poor presentation material.
Figure 1: Echo 360 video capture of a lecture
Figure 2: Echo 360 video and slide capture of a lecture
In an effort to address these issues this paper reviews a pilot scheme that uses high-quality digital
video resources to support student learning for lectures. The specialist studio facilities at the
University of Sussex enable a significant improvement on standard video capture and the creation of
bespoke teaching material aimed at digital capture enhances the student learning experience,
particularly when compared to standard capture methods. However, this scheme does have additional
resource implications, which must be considered in the larger context when implementing a University
wide system. It is the intention of this research to show that this type of capture creates significantly
568

Paul Newbury et al.
more interaction with the student and thus provide a basis for further research to address the inherent
resource implications without affecting student interaction.
2. Framework for the provision for digital video
Lecture capture is often considered to be a beneficial add-on to the actual lecture, but this leads to the
issues that have been outlined above. A more appropriate way to think of it is teaching resource
production and the following framework has been designed around this concept. There are essentially
3 key parts to the production pipeline outlined in this paper, materials development, live recording and
streaming.
2.1 Materials development
It is vital when developing teaching material that is going to be recorded that the constraints of the
recording mechanism and the delivery platform are taken into consideration. The student may view
the recorded material on a wide range of devices with varying resolutions and it is key that all material
is clear at the lowest resolution. At present the lowest resolution target device for this research is the
iPhone 3G (Figure 3).
Figure 3: iPhone streamed video (including signed translation).
Thus, there are some straightforward physical rules that need to be observed, such as specific
requirements as to minimum font size, use of colours and safe view area for all presentations.
However, special handling of content is also required. It is not possible for the presenter to provide a
significant amount of text on each slide, thus thought must be given to how textual material can be
replaced by graphical material wherever possible, as shown in Figure 4. This approach also has key
learning benefits in its own right, as outlined in Mayer’s theory of Multimedia learning. Mayer found
that on average, “students who listen to (or read) explanations that are presented solely as words are
unable to remember most of the key ideas and experience difficulty in using what was presented to
solve new problems” (Mayer 2003:126).
Figure 4: Original slide (left) with replacement slides (centre and right) covering the same material
which contain animation and audio and are specially prepared for digital capture
The significant amount of pre-production work put into the development of slides for digital capture
means that these slides are media rich and engaging. The use of the studio also means that full
569

Paul Newbury et al.
multimedia facilities are available so that sound, video and animation are common components of
lectures.
2.2 Live recording
The objective of the proposed system is that presentation material is captured as efficiently as
possible, and that means in real-time. In most systems this involves the use of a single camera used
to record the presenter and often all or part of the presentation area. Additionally, as mentioned
previously, systems usually record a separate capture of the slides straight from the presentation
machine. However, the system discussed in this paper produces high-quality engaging output by
using multiple cameras to cut between camera views in real-time. The Media Technology Lab (MTL)
in the School of Informatics consists of two broadcast studios and associated control galleries and
nonlinear editing facilities. However, for the majority of this work a single studio and gallery is used as
shown in Figure 5. Generally 3 cameras and slide capture is used in presentations, although it is
possible to get reasonable results using just a single camera and slide capture. The studio has been
built in a highly configurable manner, which enables quick conversion and setup between different
presentations.
Figure 5: The control gallery and one of multiple camera views of the presenter/screen
The use of a multiple camera set-up does involve the resource implication that a second person (the
director) is required for each lecture to cut the material as it is captured. However, this is a vast
improvement on the alternative of post-production editing from multiple camera recordings.
When reviewing lectures through the web-streaming framework, students are presented with a clear
representation of the material covered through a selection of views (including multiple views of the
presenter, screen and slide capture). The director is responsible for the views provided from the live
edit and selects material to best focus the students’ attention, therefore aiding the learning impact of
the material. This is unlike the experience that students receive from standard video lecture capture
where the whole scene is captured and it is down to the student to select the most relevant material
from the presentation (Figure 1). For the viewer the experience from the live studio recording is much
like that, for example, of the television viewers of the Royal Institution Christmas lectures.
2.3 Delivery
Captured lectures are available to students as on-demand programmes from a dedicated website.
The video-on-demand distribution system takes recorded material and makes it ready for distribution
online. As the material is recorded live, it does not require any post-production or editing so it can be
encoded and uploaded to the web within minutes of the presentation ending. Each programme is
automatically encoded into three video types shown in Table 1.
Two streams of differing quality are provided for desktop web-based viewing using an Adobe Flash
web-based player – in this case JWPlayer (Longtailvideo 2011). The Flash player is hosted within a
JavaScript web application driven by a content management system, and fed with a Real Time
Messaging Protocol (RTMP) video stream from a Wowza media server (Wowza 2011) system (see
Figure 6).
The high-quality stream offers the smoothest video, largest picture size and highest bit rate, see
Figure 7.
570

Table 1: Video encoding parameters
Video
type
High
quality
Low
quality
Mobile
(iOS)
Minimum input
requirement
720p/(25/50) HD
video
576i/25 SD video
Output
resolution
Figure 6: Streamed delivery framework
Figure 7: High-quality web page streamed video
Paul Newbury et al.
Output
frame rate
File size (average 50
minute programme)
Codec and average
bit rate
640 x 360 50/25fps 330MB h.264 - 1100
Kbits/sec
512 x 288 25fps 140MB 700 Kbits/sec
480 x 270 25fps 280MB h.264 - 1000
Kbits/sec (no Bframes)
571

Paul Newbury et al.
It is designed for high-bandwidth connections such as on-campus, but works well on good broadband
connections. It is impossible to say what a good connection is due to broadband being sold at speeds
‘up to’ a certain bit rate. However, if a speed-test reports a constant bit rate of over 2.5Mbits/sec, then
a high-quality stream will play successfully.
A low-quality stream is provided for slower connections, and also is often required during peek
Internet access times when the contention ratio of some broadband users results in a low attainable
bit rate.
A mobile stream is provided for mobile devices – specifically iOS-based (iOS 2011) devices at
present. A mobile website is driven from the same content management system as the desktop
version, with video content being handled directly by the iOS video engine. A dedicated app has been
developed that can be used instead of the mobile website. It is presently still in testing, with the
intention of releasing for the start of the 2011/2012 academic year.
2.3.1 Live delivery
The live recording production method for the on-demand content opens up the possibility of live
transmission of events. This results in programmes that look and feel the same as the recorded
programs, but which are now transmitted in real time. This gives some additional advantages over
recorded material. Firstly, it can be compelling to watch. Secondly, the audience is more forgiving of
mistakes. Thirdly, it gives the online audience the ability to interact live from either the desktop web
application, or an iOS device. Figure 8 shows a live presentation being given with a voting system in
operation.
Figure 8: Live presentation
In order to transmit live, there is a small modification to the production system (see Figure 9). The
Wowza streaming server is now fed with a single quality live h.264 RTMP stream targeted at playback
in a Flash player. From this, the Wowza server then generates a second HTTP stream targeted at
572

Paul Newbury et al.
mobile devices. Care has to be taken to ensure the originating audio and video are synchronised.
Video processing and routing systems can delay video, making it late with respect to the audio output.
Therefore an audio delay is added to resynchronise before transcoding.
User interactions are fed back to a separate interaction-server by both the desktop web browser and
a dedicated mobile app (interactions in web-based mobile video is currently limited). The interactionserver
feeds data directly into the live production environment where it can be automatically managed
or manually dealt with.
Figure 9: Live stream delivery framework
Student engagement often suffers if key lectures are missed which makes it hard for them to absorb
the taught material following that lecture even if they have access to the slides and notes. This
research found that students who do not physically attend a lecture but watch it live online or later
from a recording receive a very similar experience to those who do attended. The issue of attendance
is covered in section 3.4.
3. Pilot study - measurement of student interaction with capture material
The purpose of this pilot study is to measure student interaction with different types of recorded digital
content, in an attempt to discern the effect of the capture method on the student interaction with
recorded material. The participating classes are taken from the last three years of a level two
Multimedia Systems course, where the capture method has been different for each cohort. The initial
cohort (2007/08) was provided with online digital copies of lecture slides in .pdf and .ppt formats,
using the University’s Moodle-based course management system (StudyDirect). The following year
(2008/09) lectures were timetabled in the Media Technology Lab (Figure 5), where a full multi-camera
live edit recording could be made and provided online to the students (Figure 7). The live recording
framework outlined in section 2.2 and the delivery mechanism described in section 2.3 were used. In
the final year of the pilot study (2009/10) lectures took place in a standard lecture theatre, but were
recorded using the ECHO360 system and delivered using StudyDirect (Figure 2). The material for all
three years of this course was constructed in line with the materials development structure set out in
section 2.1.
As cohorts differ in size, makeup and aptitude between years it is hard to make definite claims about
student interaction from one cohort to the next. However, it is generally considered that qualitative
student feedback and download rates provide an insight into the student engagement with the
recorded material (Bradley 2009, Day 2006). Thus this paper assesses students’ interaction using the
following 3 key criteria:
573

Paul Newbury et al.
Download figures for the digital material – hit rate based on IP address
Qualitative student feedback – end of course questionnaire
Qualitative faculty feedback – end of course discussions
A commonly cited concern with digital capture is that it might lead to a drop in attendance for the live
lecture. Thus, attendance rates are also reviewed to see if this is affected by the provision of online
content.
3.1 Download figures
The following data shown in Figure 10 represents the material downloaded by the three student
cohorts for each of the past three years of the same level two course discussed above. Download
statistics are based on material accessed from different IP addresses, multiple accesses from the
same IP address to the same material in a short period of time are counted as a single download.
However, as students often access material from Department computers it is possible for different
students to download the same material from the same IP address over longer periods of time and
these are counted as separate downloads.
In 2007/8 the only material available for download was the lecture slides. In 2008/09 lectures were
held in the MTL studio and a full studio recording was available to the students, using the framework
described in sections 2.2 and 2.3. In 2009/10 the studio was unavailable, but the lectures were run in
an Echo360 enabled lecture theatre and a digital recording provided. Figures have been normalised
for the slight variations in cohort size between years. Also, technical problems were encountered with
the Echo360 recording on several occasions and the download results for the Echo360 recordings
have been extrapolated from the successful recordings.
900
800
Static Slides 07/08
700
600
MTL Video 08/09
500
Echo360 09/10
400
300
200
100
Figure 10: Number of student downloads per year of course
As might be expected the download rates differ significantly between the first and second years of the
study. The provision of “Television Like” video from the MTL studio has had the desired effect of
substantially increasing student download rates. The download rates for the MTL video equate to over
one download per student per lecture, although it is not possible to determine who is actually
accessing the material and how many times, it is downloaded by a single person. Interestingly,
download rates for the Echo360 captures in 09/10 are less than a quarter for those for the MTL
recordings the year before. As the lecture presentation slides were essentially identical between the
two years, this result would suggest that the difference in the recording/delivery mechanism makes a
significant effect on student use of the recorded material.
3.2 Students’ feedback
Students from several courses were polled on their opinion concerning the material captured using
the MTL studio framework outlined in sections 2.2 and 2.3. From those results: 92% had watched the
video material online and 23% rated this material as “Invaluable” with the remaining 77% rating it as
“Very Useful”. Qualitative feedback followed these results with students believing that this was a
positive aid to their learning that they would wish to see used in all their subjects. Some example
student comments:
“I find it useful to replay a lecture. It helps consolidate the information. The videos will be
useful for revision.”
“We can see the lecturer himself not just hearing his voice, have a feeling of realism and
like watching TV”
574

Paul Newbury et al.
“The slides are always very clear and understandable and it is easy to see what the
things we need to know are.”
“If you didn't quite catch something that was said or don't fully understand it, you can go
back and watch it again online rather than stop the lecture mid-flow and ask them to
repeat it.”
“They are an excellent resource - at revision they will be so much more useful than just
slides and audio.”
“You get to hear and see the full explanation of the slides and this is much more
interesting than podcasts or StudyDirect videos.” (N.B. StudyDirect video is captured
using Echo360.)
3.3 Faculty feedback
Those taking part in both the MTL studio and the Echo360 recordings found them a rewarding
experience, although some faculty found the familiar setting of the standard lecture theatre with
Echo360 setup less intimidating. However, the Echo360 capture was often frustrating and on several
occasions the video failed to record leaving just an audio and slides recording of the lecture. An
unexpected consequence of this work was the ability for a lecturer to review their own lectures,
something they found very useful when revising, updating and improving the delivery of the material.
Faculty do not normally get the opportunity to view their own lectures and modify their delivery based
on this view and they were very engaged with this aspect of the trial. Faculty were also very positive
about the required changes in material perpetration from text to more multimedia based teaching and
felt that having to think about reducing text and increasing graphics improved the lecture material
significantly. The faculty impression of the students’ engagement was that they were more interested
by the material recorded in the MTL and were excited by the studio atmosphere. Inattention issues
were reduced and there were no issues with disruptive behaviour.
3.4 Attendance figures
Although only limited research has been performed on the effect of lecture recording on student
attendance (Bradley 2009, Traphagan 2009, Harpp 2004) it is a commonly expressed concern that
providing recordings of lectures is likely to have a detrimental effect on attendance. The common
question “Why would students attend lectures when they could watch the lecture at home” is often
posed. However, previous research suggests that provision of podcast material has limited effect on
attendance (Traphagan 2009, Harp 2004). In cases where there is a drop in attendance this is often
not converted into a drop in student achievement in the course as the additional support material
alleviates this issue (Harp 2004). Although the enhanced digital media created in this research could
be considered more likely than standard podcast technologies to result in a drop in attendance the
results were found to be broadly in-line with previous studies. Attendance is taken at all lecture and
laboratory sessions. The percentage attendance figures for the above 3 years of courses can be seen
in Figure 11.
70
65
60
55
50
Figure 11: Percentage attendance figures
Static Slides 07/08
MTL Video 08/09
Echo360 09/10
Several factors affect attendance on courses from year to year. One of the key factors in attendance
on the above course was the move to lecture slots earlier in the day for the 08/09 cohort. This may
well be the reason for the slight drop in attendance at this point. The 09/10 cohort has the same
lecture times as 08/09 and attendance has rallied slightly. Although a small subset to extrapolate
from, it seems that attendance has not been affected significantly by the studio-based videoing of the
lectures. When students on courses with access to the studio recordings were anonymously asked
575

Paul Newbury et al.
“Has the fact that you can view the lectures online changed the number of lectures that you have
attended?”, 54% said no, 38% said they had attended less and 8% said they had attended more.
4. Conclusion
The student response to the provision of lecture material as digital video has been overwhelmingly
positive and similar to previous studies in this area (Bradley 2009, Bongey 2006, Woo 2008, etc.).
Indeed this type of result is generally to be expected as the provision of additional support
mechanisms are always likely to be viewed as a positive step by students. However, unlike previous
studies this pilot has attempted to assess engagement with a series of different capture methods. This
has demonstrated that, whilst all capture methods are considered positive additions, there is a
significant difference in the students’ response to static slides and Echo360 recorded video than there
is to the MTL video material. Not only are download rates significantly increased for MTL videos, but
the students comment that they find the videos interesting and “like TV” rather than the Echo360
content which they find less engaging and “harder to watch”. These results are in line with
expectations from Clark and Mayer’s work on Multimedia learning (Clark and Mayer 2011). Although
all formats followed the materials development framework (outlined in section 2.1), according to Clark
and Mayer’s principles they may have varying degrees of effectiveness, and if the students deem
particular formats to be less effective learning tools they would be less likely to use them. Clark and
Mayer provide seven principles of how to use multimedia to help learning and suggest that people
learn best when:
They have words and pictures rather than just words (Multimedia Principle)
Pictures are presented at the same time or next to the related text (Contiguity Principle)
They watch an animation with an auditory narration rather than text on screen (Modality Principle)
In the 2007/8 cohort, students only had access to the lecture slides without any audio content,
therefore losing the benefit of the Modality Principle. Based on Clark and Mayer's Contiguity Principle
the MTL video, in which slides and presenter are on screen together, is likely to be more effective
than the Echo360 version in which they are separated. Additionally, the use of multiple cameras in the
MTL studio allows multiple views of the presenter and slides which helps direct the students’ learning.
Thus, studio-based learning resources can be a more effective standalone learning tool and hence
students may be more motivated to use them.
Even though it may seem self evident that material produced in essentially a television studio will be
more engaging than that produced in a lecture theatre, the interesting result from this research is how
much more engaging this material seems. Download rates and qualitative analysis suggests that
students find the studio recordings significantly more valuable than those made in the lecture theatre
and that addressing issues on camera positioning and lighting have had a substantial effect.
As mentioned previously there is a reasonably high resource implication in the use of a studio for
recording lectures, but key findings can be drawn for this research which can be used to improve
standard lecture theatre recordings with systems such as Echo360. In particular the creation of
teaching material specifically aimed at lecture capture with the key aim of reducing textual content
and increasing graphics and animation can improve substantially the viewers’ experience of the
teaching material.
Although a significant concern of many academics, it seems that the provision of this high-quality
digital material has only a limited effect on attendance and the student performance overall is
increased.
5. Further work
So far this digital videoing pilot study has only been used in a few courses, and future work will
involve the rolling out of this framework to more of the subjects in the School of Informatics and
across the University, with the associated monitoring of student feedback and attendance. By
increasing the number of courses covered it is hoped that quantitative analysis of student results on
these courses compared to previous years can be made and significant improvements in attainment
shown.
576

References
Paul Newbury et al.
Bongey, S.B., Cizadlo, G. and Kalnbach, L. (2006) “Explorations in course-casting: podcasts in higher education”,
Campus-Wide Information Systems, Vol. 23, Issue 5, pp350-367.
Bradley, P., Summerside, C., Agar, M., Ansell, P., Humphrey, R., Knight, J., Mohammed, A., Moss, J., Watts, C.,
Wheeler, J., and Wolfendale, D. (2009) “Large scale implementation of a lecture capture system: a value
added initiative?”, ALT-C 2009 presentation.
Clark, R.C. and Mayer, R.E. (2011) eLearning and the Science of Instruction: Proven Guidelines for Consumers
and Designers of Multimedia Learning, San Francisco: John Wiley and Sons, Third Edition.
Copley, J. (2007) “Audio and video podcasts of lectures for campus-based students: production and evaluation of
student use”, Innovations in Education and Teaching International, Vol. 44, pp387-399.
Day, J. and Foley, J. (2006) ”Evaluating Web Lectures: A Case Study from HCI.” Conference on Human Factors
in Computing System 2006, pp195-200.
Echo360 Lecture Capture System (2011), [Online], Available: http://www.echo360.com/ [8 June 2011].
Harpp, D.N., Fenster, A.E., Schwarcz, J.A., Zorychta, E., Goodyer, N., Hsiao, W. and Parente, J. (2004) “Lecture
Retrieval via the Web: Better Than Being There?”, Journal of Chemical Edu, Vol. 81, n5, pp688.
iOS – Apple Mobile Operating System 4 (2011), [Online], Available: http://www.apple.com/iphone/ios4/ [8 June
2011].
JWPLayer video player (2011), [Online], Available: http://www.longtailvideo.com/ [8 June 2011].
Mayer, R.E. (2003) “The promise of multimedia learning: using the same instructional design methods across
different media”, Learning and Instruction, Volume 13, Issue 2, April 2003, pp125-139.
Panopto Focus Presentation Capture System (2011), [Online], Available: http://www.panopto.com/ [8 June 2011].
Traphagan, T., Kucsera, J.V., and Kishi, K. (2009) “Impact of class lecture webcasting on attendance and
learning”, Educational Technology Research and Development, Vol. 58, pp18-37.
Woo, K., Gosper, M., McNeill, M., Preston, G., Green, D., Phillips, R. (2008) “Web-based Lecture Technologies:
Blurring the boundaries between face-to-face & distance learning”, ALT-J. Vol. pp16-2, 81-93.
Wowza media server 2 (2011), [Online], Available: http://www.wowzamedia.com/ [8 June 2011].
577

eSubmission – UK Policies, Practice and Support
Barbara Newland 1 , Lindsay Martin 2 and Andy Ramsden 3
1 University of Brighton, Brighton, UK
2 Edge Hill University, Ormskirk, UK
3 University Campus Suffolk, Ipswich, UK
b.a.newland@brighton.ac.uk
lindsay.martin@edgehill.ac.uk
a.ramsden@ucs.ac.uk
Abstract: eSubmission is currently being implemented in universities within the UK Higher Education (HE). The
process of implementation is generally occurring in departments and Schools with institutional changes in policy
and practice following afterwards. This paper provides a strategic overview of the current situation and focuses
on key issues concerning the impact of eSubmission. The term eSubmission is used very widely to cover a range
of activities which include eSubmission, eMarking, eFeedback, eReturn as well as plagiarism deterrence and
detection. For clarity and distinction between these activities the term eSubmission is defined as online
submission of an assignment in this paper. The research was undertaken through the Heads of eLearning Forum
(HeLF). HeLF is a “network of senior staff in institutions engaged in promoting, supporting and developing
technology enhanced learning” (HeLF, 2011). It was established in 2003 and currently represents over 120 HE
institutions in the UK. It meets regularly every year at events relating to topical issues in strategic eLearning
developments. eSubmission is having an impact on the roles of academics, administrative staff and learning
technologists. It is changing the roles of academics and administrators as well as the relationship between them.
It is creating extra work for learning technologists who support both academics and administrative staff and
sometimes students using eSubmission. The technical infrastructure is in place within universities and often there
is integration with Virtual Learning Environments. eSubmission is closely related to plagiarism detection as a
digital version of an assignment is required to use the plagiarism software. In most universities plagiarism
software is available across all courses, but it is generally used on an opt-in basis by individual academics or by a
whole department. There are contrasting views on whether the plagiarism software is used for student
development or plagiarism detection. The development approach (carrot) is often recommended but in practice
the plagiarism detection (stick) approach is used. HeLF members were overwhelmingly positive to a proposed
new role for HeLF to collate and publish and share practice in areas of eLearning such as eSubmission.
Keywords: eSubmission, policy, process, change, HeLF
1. Introduction
eSubmission is currently being implemented in universities within the UK Higher Education (HE). The
process of implementation is generally occurring in departments and Schools with institutional
changes in policy and practice following afterwards. This paper provides a strategic overview of the
current situation and focuses on key issues concerning the impact of eSubmission.
The term eSubmission is used very widely to cover a range of activities which include:
eSubmission
eMarking
eFeedback
eReturn
Plagiarism deterrence and detection
For clarity and distinction between these activities the following definitions are used in this paper:
eSubmission – online submission of an assignment
eMarking – marking a paper online
eFeedback - producing online feedback which could be text, audio etc but not paper
eReturn – online return of marks
Currently, there is usually a mixture practice in the process between online and paper. For example,
students may be required to use eSubmission and hand in a hard copy. Academics may mark hard
copies but give online feedback.
578

Barbara Newland et al.
The research provides a strategic overview of the impact of eSubmission in UK universities. Key
issues which are investigated are the change in roles in this area for academics, administrative staff
and learning technologists. The technical infrastructures being used to support eSubmission and the
relationship between eSubmission and plagiarism policies. The research was undertaken through the
Heads of eLearning Forum (HeLF) and may result in a new role for this national body.
2. Methodology
In order to identify the current situation an online survey was circulated to the HeLF. HeLF is a
“network of senior staff in institutions engaged in promoting, supporting and developing technology
enhanced learning” in the UK (HeLF, 2011). It was established in 2003 and has met regularly every
year. Each HE institution can nominate one member as the Head of eLearning and the group now has
over 120 members. Heads of eLearning have an institutional perspective on eLearning in their
universities.
HeLF’s aims are to operate in an advisory, collaborative and supportive way with regard to strategic
eLearning developments. It acts as “an advisory body for national and governmental organisations on
issues relating to e-learning institutional strategy and implementation.” It is “proactive in soliciting
responses from such bodies and promoting the views of its membership” in an authoritative and
coherent manner. In its collaborative role, HeLF enables the sharing and comparing of “the strategic
implications of developing and implementing eLearning.” HeLF supports “the processes by which elearning
strategy can be effectively created, and implemented, including advice, support and cooperation
between members.” (HeLF, 2011)
A small group of HeLF members, who are currently implementing eSubmission in their institution,
developed the survey. The survey consisted of 31 open and closed questions and was undertaken in
March 2011. The data has been held anonymously and securely. Some participants were willing to
engage in further communication regarding eSubmission and consequently provided their contact
details. Quantitative and qualitative methods were used to analyse the data resulting from the survey.
3. Results
Thirty-eight members of HeLF responded to the survey, which gave a representative sample of data
with a 30% response rate. The results focus on the key issues of roles and practices, technological
infrastructure, eSubmission and plagiarism and the role of HeLF. The quotes are from HeLF members
who completed the survey.
3.1 Roles and practices
Approximately, two thirds of members thought that eSubmission has had an impact on academic,
administrative and learning technologist roles and practices, only 5% feel that it has not, but 26% are
unsure.
The impact on the roles of academics, administrators and learning technologists are being affected in
different ways in different institutions. eSubmission “bridges the technology and the pedagogy,
therefore all the above roles are affected by it.” The process has led to collaborative working across
universities. “The whole issue is proving a valuable area for bringing together academics, admin and
TEL staff to collaborate to produce workable policies and procedures.” ” The project has led to closer
working in order to join us processes.”
There have been some pedagogical changes as the “move to e-marking and feedback cannot help
but change academic practices, and many are still getting to grips with this.” eFeedback may provide
more legible, timely and comprehensive feedback to learners which enhances learning (Nicol &
Macfarlane-Dick, 2006). The introduction of eSubmission “resulted in a revisiting of the assessment
and feedback agenda in some departments, which allowed programme leaders and practitioners to
rethink.” It may lead to more consistency as the “use of grading rubrics and standard answers has led
to a more consistent marking experience, particularly when postgraduates are involved.”
There are differing views on how eSubmission has changed the balance of administrative work
between academics and administrative staff. In some universities it has “given academics more
responsibility over the submission process, and management of retrieving work for marking.”
Similarly, “Academic staff are now taking on some more administrative tasks in setting up and
579

Barbara Newland et al.
managing student submissions electronically. For some academics it has caused confusion.”
However, in other universities the administrative role is increasing as “Admin staff have undertaken a
new role which involves going in to Blackboard, checking students are in the right modules, allocating
staff to groups for marking, extracting marks from Blackboard and uploading them back into the
student information system.” Similarly, “admin staff are playing a larger role in the process, not just
the event, and academic staff are being organised by the admin teams more.” In some universities
the administrative role is decreasing it is “an admin responsibility when not using e-submission that is
no longer needed when online.” It “replaces queues at an office.”
Figure 1: Impact on academic, administrative and learning technologist roles and practices
There is a sense that this change of roles between academics and administrative staff may cause
tensions as “In some departments, administrators now 'own' more elements of the assessment cycle.
There are some dangers in the above, as it can cause an admin/academic rift, often within the same
VLE area.” In another university administrative staff are “also becoming anxious about what their role
in the process will now be.”
eSubmission has generally created more work for learning technologists and they are often key
agents of change in this area. The learning technologists “need to get an understanding of existing
assessment practices - no good simply saying "here's a wonderful tool" if academics can't be shown
how it will work for them.” They have also been “able to discuss feedback practices more with staff.”
The learning technologists often create guidance documentation and devote a lot of time to “staff
training and development as well as intensive support.” They often have to “pick up the pieces” when
things go wrong “particularly due to setups being used” that they did not recommend. In some cases
“Students with problems often contact learning technologists rather than departmental staff in the first
instance.”
The changing practices have also improved efficiency in some universities. The “critical thing is that
once you start looking at submissions and feedback, you start looking at current practice and realise
that needs to be examined!” For example, “It has given us a huge saving in the logging, distribution
and receipting of student work. It has also allowed us to back up and archive student work more
easily and has streamlined our external examining process.” Some members felt that they were too
early in this process to evaluate it. Another member said that they were “Not yet far enough
developed to say with confidence, but I suspect it will, particularly for administrative staff.”
3.2 Technological infrastructure
There are several technologies used for eSubmission and some are integrated with other university
systems such as Virtual Learning Environments (VLE) and student record systems. The major VLEs
implemented in the UK are Blackboard and Moodle (Browne et al, 2010). The technologies used by
respondents to enable eSubmission in the UK are Turnitin (either stand alone or integrated with the
580

Barbara Newland et al.
VLE), the VLE, ePortfolio software (PebblePad) or home grown solutions. This is illustrated in Figure
2.
Figure 2: Does your institution recommend particular software for eSubmission of text and non-text?
A high percentage (79%) use Turnitin integrated within the VLE for eSubmission of text, while 13%
use Turnitin stand alone. 58% use the VLE system and a small percentage uses a home grown
system (11%) or an ePortfolio system (3%). This shows the dominance of Turnitin in this area which
has probably developed from its use as plagiarism detection software. It is not possible to submit nontext
to Turnitin, so this type of eSubmission is primarily via the VLE (55%), while 13% utilise a home
grown system and 8% an ePortfolio system.
Most institutions mandate the technology to be used and there is usually more than one technology to
cope with the different file formats. One member was “quite curious about why some institutions have
gone down the path of looking at mandatory single-systems, and if they feel they have lost anything
through this, also what the benefits are.”
The technical infrastructure for eSubmission must be reliable, robust and secure as the submission of
assignments is very important for students to progress in their studies. The infrastructures in
universities support eSubmission, eFeedback (audio), eFeedback (text), eMarking and peer marking
as shown in Figure 3.
A high percentage stated that their technical infrastructure supports e-submission (89%), e-feedback
in audio format (58%), e-feedback in text format (79%) and e-marking (74%). Only a relatively small
percentage support peer marking (5%). This shows that universities are in a position to implement
eSubmission on a large scale.
The integration of systems helps with efficiency and also illustrates a level of maturity with processes.
Whereas there is a high percentage (79%) of integration of Turnitin with the VLE, only about a third
have integrated eSubmission and eGrading to their student record system as shown in Figure 4.
There were no questions to clarify the level of integration which may vary and this could be part of
future research.
The results show that eMarking has been implemented in 76% of departments. The relationship
between whether a department has implemented eMarking and the technical infrastructure being in
place is highlighted in Table 1. It shows that 58% of the departments that implemented eMarking also
felt that the technical infrastructure supported eMarking.
581

Figure 3: Technical infrastructure
Barbara Newland et al.
Figure 4: Integration of VLE (eSubmission and eGrading) with Student Record System
Table 1: Relationship between eMarking implementation and technical infrastructure
Implemented eMarking
Infrastructure support eMarking
Yes No
Yes 22 (58%) 7 (18%)
No 5 (13%) 5 (13%)
There is also a relationship between the technical infrastructure supporting eSubmission and the
impact on academic, administrative and learning technologist roles and practices. Table 2 shows
there is a correlation between whether the infrastructure to support eSubmission is in place and the
impact eSubmission has had on roles and practices – 61% felt that eSubmission has had an impact
and that the infrastructure was in place to support this.
582

Barbara Newland et al.
Table 2: Relationship between impact on academic, administrative and learning technologists roles
and practices and technical infrastructure
Infrastructure support eSubmission
3.3 eSubmission and plagiarism
Impact on roles and practices
Yes No Unsure
Yes 23 (61%) 2 (5%) 7 (18%)
No 1 (3%)
Unsure 2 (5%)
eSubmission is closely linked with plagiarism detection as electronic formats of assignments are
required to use plagiarism detection software. In most universities “TurnitinUK is integrated with our
VLE and is widely used for eSubmission because it helps to detect plagiarism.” HeLF members were
asked about plagiarism detection deployed in their institutions and whether is it across all courses.
They were also asked whether the emphasis is more on plagiarism detection (stick) or student
development (carrot). In most universities plagiarism software is available across all courses but not
necessarily used. It is generally used on an opt-in basis by individual academics or by a whole
department. For example, “Some staff use it routinely for all assignments, some staff use it when they
suspect plagiarism has occurred, some have never heard of Turnitin.” It may be deployed on all
assignments or a random sample of assignment or its use may be erratic. However, one member
stated that in their institution “We use TurnitinUK for all text based submissions. Students are given
access to their originality reports (and given training to understand and act on them) and can resubmit
an enhanced submission up to the deadline, in all bar 5% of our modules.” In another university it is
deployed across all first year modules.
There are contrasting views on whether the software is used for plagiarism detection or student
development. It appears that often the “development (carrot) approach is recommended, but the stick
approach is still implemented in some programmes.” It is “urged to use as a carrot, but currently used
as a stick.” One member stated “We recommend its use primarily in a formative way for student
development and learning around academic writing and integrity. However there is a feeling that
many staff are still using it for detecting and deterring plagiarism.” Similarly, “the clearest message
TurnitinUK is that it will detect plagiarism rather than it is a tool to assist students to improve their
academic writing skills. In staff training sessions, the emphasis is on techniques to avoid plagiarism
but this message gets drowned out somehow!” In contrast another member said “Institutional advice
is to use as a formative tool. My impression is that most staff genuinely accept the developmental
aspect, and are not just interested in the detection side.” Similarly, at another institution “It is
presented as a carrot model that will improve student academic writing and support is available to
support students having difficulty with paraphrasing, referencing etc.”
3.4 Role of HeLF
The HeLF members were asked if they think there is there a role for HeLF in collating and publishing
good practice in this area. This would be a new activity for HeLF in the way it achieves its aim of
promoting, supporting and developing eLearning. Currently, HeLF activities mainly focus of three face
to face events per year in the UK and responses to reports from government and other national
organisations. The events are organised around a theme each academic year. For example, the
theme for 2010/11 is “Strategy, Efficiency and Resource Management”. There have been events in
London, Manchester, Wolverhampton on “National Strategic Directions for eLearning” and
“Postgraduate Certificates in Education and eLearning.” HeLF has responded to papers such as the
Higher Education Academy’s consultation on the UK Professional Standards Framework (UKPSF) for
Higher Education (HEA, 2010).
The response was overwhelmingly positive to this proposed new role for HeLF to collate and publish
and share practice regarding eSubmission. Comments included “HeLF members have a role to
contribute and share good practice - that's one of the biggest benefits of the group.” “What other
group is well placed to draw together disparate work on this topic?” It was recognised that “not all
HeLF members may necessarily have the information available at their fingertips – it may be held by
Academic office, Quality Enhancement Unit, etc. instead.” “It is the collective aspect of HeLF, that
brings together a number of institutions that is important. It should be the voice of Heads of e-learning
583

Barbara Newland et al.
and I strongly believe that recommendations of a group such as this one, can make a difference to
Senior managers (as opposed to my recommendation). I would like HeLF to be involved in more of
these initiatives.”
Some members suggested collaboration with other national bodies such as JISC. JISC is a UK
national body which “inspires UK colleges and universities in the innovative use of digital
technologies, helping to maintain the UK’s position as a global leader in education.” (JISC, 2011) One
of the reasons for suggesting collaboration is the scale of the undertaking for a voluntary organisation
such as HeLF. HeLF has no income from subscriptions or events and therefore cannot employ any
staff to undertake the work. There was also a suggestion that HeLF could lobby the commercial
providers, such as Turnitin, to “meet the needs of the UK HE community.”
No-one responded that this was not a role for HeLF. Three members agreed that the work needed to
be undertaken but were unsure if it was a role of HeLF. One of these three responded “Certainly a
role for someone, not sure if it needs to be/should be HeLF, but anything which promotes good
practice and documents adoption across the UK.”
HeLF members identified their need for further information regarding the whole area of eSubmission.
They said that “support and help is desperately needed right now” and they would “like to share and
work with others.” “We all want e-submissions systems. We are struggling with, how do we have a
single point of submission integrated with University systems.” One member stated “We will inevitably
move towards making this universal at undergraduate level. Handing in bits of paper is not going to
be viable. But it has implications for standardising processes across very disparate departments, and
that - where technology is perceived to be affecting custom and practice - is a challenge.”
There was interest in sharing information on policy, process, engaging academic staff and technical
developments. This information would be useful for members as “evidence from the sector would help
members to influence practice and policy in our own institutions.” “Some general consensus would be
welcome in order to benchmark against other institutions.”
In relation to policy and practice members were interested in guidance on policy and university
regulations, guidelines for implementation and guides on using software eg Turnitin. Also, “Managers
briefings on Faculty/enterprise level adoption – consideration of risk/risk management/mitigation.”
Many members were interested in case studies of eSubmission. Members said that positive case
studies “of eSubmission in external institutions assists further adoption internally.” “Examples of
where it has been shown to work in other institutions - what needed addressing and by who, how was
this done, what are the benefits for different stakeholders to help "sell" it.” These case studies could
also include “examples of process models / work flows for managing e-submission” and identification
of administration processes which had to change. Others were interested in case studies whether
they were “successful or otherwise” to include “warts and all.”
Members were interested in “strategies for engaging academics”. There was awareness that this
move to eSubmission and eFeedback required changes in the ways in which academics work so
there was also interest in change management approaches. ”Experience from other institutions is vital
to convince academics of the benefits/deal with fears.”
There was also interest in sharing technological information about how institutions are implementing
eSubmission looking at the “pros and cons of each implementation.” “Institutional approaches to
secure infrastructure to support the variety of submission formats, including large media files.
Examples of institutional procedures for the archiving e-submission records.”
4. Conclusion
This strategic overview of the impact of eSubmission shows that it is currently an important
development in UK universities. It is changing the roles of academics and administrative staff and
creating more work for learning technologists. The technical infrastructure is in place to support
eSubmission and mainly used in an opt-in basis. There could be greater integration of VLEs,
plagiarism software and student record systems. Plagiarism policies tend to advocate the use of
plagiarism software for student development but it is often used for detection. There is positive
support for a new role for HeLF to collate and share policies, guidelines and case studies.
584

References
Barbara Newland et al.
Browne, T., Hewitt, R., Jenkins., Voce, J., Walker, R. and Yip, H.( 2010) 2010 Survey of Technology Enhanced
Learning for Higher Education in the UK, [online] UCISA,
http://www.ucisa.ac.uk/en/publications/tel_survey2010.aspx
HEA (2010) [online]
http://www.heacademy.ac.uk/assets/documents/rewardandrecog/UKPSF_Consultation_document_Nov10.p
df
HeLF ( 2011) [online] http://w01.helfcms.wf.ulcc.ac.uk/aims.html
JISC (2011) [online] http://www.jisc.ac.uk/aboutus.aspx
Nicol, D. And Macfarlane-Dick, D. (2006) Rethinking Formative Assessment in HE: a Theoretical model and
Seven Principles of Good Feedback Practice [online]
http://www.heacademy.ac.uk/assets/York/documents/ourwork/tla/assessment/web0015_rethinking_formativ
e_assessment_in_he.pdf
585

Harnessing the Internet for Authentic Learning: Towards a
new Higher Education Paradigm for the 21st Century
Abel Nyamapfene
College of Engineering, Mathematics and Physical Sciences, University of
Exeter, UK
a.nyamapfene@ex.ac.uk
Abstract: This paper discusses the use of authentic assessments in a second year undergraduate course in
communication systems and networking to help students acquire relevant and up-to-date networking skills. In the
authentic assessments presented in this paper, students solve real-life networking issues commonly faced by
practising network specialists. The Internet is used as the primary source of information, and students are
encouraged to form ad hoc teams to collaborate in gathering and analysing the available information. However,
each student is required to separately submit an independently compiled individual report documenting the
student’s solution to the problem. Turnitin, an anti-plagiarism software tool, is used to ensure that the report is the
student’s own individual effort. A rubric based on a guide developed at the Washington State University is used to
assess the student’s acquisition of critical thinking skills. Preliminary findings suggest that students generally feel
that these authentic assessments enable them to master skills that they cannot otherwise acquire in typical
lecture-based studies. However, a minority of students in the cohort under study appear to be disenchanted by
the Internet-enabled authentic assessment introduced in the course module. Whilst acknowledging that the
emerging Internet-enabled student-led learning, of which the authentic assessment approach described in this
paper belongs to, may well be the way higher education learning and teaching may proceed in the future, the
paper concludes by suggesting three pertinent questions that need to be investigated further. These questions
relate to the nature of the pedagogy for this emerging form of learning and teaching as well as the nature of the
changing roles and relationships between students, lecturers and their primary higher education institutions.
Keywords: the internet, authentic assessment, critical thinking, Turnitin, autonomous learning, higher education
learning and teaching
1. Introduction
The Internet is threatening well established norms in higher education teaching and learning in
fundamental and far reaching ways. The information transmission method of learning, as typified by
the traditional lecture method, where information habitually flows from the lecturer to the students, is
rapidly becoming outmoded. With the advent of the Internet, the course content provided by the
lecturer is no longer unique, but now stands in competition with other sources of information on the
Internet. For instance, rather than looking up a lecture’s course notes, a quick glance at Wikipedia will
quickly provide a student with the information he/she needs. Not only that, it is now possible, thanks
to the Open Educational Resources movement (Hylén, 2007), to freely access learning and teaching
content from other educational institutions as well. Notable examples of institutions that have provided
open access to their learning and teaching content include the MIT, through the MIT
OpenCourseWare website, and the Open University, through OpenLearn website.
Therefore, as a consequence of the Internet, the transmission of subject content is no longer an issue
in most higher education courses. Now, for students a more pertinent issue is how to make sense of
available information and how to choose between competing information sources in a limited amount
of time. To manage in this environment, students now need to master the techniques of decision
making, communicating effectively and solving real-life problems in real-time. These techniques
constitute the elements of critical thinking. There are several competing definitions for critical thinking,
but a generally accepted definition is the one by Scriven and Paul (2008):
Critical thinking is the intellectually disciplined process of actively and skillfully
conceptualizing, applying, analyzing, synthesizing, and/or evaluating information
gathered from, or generated by, observation, experience, reflection, reasoning, or
communication, as a guide to belief and action. In its exemplary form, it is based on
universal intellectual values that transcend subject matter divisions: clarity, accuracy,
precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and
fairness.
Critical thinking is now widely accepted as a key skill that students should acquire during their
undergraduate studies (Mulnix, 2011; Phillips and Bond, 2004). Hence for the lecturer, the advent of
the Internet means that in addition to successfully imparting mastery of discipline-specific concepts to
586

Abel Nyamapfene
students, equal attention now has to be paid to the development of teaching strategies that facilitate
critical thinking. Halpern (2001) identified key characteristics that a teaching strategy should meet if it
is to impart critical thinking skills to students. Firstly, learner should be actively involved in the learning
process. Secondly, learners should practise thinking skills in multiple settings. Thirdly, learners should
be exposed to examples that are similar to the situations in which the skills will be used. Learners
should also be able to monitor their comprehension and make appropriate adjustments to their
actions to ensure that they progress toward an identifiable goal. The strategy should also enable
learners to identify and appreciate the rationale for learning the skills in the first place. Finally, the
strategy should incorporate intrinsic motivational techniques and should make use of multiple learning
approaches.
One way of imparting critical thinking skills to students is to engage them in authentic assessments.
Gulikers et al (2004) define authentic assessment as:
An assessment requiring students to use the same competencies, or combinations of
knowledge, skills, and attitudes that they need to apply in the criterion situation in
professional life.
Such an assessment requires students to be actively involved in carrying out the specified task in a
manner similar to how they would handle it during their professional lives after leaving university. Just
like professionals engaged in their day to day tasks, an authentic assessment would also require
students to self-assess their progress and take corrective action to ensure that the desired goal is
achieved. An authentic assessment, by its very nature, will also motivate students to carry out the
specified task since it offers experiences that are relevant to professional practice.
A primary goal of authentic assessment is to impart useable knowledge to students. According to
Herrington and Oliver (2000), this is best done in a learning environment that features the following
characteristics:
Provide authentic contexts that reflect the way the knowledge will be used in real life
Provide authentic activities
Provide access to expert performances and the modelling of processes
Provide multiple roles and perspectives
Support collaborative construction of knowledge
Promote reflection to enable abstractions to be formed
Promote articulation to enable tacit knowledge to be made explicit
Provide coaching and scaffolding by the teacher at critical times
Provide for authentic assessment of learning within the tasks.
This paper reports on an individual lecturer’s attempt to use authentic assessment to enhance the
delivery of an introductory module in communication and networking technologies. The module
introduces students to the basic concepts of communication technologies that they are in everyday
contact with such as the Internet, mobile broadband, telephony and Bluetooth. The module builds up
on students’ familiarity with these technologies to develop a theoretical understanding of the
underlying telecommunication principles (Nyamapfene, 2008). The theoretical basis for this is the
suggestion by Bruner (1960) that learning is an active process in which learners construct new ideas
or concepts based upon their current or existing knowledge.
2. The course module and study context
This study was carried out on the 2010/11 student cohort of the second year course module on
Communication and Networking Technologies which is offered in the second semester at the
University of Exeter. The course module is compulsory to Electronic Engineering students and
optional to Computer Science and Information Technology and Management of Business (ITMB)
students as well as to Flexible Combined Honours students (i.e. those students who have opted for
the right to mix and match their degree course modules from any of the undergraduate courses taught
across the university). The 2010/11 student cohort comprised 10 Electronics Engineering, 1 Computer
Science, 8 ITMB and 2 Flexible Combined Honours students.
587

Abel Nyamapfene
As stated on the course homepage on the University of Exeter Moodle virtual learning environment,
the aim of the course is to provide an in-depth introduction to the fundamental principles underlying
modern communication and network technologies. The course places an emphasis on the study of
networking and communication protocols, and specifically uses the Internet as a vehicle to study
these protocols. The course module comprises 20 lecture periods of one-hour each. Lecture
attendance is encouraged, but not mandatory. All the lecture notes as well as assignments are posted
on the university’s Moodle virtual learning environment.
Assessment of the module is carried out through two tutor-marked assignments (TMA1 and TMA2),
each worth 10% of the module overall mark, one network study project worth 20% of the module mark
and a two-hour, closed-book/note examination at the end of the course worth 60% of the module
mark. The two tutor-marked assignments and the project are both summative and formative, with
students required to explore and investigate subject material that goes beyond that covered by the
lecture notes. TMA1 lays the groundwork for the project whilst TMA2 acclimatises the students to the
final examination at the end of the semester. In this study we will focus primarily on TMA1 and the
network study project.
3. Design and management of TMA1 and the network study project
The objective of TMA1 and the project is to develop and extend a key concept in the course module
beyond what would be possible through lectures and tutorials alone. For instance, in the academic
year 2009/10 these two pieces of assignment were used to enhance students’ knowledge and
appreciation of the application and importance of standards in communication and networking
systems. In TMA1 students researched on telecommunications and networking standards and
submitted an individual report. In the ensuing project, students designed a local area network for
Harrison Building, the main administrative and teaching building for Engineering, Mathematics and
Computing disciplines at the University of Exeter.
For the academic year 2010/11, the objective of TMA1 and the project was to enhance students’
knowledge and appreciation of computer and networking security. This is an important topic for IT
professionals worldwide, and given the widespread coverage of cybercrime in the popular press, it is
a very popular topic to would-be IT professionals. However, computer and networking security is a
complex subject area that would require several lecture sessions if it is to be dealt with adequately.
The high interest in computer and networking security, coupled with current student interest in the
topic, makes it an ideal candidate for learning through authentic assessment methods.
In TMA1 students researched on distributed denial of service attacks (DDOS) and submitted an
individual report. The project extended the work covered in TMA1 by requiring the students to develop
a risk assessment document for the IT network infrastructure in Harrison Building in accordance with
the National Institute of Standards and Technology (NIST) risk assessment methodology outlined in
Special Publication 800-30. For both the TMA1 and the project, students were issued with
instructional rubrics describing in detail the various levels of quality, ranging from poor to excellent,
that a student can attain in the assignment. The TMA1 rubric was based on the critical thinking rubric
by the University of Washington, whilst the project rubric was based on the risk assessment
requirements listed in the NIST Special Publication 800-30 document. In addition to serving as
standards-referenced assessment tools, instructional rubrics are designed to support student learning
and development through features such as (Goodrich Andrade, 2000):
Being written in language that students can understand
Referring to common weaknesses in students' work and indicating how such weaknesses can be
avoided
Enabling students to evaluate their works-in-progress and thereby guiding revision and
improvement
Both assignments were issued out on the first day of lectures. TMA1 was to be accomplished in three
weeks whilst the project had a time limit of eight weeks. Throughout the duration of the assignments,
time was set aside during lectures to look at and discuss online articles associated with cyber crime.
These ranged from news articles to technical and scientific journal papers posted on the Internet. All
these articles were evaluated using the instructional rubric for TMA1. In this way students learnt to be
comfortable with the process of analysing technical and scientific writings.
588

Abel Nyamapfene
Office hours were also set up to allow students to visit the lecturer to discuss difficult concepts they
came across during their research. Students could also get in touch with the lecturer through email
contact. In addition, students could also get in contact with security experts in industry as well as with
security researchers in other academic institutions. Collaborative working between the students was
also encouraged to enable them to learn from each other as well as to support each other during the
assignment. However, each student had to submit an independent individual piece of work, and this
was checked for plagiarism using Turnitin (iParadigms), an academic plagiarism detection software.
Turnitin compares each submission with other pieces of work already stored in its database. This
stored work includes assignments submitted by students at other institutions worldwide, as well as
with content that is already on the Internet. Prior to submission, students experimented with Turnitin
so that they could get a feel of its capabilities. As a consequence of Turnitin, students were forced to
produce original pieces of work, thereby contributing to the wider understanding of computer network
security.
4. Results and discussion
Students generally submitted high quality work for both TMA1 and the project. Of the 21 students
registered on the course, 12 students in TMA1 and 8 students in the project achieved a mark greater
or equal to 70%. An academic colleague who reviewed the marked work was also impressed with the
quality, even going to the extent of suggesting that at least four of the TMA1 assignments were of
journal quality. This is a remarkable achievement, given that none of the students had formal
experience of computer and networking security prior to the course.
4.1 Analysis of plagiarism
With regard to originality, for TMA1 only one student had a similarity index in the red category, with a
value of 81%. The class comprised 5 international students, with the rest being European Union or
home students. In contrast, the highest similarity index for a home or European student was 26%,
which was less than the lowest similarity index for an international student (28%). This assignment
therefore highlighted the problems of plagiarism amongst international students as highlighted by
Bamford and Sergiou (2005). However, similarity indices improved in the project coursework, with the
highest similarity index falling to 52%. In addition, international students no longer had outlier
similarity indices out of proportion with those achieved by home and European Union students. This
therefore suggests that like any assessment tasks, issues to do with plagiarism can be resolved with
sufficient feedback and tuition.
4.2 Analysis of student anonymous course evaluation
Students perception of the coursework were also analysed through the University of Exeter module
and coursework evaluation (MACE) online feedback forms that students fill in anonymously at the end
of each course. The MACE feedback form comprises 12 five-point Likert questionnaire items with a
scale ranging from 1 for the “Strongly disagree” category through to 5 for the “Strongly Agree”
category. In addition there are also separate spaces for students to write positive, negative and any
other comments they may wish to make about the course content, method of delivery and
assessment. Of the 21 students on the course, 8 students completed the MACE feedback form. Such
a low response rate is typical across the university.
The list of Likert questionnaire items used on the MACE evaluation form is as follows:
1. The module was interesting and intellectually stimulating
2. The amount of material was suitable
3. The material was at the right level of difficulty
4. The module was well structured and had clear objectives
5. The staff teaching the module were enthusiastic about the subject
6. The staff teaching the module explained things clearly
7. Good use was made of appropriate media (e.g., boards, handouts, visual aids).
589

Abel Nyamapfene
8. There were sufficient useful study resources and/or course notes
9. The staff teaching the module were supportive and helpful
10. The assessments were a fair way of testing what was learnt on the module.
11. Feedback on coursework was of sufficient quantity and quality
12. Feedback on coursework was received on time
Table 1: Student responses to the 12 Likert questionnaire items
Question StdDev Mean Definitely
Disagree
Mostly
Disagree
N/R Mostly
Agree
Definitely
Agree
1 0.97 3.75 0 1 2 3 2
2 0.93 3.13 0 3 1 4 0
3 0.99 3.38 0 2 2 3 1
4 1.50 3.50 2 0 0 4 2
5 0.43 4.25 0 0 0 6 2
6 0.99 3.63 0 1 3 2 2
7 0.97 3.75 0 1 2 3 2
8 1.11 3.63 1 0 1 5 1
9 0.66 4.25 0 0 1 4 3
10 1.17 3.88 1 0 0 5 2
11 0.97 3.75 0 1 2 3 2
12 0.50 4.50 0 0 0 4 4
Five students felt that the course was interesting and intellectually stimulating (Response to Question
1). However, whilst 4 students mostly agreed that the amount of material was suitable, 3 students
disagreed (Response to Question 2). With regard to the level of difficulty of the course, 2 students felt
that the course was difficult, 2 were non-committal, whilst 4 felt that the course was at the right level of
difficulty. Positive written responses to the course included:
Enjoyed the module lots of interesting topics covered
Best course i have taken at this university. Abel is a credit to what Exeter can offer, a
really great lecturer
Fun coursework, enthusiastic teaching.
A corresponding negative written response is:
Lots of different topics covered so hard to recall it all in the exam
From this analysis, we can conclude that most students generally found the course to be interesting.
The negative responses may indicate that there were some students whose main focus were passing
the examinations and were not overly interested in the course content. Since TMA1 and the project
were spread out through the semester, this would suggest that students were stimulated by the
directed self-learning on computer and networking security. The analysis also indicates that whilst
some students felt that the course was rather difficult, the majority were comfortable with the degree
of challenge in the course.
Turning to the organisation of the module, 2 students definitely disagreed with the assertion that the
course was well structured and had clear objectives (Response to Question 4). In addition, one
student felt that the resources available for the course were insufficient (Response to Question 8). A
corresponding negative comment was as follows:
In most lectures, the actual teaching was not very apparent. We were asked to answer
questions but we had not been taught the material very well.
In both the lectures and the authentic assessments, students were expected to research and read on
their own. Lectures were primarily used to give an overview of lecture material already posted on the
590

Abel Nyamapfene
Moodle virtual learning environment and following up on concepts that students found difficult. This is
a marked departure from the traditional approach where the lecturer spent the lecture period going
through lecture material already posted on the virtual learning environment. Whilst earlier student
cohorts could expect to sit passively in lectures, in the interactive approach adopted in this particular
semester students were expected to contribute to class discussions. Specifically, in the authentic
assessments, students had to show the initiative and come up with their own material. This again is at
variance with the traditional approach whereby students are essentially spoon-fed with all the
necessary material. However, since these students are in the minority, this would suggest that most
students would welcome an authentic assessment driven course in which they are expected to
contribute to the information and knowledge needed to do well in the course.
Students also felt that the lecturer was enthusiastic about the subject content and that he was
supportive and helpful (Responses to Question 4 and 9). As pointed out earlier on, Herrington and
Oliver (2000) are of the opinion that for authentic assessment to be optimal, students should feel that
they have ready access to expertise to guide them in their work.
Turning to the assessments themselves (Response to Question 10), seven of the respondents agreed
that assessments were a fair way of testing what was learnt on the module, whilst one student
strongly disagreed. Again, this may suggest that whilst the majority of students are likely to be
comfortable with authentic assessments, there may always be a minority who would tend to disagree.
To really go beyond the tentative statistics presented here, these findings suggest that interpretive
studies comprising interviews and focus groups may be required to shed light on exactly how student
cohorts feel towards authentic assessment.
5. Conclusion
In this paper the Internet was harnessed to enable authentic assessment of computer and networking
security in a second year university level course on communications and networking technologies.
The performance of the student cohort in the authentic assessments as well as in the end-of-course
examination suggests that Internet-enabled authentic assessment is now a viable option for university
level courses. An analysis of end-of-course anonymous student course evaluations suggests that
students appear to be motivated and excited by Internet-enabled authentic assessments. However,
there are still questions which need to be engaged with: For instance, what is the optimal pedagogy
for the Internet-based student-led learning like the one described in this paper? Secondly, in a world
where content is now widely and freely available on the Internet, what should be the nature and extent
of the lecturer’s role in student-led learning? And finally, what is the likely future nature of learning and
teaching in an environment where the lecturer and the student’s own institution are no longer the only
authoritative sources of knowledge and information in the student’s learning environment?
References
Bamford, J. and Sergiou, K. (2005) “International Students and Plagiarism: an analysis of the reasons for
plagiarism among international foundation students”, Investigations in University Teaching and Learning,
Vol. 2, No. 2, pp. 17-22.
Bruner, J.S. (1960) The Process of Education, Cambridge: Harvard University Press.
Goodrich Andrade, H. (2000) “Using rubrics to promote thinking and learning”, Educational Leadership, Vol 57,
No.5, pp.13-18.
Gulikers, J.T.M., Bastiaens, T.J., & Kirschner, P.A. (2004) “A five-dimensional framework for authentic
assessment”, Educational Technology Research and Development, Vol 52, No.3, pp. 67-86.
Halpern, D. F. (2001) "Assessing the Effectiveness of Critical Thinking Instruction" The Journal of General
Education, Vol 50, No.4, pp.270-286.
Herrington, J. and Oliver, R. (2000) “An instructional design framework for authentic learning environments”,
Educational Technology Research and Development, Vol 48, No.3, pp. 23-48
Hylén, Jan (2007) Giving Knowledge for Free: The Emergence of Open Educational Resources, Paris, France:
OECD Publishing.
iParadigms. Turnitin, [online], http://turnitin.com/static/index.php
MIT. MIT OpenCourseWare,[online], http://ocw.mit.edu/index.htm.
Moodle,[online], http://moodle.org/community.
National Institute of Standards and Technology (2002) Special Publication 800-30: Risk Management Guide for
Information Technology Systems, [online], http://csrc.nist.gov/publications/nistpubs/800-30/sp800-30.pdf.
Nyamapfene A. (2008) “From everyday playthings to the underlying theory - How I turned a ‘boring’ Course into
an ‘exciting’ Course”, UK Higher Education Academy Workshop: Novel Approaches to Promoting Student
Engagement, University Of Ulster, 30th - 30th Oct 2008.
591

Abel Nyamapfene
Mulnix, J. W. (2011), “Thinking Critically about Critical Thinking. Educational Philosophy and Theory”,
Educational Philosophy and Theory, [online],
http://onlinelibrary.wiley.com/doi/10.1111/j.1469-5812.2010.00673.x.
Petress, K. (2004) “Critical Thinking: An extended definition”, Education, Vol 124, No.3, pp. 461–466.
Phillips, V. and Bond, C. (2004) “Undergraduates’ experiences of critical thinking”,
Higher Education Research and Development Journal, Vol 23, No.3, pp. 277-294.
Scriven, M. and Paul, R. (2008) “Defining Critical Thinking, Foundation for Critical Thinking”, Center for Critical
Thinking, [online], http://www.criticalthinking.org/aboutCT/definingCT.cfm
The Open University. OpenLearn,[online], http://www.open.ac.uk/openlearn.
University of Exeter. Exeter Learning Environment, [online], http://vle.exeter.ac.uk.
592

Motivational Predictors of Academics’ Electronic:
Publishing in Nigerian Colleges of Education
Maruff Akinwale Oladejo and Adelua Olajide Olawole
Federal College of Education (Special) Oyo, Nigeria
maruvoladejo@rocketmail.com
adeluaolajide@yahoo.com
Abstract: In the modern era of digital information revolution, the application of electronics to almost every aspect
of human endeavours is on the increase. For instance, in the area of scholarly or academic journals, electronic
publishing has gained unprecedented popularity both in the developed and the developing nations. The main
thrust of the present study therefore, is to examine the predictive power of some academics’ motivational
characteristics on electronic publishing in Nigerian Colleges of Education. The study adopted the descriptive
survey design which is ‘ex-post facto’ in nature. It purposively selected the three Colleges of Education in Oyo
State,with 350 participants selected through disproportionate simple random sampling technique. Three validated
self-designed instruments were used for data collection. Six hypotheses were formulated and tested at 0.05 level
of significance. Findings revealed that attitude towards technology (r=0.67) and computer self-efficacy (r=0.56)
were significantly related to electronic publishing. Also, there were significant differences in electronic publishing
based on gender (t = 2.29, df =348, P> 0.05) and age (t = 2.22, df =348, P> 0.05) respectively. The greatest
challenges electronic publishing now faces are acceptance and continual patronage among scholars. Nigerian
College academics need to embrace digital publications and appreciate the value of this medium for scholarly
communication. Thus, College academics should exhibit positive attitudes and be more self-efficacious about
electronic publishing. Institutional administrators should put in place, necessary motivational mechanisms such
as giving free or subsidized laptops and modems to the academics, and organize regular training and re-training
workshops or seminars on internet.
Keywords: electronic publishing, attitudes, computer self-efficacy, self-concept, motivation
1. Background
In the modern era of digital information revolution, the application of electronic to almost every aspect
of human endeavours is on the increase. For instance, in the area of scholarly or academic journals,
electronic publishing has gained unprecedented popularity both in the developed and the developing
nations. In fact, the impact of the new electronic data and word processing technologies on the
publications of scientific journals cannot be over-emphasized. Academic communication has since
changed over the past decade with the arrival of the Internet as authors, scholars and researchers
now communicate nearly at the speed of thought. Thus, the systematic progress of Information and
Communication Technologies (ICTs) as well as the sharp increase in internet usage has transformed
the way scientific or academic journals are published, and findings disseminated, resulting to steady
exponential growth in the number of scholarly or academic journals. According to (Nasser and
Abouchedid 2001 ), there is a fundamental change in the world of scholarly publishing represented by
a shift away from the medium of print to electronic.
Information and Communication Technologies have therefore facilitated the publication of scholarship
on the Web. To some extent, it has altered the old epistemologies of research and re-conceptualized
ideas in forms that are quite interactive, global and instantaneous (Okerson, 1991) Ezema (in
Omotayo, 2010) remarked that electronic journal publication is a new way of stimulating the
production and dissemination of knowledge and its popularity has continued to grow since 1999s. It
has been observed that academics in developing countries are fast adapting to the Internet as a
source of information for teaching and research (Omotayo, 2010). According to her, some research
reveals the use of the Internet for things like email and browsing (Badu & Markwei, 2005; Ojedokun &
Owolabi, 2003). Several other studies have also been conducted to determine the use of e-journals
and other e-resources in Africa. For instance, Manda (in Omotayo, 2010) investigated the utilization of
electronic resources in Tanzania by academics. He reported that there was low and poor utilization of
electronic resources, due to limited access to Personal Computers (PCs), poor search skills, lack of
end-user training, slow connectivity, and budget cuts. Smith (2007)’s study in South Africa established
that the range of electronic journals in the respondents' field of interest was fairly limited due mainly to
lack of bandwidth.
In Nigeria, (Ehikhamenon 2003) found out that even though 77.5 percent of Nigerian Scientists rated
electronic journals as "important" or "very important", their rating was based on expectation rather
593

Maruff Akinwale Oladejo and Adelua Olajide Olawole
than actual use. Also, Gbaje (in Omotayo, 2010), writing on Nigerian academic libraries, reported that
only 20 percent of those sampled provide access to electronic resources. (Azubogu and Madu 2007)
observed that academic staff of the Imo State University, Owerri, Nigeria, have resorted to the use of
computer and Internet technologies to search for information because the University library lacks
funds to subscribe to scholarly and research journals. In her recent study on access, use and
attitudes of academics towards electronic journals at the Obafemi Awolowo University, Nigeria,
(Omotayo 2010)’s findings revealed that all the respondents were aware of, and have used e-journals,
though the frequency of use ranged from daily to occasional. Also, it was revealed that the majority
150 of respondents, that is, 61% prefer electronic journals, while ninety-five (39 percent) prefer print.
Those who preferred electronic journals cited their monthly subscription for Internet connectivity,
which they must enjoy, and access to faster and quicker information electronically than going
physically to the library to look for print journals as some of the reasons for their usage.
1.1 The concept of electronic publishing and publications
Electronic publishing could be described as the process of formatting and producing scholarly
journals in an electronic environment (Ken 2001). The products of e-publishing are electronic
publications, which are those publications often in form of Hyper Text Markup Language (HTML) or
Portable Document Files (PDF), word processing files (DOC), or other types of files that are uploaded
to the Internet, or distributed privately. Electronic publishing is certainly improving scholarly
communication. First, it breaks the tyranny of distance between authors and editors, reviewers, and
researchers. Second, it promotes global scholarly communication. Third, electronic publishing cost
effective as it eliminates many of the costs associated with printing and distribution Varian( Ken,2001).
Fourth, electronic journals are intrinsically archival, since it is easier to store back files of journals and
linked scholarship together with hyperlinks. Electronic publications can also be put on a physical
medium, such as a disk or CD. These can be freely distributed with information on them or sold to
individuals. The advantage to this type of medium is that distribution of the information can be more
closely regulated, though distribution is not as quick.
1.2 Motivational factors and electronic publishing
According to (Mitchell 1996:81), “Motivation represents those psychological processes that cause
arousal, direction and persistence of voluntary actions that are goal directed”. In the opinion of Steers
and Porter ( in Mitchell, 1996), when we discuss motivation, we are primarily concerned with (1) what
energizes human behaviour; (2) what directs or channels such behaviour; and (3) how this behaviour
is maintained or sustained. In the Motivational Systems Theory (MST), motivation is defined as “the
organized patterns of three psychological functions that serve to direct, energize, and regulate goaldirected
activity: personal goals, emotional arousal processes, and personal agency beliefs” (Oladejo,
2010:143).
In this study, the researchers contended that attitudes towards technology, computer self-efficacy and
academic self-concept are some of the factors capable of motivating academics to embrace epublishing.
For instance, (Webster and Hackely 1997) explained the three instructor characteristics
influencing electronic publishing among which is attitude towards technology was identified. By
attitudes, (Ajzen and Fishbein 1980) maintained that they are positive or negative evaluations of
object, people, or situation that predispose one to feel and behave toward them in positive or negative
ways. Measuring attitudes has an important role in analyzing behaviour because it is known as a fact
that there is a strong correlation between attitude and behavior (Bertea 2009). In the context of eenvironment,
a favourable attitude of academics shows a greater probability that they will accept the
modern medium of information dissemination. Furthermore, in general terms, (Bandura 1986: p. 391)
defined self-efficacy as "people's judgments of their capabilities to organize and execute courses of
action required to attain designated types of performance". The concept of computer self-efficacy
(CSE) emerged from the self-efficacy literature. It measures one’s confidence in mastering a new
technology or software with certain degree of confidence (Compeau & Higgins, 1995). (Stephens &
Shotick 2001), provided the best definition of computer self-efficacy relevant to the present study as
an individual’s belief in their ability to use technology in order to solve problems, make decisions, and
to gather and disseminate information. Also, (Foucher and Prince 2003) argued that in the field of
eLearning, students who decide to use the inter system have a sense of self-efficacy higher than
those who choose not to use the system.
594

Maruff Akinwale Oladejo and Adelua Olajide Olawole
Self-concept has also been identified as a predictor of e-publishing(Oladejo, 2010). It is the
accumulation of knowledge about the self, such as beliefs regarding personality traits, physical
characteristics, abilities, values, goals, and roles (Oladejo, 2010). Thus, Rosen (Fayombo, 2001)
described academic self-concept as self confidence which is the anticipation of successfully mastering
challenges, obstacles or tasks in academic works. It is therefore very important that attempt be made
at examining how these motivational variables predict College academics’ choice of electronic
publishing in Nigeria.
1.3 Statement of the problem
Understanding what motivate academics to use the internet as a publishing channel is a vital issue for
consideration, if Nigerian academics want to remain relevant and keep abreast of current information
and benefit from the global digital information revolution. The problem of this study therefore, is to
examine the predictive power of some academics’ motivational variables namely attitudes towards
technology, computer self-efficacy and academics self-concept on electronic publishing in Nigerian
Colleges of Education. To this end, six hypotheses were formulated and tested at 0.05 level of
significance.
There is no significant relationship between attitudes towards technology and electronic
publishing by the Nigerian College Academics.
There is no significant relationship between computer self-efficacy and electronic publishing by
the Nigerian College Academics.
There is no significant relationship between academic self-concept and electronic publishing by
the Nigerian College Academics.
Electronic publishing by the Nigerian College Academics is not significantly different based on
age.
There is no significant gender difference in electronic publishing by the Nigerian College
Academics.
Electronic publishing by the Nigerian College Academics is not significantly different on the basis
of rank.
2. Methodology
2.1 Design
The study adopted the descriptive survey design where the variables are examined “ex-post facto”.
An “ex-post facto “study is described as:
A systematic empirical inquiry in which the scientist does
not have direct control of independent variables because their
manifestations have already occurred or because they are inherently
not manipulable. Inferences about relations among variables are made
without direct interaction from concomitant variation of independent and
dependent variable, Kerlinger (in Oladejo, 2010: 264).
2.2 Sample and sampling techniques
Purposive sampling technique was adopted to select both the Colleges and the subjects. The three
Colleges of Education in Oyo State, Nigeria were therefore used for the study. This is made up of one
Federal, State and Privately-owned . Also, a total number of 350 participants , who had spent at least
five years in the service participated in the study. Three validated self-designed instruments titled
College Academics’ Attitude to Electronic Publishing Questionnaire (CAAEPQ), College Academics’
Computer Self-Efficacy Scale (CASEC) and College Academics’ Self-Concept Scale (CASCS) whose
reliability coefficients were 0.68, 0.75 and 0.78 respectively were used for data collection. These were
administered to all cadres of academic staff in the selected Colleges of Education.
595

2.3 Instrumentation
Maruff Akinwale Oladejo and Adelua Olajide Olawole
Three questionnaires were for data collection by the Researchers. These are College Academics’
Attitude to Electronic Publishing Questionnaire (CAAEPQ), College Academics’ Self-Concept Scale
(CASCS) and College Academics’ Computer Self-Efficacy Scale (CASEC) respectively. College
Academics’ Attitude to Electronic Publishing Questionnaire (CAAEPQ) was divided into two sections.
Section A contained items on College academics socio-demographic background such as age,
gender, rank/status, marital status and year of experience. Section B consisted of 15 items on
attitudes towards electronic publishing. The items were drawn on a modified four-point Likert scale of
Strongly Agree (SA), Agree (A), Disagree (D), and Strongly Disagree (SD) and carried the weights of
4,3,2,1 respectively. College Academics’ Self-Concept Scale (CASCS) was designed to seek
information College academics’ self-concept about electronic publishing. It has 10 items, which were
drawn on a modified four-point Likert scale of Most Like Me (MLM), Like Me (LM), Least Like Me
(LLM), and Not Like Me (NLM). It was scored with the weights of 4,3,2,1 respectively. Finally, College
Academics’ Computer Self-Efficacy Scale (CASEC) instrument was developed to collect information
on how efficacious College academics are about electronic publishing. It also consisted of 10 items,
drawn on a modified four-point Likert scale of Strongly Agree (SA), Agree (A), Disagree (D), and
Strongly Disagree (SD) and carried the weights of 4,3,2,1 respectively.
2.4 Validity and reliability of the instruments
All the instruments were personally designed by the researchers after going through relevant
literature. They were later given to experts in measurement and evaluation for contents, construct and
face validity. This was to determine the proper structuring, adequacy and contents validity of each of
the items in each of the instruments, and to ensure that all the instruments measured what they were
actually expected to measure. All the items in all the instruments were retained after modifications
based on the experts’ suggestions. Furthermore, a reliability study was carried out using 100
academics at the Federal School of Survey, Oyo. These academics were found appropriate for this
purpose because they share similar characteristics with the subjects of the study. They were not
included in the real study. Cronbach’s coefficient was computed based on their responses. The alpha
values obtained were 0.68, 0.75 and 0.78 for CAAEPQ, CASCS, CASEC respectively.
2.5 Administration of instruments
The researchers employed the services of Colleagues in each of the three Colleges they had earlier
contacted for the purpose. All the questionnaires were administered on College academics during the
Academics’ Union general meeting. Two hundred copies of the questionnaires were sent out to each
of the three Colleges. Four hundred and thirty-five copies of the questionnaire were returned. This
gave 72.5% rate of return However, 85 out of these questionnaires were not completely filled. The
remaining 350 copies used for the scholarship were found to be appropriately and completely filled.
2.6 Procedure for data analysis
Data were analyzed through the Pearson Product-Moment Correlation Coefficient (PMCC) and t-test
respectively. The PMCC was used to test hypothses1, 2, and 3, while t-test was used for testing
hypotheses 4, 5, and 6.
3. Findings
Finding: Table 1 shows a significant relationship between attitudes towards technology and electronic
publishing among College academics in Oyo State, Nigeria (r =0.024,df =348, P

Maruff Akinwale Oladejo and Adelua Olajide Olawole
Hypothesis 2: There is no significant relationship between computer self-efficacy and electronic
publishing by Nigerian College Academics.
Finding: Table 2 shows a significant relationship between computer self-efficacy and electronic
publishing among College academics in Oyo State, Nigeria (r =0.036,df =348, P .05
Variables N Mean SD r df Sig
Academic Self-Concept 350 27.06 7.45
Electronic Publishing 350 38.92 8.09 0.115 348 0.00
Hypothesis 4: Electronic publishing by the Nigerian College Academics is not significantly different
based on age.
Finding: Table 4 shows that there was significant gender difference in electronic publishing among
College academics in Oyo State, Nigeria (t =2.399, df =348, P

Maruff Akinwale Oladejo and Adelua Olajide Olawole
Hypothesis 6: Electronic publishing by the Nigerian College Academics is not significantly different
on the basis of rank.
Finding: Table 6 shows that there was no significant difference in electronic publishing among
College academics in Oyo State, Nigeria on the basis of rank (t =.376, df =348, P>.05). It was
observed that junior academics below the rank of Senior Lecturer cadre publish through electronic
more than their senior colleagues who are above Senior Lecturer cadre. The hypothesis was
therefore retained.
Table 6: t-test summary table showing significant difference in electronic publishing among college
academics in Oyo State, Nigeria on the basis of Rank
Variable Rank N Mean SD t-value df Sig Rmk Decision
Electronic Senior (> SL) 127 33.11 8.18 0.124
Not Do Not
Sig Reject
Publishing Junior (.05
4. Discussion of findings
Finding from hypothesis one reveals that there was significant relationship between attitude towards
technology and electronic publishing, which indicates that Nigerian College academics are favourably
disposed to embracing electronic publishing as a result of their positive attitudes towards technology.
This finding corroborates the results of studies (Golden, McCrone, Walker & Rudd 2006 ) and (Bertea
2009). For instance, (Bertea 2009) reported that the average overall attitude score of academics
towards electronic publishing is moderately high (3.67). Thus, a favourable attitude of academics
shows a greater probability that they will accept the modern medium of information dissemination.
Also, (Golden, et al. 2006 ) revealed that lecturers’ use of eLearning was associated more with their
own attitudes than with their personal institution.
This finding however, contradicts that of (Vrana, Fragidis, Zafiropoulos & Paschaloudis 2011), who
established lack of correlation between attitudes towards educational technologies and e-publishing
on the one hand and various variables such as age, educational background, employment
relationship or ownership of personal computer (PC) on the other. They asserted that this is an
indication that the attitudes revealed by the participants in their study are not related to exogenous
variables.
Hypothesis two also shows a significant and positive relationship between computer self-efficacy
(CSE) and e-publishing as reported by some previous studies. For instance, a meta-analysis of
studies published between 1998 and 2008 revealed that computer self-efficacy beliefs were positively
related to e-publishing (Multon, Bass & Laslas 2010). Computer self-efficacy beliefs were related to epublishing
(r=.38) and accounted for approximately 2% of the variance. This study also supported the
studies of Nigerians like that of Adegbola (in Oladejo, 2010) which maintained that computer selfefficacy
contributed significantly to the academics’ e-publishing . Reason for this finding might be due
to the fact that individuals with strong CSE will try out new innovations in technology and will be willing
to try to teach themselves how to use software. According to (Compeau, Higgins & Haff 1999) and
(Sein, Bostrom & Olfman 1987), a firm belief in one’s ability to deal with technology allows people to
quickly adapt related skills and provide them with the opportunity to imagine how such skills could be
applied to a variety of tasks. The results concerning the influence of computer self-efficacy may be
due to the fact that it does not reflect the actual existence of skills but the perceptions that the
individual may have of such individual’s abilities.
The present study as shown in hypothesis three establishes no significant relationship between
academic self-concept and e-publishing. This finding corroborates the study of Carpenter ( in Oladejo,
2010) that established no significant relationship between academic self-concept and e-publishing.
The reason for this finding might probably be because the participants have not formed positive
academic self-concept about publishing electronically.
Furthermore, finding from hypothesis four shows that there is significant gender difference in epublishing
(t = 2.29, df =348, P> 0.05). It therefore contradicts studies conducted by Bart (in Omotayo,
2010) that established no significant gender difference in e-publishing, but supports Parllet (in
Oladejo, 2010)’s study that found a significant gender difference in e-publishing. The rationale behind
598

Maruff Akinwale Oladejo and Adelua Olajide Olawole
this finding might not be unconnected with the fact that male academics are less occupied with home
demands which gives them enough time to work on the internet. They are also likely more efficacious
and have positive attitude towards technology than their female counterparts.
Also, there was significant difference in e-publishing based on age (t = 2.22, df =348, P> 0.05) in the
present study. In fact, younger academics patronize e-publishing than the older ones. This might be
due to the fact that the younger academics have acquired positive orientation about the importance of
e-publishing and are more technologically-inclined.
However, no significant difference was noticed in e-publishing based on rank (t =.376, df =348, P>.05)
though, academics below the Senior Lecturer rank appear better off. This is possible probably
because academics above Senior Lecturer cadre might be dominated by younger academics who, as
earlier remarked are likely to be more technologically-inclined.
4.1 Conclusion
eLearning has become an increasingly popular mode of instructions, especially in higher education
due to advances in the Internet and multi-media technologies. It has indeed, penetrated into academic
publishing. Thus, there is no doubt that scholarly academic publishing is going into a paperless
situation, whereby all information will be exchange electronically. It therefore, becomes highly
imperative for the Nigerian College academics to embrace electronic publishing. This will in no small
way, help academics in their research output and teaching activities, as well as rendering of
community services.
4.2 Policy implications
The need for eLearning policies today is vital for Higher Educational Institutions (HEIs) in view of the
potential of educational technologies and the magnitude of changes they may provoke in education
(Vrana, et al., 2011 ). The authors also contend that need for firm policies now becomes more critical
and consequential because the introduction, adoption and development of eLearning require the
contribution and teamwork of many different stakeholders within the educational Institutions. Findings
from this study therefore have certain policy implications. College Management should therefore come
up with eLearning policies that:
Academics should be made to present certain number of electronic journal publications during
promotion exercise, depending on the rank they are moving to.
All academics should subscribe to internet connectivity, and deductions be made from the source
on monthly basis. This should not even be voluntary as being practiced in some Colleges.
Also, academics should belong to one international electronically-based Association or the other.
Emphasize the development of incentives for adoption of educational technology, thus, e-
learning should be a part of the performance assessment and reward systems.
4.3 Recommendations
Based on the findings from the present study, it is hereby recommended as follows:
College academics should be encouraged to have positive attitudes towards technology and
consequently e-publishing. They should also be positively self-efficacious about electronic
publishing.
Institutional administrators should put in place, necessary motivational mechanisms such as
giving free or subsidized laptops and modems to the academics, organize regular training and retraining
workshops or seminars on internet usage for the academics.
College Management should ensure the availability of effective and efficient networked computers
in all staff offices.
The Colleges’ Libraries and Resource Centres should increase their bandwidth and electronic
journal subscription.
Younger academics, especially the male ones, should encourage and motivate the older
academics in the aspects of e-publishing.
599

References
Maruff Akinwale Oladejo and Adelua Olajide Olawole
Ajzen, I. and Fishbein, M. (1980) Understanding Attitudes and Predicting Social Behaviour, Englewood Cliffs,
New Jersey: Prentice-Hall.
Azubogu, N.C., and Madu, C. (2007) Use of Computer and Internet Technology among the Teaching Staff of Imo
State University, Owerri. H-JOLIS (Heartland Journal of Library and Information Science), Vol. 1, No. 2, pp
38-49.
Badu, E.E., and Markwei, E.D. (2005) “Internet Awareness and Use in the University of Ghana”, Journal of
Information Development, Vol. 21, No.4, pp 260-268.
Bandura, A. (1986) Social Foundations of Thought and Action: A social Cognitive Theory. Englewood Cliffs, NJ:
Prentice-Hall.
Bertea, P. (2009) “Measuring Students’ Attitude towards ELearning: A Case Study”, Proceedings of 5th
International Scientific Conference on ELearning and Software for Education, Bucharest, April 9th-10th.
Bishorp, G. T. (1985) “Electronic Publishing: To be or Not To be? “ The Quarterly Review of Biology, Vol. 60,
No.1, pp. 43-54
Burkhardt, M. E., & Brass, D. J. (1990) “Changing Patterns or Patterns of Change: The Effects of a Change in
Technology on Social Network Structure and Power” Administrative Science Quarterly, Vol. 35,No. 4, pp
104-127.
Compeau, D.R. and Higgins, C.A. (1995) “Computer Self-Efficacy: Development of a Measure and Initial Test”,
MIS Quarterly, Vol.19, No. 2, pp 189-211.
Compeau, D., Higgins, C. and Huff, S. (1999) “Social Cognitive Theory and Individual Reactions to Computing
Technology: A Longitudinal Study,” MIS Quarterly, Vol. 23,No. 2, pp. 145 -158.
Ehikhamenor, F.A. (2003) Internet Resources and Productivity in Scientific Research in Nigerian Universities.
Journal of Information Science 29 (2): 102-116.
Fayombo, G. A. (2001) “ Improving the Academic Achievement of Some Nigerian Adolescents Through Self-
Concept Grooming” African Journal of Educational Research, Vol. 7, No.1, 168-177.
Foucher, R., and Prince, I. (2003) “Motivation et Apprentissage dans des Dispositifs Ouverts de Formation,
Dans" N. Delobbe, G. Karnas et C. Vandenberghe (dir.) Évaluation et développement des Compétencesau
Travail, Vol.1. Psychologie du Travail et des Organisations, (p.517-526). Presse de l’Université de Louvain.
Golden, S., McCrone, T., Walker, M. and Rudd, P. (2006) Impact of ELearning in Further Education: Survey of
Scale and Breath. London, DfES.
Ken B, (2011) “What are Electronic Publications?” In A. Jones (Ed) Wise GEEK.
Mitchell, J. V. Jr. (1996). Interrelationships and Predictive Efficacy for Indices of Intrinsic, Extrinsic and Self-
Assessed Motivation for Learning. Journal of Research and Development Education, Vol. 25, No. 3.pp149-
155.
Multon, F. D., Bass, S. D., & Laslas, K.W. (2010) “Computer Self-Efficacy Beliefs and Academic Performance”
Journal of Counselling Psychology, Vol. 4, No. 6, pp 13-24.
Nasser, R. and Abouchedid, K. (2001) Problems and the Epistemology of Electronic Publishing in the Arab
World: The Case of Lebanon by Ramzi Nasser and Kamal Abouchedid First Monday, Vol. 6, No.9
(September 2001), [online] URL:http://firstmonday.org/issues/issue6_9/nasser/index.html
Ojedokun, A.A., & Owolabi, E.O. (2003) “Internet Use for Teaching and Research in Botswana” African Journal of
Library, Archives, and Information Science, Vol. 13, No.1, pp 43-53.
Okerson, Ann. (1991)”The Electronic Journal: What, Whence, and When?” The Public-Access Computer
Systems Review , Vol. 2, No. 1, pp 5-24. [online] http://bubl.ac.uk/journals/lis/oz/pacsr/v02n0191.htm
Oladejo, M.A. (2010) A Path-Analytic Study of Socio-Psychological Variables and Distance Learners’ Academic
Performance in Nigerian Universities. Doctoral Thesis, University of Ibadan, Nigeria.
Omotayo, B O ,( 2010) Access, Use, and Attitudes of Academics Toward Electronic Journals: A Case Study of
Obafemi Awolowo University, Ile-Ife,Nigeria.
Sein, M., Bostrom, R. and Olfman, L. (1987) “Training End-Users to Computers: Cognitive, Motivational, and
Social Issues,” Information Systems and Operations Research, Vol. 25, No. 2, pp. 236-254.
Smith, J.G. (2007) The Impact of Electronic Communications on the Science Communication Process:
Investigating Crystallographers in South Africa. IFLA Journal, Vol. 33, No. 2, pp 145-159.
Stephens, P. and Shotick, J. (2001) “Computer Literacy and Incoming Business Students: Assessment, Design
and Definition of a Skill Set,” Issues in Information Systems, Vol. 2, No. 4, pp. 460 – 466.
Webster, J., and Hackley, P. (1997) “Teaching Effectiveness in Technology-mediated Distance Learning,”
Academy of Management Journal, Vol. 40, No. 6, pp 1282-1309.
Vrana, V.; Fragidis, G.; Zafiropoulos, C.; & Paschaloudis, D. (2010) “Analysing Academic Staff and Students’
Attitudes towards the Adoption of ELearning”, Paper Delivered at the International Forum of Educational
Technology & Society (IFETS), July.
600

Psycho-Social Predictors of Students With Disabilities’
eLearning: Usage at the Federal College of Education
(Special), Nigeria
Adelua Olajide Olawole and Maruff Akinwale Oladejo
Federal College of Education (Special), Oyo, Nigeria
adeluaolajide@yahoo.com
maruvoladejo@rocketmail.com
Abstract: There has been growing interest in the education of students with disabilities in the recent times. This
is contingent on the fact that students with disabilities constitute distinct population whose educational needs are
different from those students without disabilities. Also, the application and usage of Information and
Communication Technologies (ICTs) as indispensable force to aid teaching-learning processes has equally been
on the increase. Instructional use of information and communication technologies carries a peculiar importance
as they help in meeting the instructional needs of students with disabilities who cannot follow the requirements of
normal educational processes. Thus, information and communication technologies have become frequently used
devices in modern instructional settings by all stakeholders, students with disabilities inclusive. These
technologies regulate, direct, guide and shape students with disabilities’ academic activities and social
interactions. Consequently, the increased use of these technologies and the recent developments in adaptive
software and hardware have enabled students with disabilities to participate in, and do things that used to be
difficult or impossible for them. Several studies have been carried out on eLearning usage, but not much on
students with disabilities in Nigeria. This study therefore, provided a causal explanation of students with
disabilities’ eLearning usage through the analysis of some psycho-social variables such as gender, nature of
disability, computer self-efficacy, technological confidence and attitude towards technology. The study adopted
descriptive research design of the “ex-post facto” in nature. The Federal College of Education (Special), Nigeria
was purposively selected. Two hundred and fifty participants were selected through simple random sampling
technique. Three validated self-designed instruments namely Students’ Computer Self Efficacy Scale (r =0.75),
Attitudes towards Technology Questionnaire (r =0.86), Students’ Technological Confidence Scale (r =0.75) were
used to collect data. Two research questions and hypotheses each were answered and tested to pilot the study.
Regression analysis and t-test were used for data analysis. The five factors combined accounted for 17% of the
total variance in students with disabilities’ eLearning usage. The order of magnitude in terms of relative
contributions were computer self-efficacy (β=.048), technological confidence (β=.041), attitude towards
technology (β=.038), gender (β=.025) and nature of disability (β=.018). Furthermore, there were significant
differences between nature of disability (t=2.29, df= 248, P

Adelua Olajide Olawole and Maruff Akinwale Oladejo
Asuncion, Chwojka, Barile, Nguyen, Klomp, & Wolforth 2009) maintained that the increased use of
these technologies and the recent developments in adaptive software and hardware have enabled
students with disabilities to participate in, and do things that used to be difficult or impossible for them.
For example, it has allowed people who are blind to read using text-to-speech technology, people
who are deaf to communicate using chat programs, and people with difficulties using their hands or
arms to write and communicate using dictation software (Fichten, Asuncion, Barile, Fossey, & De
Simone 2000).
(Burgstahler 2003) lists the benefits of implementing these technologies for students with disabilities
as follows:
Maximizes independence in academic and employment tasks;
Increases participation in classroom discussions;
Helps students gain access to peers, mentors and role models;
Helps them self-advocate;
Provides them with access to the full range of educational options;
Helps them participate in different experiences not otherwise possible;
Provides them with the opportunity to succeed in work-based learning experiences;
Secures high levels of independent living;
Prepares them for transitions to college and careers;
Gives them the opportunity to work side-by-side with peers;
Helps them enter high-tech career fields; and
Encourages them to participate in community and recreational activities.
1.1 Electronic learning defined
In reference to (Homan & Macpherson 2005), the emergence of eLearning is well documented,
however , what constitutes eLearning is less well defined. Be that as it may, (Homan & Macpherson
2005) described the term “eLearning” to cover any electronic learning material from CD-ROMs on
stand alone Personal Computers (PCs) to intranet/internet networked systems with downloadable and
interactive material. According to (Hall & Snider 2000), eLearning is the process of learning via
computers over the Internet and Intranets. In general, e- learning is defined as the use of Internet and
digital technologies to create experiences that educate fellow human beings (Horton 2001). However,
it was (Jackson 2001), who categorized eLearning into two namely: “technology-enhanced
eLearning”, in which technology is used as a supplement to traditional on-campus learning, and
“technology-delivered eLearning”, which is related with distance education.
1.2 Psycho-social factors and eLearning
Students with disabilities are faced with some psychological cum social hindrances , capable of
preventing them from participating and gaining fully in electronic learning environment. For them to be
able to fit in properly, they are expected to have positive attitudes towards eLearning and be
technologically confident. For this reason, determining these types of psychological characteristics of
students with disabilities is extremely important to be able to assist them in their academic work in
electronic environment.
(Saade & Kira 2009) argued that in the context of eLearning, a student’s ‘likelihood of use of a specific
system such as the Learning Management System (LMS)) is jointly determined by several psychosocial
factors among which is their attitude toward using the system. The need to investigate the
attitudes of SWDs is highly consequential. This is because attitude has been seen as a mental state
that exerts influence on a person's response to people, objects and situations, Shannon (in Olaoye,
2005). (Ajzen and Fishbein 1980) maintained that attitudes are positive or negative evaluations of
object, people, or situation that predispose one to feel and behave toward them in positive or negative
ways. Measuring attitudes therefore, has a vital role in analyzing behaviour because it has been
established that there is a strong correlation between attitude and behaviour (Bertea 2009). In the
context of e-environment, a favourable attitude of academics shows a greater probability that they will
602

Adelua Olajide Olawole and Maruff Akinwale Oladejo
accept the modern medium of information dissemination. Attitudes towards technology can therefore
predict the electronic usage or otherwise by students with disabilities.
Also, Computer Self-Efficacy (CSE) has been identified as another psychological factor that can
determine students with disabilities’ eLearning usage. According to ( Bandura 1986: p. 391), general
self-efficacy (GSE) refers to "people's judgments of their capabilities to organize and execute courses
of action required to attain designated types of performance". The concept of computer self-efficacy
emerged from the general self-efficacy literature. Computer Self-Efficacy (CSE) therefore according to
(Compeau & Higgins 1995), measures one’s confidence in mastering a new technology or software
with certain degree of confidence. Also, (Stephens & Shotick 2001), opined that computer selfefficacy
is an individuals’ beliefs in their ability to use technology in order to solve problems, make
decisions, and to gather and disseminate information. Also, (Foucher and Prince 2003) argued that in
the field of eLearning, students who decide to use the inter system have a sense of self-efficacy
higher than those who choose not to use the system.
Furthermore, how technological confident one is, needs to be considered while discussing electronic
usage. This is because the technological confidence of one has been reported to be a determinant of
electronic usage (Akinniyi, 2010). In fact, (Akinniyi 2010) established that individual’s technological
confidence is positively and significantly related to such individual’s eLearning adoption and usage
(r=.023). The Researchers were therefore inclined to examine further, the relationship that exists
between these psycho-social variables with respect to students with disabilities’ eLearning usage,
which appears to have received little attention in Nigeria. This becomes highly important because
attention now shifts to inclusive educational system, whereby students with or without disabilities
receive instructions in regular educational Institutions.
1.3 Statement of the problem
The issue of acceptance of eLearning technology by learners has attracted many researchers in
information systems. However, it appears that only few research have investigated the influence of
psycho-social factors on the acceptance and usage of eLearning technology by students with
disabilities in Nigeria. Hence, the problem of this study is to examine the predictive power of some
psycho-social factors on eLearning usage by students with disabilities in Nigeria. To this end, the
following research questions and null hypotheses were answered and tested in the study:
Research Question 1: To what extent would the selected factors namely gender, nature of disability,
computer self-efficacy, technological confidence and attitude towards technology, when taken
together, predict the eLearning usage of students with disabilities in Nigeria?
Research Question 2: What are the relative contributions of each of the selected factors to the
prediction of eLearning usage of students with disabilities in Nigeria?
Hypothesis 1: There is no significant gender difference in eLearning usage of students with
disabilities in Nigeria.
Hypothesis 2: There is no significant difference in eLearning usage of students with disabilities based
on the nature of disability in Nigeria.
2. Methodology
2.1 Design
The adopted research design for the present study is the descriptive survey which is “ex-post facto” in
nature. An “ex-post facto “study according to a scholar is described as :
A systematic empirical inquiry in which the scientist does
not have direct control of independent variables because their
manifestations have already occurred or because they are inherently
not manipulable. Inferences about relations among variables are made
without direct interaction from concomitant variation of independent and
dependent variable, Kerlinger (in Oladejo, 2010: 264).
603

2.2 Sample and sampling techniques
Adelua Olajide Olawole and Maruff Akinwale Oladejo
The Federal College of Education (Special) (FCES), which happens to be the first and only College of
Special Education not only in Nigeria, but also in the whole of Sub-Sahara Africa, was purposively
selected for the study. Also, a total number of 250 students with hearing and visual impairments was
selected purposively.
2.3 Instrumentation
One instrument titled ‘Students With Disabilities’ Electronics Usage Questionnaire’ (SWDEUQ) was
used for the study. It consists of four sections. Section A contained items on the socio-demographic
background of students with disabilities such as age, gender, level, nature of disability. Section B
consisted of 10 items on attitudes towards electronic usage. The items were drawn on a modified
four-point Likert scale of Strongly Agree (SA), Agree (A), Disagree (D), and Strongly Disagree (SD)
and carried the weights of 4,3,2,1 respectively. Section C is on students with disabilities’ technological
confidence. It has 10 items, which were drawn on a modified four-point Likert scale of Most Like Me
(MLM), Like Me (LM), Least Like Me (LLM), and Not Like Me (NLM). It was scored with the weights of
4,3,2,1 respectively. Finally, computer self-efficacy scale beliefs of students with disabilities is in
Section D. It is made up of 10 items, drawn on a modified four-point Likert scale of Strongly Agree
(SA), Agree (A), Disagree (D), and Strongly Disagree (SD) and carried the weights of 4,3,2,1
respectively. A reliability study was carried out using 85 students with other forms of disabilities
besides hearing and visual impairments, who were not used for the main study. Cronbach’s coefficient
was computed based on their responses. The alpha values obtained was 0.75.
2.4 Administration of instrument
The researchers collected data during specially scheduled lecture hours. All the questionnaires were
purposively administered on students with hearing and visual impairments. General Class
Representatives were used as Research Assistants. Three hundred copies of the questionnaire were
given out. Two hundred and eight-six copies of them were returned. This gave 95.5% rate of return
However, 36 out of these questionnaires were not completely filled. The remaining 250 copies found
to be appropriately and completely filled were used for the study.
2.5 Method of data analysis
Multiple Regression Analysis (MRA) and t-test were used for data analysis. MRA was used to answer
the two research questions, while t-test was used to test the two hypotheses. The hypotheses were
tested at 0.05 level of significance.
3. Results
Research Question 1: To what extent would the selected factors namely gender, nature of disability,
computer self-efficacy, technological confidence and attitude towards technology, when taken
together, predict the eLearning usage of students with disabilities in Nigeria?
Result
To provide answer to this question, multiple regression analysis of eLearning usage of students with
disabilities on the five psycho- social constructs was carried out. The total contribution of the five
explanatory variables to the prediction of the criterion variable is shown in Table 1.
Table 1: Summary of regression of eLearning usage by students with disabilities on the selected
factors
R
.152
R Square
.020
Adjusted R Square
.012
Standard Error of the Estimate
Table 1 shows that the five psycho- social factors namely gender, nature of disability, computer selfefficacy,
technological confidence and attitude towards technology have a joint positive multiple
correlation with eLearning usage of students with disabilities (R=.152). This implies that the five
604
1.334

Adelua Olajide Olawole and Maruff Akinwale Oladejo
factors are quite relevant and consequential towards the determination eLearning usage by students
with disabilities.
Furthermore, the table reveals that the five factors explained 2.0% of the total variance in eLearning
usage of students with disabilities (R Square =.020). By implication, the remaining 98% is due to other
factors (excluded in the present study) and residuals.
However, in order to determine whether or not, the R Square value of .020 obtained is significant, the
Analysis of Variance (ANOVA) was performed. This is shown in the Table 2.
Table 2: Analysis of variance of the regression analysis
Source of Variance Sum of
Squares
Regression
118.640
Residual
3877.391
Total
3996.032
*Significant at p

3.1 Result
Adelua Olajide Olawole and Maruff Akinwale Oladejo
Table 4 presents information on hypothesis 1 as measured by t-test to determine the significance or
otherwise of the difference between the eLearning usage of male and female students with
disabilities. The result shows a mean of 2.45 for female students with disabilities compared with a
mean of 2.61 from male counterparts. This finding indicates that there was gender difference in the
eLearning usage of male and female students with disabilities (t = 2.21, df =248, P

Adelua Olajide Olawole and Maruff Akinwale Oladejo
(PDAs). Personal discussions also reveal that most of the hearing impaired students have their own
personal laptops. Thus, they are used to working on the internet like browsing, sending e-mails and
so on at the College’s National Resource Centre for the Disabled, where there are provisions for
Information and Communication Technologies (ICTs) for academic activities.
Finally, the eLearning usage of students with disabilities was significantly different on gender basis
(t=2.21, df=248, P

Adelua Olajide Olawole and Maruff Akinwale Oladejo
Fichten, C. S., Asuncion, J., Barile, M., Fossey, M., and De Simone, C. (2000) “ Access to Educational and
Instructional Computer Technologies for Post-Secondary Students with Disabilities: Lessons from Three
Empirical Studies”, Journal of Educational Media, Vol. 25, No.3, pp 179-201.
Foucher, R., and Prince, I. (2003) “Motivation et Apprentissage dans des Dispositifs Ouverts de Formation,
Dans" N. Delobbe, G. Karnas et C. Vandenberghe (dir.) Évaluation et développement des Compétencesau
Travail, Vol.1. Psychologie du Travail et des Organisations, (p.517-526). Presse de l’Université de Louvain.
Georgiev, T., and Gobachev, A. (2004) M-Learning – The Modern Stage of ELearning. International Conference
on Computer Systems and Technologies . [online] Retrieved January 04,2005 from
http://edet.ecs.ru.acad.bg/cst05/ Docs/sIV/426.pdf.
Hall, B., and Snider, A. (2000) Glossary: The Hottest Buzz Word in the Industry, Learning, Vol. 44, No. 4, pp 85-
104.
Homan, G., and Macpherson, A. (2005) “ELearning in the Corporate University” Journal of European Industrial
Training, Vol. 29, No. 1,pp 75-99.
Jackson, R.H. (2001) Defining ELearning - Different Shades of Online Web-Based Learning Resources. [online]
Retrieved on June 25, 2005 from http://www.outreach.utk.edu/weblearning/.
Newton, R. (2003) “Staff Attitudes to the Development and Delivery of ELearning” New Library World. Vol.
104,No.1193, pp 412-425.
Odabasi, H.F.; Kuzu, A.; Girgin, C.; Cuhadar, C.; Kiyici, M. and Tanyeri, T. (2009) “Reflections of the Hearing
Impaired Students on Daily and Instructional PDA Use”, International Journal of Special Education, Vol. 24,
No. 1, pp8-19.
Saade, R., and Kira, D. (2007)”Mediating the Impact of Technology Usage on Perceived Ease of Use by Anxiety”
Computers and Education, Vol. 49, No.4,pp 1189-1204.
Smith, Q. P., and Bendev, P. (2003)”The Use of Personal Digital Assistants for Learning: A Review of Literature”,
Learning and Skills Development Agency, Vol.. 4, No. 3, pp 95-105.
Stephens, P. and Shotick, J. (2001) “Computer Literacy and Incoming Business Students: Assessment, Design
and Definition of a Skill Set,” Issues in Information Systems, Vol. 2, No. 2, pp. 460 – 466.
Vrana, V.; Fragidis, G.; Zafiropoulos, C.; and Paschaloudis, D. (2010) “Analysing Academic Staff and Students’
Attitudes towards the Adoption of ELearning”, Paper Delivered at the International Forum of Educational
Technology & Society (IFETS), July.
608

An Integrated Environment for Providing Learning Style
Information in a Unified Manner
Fatemeh Orooji, Fattaneh Taghiyareh and Zahra Rahimi
Department of Electrical and Computer Engineering, Tehran University,
Tehran, Iran
f.orooji@ut.ac.ir
ftaghiyar@ut.ac.ir
z.rahimi@ut.ac.ir
Abstract: Learning styles inventories provide insights into student perceptions about how they prefer to learn.
Since learning styles have notoriously some inconsistencies and invalidities, it is unreliable to base instructional
decisions on just one learning style preferences. This paper introduces a new approach to integrate various
learning style modelers in a unified system. The unified system contains some predefined phases in modeling
each style’s dimensions and some representing suggestions in order to visualize modeler results. Four popular
learning styles along with their specific dimensions and report representations have been studied to demonstrate
how to synthesize various inventories into a single representation of learners’ learning styles. Utilizing integrated
environment in the University of Tehran, department of electrical and computer engineering, has revealed some
dependencies between a modeler dimensions as well as some correlation between two modelers' dimensions.
The implication of this effort is to provide advice about how web-based instructions can be modified to
accommodate learners’ differences.
Keywords: user model, learning style, learning style questionnaire, web-based educational system, eLearning
1. Introduction
In the past decades, the rapid growth of the internet has brought a great deal of changes in our
educational environment, demanding continuous access to education. These web-based educational
systems need to consider different characteristics of a large number of students, constructing their
user models. User model represents essential information about each user in two dimensions: first,
viewing him as an individual considering the user’s knowledge, interests, goals, background, and
individual traits; second, modeling his/her context of work which consists of location, time of day and
platform. Individual traits are stable and usually determined using psychological tests, consist of
personality traits, cognitive styles, cognitive factors and learning styles (Brusilovsky and Millán, 2007).
Learning style (LS) shows how people learn and how they prefer to organize and represent
information. People have different strategies in managing information and different ways of
implementing these strategies. Considering the LS of each learner individually, provides facilities to
adapt the content (form, structure, presentation order of learning activities and choices of these
activities) to improve the learning results, which is impossible for a teacher in a traditional educating
system with a group of learners (Reed et al., 2000). There are several studies on some pedagogically
personalized learning contents based on learners’ LSs, indicating differences in performance between
matched and non-matched learners (Siadaty and Taghiyareh, 2007). However, there are still some
questions about the consistency of visual, auditory and kinaesthetic preferences and the value of
matching teaching and LSs (Coffield et al., 2004).
A considerable amount of research has been carried out to clear what is the best method to capture
learners’ LS. LSs can be detected through learner’s profile as well as his behaviors, for example his
navigation behaviour in web-based educational system (Bousbia et al., 2010). In addition, using a
questionnaire is another acceptable approach to detect user LS. Naturally almost all of known models
have proposed their own questionnaire independently (Felder and Silverman, 1988) (Grasha and
Riechmann, 1975) (Briggs Myers, 1962). It is noticeable that there are more than 70 definitions of the
LS concept and many researchers have tried to evaluate them showing the relationships between the
LS identified by the instrument and students' actual learning. It has been claimed that since “a reliable
and valid measure of learning styles has not yet been developed, it will be more useful to focus on
learners’ previous experiences and motivation” (Coffield et al., 2004).
This research aimed to be beyond any individual LS by gathering different LS models together and
unifying their information via a proposed approach. This unified modeling system provides a complete
report of learners’ LSs in adoption to integrated modelers, which may bring about better self-
609

Fatemeh Orooji et al.
awareness and lead to more organized and effective approaches to teaching and learning. To
address this issue, our interest concerned providing a unified methodology to define and represent
various models’ dimensions through predefined construction phases. Our depicted mechanism has
illustrated the selected models in a similar structure and moved toward a comprehensive LS provided
for each user based on different aspects of utilized modelers in addition to some facilities to compare
their results in some understandable representations. In this way, there is a possibility to have a more
precise picture of each learner decreasing the amount of uncertainty which he/she will be interfaced
by trying just one modeler (due to the modeler inconsistency and unreliability (Coffield et al., 2004)).
The paper is organized as follows. We begin with a brief overview of LS theory in section 1 followed
by a description of some known LSs in details. Our proposed approach for defining a LS and its
different dimensions is then introduced. Section 4 is devoted to the depiction of our unified LS
modeling system to point out some of its features. Some outcomes of utilizing the proposed system
are explained in Section 5. Finally, we present some conclusions and recommendations for further
study.
2. Background
Learning style is one of learner individual traits typically defined as the way people prefer to learn,
representing their habitual manner of acquiring knowledge and skills via studying or experiencing
(Coffield et al., 2004). Understanding LS enables a learner to improve their self-awareness and metacognition
skills, to develop his/her natural approaches to learning, and even to provide more capacity
to learn in ways that he/she may require more efforts. If students know about their LSs and
preferences, they can select better choices about learning tasks they should engage with. On the
other hand, knowing the LS may enrich instructors’ information about their students, leading them to
develop a range of teaching strategies to engage learner individual strengths. Some studies have
demonstrated benefits of adapting learning contents and activities based on learners’ LS (Siadaty and
Taghiyareh, 2007) while some other researchers have some doubts about the correctness and
necessity of such adaptation (Coffield et al., 2004).
2.1 LS modeling approaches
Generally, there are two learner modeling approaches as follows (Brusilovsky and Millán, 2007):
Collaborative: In this approach, users explicitly provide information about themselves via filling out a
questionnaire. Although this approach provides reliable information, most of earners do not like
spending a considerable amount of time to fill corresponding questions. Almost all LS modelers
introduce their special questionnaires, some explained in following sections.
Automatic: In this approach, the system implicitly gathers any information needed while users are
working/learning. Class discussion/student reflection, instructor observations, cognitive sciences or
artificial intelligence are all utilized in this approach, no need to ask implicit questions.
2.2 Learning style main parameters
Learning preferences related to different LSs can be summarized as follows (Popescu et al., 2007):
Environment, Physical dimensions, Perception modality, Field dependence/field independence,
Processing information, Reasoning, Organizing information, Motivation, Persistence, Pacing, Social
aspects, Study organization, Coordinating instance.
Also, there is an "Onion Model" which categorizes LSs into four layers (E.Popescu et al., 2007):
Personality Models focus on the personality traits of the learner and the way they influence the
learning process.
Information Processing Models focus on the processes of acquiring, ordering, and engaging with
information.
Social Interaction Models focus on the collaborative aspects of the learning process.
Instructional Preference Models focus on the environmental, emotional, and sociological
preferences of the learner.
610

2.3 Learning Strategies
Fatemeh Orooji et al.
On the other hand, there are several competing theories about how people learn. To give a highly
simplified overview, the theories cover three main aspects of how people study:
Perceiving information (auditory, reading): When a person gathers information about the world around
him/her (including needed data to study), he/she may employ all of his/her senses unequally. The
VARK system assesses how much people rely on: Visual (sight), Auditory (hearing), Reading, and
Kinesthetic (other sensations like touch and temperature as well as movement). People say things like
‘I’m an auditory learner’ (meaning that they are comfortable absorbing information which they have
heard or discussed); or ‘I’m a Kinesthetic learner’ (if they prefer to learn through practical classes and
hands-on activities, rather than by reading books and listening to lectures). In fact, we use all of our
senses to absorb information.
Processing information (abstract, logical, sequential, and reflective): Once a person has acquired the
information (by listening, reading, etc.), he/she then processes it mentally, as he/she thinks about it
and memorizes it. He/she will have a natural preference for how he/she:
Grasp information: does he/she prefer to deal with abstract concepts and generalizations, or
concrete, practical examples?
Order information: would he/she rather receive facts in a logical, sequential way (to build up a
picture one step at a time), or with an overview straight away (to show the big picture first, then
the details)?
Engage with information: does he/she prefer active experimentation or reflective observation?
Organizing/presenting information (analytic, verbal): Finally, there is how a person chooses to share
information with others. He/she will prefer how he/she:
Organise information: with a holistic overview, or with detailed and logical analysis
Present information: verbally or using images.
3. Selected models for proposed approach
We have provided a mechanism to build an understandable and integrated picture of learner’s LS.
This picture puts models dimensions together and shows their values via a graphical interface. The
graphical representation of models information can support the development of students’ selfawareness
and meta-cognition skills. Students who know how they learn, make more informed
choices about how to engage in learning(Coffield et al., 2004).
We wanted to depict a unified approach for different phases of dimensions definitions as well as their
inferring and representing mechanisms. Since there are different dimensions of a learner's
characteristics and behaviors concentrated in a model, it is necessary to manage all dimensions in a
similar way. In addition, the inferring method of each model may be different from the others. Thus, it
is a real demand to provide some consistent and comparable inferring methods. Furthermore, the
modelers utilize different representing mechanism. We have unified these mechanisms in order to
provide a better recognition of learners' characteristics.
Our integrated environment supports some types of models that their representing and inferring
approaches are understandable by proposed mechanism. Although there are very few robust studies
which offer, for example, reliable and valid evidence and clear implications for practice based on
empirical findings as (Coffield et al., 2004) has claimed, we have selected some models which are
more applicable in web-based educational systems as (Graf, 2007) indicated. Models consist of: Kolb
and Felder-Silverman LS inventories which indicate ‘Flexibly stable’ LSs, Myers-Briggs Type Indicator
(MBTI) which considers LSs as relatively stable personality type, and Grasha-Riechmann LSs which
detects LS related to approaches and strategies(Coffield et al., 2004).
3.1 Myers-Briggs Type Indicator (1962)
Myers-Briggs Type Indicator (MBTI) is a personality test which includes important aspects for
learning(Briggs Myers, 1962). Derived from psychologist Carl Jung's theory, MBTI classifies students
according to their preferences on these four scales:
611

Fatemeh Orooji et al.
Figure 1: The Myers-Briggs Type Indicator 4x4 grid structure
Extraverts/Introverts: refers to the orientation of a person; attitude is on the surroundings (like
other people) or on his/her own ideas and thoughts.
Sensors/Inductors: the way people prefer to perceive data; from their five senses or from
unconscious.
Thinkers/Feelers: the base of judgment; logical connections ( “true or false” and “if-then”) or
feeling refers to humanistic evaluation (“more-less” and “better-worse”)
Judgers/Perceivers: prefer step-by-step approach, seek closure even with incomplete data or
being more flexible, and keep all options open.
3.2 Grasha-Riechmann LSs (1975)
The Grasha-Riechmann learning style model is based on students’ responses to actual classroom
activities and identifies three bipolar dimensions(Grasha and Riechmann, 1975):
Participant/avoidant: wish to become involved in the classroom activities or avoid
Collaborative/competitive: the motivation behind a student’s interactions with others; corporative
or competitive
Dependent/independent: attitudes toward teachers; desire freedom or control in the learning
environment
Grasha argues that this situation-specific approach is more likely to be reliable and can help faculty
identify teaching techniques to address different LSs.
Figure 2: Our depiction for Grasha-Riechmann LSs
612

3.3 Kolb’s theory of LSs (1984)
Fatemeh Orooji et al.
According to Kolb(Kolb, 1984), the learning cycle involves four essential processes which each
person needs to complete and its LS Inventory shows where he/she prefers to enter the learning
cycle (Hashim, 2003). The four-stage cycle consists of:
Concrete Experience (CE): being involved in a new experience
Reflective Observation (RO): watching others or developing observations about own experience
Abstract Conceptualization (AC): creating theories to explain observations
Active Experimentation (AE): using theories to solve problems make decisions
Figure 3: Kolb’s LSs
In this theory, there are four LSs –diverging, assimilating, converging and accommodatingdetermined
by the combinations of two preferred styles of perceiving and processing, for which Kolb
used the terms: Diverging (CE/RO), Assimilating (AC/RO), Converging (AC/AE), Accommodating
(CE/AE).
3.4 Felder- Silverman LS Model (1988)
They show how learners prefer to process, perceive, receive and understand, by using scales from
+11 to -11 for each dimension (including only odd values) which facilitate describing the LS
preferences in more detail (Felder and Silverman, 1988):
Figure 4: Our depiction of Felder- Silverman LS Model
613

Fatemeh Orooji et al.
Sensing/intuitive: oriented toward facts and procedures or toward theories and meanings (taken
from the Myers-Briggs Type Indicator and has also similarities to the sensing/intuitive dimension
in Kolb’s model)
Visual/verbal: prefers visual representations or written and spoken ones
Inductive/deductive: inductive learner (prefers presentations that proceed from the specific to the
general) and deductive learners (prefers presentations that go from the general to the specific)
Active/reflective: learns by trying things out, working with others or learn by thinking things
through, working alone (analogous to the respective dimension in Kolb’s model)
Sequential/global: gain understanding in linear steps or in large jumps, absorbing material almost
randomly without seeing connections
3.5 Conclusion
Table 1 shows comparison between 'Myers-Briggs', 'Kolb' and 'Felder-Silverman' learning styles
model from view point of 'Orientation to life processing', 'perception decision making', 'perception
attitude to outside world', 'input', 'organization' and 'understanding' (Montgomery and Groat, 1998).
Table 1: Comparison of 'Myers-Briggs', 'Kolb', 'Felder-Silverman' models
Described LSs may have some dimensions in common while they have considered different labels for
those similar dimensions. Since there is no comprehensive LS modeler, it may be a good idea to
design an integrated LS modeling system, which can provide some facilities toward that purpose.
Integrated environment introduced in this paper, consists of different tools for managers to define their
desired modelers, as well as some useful interfaces for users to interact with system easily.
4. Unified LS modeling system
We have provided an innovative integrated modeling environment in order to accept different kinds of
inventory and provide some feasibility to show their reports in the same way. This paper attempts to
synthesize various Learning Styles inventories into a single representation of learners learning styles.
The implication of this effort is to provide advices about how web-based instructions can be modified
to accommodate learners’ differences.
The proposed system can accept modelers as long as they can be illustrated in adoption to a
predefined methodology. Administrators have been enabled to add desired modelers easily through
some predefined phases as well as to access some valuable reports about their users. On the other
hand, learners who answer some questionnaires in this integrated system can see their reports in a
unified and understandable layout. Comparing these reports, makes it possible for a learner to build a
more completed picture of him/herself, which can be considered as a useful meta-cognition skill. In
addition, all users have been informed of some overall measurements about the whole results like the
average value of each modeler dimensions. They also receive some modeler or person oriented
reports explained in the next sections.
614

Fatemeh Orooji et al.
4.1 System predefined methodology for introducing a new questionnaire
Each model dimension and their computational measures are defined via a systematic approach. The
approach contains some predefined phases to determine type of a questionnaire, add its questions,
determine its dimensions, and select its type of representation, explained in the following subsections.
Predefined phases enriched by some complimentary guides, enable managers and instructors to
define desired learning styles easily.
4.1.1 Type of questionnaire
The first phase of introducing a new questionnaire; proposed by unified LS system; is devoted to
determining type of questionnaire, as illustrated in Fig 5. By analyzing described questionnaires
modelers, we have categorized modelers as multiple choices, correct/incorrect, and numerical
questionnaires, based on their type of questions. Correct/incorrect questionnaire has some questions
with a simple yes/no answer while multiple choices questionnaire contains questions that ask user
opinion about the choices. Numerical questionnaire is a little more complicated since they demand
user to assign a number showing the value of his/her agreement with each questions.
Figure 5: First phase: choose type of questionnaire
4.1.2 Adding questions
Each questionnaire has a number of questions (40-80) asking about learner's personal
characteristics. These questions are determined in the second phase either by importing all of them
once (as an Excel table) or by typing questions one by one.
4.1.3 Questionnaire dimensions
Each questionnaire has some specific dimensions that represent the main aspect focused by modeler
designer. Third phase provides an interface to introduce questionnaire dimensions and reports. In this
phase, dimensions are introduced, and some questions are assigned to each dimension. In addition,
the evaluation method of each question is determined (whether it is needed to count number of ‘a’/‘b’
or it is just ‘true’/’false’). It needs to import modeler final text report, too. Prepared by experts, this text
recommends some useful guidelines for different types of learners. When a learner finishes answering
modeler questions, his/her model will be recognized and its related recommendations will be
presented.
4.1.4 Representation of modeler report
Shape of questionnaire report is determined at the last phase, where modeler scoring methods and
opposite dimensions are stated. First, some questionnaires categorize their numerical scores
substituting values by some predefined labels (low, medium or high). In this situation, each label has
615

Fatemeh Orooji et al.
lower and upper levels that should be assigned below the label of "Assign numbers to dimensions" in
0. It also shows that different colors have been proposed for different margins to make them more
visually understandable. Second, each modeler may either represent its dimensions independently or
organize them as some opposite dimensions. Some questionnaires like Felder-Silverman LS have
dimensions that are two by two against each other and must be placed faced by. In this situations,
opposite dimensions will be represented as two ends of one axis, shown in 0as "Opposite
Dimensions".
Figure 6: Phase 4: determine representation of modeler report
4.2 System management tools
Proposed unified LS modeling system provides some valuable feasibilities enabling administrators
and designers to introduce a new questionnaire as well as altering and removing available modelers.
It is possible to define a new questionnaire via the predesigned phases explained before or edit
desired features of an available modeler easily. In addition, administrator receives some management
specific reports consist of the last version of all activities and the number of users participated, their
LSs reports, group LS reports and the average scores of each group.
4.3 System reports
Perhaps the most important outcome of this study is that it provides an opportunity for a unified
information representation despite the differences between modelers. When learner completes a
questionnaire, system analyzes his answers and makes a graphical report representing achieved
results in an understandable manner. Our integrated environment reports the name of user’s LSs and
his/her scores in a similar manner. In addition, system provides some recommendations based on
some given instructional rules due to learner recognized model. To motivate students, we consider
each learner situation among the others and compare their learning styles to the average of their
community. There are also some other possibilities to compare learners recognized styles against
616

Fatemeh Orooji et al.
their popular instructors’ ones which may make them more enthusiastic about their instructors and
related course.
5. Results
The proposed system has been utilized in the university of Tehran, department of electrical and
computer engineering, ECE. At the first step, students of one course have been demanded to fill FLSI
and MBTI questionnaires. Almost 40 students have been participated in this study; most of them are
at the second year of computer engineering. After processing the results, 25% of students, who had
not filled the questionnaire completely, were detected as outliers and put aside.
5.1 Statistical reports
Our unified LS modeling system make it available to see the results in a comparable manner. Table 2
shows the average and variances of each questionnaire dimensions. All dimensions have averages
about 5-6 except FLSI Visual and MBTI Judgment dimensions which have a greater average. In
addition, the variances of all dimensions have values between 4.5 and 5.5 except MBTI Sensing
dimension, which has a smaller one, and MBTI Extroversion dimension, which has a greater one. It
shows that this group of students are either highly or lowly extrovert while they are less variant in their
sensing aspect of their characteristics.
Table 2: Statistical Reports of FLSI and MBTI Dimensions
FLSI
Dimension Active Sensing Visual Sequential
Average 5.71 5.54 6.82 5.14
Variance 5.47 5.29 5.19 4.42
MBTI
Dimension Extroversion Sensing Thinking Judgment
Average 5.11 5.78 5.63 7.15
Variance 8.56 3.79 4.63 4.59
5.2 Correlation analysis
Trying to check the consistency and validation of different modelers, we have designed and
implemented our unified LS modelers. We wanted to check whether there is any relationship between
modelers' dimensions. Table 3 shows the results of the experiments.
Table 3: Correlation between FLSI and MBTI dimensions
Pearson coefficients between FLSI dimensions
Active & Sensing Active & Visual Active & Sequential
Sensing &
Visual
Sensing &
Sequential
Visual &
sequential
0.16 0.14 0.08 -0.12 0.65 -0.06
Extroversion &
Sensing
Extroversion &
Thinking
Pearson coefficients between MBTI dimensions
Extroversion &
Judgment
Sensing &
Thinking
Sensing &
Judgment
Thinking &
Judgment
0.41 -0.49 0.1 -0.30 0.26 0
Sensing &
Sensing
Pearson coefficients between FLSI & MBTI dimensions
Active &
Extroversion
Active & Judgment
Visual &
Sensing
Sensing &
Extroversion
Sequential &
Thinking
0.3 0.55 -0.04 -0.57 0.17 -0.36
We expected that there would be no correlation between a modeler dimensions since each dimension
has a special psychological objective, and as a result, it is naturally expected to be independent from
617

Fatemeh Orooji et al.
the other dimensions. Data gathered in this study has violated this assumption somehow; especially it
has shown that MBTI dimensions are strongly correlated highlighted in the middle part of Table 3.
Furthermore, we expected that there would be some acceptable dependencies between modelers'
similar dimensions since they want to guess approximately the same objectives. Analyzing data has
revealed such positive correlations between FLSI and MBTI Sensing dimensions (0.3), and FLSI
Active & MBTI Extroversion dimensions (0.55). In addition there are negative correlations both
between FLSI Visual and MBTI Sensing dimensions (-0.57) and between FLSI Sequential and MBTI
Thinking dimensions (-0. 36).
5.3 Discussion
We have examined two famous modelers in order to check their inter-modelers and intra-modelers
correlations. Experiments have shown that unexpectedly, some MBTI dimensions are strongly
correlated while some FLSI and MBTI are not correlated as much as expected. These results shows
that there is some considerable issues in using just one modeler as a base of adaptation mechanisms
since there may be some inconsistency among its results as (Coffield et al., 2004) has emphasized. It
may be a good idea that some psychologists see the results and try to find some understandable
explanation. At least it has revealed that those LS modelers are not reliable as people expected,
needing to be modified or redesigned for a more reliable usage in investigating learners
characteristics.
Furthermore, since the area of study is limited to some specific students, it may be considered as a
hypothesis that the field of engineering might have lead these outcomes and if we expand the area of
research we may have some more general results.
6. Conclusion
There is a large number of learning styles (LS) introduced by psychologists and have been used in
various web-based educational systems. Some studies have emphasized on the benefits of matching
teaching strategies and learner LSs while some other researchers have explained their doubts about
the value of this kind of matching (Coffield et al., 2004). In addition, the various definitions of LS
concepts make it hard to understand and compare their reports against each other. Each model
incorporates its specific dimensions, and as a result, the generated reports will have specific
representation, which is a challenge for combining a full 360-degree view of a learner.
This paper proposed a new framework for modeling different learning styles attempting to synthesize
various LS inventories into a single representation of learners LSs. Proposed approach allows
administrators to define various LS modelers through its predefined constructing phases. Answering
available questionnaires, learner receives some comprehensive reports consist of his/her recognized
LSs due to different aspects of different models. Considering pedagogical instruction, system provides
some recommendations based on each learner detected LSs. These reports may help learners to
improve their meta-cognition skills and to plan some more efficient strategies for their studies. They
may be considered as guidelines for system designers and course instructors in order to be
implemented in a content adaptation mechanism. In addition, the proposed system has provided
some representation facilities to show the situation of each learner among the others, enabling
him/her to compare his/her detected LSs reported by different models and evaluate the results.
Utilizing the proposed system in an educational environment has revealed some dependencies
between a modeler dimensions as well as some inconsistencies among two modelers' dimensions.
Results make it more emphasized that recognition of a person’s learning styles really needs more
than one modeler. Since there are some serious comments about modelers validities and
consistencies (Coffield et al., 2004), integrating modelers' reports may decrease the effects of their
insufficient theoretical basis, providing learners a more reliable knowledge about their characteristics.
This study was limited to some popular learning style modelers that are more suitable in web-based
learning systems, neglecting the others at this phase. Since there is no comprehensive LS
questionnaire, learner is recommended to fill out different questionnaires, and as a result, this
constraint may lead to a decrease in his/her satisfaction. It seems this area of research really needs
additional studies made by educational psychologist to construct an inclusive questionnaire, capable
to detect all aspects of learner characteristics. In addition, to have more accurate and more reliable
results, it is necessary to demand a large number of learners to participate in the future steps of this
618

Fatemeh Orooji et al.
study. In this way, experiments outcomes will be considered in modeling learners’ preferences and
behaviors in an educational simulation and modeling tool prepared before (Orooji et al., 2010).
Future work will deal with incorporating information about relationships between learners’ learning
styles and their other characteristics like field of study. In this way, students’ evaluation results will be
put together to detect special LS modelers which fit better to each category of students. In this way,
identified LS modelers will be used in an automatic selection of the most effective questionnaire due
to each leaner. Future work will also deal with analyzing the concept for providing adaptivity in more
detail based on the dominant LS distinguished for each user. Unified LS modeling system provides
some valuable guides for course instructors and system designers to provide some more effective
content adaptations. Another direction of future work will be constructing a hierarchical concept map,
a combination of different parts of LS modelers able to cover all styles, to be considered as a basis for
integration mechanism. It really needs to understand the correlation between different LSs.
Acknowledgments
We thank Ms Fatemeh Noorani for her unpublished comprehensive lecture.
References
Bousbia, N., Rebaï, I. & Labat, J.-M. (2010) Analysing the Relationship between Learning Styles and Navigation
Behaviour in Web-Based Educational System Knowledge Management & E-Learning: An International
Journal, 2, 400.
Briggs Myers, I. (1962) The Myers-Briggs Type Indicator. Consulting Psychologists Press. Palo Alto, CA.
Brusilovsky, P. & Millán, E. (2007) User Models for Adaptive Hypermedia and Adaptive Educational Systems.
The Adaptive Web. Springer-Verlag Berlin Heidelberg.
Coffield, F., Moseley, D., Hall, E. & Ecclestone, K. (2004) Learning Styles and Pedagogy in post-16 learning: A
systematic and critical review. Learning & Skills research centre,
.
Popescu, E. Trigano, P. & Badica, C. (2007) Towards a Unified Learning Style Model in Adaptive Educational
Systems. International Conference on Advanced Learning Technologies, ICALT.
Felder, R. M. & Silverman, L. K. (1988) Learning and Teaching Styles in Engineering Education. Engineering
Education, 78 674-681.
Graf, S. (2007) Adaptivity in Learning Management Systems Focussing on Learning Styles. Faculty of
Informatics. Vienna, Vienna University of Technology.
Grasha, A. F. & Riechmann, S. W. (1975) Student Learning Styles Questionnaire. Cincinatti, OH, University of
Cincinatti, Faculty Resource Center,.
Hashim, H. E. D. K. (2003) Online learning style and e-learning approaches. Australasian Computing Education
Conference (ACE2003).
Kolb, D. (1984) Experiential learning: experience as the source of learning and development, New Jersey:
Prentice Hall.
Montgomery, S. M. & Groat, L. N. (1998) Student Learning Styles and Their Implications for Teaching. CRLT
Occasional Papers, The University of Michigan, No. 10,
.
Orooji, F., Sarjoughian, H. S. & Taghiyareh, F. (2010) Modeling & Simulation of Educational Multi-Agent Systems
in DEVS-Suite. International Symposium on Telecomunications with emphesis on Information &
Communication Technology (IST).
Popescu, E., Trigano, P. & Badica, C. (2007) Towards a Unified Learning Style Model in Adaptive Educational
Systems. International Conference on Advanced Learning Technologies (ICALT).
Reed, W. M., Oughton, J. M., Ayersman, D. J., Ervin, J. R. & Giessler, S. F. (2000) Computer experience,
learning style, and hypermedia navigation. Computers in Human Behavior, 16, 609-628.
Siadaty, M. & Taghiyareh, F. (2007) PALS2: Pedagogically Adaptive Learning System based on Learning Styles.
7th IEEE International Conference on Advanced Learning Technologies (ICALT).
619

Using Lifeworld-led Multimedia to Enhance Learning
Andy Pulman, Kathleen Galvin, Maggie Hutchings, Les Todres, Anne Quinney,
Caroline Ellis-Hill and Peter Atkins
School of Health and Social Care, Bournemouth University, UK
apulman@bournemouth.ac.uk
Abstract: In 2010, the School of Health and Social Care developed a collaborative lifeworld led transprofessional
curriculum for health and social work disciplines harnessing technology to connect learners to a wider view of
evidence based practice. The purpose was to increase use of technology-enhanced learning, introduce lifeworldled
philosophy to the curriculum, release staff potential, and expose students to research undertaken within the
School. Delivered to over 600 undergraduate students from community development, midwifery, nursing fields,
occupational therapy, paramedic science, physiotherapy and social work, the Exploring Evidence to Guide
Practice Unit was facilitated by a number of resources including lectures, group work and a variety of web-based
learning materials. Central to the unit were seventeen web-based case studies which included the human
experience of the impact of specific illnesses (such as stroke and living with dementia) and more general
experiences (such as social isolation and homelessness). Each case study provided stories and poems,
qualitative and quantitative research and policy and practice issues related to specific topics. At the heart of the
philosophy underpinning the case studies and unit was an opportunity for students to integrate understandings
about different kinds of knowledge for practice, conventional evidence, understandings about the person’s or
service user’s experience and the student’s own insights that came from imagining ‘what it was like’ for the
person experiencing a condition or situation and encountering human services (Galvin and Todres 2011). The
project built on the successful development of Wessex Bay, a virtual community of case scenarios, used as
problem-based triggers to engage students in learning activities relating to the residents (Pulman, Scammell and
Martin 2009). This paper discusses the development of the web based case studies and how they integrated
visual and audio materials with the aim of enhancing the lifeworld experience of students.
Keywords: Lifeworld, technology, transprofessional, web, humanising care, healthcare
1. The Lifeworld
The rise of technology could perhaps be seen as a distraction from compassion in the caring
professions (Todres et al. 2009). However, technology can also be harnessed for educational
development to support compassion in practice. In their article on the concept of Lifeworld, Todres,
Galvin and Dahlberg (2007) listed ideas about the values and philosophy of life world-led care which
could translate into practice. These included phenomenological and narrative studies which deepened
insights into Lifeworld phenomena such as palliative care (Seymour and Clark 1998), myocardial
infarction (Johansson, Ekebergh and Dahlberg 2003) and non-caring encounters in an emergency
unit (Nyström, Dahlberg and Carlsson 2003). They also noted ways of disseminating qualitative
research findings to make them valuable to users and help to deepen professionals’ Lifeworld
understanding (Ziebland 2004). Todres et al. (2007) also highlighted the central foundations of life
world-led care: its humanising value, the holistic contextuality of Lifeworld experience, and its benefits
of experiential credibility and citizen empowerment.
2. Integrating Lifeworld with technology - caring for a partner with Alzheimer’s
In 2006, staff from the School of Health and Social Care (HSC) at Bournemouth University were
investigating ways to make the Wessex Bay Simulated Community more innovative and interactive
(Pulman, Scammell and Martin 2009). One approach concerned integrating the methods and themes
of Lifeworld with Hypertext narrative into an interactive educational resource concerning the
experience of Alzheimer’s. The project began as a text-to-screen adaptation of Todres and Galvin’s
2006 article, which aimed to contribute to a deeper understanding of caring for a partner with
advancing memory loss with particular concern for communicating findings in evocative and empathic
ways. In their article Todres and Galvin (2006) suggested that by engaging with descriptions and
interpretations offered, carers, professionals and family could be better equipped to understand the
issues discussed. The initial project idea was centred on an interactive hypertext narrative utilising the
transposition model of adaptation (Pulman, Galvin and Todres 2010) aiming to generate a deeper
understanding of six related phenomena within an intimate carer’s journey. The carer - Mervyn
Richardson - who featured in the article (Todres and Galvin 2006) was approached about participating
in the project and agreed to work in conjunction with HSC. He also gave permission to use personal
photos, diary extracts and video clips from the award winning A Sweeter Pill to Swallow: Beryl’s Story
(Richardson 2002).
620

Andy Pulman et al.
After producing two prototypes, a third iteration - The Carers World - was created (Pulman, Galvin
and Todres 2010) - designed to give users greater knowledge about, and insight into, the challenges
of caring for people with Alzheimer’s. The package was based on the idea that learning to become a
competent professional required not only knowledge about the condition and its treatment, but also
what it was like to be with a person who was suffering from the condition in their everyday lives. An
important access to this understanding might come from the experience of family carers who lived
with them throughout the whole journey (Galvin, Todres and Richardson 2005). The aim of the
resource was to offer users opportunities to engage not only with technical knowledge about the
condition, but more importantly with understandings of the world of the carer. Imagining more deeply
what the world of the carer might be like might provide directions for enhancing person-centred
practice which required situated judgement and ethical sensitivity. This capacity is important when
applying evidence to unique situations that require imagination and sensitivity. Work on The Carer’s
World was completed in Spring 2008.
3. Integrating Lifeworld with technology - exploring evidence to guide practice
Pulman, Galvin and Todres (2010) had noted that there was exciting potential in working with
healthcare professionals in their education using new media to focus on the experiences of people in
ways that could help them understand and learn from issues and personal views, becoming more
effective and empathetic in supporting people. The use of interactive media to convey Lifeworld
stories around diseases and their effects also offered a uniquely potent way of harnessing public
awareness and engaging policy makers, practitioners, students and carers in health and social care
issues. With the potential to work with the public and health and social care professionals in exploring
society’s big issues in interesting new ways, it was hoped that other innovative resources based on
the lifeworld framework could be created and utilised within the HSC undergraduate and postgraduate
curriculums. During 2010, HSC began development of a collaborative lifeworld led transprofessional
curriculum for health and social work disciplines through harnessing technology to connect learners to
a wider view of evidence based practice. The purpose was to increase use of technology-enhanced
learning, introduce lifeworld-led philosophy to the curriculum, release staff potential, and expose
students to research undertaken within HSC. The project was supported by the Higher Education
Academy (HEA) Discipline-focused Learning Technology Enhancement Academy through the HEA
Subject Centre for Social Policy and Social Work (SWAP). The aim was to capitalise on the School’s
expertise and scholarship in utilising a range of evidence around lifeworld service user and carer
stories (demonstrated in the creation of The Carer’s World) in the form of clips from television and
radio programmes, films, podcasts, poetry, drama and narrative case studies, and associated
evidence from journal articles and policy documents, to immerse health and social care students in
the lifeworld of the people they might encounter in their future professional roles.
Personal
resources
Life
resources
Figure 1: The unit philosophy
The experience of the person
First person accounts
621
Technical
knowledge
Research
evidence

Andy Pulman et al.
A key feature of the project was to raise student awareness of a range of evidence, including
narratives and material from the arts and humanities to consider how practice could be guided. In this
way HSC wished to introduce students to evidence in a situated way, embedded in practice and to
make transprofessional learning explicit. At the heart of the philosophy underpinning the educational
resource (Figure 1) was an opportunity for students to integrate understandings about different kinds
of knowledge for practice, conventional evidence, understandings about the person’s or service user’s
experience and the student’s personal insights that come from imagining ‘what it was like’ for the
person experiencing a condition or situation and encountering human services (Todres 2008, Galvin
2010, Galvin and Todres 2011).
The project also built on the successful development of Wessex Bay, a virtual community of case
scenarios representing different service user and carer perspectives, used within the previous
interprofessional education (IPE) curriculum (Pulman, Scammell and Martin 2009) as problem-based
triggers to engage students in learning activities relating to the residents (Scammell, Hutchings and
Quinney 2008, Quinney, Hutchings and Scammell 2008, Hutchings, Quinney and Scammell 2010).
This resource comprised a virtual seaside town containing homes and community facilities and
populated by people accessing health and social care services. Evolving case scenarios linked to
curricular outcomes were used to engage students in blended learning activities relating to residents.
Working in interprofessional groups, students developed client-centred assessment and intervention
strategies considering many issues such as professional roles and multi-agency working which
increased not only their understanding of services available but also the realities of collaborative
working.
The key beneficiaries of this most recent initiative would be HSC students, with the focus on creating
an undergraduate intermediate (Level I, Year 2), transprofessional unit to be delivered to over 600
students in consecutive groups of 300 from nursing fields, midwifery, social work and community
development, occupational therapy, physiotherapy, operating department practice, and paramedic
practice. The project implementation impacted directly on working practices within HSC and the wider
University with key stakeholders including academics, programme leaders, the School’s management
team, and professional staff in advisory and support roles within the School and centrally, including
web development, educational and learning technology, academic staff development, quality and
enhancement and examination co-ordination. The scale of transformational change was considerable
given that all professional groups in the School were participating, with forty staff taking part as
academic developers, champions and facilitators.
In the first quarter of 2011 the unit went live, facilitated by a number of resources including lectures,
group work and a variety of web-based learning materials. Over five weeks groups of between five
and nine students worked together on one of seventeen different case studies initially exploring
evidence from poems, stories and qualitative research (what might be expressed in a metaphorical
way as ‘knowledge for the Heart’); then exploring quantitative research and policy (‘knowledge for the
Head’) and finally considering how the evidence could be integrated with their other studies and own
experience to inform their practice (‘knowledge for the Hand’). This was supported by podcasts
describing research terminology and student managed guided learning activities. Students wrote
weekly individual blogs and a final group blog which was assessed (50%) and also undertook an
online multiple choice examination (50%) during the final week of the unit. By building a lifeworld
framework which informed and incorporated rational ‘head’ knowledge it was felt students were more
likely to develop the confidence to draw on evidence not only from research and policy documents,
but also to value the stories of service users and their own human experiences to create the
judgment-based care (Polkinghorne 2004) needed in complex caring situations.
4. Case study development
Central to the overall philosophy of the unit was the creation of seventeen different web-based case
studies (Figure 2) which included the human experience of the impact of specific illnesses (such as
stroke and living with dementia) and more general experiences (such as social isolation and
homelessness). A wide variety of cases was required to cope with the large cross-section of health
and social care students taking the unit.
From the beginning of 2010, academic champions for each case study were identified and invited to
complete a pro-forma with initial ideas. They were prompted to look at The Carer’s World as this might
give them some useful ideas on how they might want their case to be set out. Academics were also
622

Andy Pulman et al.
advised that at the beginning of the creation process, the information supplied and the layout or
design was not finalised, so that they could build their case studies iteratively before the actual live
date. They were also requested to include a narrative from the person's perspective which could be
written, or be from a video clip from Healthtalkonline (2011) or other similar source.
Figure 2: Partial case list
The clips could be supplemented by the use of short extracts from films which had been on television
or from television or radio documentaries that were about the wider issues in the case. Poetry or
extracts from literature or diaries or images could be used, with the only limit being the imagination.
The aim was to build up knowledge from many sources around the central narrative. The sources also
needed to include three or more academic articles with a variety of quantitative approaches and three
or more academic articles using qualitative approaches to highlight peoples' experiences and
perspectives, the film clips and poetry as well as links to any interesting policy documents and
relevant websites (such as associated charities).
HyperText Mark up Language (HTML) - a language used to construct, arrange and present text and
graphics through a user’s web browser - was used for the creation of each case study. Internal audio
visual elements were supported by using Apple’s Quicktime technology for video clips and the MP3
audio file format - based on MPEG technology which creates a very small file suitable for streaming or
downloading over the Internet - for audio clips and podcasts. Once all cases had been drafted, the
central project team were able to look at each one in turn to ensure that the student experience would
be equitable whichever case was studied. Individual sections were continuously revised and redrafted
before the live date to ensure there was an acceptable level of standardised content across each one.
5. Case Study 16: The experience of the impact of Dementia
Looking in more detail at Case Study 16 - The experience of the impact of Dementia (Figure 3), the
paper now discusses the individual case components.
The Homepage of each case served two purposes: aiming to guide the user simply and effectively to
the various learning tools and also highlighting a current campaign about the topic under discussion –
623

Andy Pulman et al.
in this case “Putting Care Right” (Alzheimer's Society 2011), a campaign to ensure that people with
dementia have access to good quality care whether at home, in a care home or on a hospital ward.
Figure 3: Case 16: Part of the “Imagine This” section
The Getting Started section set the scene for the students, describing what they would have the
opportunity to experience:
Learning about a range of issues in dementia.
Watching and listening to video clips of people talking about their experience.
Imagining what it might be like caring for someone they loved who had dementia and also
imagining what it might be like for the person with dementia through a range of immersive
materials.
Imagining what dementia might feel like by reading Haiku poetry written by people with dementia.
This section also discussed the other parts of the case study and most importantly reminded students
of the three qualitative and three quantitative papers that they would need to read to support their
experience and understanding. The Background section contained introductory information on the
condition or situation - in this case a definition of dementia and some suggested additional reading
which included journal articles and fact sheets from the Alzheimer’s Society. For each case, these
might be demographic papers, research pointing to the extent of the situation or condition and the
difficulties it presented, or scientific, biomedical information about the case/condition, where relevant.
The focal point of each case study was the Imagine This section where as well as resources, there
were also a number of scenarios that aimed to give students some insight and information on the
experience of the impact of Dementia. For the Dementia Case Study, there were four different
scenarios and a learning activity involved choosing one particular scenario to investigate in more
depth. The first scenario contained a character from Wessex Bay (Pulman, Scammell and Martin
2009) and was an extension of the approach tested in the previous IPE curriculum. This character,
Duncan Galloway, is a 62 year old man who has dementia and arthritis. Ellen, his wife of 30 years, is
caring for Duncan. Their daughter lives close by but is busy with her four children. A situation is
624

Andy Pulman et al.
described in detail and the students are asked to consider what can be done at a particular moment in
time.
The second scenario featured the first person to be diagnosed with Alzheimer’s - Frau Auguste Deter,
a 51-year-old woman who was admitted to hospital and examined by Dr. Alois Alzheimer. Because of
her age, Deter was diagnosed with presenile dementia; today, the diagnosis would be early-onset
Alzheimer’s Disease, which is defined as development of the condition before the age of 65. Deter
died in April 1906, aged 55. Psychiatrists rediscovered Deter’s medical records in 1995, in the
archives at the University of Frankfurt. The 32-page file contained her admission report, and three
different case histories, including notes written by Alzheimer himself. An extract from the file, written
by Alzheimer in 1901 is highlighted and the students are asked to imagine themselves back in that
time and consider their reflections on his notes.
The third scenario concentrated on five Visions of Dementia portrayed through audio visual materials
which could be experienced by the students:
Malcolm and Barbara - A Love Story (Watson 1999) won critical acclaim for the moving account
of Malcolm and Barbara Pointon who had their lives radically altered when Malcolm developed
Alzheimer's. Paul Watson's film told the story of how love sustained a relationship in even the
most difficult of times. After spending eleven years with the couple, Malcolm and Barbara - Love’s
Farewell (Watson 2007) followed Malcolm as Alzheimer’s took control of his body, mind and
marriage. This second film followed Malcolm just after he was diagnosed with the disease aged
51 through to his final conscious moments.
Being Together (Cash 2009) is a film produced by Marilyn Cash who has worked with older
people in both the voluntary and statutory sectors; in social care and as a researcher. Her
research explored how the findings of qualitative research could be utilised to improve the quality
of life for people with dementia and their carers. Being Together was made as part of Marilyn's
PhD from Bournemouth University and related to the experience of living with dementia from both
partners’ perspectives in a relationship.
A Sweeter Pill To Swallow - Beryl's Story (Richardson 2002) is a documentary which tells the
personal story of Mervyn Richardson's fight to get a prescription of Galantamine for his wife Beryl
who was diagnosed with Alzheimer's disease. An exposé of the immense difficulties faced by
Alzheimer's disease sufferers and their carers in obtaining anti-dementia medication, the film
highlighted the first patient in Dorset to receive an NHS prescription for Galantamine, a drug that
Mervyn would like to encourage.
In 2003, the director Josh Appignanesi wanted to make a short film based on his experiences of
visiting his grandmother - a Jewish refugee - in a care home when she had dementia. During the
war, she had lost touch with her brother and his fate was never discovered, which became a key
theme in her experience of dementia. The resulting film - Ex Memoria (Appignanesi 2006) - takes
the viewer through a day in the life of Eva, putting them face to face with her experience of old
age and dementia. It has been distributed to careworkers, residential home managers, charities
and specialists in the field, as part of a Wellcome Trust scheme.
The Lion’s Face (2011) is a tale of love, loss and family with music by Elena Langer and words by
Glyn Maxwell. When a man loses his way home it signals an irreversible return to childhood.
Compassionate, heartfelt, strikingly dramatic and often witty, the work is an original, richlytextured
story on ageing, memory and the incomprehension of getting old in the minds of the
young. Working in partnership with Professor Simon Lovestone’s team at the Institute of
Psychiatry, Kings College London, The Opera Group developed the opera over the course of two
years to find a way of creatively exploring the experiences of the person with Alzheimer’s, their
carers and a research scientist. A BBC Wales news report on the drama and a link to the
company’s website offer a different perspective rather than featuring the actual opera.
The fourth scenario offered a viewpoint from the world of poetry. Haiku poetry celebrates the beauty
of the ordinary moment. Derived from the Japanese tradition, it is characterised not only by simplicity,
but also by reverence for nature. Philomene Kocher began writing haiku in 1991 and her work has
been published internationally, most recently in a Canadian haiku anthology (Kocher 2008). In 2008,
she completed her Master of Education studies in which she explored haiku as a way of connection
with persons with dementia. Her haiku can be read by students in the September 2007 archives of the
Daily Haiku website.
625

Andy Pulman et al.
The Research Evidence section of the case study provided deep library links to the three qualitative
(evidence that gives a rich perspective of the condition or situation) and three quantative (findings
about the condition or situation which are technical evidence) papers which each student was
required to access to compare, explore and learn from a range of published research evidence
embedded within a practice issue relevant to their field and consider how these research studies
could usefully guide practice.
From the Research Process and Methods section, students could access a range of supplementary
resources and material to facilitate their learning about research approaches and methods. There
were links to the Thinking about Research guide - a web based resource which provided a brief
overview of the research process and acted as a reference document to introduce students to core
terms and processes. There were also links to 45 short podcasts which were designed for students to
listen to in conjunction with other unit materials. The podcasts featured Professors discussing and
explaining a variety of key research approaches, methods, and aspects of the papers that students
would be accessing in their case study. Students were also able to access a range of Policy
Documents which were relevant to their case and additional interesting information in Useful Links. In
the Dementia case study this included links to show students what charities and national and
international agencies were currently focusing on in relation to dementia, such as The Alzheimer's
Society and the Social Care Institute for Excellence’s Dementia Gateway.
6. The impact of integrating Lifeworld with technology
Work is currently being undertaken to fully evaluate the student experience of the various aspects of
the Exploring Evidence to Guide Practice unit. This will be disseminated in more detail by members of
the project team later in the year. However, early indications from some of the group blog postings
undertaken as part of the unit highlighted the powerful effect that the case studies had on student
perceptions:
One student felt that although it was important to learn the facts of dementia, to understand the
illness through the eyes of someone with the condition was an important opportunity as
understanding patient’s fears and how their life changed and how they coped with those changes
were very important to the way that individual health professionals cared.
Another student had not personally worked with someone with dementia or looked closely into the
condition itself and was shocked at how little they actually knew. They had expected the primary
caregiver to experience feelings of anger, stress, upset and confusion, but reflected that to watch
the person with condition go from an active, happy, loving person to an aggressive person that
could do very little for themselves in a short space of time as portrayed in Malcolm and Barbara -
Love’s Farewell (Watson 2007) was shocking and hard hitting
The film Ex Memoria (Appignanesi 2006) affected another student concerning the sort of care that
some people received, and how important it was to think about the person behind the disease.
They found that many of the resources had reminded them of a particular person they’d met or
looked after in the past, and that this had affected them more than some of the other narratives
had.
When one student attempted to imagine what it was like they felt quite upset because even after
years of trying to understand the disease they acknowledged that it was still forgotten that,
regardless of the condition, people with dementia were first and foremost individuals. They
reflected that this feeling would guide them in future practice to always see the person behind the
condition.
Murray (1997) felt that the right stories could open hearts and change who we were. By exploring
narratives and qualitative research (to obtain ‘knowledge for the Heart’); exploring quantitative
research and policy (‘knowledge for the Head’) and then reflecting on how that evidence could be
integrated with other studies and their own experience to inform practice (‘knowledge for the Hand’) it
is hoped that this unique lifeworld approach will continue to assist and benefit health and social care
professionals of all disciplines in their holistic understanding and further help them to guide and
improve theirs and others future practice.
References
Alzheimer's Society. (2011) “Putting care right”, [online], Alzheimer's Society,
www.alzheimers.org.uk/site/scripts/home_info.php?homepageID=58
Appignanesi, L. (2006) Ex Memoria, Film.
626

Andy Pulman et al.
Cash, M. (2009) Being Together, Film.
Galvin, K. (2010) “Revisiting caring science: some integrative ideas for the ‘head, hand and heart’ of critical care
nursing practice”, Nursing in Critical Care, Vol 15, No. 4, pp 168-175.
Galvin, K and Todres, L. (2011) “Research based empathic knowledge for nursing: a translational strategy for
disseminating phenomenological research findings to provide evidence for caring”, International Journal of
Nursing Studies, Vol 48, No. 4, pp 522-530.
Galvin, K., Todres, L. and Richardson, M. (2005) “The intimate mediator: a carer's experience of Alzheimer's”,
Scandinavian Journal of Caring Sciences, Vol 19, pp 2-11.
Healthtalkonline. (2011) “People's stories: see, hear and read their experiences”, [online], Healthtalkonline,
www.healthtalkonline.org/
Hutchings, M., Quinney, A. and Scammell, J. (2010) The utility of disruptive technologies in interprofessional
education: negotiating the substance and spaces of blended learning. In Bromage, A. et al (eds.)
Interprofessional eLearning and Collaborative Work: Practices and Technologies, Hershey, PA: IGI.
Johansson, A., Ekebergh, M. and Dahlberg, K. (2003) “Living with experiences following a myocardial infarction”,
European Journal of Cardiovascular Nursing, Vol 2, No. 3, pp 229-236.
Kocher, P. (2008) Carpe Diem - Anthologie Canadienne Du Haïkus, Les Éditions David and Borealis Press.
The Lion’s Face. (2011). “The Lion's Face - Music Meets Science in a New Opera”, [online], The Opera Group,
thelionsface.wordpress.com/
Murray, J. H. (1997) Hamlet on the Holodeck (2nd print), Cambridge MA, MIT Press.
Nyström, M., Dahlberg, K. and Carlsson, G. (2003) “Non-caring encounters at an emergency unit – a lifeworld
hermeneutic analysis of an efficiency-driven organization”, International Journal of Nursing Studies, Vol 40,
pp 761-769.
Polkinghorne, D. (2004) Practice and the Human Sciences: The Case for a Judgment-based Practice of Care,
Albany, N.Y. State University of New York Press.
Pulman, A., Galvin, K. and Todres, L. (2010) "The Carers World - an interactive reusable learning object",
Dementia, Vol 9, No. 4, pp 535-547.
Pulman, A., Scammell, J. and Martin, M. (2009) “Enabling interprofessional education: the role of technology to
enhance learning”, Nurse Education Today, Vol 29, No. 2, pp 232-239.
Quinney, A., Hutchings, M., and Scammell, J. (2008) “Student and staff experiences of using a virtual community,
Wessex Bay, to support interprofessional learning: messages for collaborative practice”, Social Work
Education, Vol 27, No. 6, pp 658-664.
Richardson, M. (2002) A Sweeter Pill To Swallow - Beryl's Story, Film.
Scammell, J., Hutchings, M. and Quinney, A. (2008) “A virtual practice community for student learning and staff
development in health and social work inter-professional education; changing practice through
collaboration”, Project Report. Higher Education Academy and Bournemouth University,
www.health.heacademy.ac.uk/publications/miniproject/scammell08.pdf
Seymour, J. and Clark, D. (1998) “Phenomenological approaches to palliative care research”, Palliative Medicine,
Vol 12, pp 127-131.
Todres, L. (2008) “Being with that; the relevance of embodied understanding for practice”, Qualitative Health
Research, Vol 18, No. 11, pp 1566-1573.
Todres, L. and Galvin, K. (2006) “Caring for a partner with Alzheimer’s disease: Intimacy, loss and the life that is
possible”, International Journal of Qualitative Studies on Health and Well-being, Vol 1, pp 50-61.
Todres, L. and Galvin, K. (2008) “Embodied Interpretation: A novel way of evocatively re-presenting meanings in
phenomenological research”, Qualitative Research, Vol 8, No. 5, pp 568-583.
Todres, L., Galvin, K. and Dahlberg, K. (2007) “Lifeworld-led healthcare: revisiting a humanising philosophy that
integrates emerging trends”, Medicine, Health Care and Philosophy, Vol 10, No. 1, pp 53-63.
Todres, L., Galvin, K. T. and Holloway, I. (2009) “The Humanization of Healthcare: A Value Framework for
Qualitative Research”, International Journal of Qualitative Studies on Health and Well-being, Vol 4, No. 2,
pp 68-77.
Watson, P. (1999) Malcolm and Barbara - a Love Story, ITV, 24 June 1999. Television Documentary.
Watson, P. (2007) Malcolm and Barbara - Love’s Farewell, ITV 1, 8 August 2007. Television Documentary.
Ziebland, S. (2004) “The importance of being expert: the quest for cancer information on the internet”, Social
Science and Medicine, Vol 59, No. 9, pp 1783-93.
627

The Project Mobile Game Based Learning
Thomas Putz
evolaris next level GmbH, Austria
thomas.putz@evoalris.net
Abstract: The specific aim of the project “mobile game-based learning” - supported by the EU within the 6th
framework programme - was to design, develop and pilot a prototype game platform that might be used to
efficiently develop games for m-learning. The basic idea is to use the mobile phone to implement games bridging
the real and virtual world. These games are firstly intended to directly support learning via opportunities to
develop knowledge and cognitive skills in an exciting and inspiring – and hence in a highly emotional – way, and
secondly to indirectly motivate users to refer to other media for learning purposes. In general, all students liked
the games and also the mGBL platform. All indicators show that students like to use the games in a real tertiary
education environment. Some of the students requested usage of the resource in other university courses. Many
of them pointed out the efficiency, flexibility and ease of use of the platform. The new experience, fun and
playability of the games gave them additional motivation. More than half of the students stressed that they
learned more by playing the games, paid more attention while playing the games, and were more engaged when
using the “learning by playing” method.
Keywords: eLearning, mLearning, serious games, game based learning, mobile phones
1. The project “mobile Game Based Learning”
The project mobile Game Based Learning (mGBL) coordinated by evolaris next level GmbH
Has been implemented from October 2005 until December 2008,
Has been conducted by 30 researchers from 11 project partners from 5 European countries
(Great Britain, Italy, Croatia, Austria and Slovenia),
Used nearly 600 person-moths resources with a budget of 2.5 Mio EUR,
Was supported by the EU under the FP6 IST.
The overall goal of the project was to improve the effectiveness and efficiency of learning in the target
group of young people (aged 16 – 24) through the development of innovative learning models based
on mobile games. The biggest challenge in this project was to communicate content from different
fields in a motivational, inclusive and emotional way. As the most personal and emotional
communication channel the mobile phone was used to establish the link between learners and
teachers.
The specific aim of the project was to design, develop and pilot a prototype game platform that might
be used to efficiently develop games for m-learning. The basic idea is to use the mobile phone to
implement games bridging the real and virtual world. These games are firstly intended to directly
support learning via opportunities to develop knowledge and cognitive skills in an exciting and
inspiring – and hence in a highly emotional – way, and secondly to indirectly motivate users to refer to
other media (e.g. “classic” libraries, scripts, etc.) for learning purposes.
2. The mGBL platform
The mGBL platform is the core result of the mGBL project. This software platform enables the cost
effective development of mobile learning games, the planning, the deployment, the management, the
reporting as well as the control of those games for m-learning.
The mGBL platform is a comprehensive integration system for game-based learning. It supports the
full cycle of mobile game-based learning from the selection of games and game styles to game
authoring with the definition of learning content right through to the practical administration of game
play and its results. Individual mobile game software is replaced by an integrative approach to mobile
learning. Different server- and client-side state-of-the-art technologies are utilised according to the
needs of different game concepts and user preferences. This has been shown in practice to be very
important for acceptance and learning success. The administrative web user interface with its unique
built-in interaction modelling system allows users with limited technical expertise to manage games
and game players.
628

PC
Client
Game
Authoring
Game
Style
Selection
Interaction
Processing
Reporting
Logging
O/R Mapping
Database
Download Tool
Web Browser
Figure 2: Architecture of the mGBL platform
Thomas Putz
Web Interface
Session Management
Security
UI Libraries
Games
User Management
Game Management
SMS, MMS Mobile
Device
WAP Browser SMS / MMS
Asynchronous
Communication
Sender
Receiver
Message Queue
Dispatcher
Mobile Deployment
In addition to these fairly conventional features, the platform facilitates flexibility and communication.
The rich diversity of mobile game types requires high configurability and connectivity. Therefore, the
platform has been designed to be not just a monolithic application providing a fixed set of features,
but to be modular and, to a high degree, extendable. An innovative game modelling system is
available in the platform administrative user interface for handling incoming and outgoing SMSs,
MMSs and e-mail messages and for responding in ways that suit the needs of diverse game types,
including all-pervasive games. Web interfaces for interactive message sending and blogging are
provided in order to further support communication between teachers and game participants. Most
parts of the software modules are available as open source code with the EUPL 1.0 license.
3. The mGBL project delivered 3 different mobile game templates
3.1 Hybrid quiz simulation game template: “Ahead of the Game”
Figure 3: Screenshot of “Ahead of the Game”: Fastest First – quiz component
629
J2EE Server

Thomas Putz
A hybrid game, consisting of 2 components
First, knowledge is tested
Then there are opportunities to deal with challenges in simulated crisis situation
An aspect of the learning games is the stimulation of learning through activity, Players are forced to
make their decisions quickly in Fastest First!, because only the fastest have a chance to reach the
next level in Crisis!, because the situation rapidly deteriorates without appropriate interventions
Learning goals are decided by the game author:
Content goals: e.g. Facts (after Prensky, 2001)
Activity goals: e.g. Questions (after Prensky, 2001)
Process goals: e.g. Remember (after Anderson and Krathwohl, 2001)
Figure 4: Screenshot of “Ahead of the Game: Crisis!”
In general all the professors liked Game 1 (Ahead of the Game) and also the mGBL-platform and the
Game Authoring Tool. All professors would like to test the game for their learning purposes. All of
them have the opinion that Game 1 could bring an added value for their learning purposes. The
professors are not sure whether the games will be edited and authored by themselves or if they will
need support by technical persons. All professors would like to use the game for their learning
purposes. All of them have the opinion that games could provide an added value for their learning
purposes.
All indicators show that student like to use the game in real high education environment. Some of the
students asked for possible usage in other courses at the university. Using the game for final course
exam was very well accepted by the students. Many of them pointed out the efficiency, flexibility and
630

Thomas Putz
easiness of use of the platform. New experience, fun and playability of the game gave them additional
experience. More than half of the students stressed that they have learned more by playing the game,
paid more attention while playing the game, and that they are more engaged when using “learning by
playing” method.
Table 1: Results from 137 students and 13 teachers
Game Characteristics Students Teachers
user friendliness 32% sufficient 90% sufficient, 40% of which excellent
usefulness of information 26% good, 23% sufficient 30% excellent, 30% good, 20% sufficient
appeal 25% sufficient 20% excellent, 40% good
interface 21% good, 24% sufficient 60% good, 20% sufficient, 20% poor
clearness of objectives 23% good, 22% sufficient 30% excellent, 30% good, 10% sufficient
enjoyment 24% sufficient, 19% poor 25% excellent, 40% good, 30% sufficient
3.2 Board game template: “Mogabal”
An adventure game version that enables players to move around the board freely and interact with
objects seen around. Events are triggered when a player falls on a board cell or when s/he ‘touches’
an object on the map. The player has two main aims: discovering about different work contexts, by
engaging in game events enhancing their characteristics profile, through making informed choices,
astute choices, making alliances or by pure good luck!
Learning goals are decided by the game author.
Content goals: e.g. Facts, Judgement, Theories, Systems (after Prensky, 2001)
Activity goals: e.g. Experimentation (after Prensky, 2001).
Process goals: e.g. Evaluating, Creating (after Anderson and Krathwohl, 2001).
MOGABAL can be seen as a type of role-play game.
Figure 5: Screenshot of “Mogabal”
631

3.3 The different ‘events’ supported:
Thomas Putz
Quiz: a text and one or more options to choose from
Decision Tree: similar to Quiz, however various choices have no immediate reward but link to a
subsequent event (which can be any event type). This allows construction of complex simulations of
chains of choices or decisions
Conditional decision tree: similar to Decision Tree, but some of the possible choices are available
and visible to the player only under particular conditions
Simple: text message that can be used as a ‘leaf’ of a decision tree or as a simple random event
Multimedia: opens a multimedia resource then links to a subsequent event. Can be used to enhance
the graphic aspect of the events or to insert audio/visual elements in decision trees
Set internal variables value: The game holds an internal array. This can be used as a sort of ‘state
machine’ for complex event correlations
‘Case of’ tree: event structured just like the CASE instruction in programming languages, such as
SQL. Different events are activated according to current value of one of the internal variables
Null event: game contents logic may require an ‘empty’ event
Game Over: event overriding the normal ‘game-over’ rules
A trial was held a whole semester usage of the mGBL game Mogabal during the course “Logistics“ at
the Faculty of Maritime studies, University of Rijeka. This Game has been used to test the knowledge
of students during the semester, and also, for the first time ever, mobile game was used for the final
course exam.
3.4 Pervasive game template: “Get Real!”
Teams of learners engage and re-engage with a real world critical situation
Planning and undertaking learning activity in competition with other teams
Afterwards they reflect on the processes they were engaged in
Learning goals are decided collaboratively by the teacher and their group of students
Content goals: e.g. Making choices (after Prensky, 2001).
Activity goals: e.g. Skills, Judgement; Process, Procedures; Observation (after Prensky, 2001).
Process goals: e.g. Analyse, Evaluate, Create (after Anderson and Krathwohl, 2001).
The phases of the competition are linked to Kolb’s (1984) learning cycle (simplified terminology: Race,
1994):
‘WANTING’: Planning a game to fit their own learning agenda.
‘DOING’: Engaging with a real world critical situation; investigating and proposing solutions.
‘FEEDBACK’: Using system feedback and peer feedback.
‘DIGESTING’: Reflecting in and on action (double loop learning, Argyris and Schön, 1976), i.e.
reflecting
The mGBL prototype ‘Get real!’ has been developed as a ‘pervasive’ game, i.e. it is a multi-user
mobile game that uses mobile technologies to bridge virtual world with real world activities (c.f.
Benford et al., 2005).
It is a true game:
632

Thomas Putz
The player struggles to meet challenges in an uncertain situation (c.f. Fabricatore, 2000, Prensky,
2001, Salen and Zimmerman, 2004), using the phone to co-operate and collaborate with others in
identifying critical issues and proposing solutions (c.f. Senge, 1990, Small, 2000).
The activity takes place in a context directly relevant to their course of study (c.f. Knowles, 1990).
Feedback takes various forms (c.f. Race,1994, Prensky, 2001): system score, based on time
taken, and teacher feedback, geared to generic learning objectives (Anderson and Krathwohl,
2001) and any specific intended learning outcomes that the teacher may negotiate with the
students. There is also the option of supporting peer feedback in an in-game blog discussion.
Figure 6: Gameplay “Get Real”
Figure 7: Inquiry based learning
633

4. Pedagogical impacts
Thomas Putz
It was a main goal of the mGBL project to develop methods and tools that enable learning in a playful
and emotional way. To this end, WP3 developed innovative learning models, which have
subsequently been implemented prototypically as game templates. These game templates have been
evaluated from both the learners’ and teachers’ perspective and proved to fully support the overall
goal. For example, the results of a empirical study with approx. 100 students for game template three
(Get Real!) reveal that it leads to higher energetic activation, more positive emotions, more positive
attitudes towards learning content, and more efficient knowledge transfer than a conventional case
study approach. For the first time ever a mobile learning game has even been used to hold exams at
the University of Rijeka – with very positive reactions from all side. This also exemplifies the dramatic
impact the models and technologies developed in the course of the mGBL project can have in all
phases of teaching and learning.
5. Impacts of innovative new services
The mGBL project has proven that it is not always necessary to deploy the most advanced 3D
graphics and cutting-edge handsets in order to ensure a good user experience. In contrast, a practical
"low tech – high involvement" approach is in many cases much more suitable in the learning context,
especially when considering younger people in formal education. It is of utmost importance that the
systems can be used by all pupils and students and that it does not require special devices, which
may not be available or affordable for some target groups. The mGBL project thus developed a
platform which fully supports wide-spread standards and makes minimal demands regarding device
capabilities on the client side (Java, http, SMS/MMS). The platform exhibits maximum flexibility
regarding contents and can easily be adapted to individual needs without very specific know-how. It is
available as Open Source under the European Public License (EUPL) model at SourceForge and its
modularity and ease of use should give rise to a considerably large community both contributing new
functionalities and contents and at the same time improving the software quality. As could already be
seen from first exploitation activities (e.g. the trials at the University of Vienna, or an exploitation
workshop with mobilkom Austria representatives), hosting the platform as an application provider and
offering additional services (such as user acceptance research or community building services) are
promising business models (where also advertising may play a vital role for revenue generation),
which shall lead to a broader uptake of the new learning models in the future.
6. Impacts on target groups
The main target group, i.e. younger people aged 16-24, benefits greatly from the new learning models
developed in the course of the project. As the empirical results show these models can not only
support more efficient knowledge transfer then for example a conventional case study under certain
conditions, but also leads to more positive emotions and especially high flow values when using the
mobile learning games. The strong flow experience points to a high degree of intrinsic motivation in
the learner and shows that the game is being played for the game itself and not due to an external
incentive, e.g., a good grade.
7. Contribution to standards
The mGBL project has contributed to the raising of standards in two domains:
A Pedagogical and game models
B Learning standards
7.1 Pedagogical and game models
The three game templates (also known as models) produced by the mGBL consortium reflect generic
educational goals (Anderson and Krathwohl, 2001) and between them cater for three different
learning paradigms (experiences of learning):
Instructionist (Tolman, 1932), student as consumer. Focus on content, throughput, quality of
student on exit (e.g. Game model 1).
Revelatory (Bruner, 1973): student as explorer, team-worker. Focus on discovery (e.g. Game
model 2).
634

Thomas Putz
Conjectural (Kolb, 1984): student as producer, collaborator. Focus on output (e.g. Game model
3).
The games themselves are first and foremost games since without their entertainment value their use
as educational devices would be impaired. The game templates use J2ME which make the games
independent of a manufacturer-specific operating system. Game template 3 also makes use of Wi-Fi
Internet enabled handsets. These innovations place the games at the cutting edge of mobile game
developments.
7.2 Learning standards
To improve standardisation, interoperability and reusability of mobile learning games we have
undertaken the following improvements:
Game content for game 1 and game 2 is XML structured and thus substantially enhances possibilities
for interoperability and reusability.
Implementation of Dublin Core element dataset for the meta-tagging of mobile games with the Dublin
Core Initiative learning standard which comply with following standards:
ISO Standard 15836-2003 of February 2003 [ISO15836]
ANSI/NISO Standard Z39.85-2007 of May 2007 [NISOZ3985]
IETF RFC 5013 of August 2007 [RFC5013]
8. Overall feedback
In general, all students liked the games and also the mGBL platform. All indicators show that students
like to use the games in a real tertiary education environment. Some of the students requested usage
of the resource in other university courses. Overall, the students were very enthusiastic about using
the game Mogabal for their final course exams. Many of them pointed out the efficiency, flexibility and
ease of use of the platform. The new experience, fun and playability of the games gave them
additional motivation. More than half of the students stressed that they learned more by playing the
games, paid more attention while playing the games, and were more engaged when using the
“learning by playing” method.
All of the professors who participated in the trial would like to use the games for their learning
purposes. All of them are of the opinion that games could provide added value for their learning
purposes.
The mGBL games were complex in terms of the pedagogy and organisation involved. Despite this,
from the analysis which we performed it is clear that both teachers and students found them to be
challenging and enjoyable experiences. Most important of all, there is clear evidence that significant
learning took place and at least some of that learning was retained.
The mGBL project has proven that it is not always necessary to deploy the most advanced 3D
graphics and cutting-edge handsets in order to ensure a good user experience. In contrast, a practical
"low tech – high involvement" approach is in many cases much more suitable in the learning context,
especially when considering younger people in formal education. It is of utmost importance that the
systems can be used by all pupils and students and that it does not require special devices, which
may not be available or affordable for some target groups.
9. Where to find additional information
www.mg-bl.com
http://mgbl.sourceforge.net/
www.evolaris.net
References
Anderson, L.W. and Krathwohl, D.R. (Eds.) (2001). A taxonomy of learning, teaching, and assessment: A revision
of Bloom's taxonomy of educational objectives. Longman, New York.
Argyris, C. and Schön, D. (1978). Organizational learning: a theory of action perspective. McGraw-Hill, New York.
Benford, P., Magerkurth, C. and Ljungstrand, P. (2005). Bridging the Physical and Digital in Pervasive Gaming. In
Comm. ACM 48 3, p. 54–57.
635

Thomas Putz
Fabricatore, C. (2000). Learning and Videogames: an unexploited synergy. At:
www.learndev.org/dl/FabricatoreAECT2000.PDF.
Kolb, D. A. (1984). Experiential Learning. Experience as the source of learning and development. Prentice-Hall,
Inc., New Jersey.
Mitchell, A., Doherty, M., Millwood, R., Mininel, S., Inchingolo, P., Vatta, P., Parvu, A. (2007) On the Edge of
Design for Mobile Game-Based Learning. International Journal of Advanced Technological Learning, In
Press
Prensky M. (2001). Digital Game-based Learning. McGraw Hill, New York.
Race, P. (1994). The Open Learning Handbook. Kogan Page, London.
Salen, K. and Zimmerman, E. (2004) Rules of play: game design fundamentals. London: MIT Press
Senge, P. (1990) The fifth discipline: the art and practice of the learning organization. London: Random House.
Cited by Smith, M.K. (2001). Peter Senge and the learning organization. Infed.
Small, P. (2000) The Ultimate Game of Strategy. London: FT.COM ISBN: 027364999X.
636

Using the Common Cartridge Profile to Enhance Learning
Content Interoperability
Ricardo Queirós 1 and José Paulo Leal 2
1
CRACS & DI-ESEIG/IPP, Porto, Portugal
2
CRACS & INESC-Porto LA, Faculty of Sciences, University of Porto, Portugal
ricardo.queiros@eu.ipp.pt
zp@dcc.fc.up.pt
Abstract: The concept of Learning Object (LO) is crucial for the standardization on eLearning. The latest LO
standard from IMS Global Learning Consortium is the IMS Common Cartridge (IMS CC) that organizes and
distributes digital learning content. By analyzing this new specification we considered two interoperability levels:
content and communication. A common content format is the backbone of interoperability and is the basis for
content exchange among eLearning systems. Communication is more than just exchanging content; it includes
also accessing to specialized systems and services and reporting on content usage. This is particularly important
when LOs are used for evaluation. In this paper we analyze the Common Cartridge profile based on the two
interoperability levels we proposed. We detail its data model that comprises a set of derived schemata referenced
on the CC schema and we explore the use of the IMS Learning Tools Interoperability (LTI) to allow remote tools
and content to be integrated into a Learning Management System (LMS). In order to test the applicability of IMS
CC for automatic evaluation we define a representation of programming exercises using this standard. This
representation is intended to be the cornerstone of a network of eLearning systems where students can solve
computer programming exercises and obtain feedback automatically. The CC learning object is automatically
generated based on a XML dialect called PExIL that aims to consolidate all the data need to describe resources
within the programming exercise life-cycle. Finally, we test the generated cartridge on the IMS CC online
validator to verify its conformance with the IMS CC specification.
Keywords: eLearning, standards, interoperability
1. Introduction
Computers and the Internet have broadened the learning experience from the four walls of the
classroom to the remote delivery of instructions and educational content by digital means known as
eLearning. The evolution of eLearning in the last decades has been astonishing through the inclusion
of new paradigms and new platforms exploring and producing a great variety of solutions for the
enrichment of the educational experiences. In spite of their number, these platforms live at the
expense of the business model adopted and the standardization level supported. Standards can be
viewed as "documented agreements containing technical specifications or other precise criteria to be
used consistently as guidelines to ensure that materials and services are fit for their purpose"
(Bryden, 2003). In the eLearning context, standards are generally developed for the purposes of
ensuring interoperability and reusability in systems and in the content and meta-data they manage.
Many standards were created by educational organizations to enhance interoperability, reusability,
and customization of digital learning content, assessments, collaborative discussion forums, and a
diverse set of learning applications.
In this paper we focus on the IMS Common Cartridge. The Common Cartridge provides a standard
way to represent digital course materials for use in online learning systems so that such content can
be developed in one format and used across a wide variety of learning systems. We analyze the
Common Cartridge profile based on its interoperability levels: content and communication. In the
former, we detail its data model composed by a set of schemata referenced on the CC schema. In the
latter, we explore the use of the IMS LTI specification to enable web service invocation and data
exchange among distributing learning applications.
The study was the starting point to evaluate the applicability of IMS CC to describe programming
exercises in a network of eLearning systems where students can solve computer programming
exercises and obtain feedback automatically. Networks of this kind include systems such as learning
management systems (LMS), evaluation engines (EE), learning objects repositories (LOR) and
exercise resolution environments (ERE). Our strategy to achieve the interoperability among these
tools is based on a shared definition of a programming exercise as a Learning Object. This shared
definition is formalized using the IMS CC specification. The CC learning object is automatically
generated based on a XML dialect called PExIL. This XML dialect aims to consolidate all the data
need to describe resources that are useful on the programming exercise life-cycle. To test this CC
637

Ricardo Queirós and José Paulo Leal
learning object we introduce the IMS CC conformance framework used to verify the compliance of the
cartridges with the IMS CC specification and additional constraints (e.g. static, dynamic and
conditional constraints).
The remainder of this paper is organized as follows. Section 2 traces the evolution of standards on
the eLearning realm. In the following section we detail the IMS CC information profile highlighting the
Common Cartridge structure and the Learning Tools Interoperability specification. Then, we test the
applicability of the IMS CC interoperability levels based on the definition of a new CC learning object
for representing programming exercises and we test the cartridge structure using the IMS validator.
Finally, we conclude with a summary of the main contributions of this work and a perspective of future
research.
2. eLearning standards
In the last decade practitioners of eLearning started valuing more the interchange of course content
and learners' information, which led to the definition of new standards. These standards are generally
developed to ensure interoperability and reusability in content and communication. In this context,
several organizations (IMS, IEEE, ADL, ISO/IEC) have developed specifications and standards
(Friesen, 2005). These specifications define, among many others, standards related to learning
objects (Rehak and Mason, 2003), such as packaging them, describing their content, organizing them
in modules and courses and communicating with other eLearning systems.
Packaging is crucial to store eLearning material and reuse it in different systems. The most widely
used content packaging format is the IMS Content Packaging (IMS CP, 2007). An IMS CP learning
object assembles resources and meta-data into a distribution medium, typically an archive in zip
format, with its content described in a manifest file in the root level. The manifest file - named
imsmanifest.xml - adheres to the IMS CP schema and contains several sections such as Metadata to
describe the package as a whole and Resources to refer to resources (files) needed for the manifest
and metadata describing these resources.
The metadata included in the manifest uses another standard - the IEEE Learning Object Metadata
(LOM, 2002). The IEEE LOM is a data model used to describe a learning object. The model is
organized in several categories that cover general data, such as title and description, technical data
such as object sizes, types and durations, educational characteristics and intellectual property rights,
among many others. These categories are very comprehensive and cover many facets of a LO.
However, LOM was designed for general LO and does not to meet the requirements of specialized
domains, such as the automatic evaluation of programming exercises. For instance, there is no way
to assert the role of specific resources, such as test cases or solutions. Fortunately, IMS CP was
designed to be straightforward to extend through the creation of application profiles.
The term Application Profile generally refers to "the adaptation, constraint, and/or augmentation of a
metadata scheme to suit the needs of a particular community". A well know eLearning application
profile is SCORM (SCORM, 2004) that extends IMS CP with more sophisticated sequencing and
Contents-to-LMS communication.
The IMS GLC is also responsible for another application profile, the Question & Test Interoperability
(QTI, 2006) specification. QTI describes a data model for questions and test data and, since version
2.0, extends the LOM with its own meta-data vocabulary. QTI was designed for questions with a set
of pre-defined answers, such as multiple choice, multiple response, fill-in-the-blanks and short text
questions. Although long text answers could be used to write the program's source code, there is no
way to specify how it should be compiled and executed, which test data should be used and how it
should be graded. For these reasons we consider that QTI is not adequate for automatic evaluation of
programming exercises, although it may be supported for sake of compatibility with some LMS.
Recently, IMS Global Learning Consortium proposed the IMS Common Cartridge (IMS CC, 2011) that
adds support for several standards (e.g. IEEE LOM, IMS CP, IMS QTI, IMS Authorization Web
Service) and its main goal is to shape the future regarding the organization and distribution of digital
learning content.
The standardization of the learning content it is not enough to ensure interoperability, which is a major
user concern with the existing systems (Leal and Queirós, 2010). In the last few years there have
been initiatives (Holden, 2004) to adapt Service Oriented Architectures (SOA) to eLearning. These
638

Ricardo Queirós and José Paulo Leal
initiatives, commonly named eLearning frameworks, had the same goal: to provide flexible learning
environments for learners worldwide. While eLearning frameworks are general approaches for
eLearning system integration, several organizations proposed service oriented approaches
specifically targeted to the LMS (IMS Digital Repository Interoperability – IMS DRI) and Repositories
(IMS LTI).
The IMS DRI specification provides a functional architecture and a reference model for repository
interoperability composed by a set of recommendations for common repository functions, namely the
submission, search and download of LOs. It recommends the use of web services to expose the
repository functions based on the Simple Object Access Protocol (SOAP) protocol, defined by W3C.
Despite the SOAP recommendation, other web service interfaces could be used, such as,
Representational State Transfer (REST) (Fielding, 2005).
A common interoperability standard that is increasingly supported by major LMS vendors is the IMS
Learning Tools Interoperability (IMS LTI) specification. It provides a uniform standards-based
extension point in LMS allowing remote tools and content to be integrated into LMSs. The main goal
of the LTI is to standardize the process for building links between learning tools and the LMS. The
IMS launched also a subset of the full LTI v1.0 specification called IMS Basic LTI. This subset
exposes a unidirectional link between the LMS and the application. For instance, there is no provision
for accessing run-time services in the LMS and only one security policy is supported (IMS BLTI,
2010).
3. The IMS Common Cartridge profile
The IMS Common Cartridge specification defines an open format for the distribution of rich webbased
content. Its main goal is to organize and distribute digital learning content and to ensure the
interchange of content across any Common Cartridge conformant tools. The latest revised version
(1.1) was released in May 2011. The IMS CC package organizes and describes a learning object
based on two levels of interoperability: content and communication as depicted Figure 1.
Figure 1: Common Cartridge content hierarchy
In the content level, the IMS CC includes two types of resources:
Web Content Resources (WCR): static web resources that are supported on the Web such as
HTML files, GIF/JPEG images, PDF documents, etc.
Learning Application Objects (LAO): special resource types that require additional processing
before they can be imported and represented within the target system. Physically, a LAO consists
of a directory in the content package containing a descriptor file and optionally additional files
639

Ricardo Queirós and José Paulo Leal
used exclusively by that LAO. Examples of Learning Application Objects include QTI
assessments, Discussion Forums, Web links, etc.
In the communication level the cartridge describes how the target tool of the cartridge (usually a
LMS) should communicate with other remote web applications using the IMS Basic LTI specification.
Both levels enhance the interoperability of the cartridge among a network of eLearning systems. In
this scope a new IMS CC specification feature is introduced to support authorization at two levels:
either the whole cartridge can be protected or individual resources can be protected. In the following
subsections we detail the most important elements of the CC content hierarchy.
3.1 3.1 Content Packaging
The Common Cartridge builds upon a profile of the IMS Content Packaging (CP v1.2 schema). The
following figure provides a view of the CC manifest.
Figure 2: Common Cartridge package
The manifest is composed by four sections: metadata, organizations, resources and authorizations.
The Metadata section is used to store the cartridge metadata restricted to a loose binding of LOM
elements based on the Dublin Core (DC) specification. The Organization section will be used to
represent the Common Cartridge Folder content type as a structural approach to organize content.
The Resources section will be used to refer assets included in the cartridge.
3.2 Metadata
The manifest includes the LOM standard to describe the cartridge and the learning resources
included in the cartridge package.The metadata could be include at two levels: cartridge and
resources. At the cartridge level we must use metadata according to the Common Cartridge profile of
the IEEE LOM (loose binding) which describes the range of a mapping from the core elements of the
Dublin Core specification v1.1 to IEEE LOM. At the resources level we could use the original IEEE
LOM namespace.
640

Ricardo Queirós and José Paulo Leal
There will be scenarios where resources may need to be specified within the organization, but should
not be made visible in player mode upon default import of the cartridge. One such situation is the
inclusion of a solution program within the cartridge. To meet these needs, the common cartridge
supports the optional “roles” meta-data associated with the resource in the manifest file. The
supported roles in the IMS CC version 1.1 are: Learner, Instructor and Mentor. If case of absence of
the role the resource would be viewable by all users.
3.3 Authorization
An IMS CC learning object supports authorization at three levels: on cartridge import, on cartridge
usage and on usage of specific resources in the cartridge. The mechanism by which the authorized
access to particular resources is enforced by the tool is not defined by the profile. The following code
shows the use of the authorization element that extends the manifest to protect the learning object as
a whole.
12345
http://publisher.com/authme
Note that the address of the web service must support the Authorization Web Service as described in
IMS Common Cartridge Authorization Web Service. If the access attribute of the authorizations
element is set to resource then the individual resources which need to be protected are specified by
adding the protected attribute to each resource.
3.4 Questions and tests
The Common Cartridge uses the IMS QTI specification as a data model for questions and tests. This
specification is represented on the manifest through two LAO resource types: assessments and
question banks.
An assessment represents an ordered question set (e.g. Multiple Choice, True/False, Fill in the
Blanks, Pattern Match, and Essay) and may include optional attributes (e.g. number of attempts, time
limit and whether late submission is allowed) that apply to the set as a whole.
A question bank can embed any of the question types supported by the CC v1.1 profile of QTI. Only
one question bank can optionally be included in a cartridge.
3.5 Web links
A Web Link is a LAO resource that extends a standard HTTP link. The extension comprises a title and
a URL describing a set of window open features such as the dimensions of the window. This
approach allows the cartridge to minimize its storage space and to have content updates after
distribution. Web links are described in a descriptor file as follows:
The C Language
641

3.6 Discussion Topics
Ricardo Queirós and José Paulo Leal
A Discussion Topic is a LAO resource used to initiate a discussion activity. Upon import, the
discussion topic content is stored by the tool using its own internal representation. As the cartridge
content is added to an actual course, an associated discussion topic is created in the default tool
discussion forum. Discussion topics are described in a descriptor file as follows:
The Psychology of Faces
Differences from LMS and CMS? <br/> <img src="$IMS-CC-
FILEBASE$/images/img01.jpg"/>
The Discussion topic schema supports the use of plain text or HTML for the discussion content and
allows the attachment of other resources through the use of the attachment element.
3.7 Basic LTI
A Basic LTI resource refers to an XML file that contains the information needed to create a link in a
Tool Consumer (e.g. LMS). Upon the user’s click, the execution flow passes to a Tool Provider along
with contextual information about the user and Consumer. The Basic LTI link is defined in the
resource section of an IMS Common Cartridge as follows:
The href attribute in the resource entry refers to a file path in the cartridge that contains an XML
description of the Basic LTI link. A BLTI link contains several elements. The most important are: the
title and description elements contain generic information about the link; the custom and
extensions elements allow the Tool Consumer to extend the basic communication data; the
launch_url element contains the URL to which the LTI invocation is sent; the
secure_launch_url element is the URL to use if secure http is required.
4. Applicability to automatic evaluation
In order to test the applicability of these interoperability levels to the automatic evaluation we define a
new CC learning object for representing programming exercises. These LOs are exchanged in a
network of eLearning systems where students can solve computer programming exercises and obtain
feedback automatically. Networks of this kind include systems such as LMS, evaluation engines (EE),
learning objects repositories (LOR) and integrated development environments (IDE). This specialized
cartridge includes a LTI descriptor referenced in the manifest by LAO resource. The use of this LTI
descriptor will allow a secure integration of the cartridge from the place where it is referenced (e.g.
LMS) to the place where it will be used (e.g. ERE).
4.1 PExIL
The Question and Tests Interoperability (QTI) is used on IMS CC to describe simple exercises.
However, QTI was designed for questions with predefined answers and cannot be used for complex
evaluation domains such as the programming exercise evaluation. A programming exercise requires
a collection of files (e.g. test cases, solution programs, exercise descriptions, feedback) and special
data (e.g. compilation and execution lines). These resources are interdependent and processed in
different moments in the life-cycle of the programming exercise. To address these issues we create a
XML dialect (Queirós and Leal, 2011) – called PExIL (Programming Exercises Interoperability
Language) – whose aim is to consolidate all the data required in the programming exercise life-cycle,
from when it is created to when it is graded, covering also the resolution, the evaluation and the
feedback.
642

4.2 Generation of an IMS CC package
Ricardo Queirós and José Paulo Leal
In order to validate the PExIL usefulness we created a tool (named PexilUtils) to generate several
resources related to the programming exercise life-cycle (e.g. exercise descriptions in multiple
languages, test cases, feedback files). The LO generation is depicted in Fig. 3.
Figure 3: Common Cartridge package generation
The Generator tool uses as input a valid PExIL instance and a program solution file and generates 1)
an exercise description in a given format and language, 2) a set of test cases and feedback files and
3) a valid IMS CC manifest file. Then, a validation step is performed to verify that the generated tests
cases meet the specification presented on the PExIL instance and the manifest complies with the IMS
CC schema. Finally, all these files are wrapped up in a file (with the .IMSCC extension) and deployed
in a Learning Objects Repository.
From the several manifest sections included in the last version of the IMS CC specification only two
were addressed in the generation phase: metadata and resources. The former is fed by the generator
using a binding of the PExIL textual elements (e.g. title, authors, date) to the corresponding LOM
elements. The latter contains a list of references to other files in the archive (resources) and
dependency among them. The next figure shows the resources section of the generated CC manifest.
The resources section starts with a LAO resource (1) pointing to the PEXIL descriptor. This file is
responsible for the automatic generation of all the other files included in the package (with the
exception of the solution program and images). The description of the exercise is included on the
manifest as a WCR resource (2). This type of resources can be automatically rendered by the
browser without any additional processing. The program solution (3) is associated with metadata
since this resource should not be made visible in player mode to the students and will be used only to
regenerate test cases and in the evaluation phase of the programming life-cycle. The test cases are
depicted as LAO resources (4) comprising the PEXIL descriptor and a pair of input and output files
referenced by dependency elements and defined individually as resource objects (5). Finally, the BLTI
link is included as a LAO resource (6). This link points to a XML file that includes all the data needed
to integrate the cartridge in a LMS-web application communication.
643

Ricardo Queirós and José Paulo Leal
Figure 4: Resources section of the IMS CC LO manifest of a programming exercise
4.3 Validation
In this subsection we report on our efforts to validate the IMS CC cartridges previously generated
using the IMS validator at http://validator.imsglobal.org. This system validates cartridges for
conformance with the IMS Common Cartridge v1.0 and/or v1.1 specification. In the validation process
the IMS CC Validator test the whole cartridge verifying the following type of constraints:
Static: the parameters (e.g. file names) are fixed in the profile (e.g. imsmanifest.xml must
exist at the root of the package)
Dynamic: the parameters are taken from an instance document in the package (e.g. href
attribute of a resource element must point to a QTI file)
Conditional: the constraint depends on a condition (e.g. If parameter ‘contenttype’ is
‘question’ then the href attribute must point to a QTI file).
The cartridges generated from PExIL instances using the methodology presented in the previous subsection
passed all tests performed by the validator.
644

5. Conclusion
Ricardo Queirós and José Paulo Leal
In this paper we present a comprehensive study on the new IMS CC specification. For this study we
analyzed this specification at two interoperability levels: content and communication. At the content
level we studied the specifications of LAO resources, such as QTI assessments, Discussion Forums,
and Web links. At the communication we studied the IMS BLTI has a means to use the cartridge in a
LMS-web application communication. Both levels support authorization through the use of the IMS
Common Cartridge Authorization Web Service.
The applicability of IMS CC was tested in a specialized domain: the automatic evaluation of
programming exercises. In this scope we defined a new CC learning object for representing
programming exercises. These learning objects are automatically generated from a PExIL instance -
a XML dialect to describe data for the programming exercises life-cycle. Finally, the generated
cartridge was validated using the IMS CC validator to assure the conformance of the cartridge with
the IMS CC v1.1 specification.
Our major conclusion is that the IMS CC has a number of improvements when compared with the
base specification, the IMS CP. From the standpoint of our intended use, the automatic evaluation of
programming exercises, the most relevant features are the BLTI support and the access control at the
resource level. The Basic LTI (BLTI) will be instrumental in binding a programming exercise with an
environment where the student can resolve it; it can be used to launch exercise resolution
environments (ERE) from the LMS with a specific exercise. The access control at the resource level
enables an eLearning system (e.g. a LMS) to present the problem solution to the teacher while hiding
it from students.
We are currently finishing the development of the generator engine to produce a LO compliant with
the IMS CC specification. This tool could be used as an IDE plug-in or through command line based
on a valid PExIL instance. A valid PExIL instance can be integrated in several learning scenarios
where programming exercises are used, from curricular to competitive learning. For future work we
intend to support the PExIL definition in the crimsonHex repository (Leal and Queirós, 2009) – a
repository of programming problems.
References
Advanced Distributed Learning (ADL) (2004) “Scorm 2004 4th Edition”. Available online at http://www.adlnet.gov
Bryden, A. (2003) “Open and global standards for achieving an inclusive information society”. International
Organization for Standardization. Available online at http://www.iso.org/iso
Fielding, Roy T.; Taylor, Richard N. (2002-05) “Principled Design of the Modern Web Architecture”, ACM
Transactions on Internet Technology (TOIT) Association for Computing Machinery 2 (2): 115–150.
Friesen, N. (2005) “Interoperability and Learning Objects: An Overview of E-Learning Standardization”.
Interdisciplinary Journal of Knowledge and Learning Objects.
Holden, C. (2004) “What We Mean When We Say “Repositories User Expectations of Repository Systems””. In:
Academic ADL Co-Lab.
IEEE LOM (2002) “IEEE Standard for Learning Object Metadata” IEEE 1484.12.1-2002 Available online at
http://www.ieeeltsc.org/standards/1484-12-1-2002/
IMS BLTI (2010) “IMS Basic Learning Tools Interoperability Specification” - Version 1.0 Final Specification,
Available online at http://www.imsglobal.org/lti/blti/bltiv1p0/ltiBLTIimgv1p0.html
IMS CC (2011) “IMS Common Cartridge Profile” v1.1 Final Specification Available online at
http://www.imsglobal.org/cc/
IMS-CP (2007), “IMS Content Packaging Specification Primer”, Version 1.2 Public draft Available online at
http://www.imsglobal.org/content/packaging
IMS QTI (2006) IMS Question and Test Interoperability v2.1 Draft specification, Available online at
http://www.imsglobal.org/question/index.html
Leal, J.P. and Queirós, R. (2010) “eLearning Frameworks: a survey”. Proceedings of International Technology,
Education and Development Conference.
Leal, J.P., Queirós, R. (2009) “CrimsonHex: a Service Oriented Repository of Specialised Learning Objects”. In:
ICEIS 2009: 11th International Conference on Enterprise Information Systems.
Rehak, D. R., Mason, R. (2003) “Keeping the learning in learning objects”, in Littlejohn, A. (Ed.) Reusing online
resources: a sustainable approach to e-Learning. Kogan Page, London, 2003. (pp.22-30).
Queirós, R. and Leal, J.P. (2011) “PExIL: Programming Exercises Interoperability Language”, In 9th Conferência
- XML: Aplicações e Tecnologias Associadas (XATA).
645

The Design and Development of an eLearning System
Based on Social Networking
Andrik Rampun and Trevor Barker
Department of Computer Science, University of Hertfordshire, UK
y.rampun@herts.ac.uk
t.1.barker@herts.ac.uk
Abstract: For many people the internet “eLearning” experience can be rather lonely, lacking as it does, the
opportunity for interaction between students and the sense of community and sociability of a well delivered live
lecture. As a result the potential benefits from eLearning may fail to materialise because of this lack of student
stimulation, involvement and interaction. The basic premises of this project was that students prefer an audio
visual rather than solely text based medium, and prefer sociable rather than solitary individual study. It is
hypothesised that social networking which builds virtual communities allows connectivity, interaction and in the
context of eLearning holds the promise of making learning a more pleasurable interactive process. Based on
investigation current issues of eLearning and discovering the possibility of eLearning in social networks, the
authors describe the development and evaluation of an online eLearning application intended to enhance the
interaction levels of learners in a virtual eLearning environment. It was hoped that eLearning could be more
enjoyable by providing a friendly internet forum where students were able to post problems and ideas connected
with their subject domains, pose and answer peer questions, thus generating lively interactive chat, whilst
simultaneously developing communication and analytical and support skills. A prototyping process model was
applied in designing the application to maximise the user involvement during requirement gathering and design.
In the paper, the design process consists of five iterations is described. The application was developed based on
the concept of two popular social networking websites, Youtube and Facebook. It was our hypothesis that social
network websites have several drawbacks in terms of their educational use. These potential drawbacks are
described in the paper. The ways in which the authors allowed for these during the design and development
phases of this project is also explained. The application was evaluated using four evaluation methods,
observations, interviews, questionnaires and database records analysis. The results of this evaluation are
presented in the paper. It was found that learners had a positives experience of using the application and based
on learners’ comments we present a discussion on the possible reasons for this.
Keywords: Web 2.0, eLearning, application development, social networking, multimedia
1. Introduction
The popularity of social networks is increasing and, according to figures from In-Stats reports
produced by market analysts, there were 4.5 billion active social networking accounts in 2010 (Portet,
2011). Over the last few years, the popularity of social networks websites soared and opening of
accounts has become the norm for the younger generations. This has triggered numerous debates
about the possibility of making them a platform for online learning.
The idea of this paper is to explain the design, development and evaluation of an online eLearning
application based on social networks. The application is a web-based tool that allows students to
learn over the internet. But compared with many existing eLearning options, it is intended to increase
the interactions between users and deepen their engagement in their learning, so enriching the
process. This gives the potential to increase the efficiency of learning we argue. The application has
been developed based on the concept of social network models, Facebook (Facebook, 2011) and
Youtube (Youtube, 2011).
In order to understand the issues involved and to obtain the requirements of the system, extensive
research on the current eLearning approaches and the possibility of social networking as an
eLearning platform was undertaken by the authors. The main concern in eLearning is communities of
learners are difficult to form because they separated by distance which implies that each student
learns individually without the benefit of stimulating face to face discussion (Rennie and Mason,
2008). Therefore, for many people traditional “eLearning” via the internet can be rather lonely, as it
lacks the opportunity for interaction between students and the sense of community and sociability. As
a result the potential benefits from eLearning may fail to materialise because of this lack of student
stimulation, involvement and interaction.
After looking at the features of social networking and reading the literature, the authors were
convinced that it has the ability to form communities of learners to make eLearning more interesting
and fun by improving communication and interaction between students as well as the engagement
646

Andrik Rampun and Trevor Barker
level in virtual community. This suggests possible enhancements to make eLearning processes more
effective by improving the communication and interactions by applying the main features of social
networking. Therefore an extensive analysis of the weaknesses and strengths of social networks from
the eLearning perspective and student’s-teacher’s perspective is vital.
If education and learning systems are to use social networks such as YouTube, and Facebook, we
suggest they must be very carefully designed so that the learning goals are central, and various
pitfalls avoided. A prototyping process model (Evolutionary method) has been applied during the
design and development process to maximise user involvement to ensure the final product meet
user's requirements (Rogers, 2008). In order to test and evaluate the system, the researchers
undertook qualitative evaluation of the system with a small group of learners. The evaluation covered
4 aspects; the application’s usability, the user’s attitude while performing a task, whether the
application achieves its goals as an eLearning application and the functionalities of the application.
The participants were required to provide feedback on each of these aspects, in order to provide
information which was the key to the overall judgment as to whether the application had the potential
to perform as a satisfactory eLearning platform.
1.1 The current issues of eLearning approaches
More than 30 articles, journals and case studies were reviewed by the authors to investigate the
current issues of eLearning approaches. In addition questionnaires were distributed to learners and
70 responses were received. The respondents’ additional information such as age and grade level is
similar to the intended user of the system. As a result, the authors considered that many eLearning
approaches have had varying degrees of success but often have reduced effectiveness due to lack of
helpful interaction and communication among the users. According to Blinco et al. (2004), one of the
main issues in much conventional eLearning, apart from the content issue, is that the interface
between the user and the material can be uninteresting, the material itself is often difficult to
understand and the approaches do not persuading thus cause the learning process to become boring
and the learner to lose motivation. In fact, based on our survey findings, text-based traditional
approaches such as forums, wikis and blogs can be quite boring because of the low degree of
interaction (Shepherd, 2002). We hypothesise that in order to engage the attention of learners,
learning should be fun and interesting. However, case studies and the survey result showed that
reading thousands of words on computer screen (blogs and wikis) could be quite boring and stressful.
This is supported by the survey result when approximately 80% of the respondents agreed that textbased
materials are not the best medium for explanations in various fields of education and especially
when it comes to courses with a large component of practical work.
Therefore using multimedia delivery methods such as video and audio should be expected to
enhance the interaction and communication as well as the engagement level in a virtual community.
This is because study materials can be delivered via lecture, mentoring, modelling, presentation and
demonstration (Baird and Fisher, 2005). Mantyla (2001) added the materials also can be delivered
through simulation and small group discussions. This means that multimedia can deliver lectures in
various different ways which offer a better selection to gain knowledge. Several other main issues
such as lack of historicity elements, restricting the ability to organise the information and irrelevant
content were cited by approximately 90% of the respondents in the survey. These were consistently
cited as important elements which could be useful in eLearning.
1.2 The benefits of social networking to education
Social networks have its advantages to education. According to Thomas (2010), Bishop, a director of
the eLearning company, ‘Glamorgan Blended Learning’, commented that social networking is a
persuasive, adaptable and sociable system that could lead to more interesting virtual learning
environments. Social networking allows learners to procure information quickly and easily, on any
subjects (Iloveindia, n.d). For example users within a student's network can share information they
have brought in from different websites. Therefore, the student can get notification of others' research
in areas of shared interest. In this way they are able to help each other to keep up to date with current
information as long as they are connected with other users (Bedel, 2009). The bigger the network, the
more information that can be gathered and the more frequently the information shared is updated.
Moreover, social networking encourages users to work in collaboration rather than on their own,
increasing group working skills (iloveindia, n.d). In fact, it also can improve the communications skills
647

Andrik Rampun and Trevor Barker
as well as collaborative with teachers (Xomba, 2010). In addition, it gives the chance to all people to
be content creators, managers and distributers (Digizen, n.d). Users can create (role as a content
creator), upload (role as a distributer), edit and organise (role as a manager) the materials. These
activities support and develop users' creativity and active participation. Additionally, social networking
allows a learner to connect with many other learners from different backgrounds, cultures and
experiences. According to Banks et al. (2001) in the University of Washington, “learning occurs when
there is diversity in the environment”.
2. The design and implementation process
2.1 The requirements
The application's functional requirements were identified during the investigation of current issues of
eLearning described in the previous section. A summary of the application's requirements identified
by the authors is presented in table one below:
Table 1: List of functional requirements
Number Requirements
1 User login and logout
2 User register account
3 User forgot password (change password)
4 Search materials (videos)
5 Create topic discussion
6 Upload material(video)
7 Update video’s details
8 Update user’s profile
9 Send message
10 Reply message
11 Save video into user’s folders (favourite folder, watch later folder, etc)
12 Request to joint network/approve/reject requests
13 Create quiz to be attached with a video
14 Create notice on blue board.
15 Download video/materials note
16 Post comments to a video
17 Like and dislike video
18 Reply topic discussion
19 Red flag notification
20 Notifications on dashboard
21 Video statistics
22 Delete messages
23 Delete videos
24 Delete topic discussion
25 Edit topic discussion
26 Table paging
27 Display friends suggestion
28 Display current network’s activities on user’s dashboard
29 Display current blue board contents on user’s dashboard
30 FAQ pages
31 Tool tips for each of the related images, links, menus, etc
648

Andrik Rampun and Trevor Barker
The authors also suggest a set of non functional requirements of the system based on a consideration
of usability (Miller, 2002). This would improve the interaction level between learners and the
application. Therefore increase user’s satisfaction through:
Consistency
Controllable by users
Effectiveness and efficiency
Memorability
Application’s usability design
Understandability
Ease of learning
2.2 Application design
There were five iterations involved in the application design phase and each of the iterations involved
four stages. Here is the general process how the application was designed. The first stage was “quick
design” which is a stage where the developer plans specific parts of the application in outline. Some
designers prefer to sketch the initial design on a paper. From the frames templates created, the
developer was able to convert it using HTML and CSS mark-up languages to make it viewable on
internet browsers. The designs were then available for formative evaluation in a user centred
approach in order to gain feedback. User would write down what he/she liked and what he/she did not
like about the design in a feedback session. During the feedback session, new requirements were
discovered as well as design improvements suggested. Based on the feedback from the user, the
researcher analysed and refined the new design for the existing requirements and also developed a
set of new requirements. Figures 1, 2 and 3 are an example of sequence on one of the prototypes
was developed.
Figure 1: Sketched design for video player
During the first iteration, the developer constructed the designs for all the basic requirements
gathered in the previous phase. Firstly, the designs were sketched using the Microsoft Word
application then converted into mark-up languages using HTML and CSS, so the designs were
viewable on internet browsers. By the end of the process, Microsoft Visual Basic (VB) scripts were
649

Andrik Rampun and Trevor Barker
added to develop the various prototype systems as dictated by the requirements and feedback. Once
all the prototypes were completed, they were exposed to the user to gain additional feedback and
also to identify any new requirements that presented themselves.
Figure 2: Example of VB.NET static code
Figure 3: The prototype for the video player
The basic requirements in the first phase were login and logout pages, signup page, and recovery
password and search functionality. By the end of the first iteration, new requirements were gathered
and those requirements were based on user feedback and suggestions from the developer. The new
requirements were brought to the second iteration and the prototypes were developed in this stage.
The prototype development process was exactly the same in the previous phase but with additional
requirements. Once the prototypes were completed, they were again exposed to the users in
additional sessions to gain feedback. This process was continued until the fifth iteration where the
users and developers were satisfied with the designs.
650

2.3 The implementation
Andrik Rampun and Trevor Barker
After the design phase had been completed and achieved the required standard, the developer
initiated the application development stage of the project. The application mainly has been developed
using Microsoft VB.Net under the Visual Studio 2008 environment on the Windows Vista platform.
VB.Net is known as a 4 th generation language which helps to quicken the development process, and
also it is easier to learn compared to some programming languages such as Java and C++. However,
JavaScript was also used on some of the pages, for example to assist in the paging process. The
purpose of using JavaScript in this application is the fact that it is a powerful script for very efficient
and effective tasks, especially paging and form validation. Moreover, AJAX technology
(webhostdesignpost, n.d) has been used to help the application perform some of the tasks more
effectively and efficiently. For example, VB.Net is unable to refresh a specific part of a page without
using AJAX technology. Therefore, AJAX was widely used in developing the application in addition to
VB.Net.
In addition to these technologies, an Open Source technology was also used. For example,
“JWPlayer” is an Open Source video player which is customisable. “JWPlayer” is not only easy to use,
but very easy to install and deploy on the server. The main advantage of “JWPlayer” is that the
developer does is not required to install Macromedia Flash or Window Media Player component in the
Visual Studio environment. The player is able to display many kinds of video format such as .AVI,
.MP4, .MPG and others. In the last part of this stage, HTML and CSS integration was undertaken.
HTML and CSS are two mark-up languages which are mandatory in any web application
development.
The database side of the application was designed using standard techniques and the Microsoft
Access 2007 was used as a database for the application. During the development process several
software engineering practices were employed to ensure the application’s reliability such as compile
and smoke test frequently and manage testing as a continuous process (SPMN, 2010). The
developer additionally maximised user involvement during the implantation phase in order ensure that
the requirements and designs were fully synchronised. The figures 4 and 5 below are example screen
shots of the application.
Figure 4: Screen shot to display a video material
651

Figure 5: Screen shot of a user’s dashboard
3. Evaluation
Andrik Rampun and Trevor Barker
The evaluation objectives were:
To evaluate the importance of “historicity” element in eLearning. Five participants were selected
and given five days to use the application. They were required to create their accounts in the
application. Once they have created, they had five days to use the application and make
interactions with the other users and materials. They can upload videos, comment videos, create
forum discussion, create quiz, etc. By the end of period, the authors performed the following
methods to evaluate the importance of historicity element in the application.
Database record analysis – This was achieved by recording and analysing the interactions made
by users as they used the application in order to achieve their objectives and goals. This was
instrumental in understanding the importance of the user’s history function in the application.
Think aloud – After the end of the evaluation period, participants were required to write a short
answer for a set of questions given.
To evaluate the ability of a social networking for interactive eLearning environment to increase the
interaction levels among the users, as well as with the materials. The same five participants were
given set of questionnaires and required to write a short answer for a set of the questions given to
them.
To discover how virtual appreciation elements can motivate students to get involved in a team.
The authors asked the same participants by distributing set questionnaires and required to write a
short answer for set of the questions given to them related to this issue.
To evaluate whether the application achieved its overall goals or not
The application’s goals were:
652

Andrik Rampun and Trevor Barker
To increase users’ interactions and engagement level among the users and materials within a
virtual learning community by providing a social networking based eLearning environment.
To increase the learners’ motivation to participate in a collaborative team within a virtual learning
community.
To make the learning process more interesting and fun by using Video and Audio instead of text
and images.
3.1 Findings
3.1.1 The importance of “historicity” element in eLearning
Based on the positive feedback obtained, it is suggested that it is one of the key features to be
integrated within an eLearning application of this type. One of the key aspects of this feature is that it
was able to show the relative popularity of any posting, (materials or other content) by means of a
tally count, according to the frequency with which a particular posting is viewed by other learners. This
popularity can be viewed on a user’s dashboard, and will tend to steer e-learners towards popular
(and arguably useful) material. Interesting material would tend to be selected more often by users and
would often lead to a further discussion (through comments posted) among the learners. As we can
see in this process, this feature can, in a more subtle way, encourage learners to discover something
new as well as to get involved in a virtual group discussion, developing the friendly, virtual community
that was one of our objectives at the start of this research as being desirable in an eLearning system.
One of the participants commented that by doing this learners could acquire “bonus” knowledge even
though the initial intention was to find different information. In fact, it's believed that this feature could
also be especially helpful to those who need direction from others, being non-self starters themselves.
Moreover, this feature also allows learners to look back and view a record of useful information at
anytime. The “useful information/activities” might include what they have done in the past, such as
items they have revised, videos they have watched, documents they have created and issues they
have discussed. All those activities can help to sustain the learning process through “memory
support” prompted by the historical “threads”. In fact, these “threads” we argue, can also assist by
making it quicker to access, organise and recall information. However, it is true to say that privacy is a
potential issue that has to be considered. This is because not every learner may like to be “watched”
or get “noticed” for what they did in the past. This is an issue for future research.
The second data analysis was based on the application's database record. The application recorded
the number of clicks made during the whole history of the interactions on pages and the also records
the number of clicks made using the search field at the top of the application page. The purpose of
the comparison was to measure how important the history page is in helping learners to find useful
information related to their studies instead of using the search function.
Variable a = number of clicks made through the history page --- N(a)
Variable b = number of clicks has been made using search field --- N(b)
The researcher assumes if N(a) > N(b) means users are mainly relying on feature A to find
information within the application. Otherwise users are mainly relying on feature B as their main tool
to find information within the application. The finding was that:
N (a) = 114 clicks
N (b) = 82 clicks
The records show that during the evaluation period the users made 114 clicks through the history
page displayed on dashboard whilst 82 clicks were made by the users through search field. This is
interpreted as showing that the user’s history page is a very important alternative tool for finding
information within the application and indeed that users are attracted more using the history feature
more than the search functionality. Therefore, even though both of those features serve different
purposes it clearly shows that users will use history page not only to retrieve previous action but to
seek info similar to the search function.
653

Andrik Rampun and Trevor Barker
3.1.2 The ability of a social networking based eLearning environment to increase the interaction
levels among the users, as well as with the materials
According to the feedback from the evaluators, the application had great potential to increase the
interaction levels among users as well as between users and the materials. It was likely however that
the application would benefit from a few improvements in certain respects. For example too many
interactions may cause interference and distractions, especially for users who prefer to learn at their
pace and space with just the minimum interactions. However, the application was able to increase
both the users’ interactions with other users and also the number of comments made on materials.
This was because users were able to contribute videos, images and text based notes, all of which can
be downloaded onto hard-disk for further use. Because of this case we suggest that learners have
more opportunity for interactions with the materials. This is unlike the materials available on YouTube
for example where users do not have the ability to download the videos directly from the website
itself. This will likely minimise the learner’s ability to control the materials themselves.
One evaluator commented that the application’s ability to suggest and display relevant videos is a
useful feature to enhance the interaction levels between users and the materials within the
application. The videos suggestion function allows the user to find more about the links provided. This
feature therefore was seen to enable users to find a range of answers/techniques to solve a particular
question. In fact, the freedom given to the users to personally organise materials is seen as another
way to increase the interaction level between a user and a material.
3.1.3 To discover how Virtual appreciation elements can motivate students to collaborate
The feedback also indicated that virtual appreciation highly motivates users to get involved in a
collaborative team, leading to more interactions not only among the users but between users and the
materials. For example kudos points would likely lead to a healthy competition if the users were
ranked in a league within his/her fellow network members. This would allow learners to see their
relative positions in the network and to understand how they might be able to improve his/her current
position in a league. This sort of healthy competition and friendly rivalry might be able to encourage
users to publish and share more information, as well as to get involved in many discussions. By doing
these activities, they will collect more points so improve their current position in a league.
Additionally, the “like” and “dislike” buttons available to learners were also recognized as very useful
features in online eLearning. All the participants reported that the use of these buttons encouraged
them to share their videos with the community. For example if many users “liked” a video, it means
the video’s content has good qualities and is acceptable. The “like” feedback encourages the
uploading of more videos for sharing. It is likely that this kind of activity ensures that the learner is
engaging with a community rather than just watching the videos suggested by others.
The feedback received also show that learners wanted to be recognised in a virtual learning
community. One of the ways is through a distinctive user profile in the community. For example, in
this application, as users contribute more content into the community, his/her “kudos points” would
increase. In this case the researcher received a response from one of the participants to say how she
liked the feature because the “kudos points” on her profile not only made her more confident in
helping others but encouraged her to be more active, for example by replying to comments on the
forum and uploading videos.
3.1.4 Evaluation whether the Application achieved its Goals or not
The overall analysis based on the responses received from the evaluators suggests strongly that the
application has successfully achieved its goals. A social networking based eLearning environment
was able successfully to increase interaction among the users in the virtual community and also
increased the engagement level between users and the materials’ contents. The combinations of
forum discussion and learning through video and audio made the learning process more interesting
and fun. It was seen that this combination was beneficially blended together in an on online eLearning
application rather than in a text based or only video based application.
The engagement level among users and materials increased when users were given the freedom and
ability to organise the videos into their own personal folders. For example, the application allowed
users to save videos into any folder of their choice (‘watch later folder’, ‘my document folder’, ‘saved
video folder’ and ‘favourite folder’). We argue that learning through video and audio also is likely to
654

Andrik Rampun and Trevor Barker
enhance the learners’ engagement in a particular topic because it is often easier to understand a
practical task with video than by text based, information alone.
The interactions among the users increased because they were able to see what the other learners
had done in the past. Those activities were displayed on the user’s dashboard and they could
persuade a user to follow what the others were doing by clicking the link history displayed on his/her
dashboard. Interactions among users often occurred when users were discussing a particular topic
and exchanging opinions with each other by posting their comments on the page.
From the feedback we learnt that participants enjoyed using the application because they had the
ability to create quizzes and attach notes together with the videos uploaded by them. The evaluators
commented that in this way learners are not only playing their role as a learner but also the role as a
teacher. This we argue, motivated them not only to get involved in a learning community but in a
virtual teaching and learning community.
We are able to view this application as an eLearning environment with the motivation to learn and
teach, with deep collaboration and virtual appreciation from the other users in the community.
Appreciation can be shown through the dislike and like buttons. User reputation can be shown
through reputation points making the application e -learning process more fun and interesting.
Displaying a user’s position in a league within his network can create a healthy competition as users
will tend to try to increase their reputation points through greater participation, such as uploading
more informative videos, creating more topic discussion, helping others to answer questions, etc.
4. Discussion
The evaluation results and findings suggest that a social networking application such as developed
here, has the potential to improve eLearning on the internet compared with purely text-based existing
eLearning options. Such an application can increase the interaction levels both between different
learners and also between a learner and the study materials. In fact, the authors found that allowing
learners to revisit their tracks on the eLearning application is a feature particularly welcomed by the
participants, being very helpful for information retrieval. According to Lamming and Newman (1991),
“much information is hard to retrieve the need to do so was not foreseen at the time the information
was stored”. However, the user's "trail" can support the retrieval process because it contains a
sequence of logical steps and pieces of information which can facilitate recall and memorisation.
Hailpern et al. (n.d) in their article about improving recall with contextual searches stated that user's
activity history acts as a “memory support” also because it contains pieces of information such as
keywords and dates.
When a learner can retrieve the information previously studied, this is likely to make the learning
process more interesting. One of the challenges in eLearning is how to help those learners with
problems of information retention over time which may be hours, days or longer. Therefore a learner
might choose to make use of the historicity elements in the application, knowing in a particular case
that they can increase ease of learning and retention, and so also increase their motivation to learn
(ETTAD, n.d). Therefore the historicity elements provide a promising enhancement in eLearning to
assist learners who are having retention problems.
If a learner achieves his/her learning objective more easily, it seems likely that learning will be more
interesting, fun and enjoyable, giving an increased incentive to learn. In eLearning, motivation could
be influenced by means of several factors such as appreciation, intrinsic level of interest of the topic,
and through competition among the learners. Based on the feedback gained during the evaluation
process, showing appreciation (in this case through comments, “kudos points”, like and dislike
buttons) is an effective way to motivate learners. One of the participants pointed out that as he
accumulated more “kudos points” from other learners, this visible expression of appreciation
increased his enthusiasm and readiness to get involved in yet more discussion by helping others with
answers, clarifications and comments regarding their postings. So incorporating means of expressing
appreciation can create a virtuous circle motivating learning. Therefore, the authors suggest that
integrating features that enable users to express their emotion is a important way to increase the
incentive to learn.
Kudos points might also be used to create healthy competition among learners within a virtual
community. Indeed if the points were depicted in the form of a league table, for many learners this
655

Andrik Rampun and Trevor Barker
could create a constructive sense of competition because they will strive to get more “kudos points”
by participation in discussion or contributing and sharing materials with others. It is possible however
that this competition might be counter-productive for some learners. We will investigate this issue in
future research. This competition could potentially increase a learner’s participation and engagement,
not only in a particular group discussion but in the whole community. By involvement with more
groups there are likely to be more interactions, leading to a greater spectrum of views and diversity of
opinions, leading to a richer "debate". Diversity of expressed views may give students more
alternative personal routes to understanding and learning and options to choose those which are the
best for them. Therefore, this tally with an article entitled “diversity within unity” by Banks et al. (2001)
the University of Washington, arguing that learning occurs when there is diversity in the environment.
In light of these findings, it will clearly be desirable to enhance the evaluation of the application by
using a greater number of participants than the ten participants in this study. In conclusion, this study
strongly suggested that social networking has the potential to revolutionise the current eLearning
approaches such as portal based eLearning, blogs, wikis and forum discussion. In fact, social
networking technology has the potential to shift the learning process from "formal" to "informal"
making learning a more relaxed, pleasurable and interactive process. This would allow students to
schedule and plan a learning period so that their session is tailored for a specific time allocation. This
is in accordance with one assumption related to the cognitive theory of multimedia learning which is
due to the capacity of working memory; learners can only process a limited amount of information in
each channel at one particular time (The eLearning coach, 2010). Other than that, the authors
consider that the social networking approach allows learners to develop their knowledge for
themselves as he or she learns in a constructivist way.
References
Baird, D.E. and Fisher, M. (2005) ‘NEOMILLENNIAL USER EXPERIENCE DESIGN STRATEGIES: UTILIZING
SOCIAL NETWORKING MEDIA TO SUPPORT “ALWAYS ON” LEARNING STYLE’ Educational
Technology systems. Vol. 34(1) pp.5-32, 2005-2006. http://gatortracks.pbworks.com/f/social+media.pdf
[accessed 26th April 2011].
Banks, J.A., Cookson, P., Gay, G., Hawley, W.D., Irvine, J.J., Neito, S., Schofield, J.W., Stephan, W.G. (2011)
‘Diversity within Unity: Essential Principles for Teaching and Learning in a Multicultural Society’.
http://www.cwu.edu/~ectl/diversity/wastate/diversityunity.pdf [accessed 2nd March 2011].
Bedel, J.T. (2009) ‘The Case for Social Networking in Schools’ JTB consulting: Create new connections, simply.
http://jasontbedell.com/the-case-for-social-networking-in-schools [accessed 2nd March 2011].
Blinco, K., Mason, J., McLean, N., Wilso, S. (2004) Trends and Issues in ELearning Infrastructure Development.
http://www.jisc.ac.uk/uploaded_documents/Altilab04-infrastructureV2.pdf [accessed 27 th April 2011 9.17
am].
Digizen (n.d), ‘Social networking-report-benefits’ Digizen: Young people and social networking services.
http://www.digizen.org/socialnetworking/benefits.aspx [accessed 2nd March 2011].
ETTAD (n.d) ‘eLearning/ICT Activities and difficulties with Memory/Recall’ ETTAD: Enabling Teachers
and Trainers to improve Accessibility of Adult Education. http://uk.ettad.eu/learning-categories-andchallenges/eLearning-ict/eLearning-ict-and-memory-recall
[accessed 25 th May 2011].
Facebook (2011) http://www.facebook.com/ [accessed 31 st May 2011].
Hailpern, J., Jitkoff, N., Warr, A., Karahalios K., Sesek R., Shkrob N. (n.d) ‘YouPivot: Improving Recall with
Contextual Search’ Social Spaces: Social Spaces Group. http://social.cs.uiuc.edu/papers/pdfs/youPivot-
CHI2011.pdf [accessed 24 th May 2011].
Iloveindia (n.d) ‘Pros And Cons Of Social Networking Sites’ iloveindia: Lifestyle Lounge.
http://lifestyle.iloveindia.com/lounge/pros-and-cons-of-social-networking-sites-9204.html [accessed 2nd
March 2011].
Lamming M.G and Newman W.M (1991) ‘Activity-based Information Retrieval: Technology in Support of Personal
Memory’ http://www.lamming.com/mik/Papers/air.pdf [accessed 24 th May 2011].
Mantyla K. (2001) Blending eLearning. American Society for Training and Development.
http://books.google.co.uk/books?id=NsBejLOYbmQC&pg=PA117&dq=problem+with+text+based+elearning
&hl=en&ei=0IReTfM1x4qFB9XL8YYO&sa=X&oi=book_result&ct=result&resnum=4&ved=0CE0Q6AEwAw#
v=onepage&q=problem%20with%20text%20based%20eLearning&f=false [accessed 18 th February 201].
Miller, R.H. (2002) ‘ELearning usability checklist’ www.stcsig.org/usability/resources/toolkit/eLearningchecklist.doc
[accessed 31 st May 2011]
Portet,S. (2011) ‘Social Network Accounts Outnumber People On Earth’ socialnetworkingwatch.com.
http://www.socialnetworkingwatch.com/2011/04/social-network-accounts-outnumber-people-on-earth.html
[accessed 27 th April 2011].
Rennie, F. and Mason, R. (2008) ELearning and social networking handbook. Routledge.
http://books.google.co.uk/books?hl=en&lr=&id=KMjh7MCEcYYC&oi=fnd&pg=PP1&ots=hbrBemHS4J&sig=
7nEQpPtPORCx3txVln4DsxZdgfc#v=onepage&q&f=false [accessed 26th April 2011].
656

Andrik Rampun and Trevor Barker
Rogers, B. (2008) ‘Prototyping’ Saint Xavier University. http://web.sxu.edu/rogers/sys/prototype.html [accessed
31 st May 2011]
Shepherd, C. (2002) ‘Becoming an online media mogul’. http://www.fastrakconsulting.co.uk/tactix/Features/mediamogul.htm
[accessed 14 th February 2011].
SPMN (2010) ‘16 CRITICAL SOFTWARE PRACTICES TM for Performance-based management’ SPMN: Software
Program managers network. http://www.spmn.com/16CSP.html [accessed 31 st May 2011]
The eLearning coach (2010), ‘Learning Theory and Multimedia’ The eLearning coach: Tips, advice and reviews
for online learning. http://theelearningcoach.com/learning/learning-theory-and-multimedia/ [accessed 15 th
February 2011].
Thomas, H. (2010) ‘Social Networking in eLearning: Beneficial or Unproductive?’ Trivantis: High definition of
eLearning. http://www.trivantis.com/uk/blog/elearning/social-networking-elearning-beneficial-or-unproductive
[accessed 1st March 2011].
Web host design post (n.d) ‘What is Ajax Technology? | New 2.0 Technology‘ The Web host design post: put the
pieces together. http://www.webhostdesignpost.com/website/webtechnology-ajax.html [accessed 31 st May
2011].
Xomba (2010) ‘Educational Benefits Of Social Networking Websites’ Xomba: Computers and Internet.
http://www.xomba.com/educational_benefits_social_networking_websites [accessed 2nd March 2011].
Youtube (2011) ‘Boardcast yourself’ http://www.youtube.com/ [accessed 31 st May 2011].
657

Kansei Design Model for eLearning: A Preliminary Finding
Fauziah Redzuan 1, 2 , Anitawati Mohd Lokman 2 , Zulaiha Ali Othman 1 and Salha
Abdullah 1
1 Universiti Kebangsaan Malaysia (UKM), Bangi, Malaysia
2 Universiti Teknologi MARA (UiTM), Shah Alam, Malaysia
fauziahr@tmsk.uitm.edu.my
anita@tmsk.uitm.edu.my
zao@ftsm.ukm.edu.my
sa@ftsm.ukm.edu.my
Abstract: Positive emotion plays an important role in learning. Previous researchers have emphasised the
importance of positive emotion for effective learning. Although some researchers have documented the
association between emotion and eLearning, few studies have addressed users’ emotional experience during
their interaction with eLearning material. Therefore, this research paper highlights the importance of emotion and
aims to associate users’ emotional experience in eLearning in a convergence of interface, content, and
interaction design, of the learning material. In an experimental setting, we adopt the Kansei Engineering (KE)
methodology that has been proven successful in associating emotion and product design characteristics. The
Kansei Engineering technique is used in this research to translate the users’ Kansei (feeling and desire) into the
design elements of the online course. The instruments used in this research include ten online database courses,
and 478 adjectives have been used to represent the users’ emotional experience. Respondents are comprised of
36 undergraduate students from a public institution of higher learning, in Malaysia. The findings led to the
development of a Kansei design model, which aims to provide instructors and designers with clues for
engineering a positive emotional experience, for students learning in an online environment. The positive
emotional experience is targeted to facilitate, not only the capable students, but also the at-risk students, by
enhancing their learning experience. Recent research has also highlighted the need for new techniques to
identify at-risk students, as well as to support their learning. Therefore, identifying a good emotional design for
eLearning will hopefully assist better learning, not only for the good student, but also, and more importantly the
at-risk student. This paper presents the preliminary findings of the first experiment conducted by the researchers.
Results reveal the key adjectives for describing emotional experience in online learning, as well as the specific
design elements of the online course, associated with these emotions. Additionally, this paper briefly discusses a
proposed model for positive emotional experience in online learning; a description of the Kansei Engineering
technique adopted for this study; the analysis and findings, as well as a brief explanation of future research
directions.
Keywords: online course, Kansei Engineering (KE), emotion, design elements, eLearning, emotional experience
1. Introduction
Emotion plays an important role in learning. As emphasised by various researchers, positive emotions
are crucial for effective learning to take place. Despite critiques claiming that emotion should be
neglected in the learning process, many other researches support the idea that positive emotion is
important for effective learning.
However, although there are some researchers who associate emotion and eLearning, researches
that address users’ emotional experience in eLearning are still in the early stages of development.
The work reported in this paper presents the results of the preliminary study, performed to provide
input to the ultimate aim, which is to discover the association of users’ emotional experience in
eLearning with a convergence of interface, content, and interaction design, of the online learning
material.
The following sections are placed in order of what has been said in the literature on emotion in
eLearning, issues in eLearning, emotional experience in web-based applications, Kansei Engineering,
a brief description of the research model, followed by the research method, and the analysis and
discussions from the results of the evaluation experiment. Finally, the research summary and future
work directions are given.
2. Emotion in learning
Emotion is still somewhat neglected in learning, as researchers only tend to emphasise on cognition
and rationality. It is only quite recently that the trends have changed, and more researchers are
including emotion in their research.
658

Fauziah Redzuan et al.
According to Clapper (2010), positive emotion can be positively associated with learning. Recent
brain research also indicates that emotions are essential to learning (Rager 2009). In order for
learning to take place, the attention of the learner must first be engaged, and emotional responses
can ‘trigger’ the learners’ attention (Rager 2009: 29). Rager (2009) also suggested that emotional
issues could not be ignored and should be included in all types of learning, and stressed that more
research is needed in this area.
Many researchers highlight the importance of positive emotion in learning, such as in becoming more
efficient, and creative problem solvers, Hirt et al (in Capota, van Hout and van der Geest 2007); to
affect information processing and learning outcomes (Domagk, Schwartz and Plass 2010); to promote
knowledge construction and problem solving (Um, Song and Plass 2007); to improve creativity and
flexibility, Isen (in Chaffar and Frasson 2005); in motivating students (Wang et al 2010); and to help in
performing difficult tasks (MacFadden 2005). However, negative emotion poses several risks
including making simple tasks difficult (MacFadden 2005) and impairing individuals’ abilities to think or
learn efficiently, Goleman (in Chaffar and Frasson 2005).
However, de Jong (2009) implied that emotion in learning is rediscovered. This renewed discovery
may be based on the direction of research from being teacher-centred to learner-centred. This is a
new light to emotion in education, and rejects MacFadden’s (2005) argument that emotion has been
neglected in education and online education, which was reported due to the heavy emphasis on
cognition and rationality. According to him, historically, emotion has not been very appropriate in
education (MacFadden 2005). Additionally, MacFadden also asserted that more investigation was
needed to identify ways that the focus on emotions can inform and strengthen the web-based learning
experience (MacFadden, 2005).
Linnenbrink-Garcia and Pekrun (2011) also highlighted the importance of studying emotions,
especially in the context of a student’s engagement and learning. These authors emphasised that
research on emotions was still limited in educational psychology, a point that was previously stressed
by (Domagk, Schwartz and Plass 2010).
In conclusion, it is evident from the above literature that emotions, especially positive emotions, are
indeed very important and powerful in engaging learners either traditionally or online. Hence, putting
this argument forward, the research reported in this article was conducted in order to investigate the
emotional experience that students would have in online learning, looking into the possibilities to
cluster significant emotions and thus, distinguish between positive and negative emotions.
3. Problems in eLearning
Three important problems that are uniquely associated with the design within the eLearning
environment have been identified from the literature. The first problem, as highlighted by (Vrasidas
2004), focuses on the eLearning design itself, with regard to some problems in the Learning
Management System (LMS). The second problem is associated with the course design in the
eLearning environment, as discussed by (Yang and Cornelius 2004; Fisher and Wright 2010). The
third issue concerns the design of the learning material in the online course, as discussed below.
Although many issues remain, especially related to design in the eLearning environment, the focus of
this research is only on the emotional experience evoked by the design of the learning material in the
online environment.
One of the general problems in web-based design, according to Lokman (2011), is that web-based
design always produces a conflict between the designer’s specification and the user’s conformance.
Her argument agrees with that of Liu’s (2007) that posited that there is a discrepancy between the
designs of eLearning and the learners’ need or preferences, which results in poor learning among
students. This idea is further supported by Stephenson et al (2007), who state that designers should
employ the principles of a user- or learner-centred design. Even though many eLearning materials do
employ multimedia elements, as well as interactivity, these elements still do not meet the expectations
of the user, as noted by Greitzer (in Stephenson et al 2007). The root cause of this problem may be
poor design, organisation of the content or usability.
Furthermore, van Schaik and Ling (2008) also emphasised that poorly designed pages can rapidly
turn users away. O'Brien and Toms (2008) also stated; “a Web interface that is boring, a multimedia
659

Fauziah Redzuan et al.
presentation that does not captivate users’ attention ….are quickly dismissed with a simple mouse
click”. O'Brien and Toms (2008) also argued that, in order for technologies to be successful, they must
not only be usable, but also engage users. Accordingly, previous researchers also noted the difficulty
of designing online learning interfaces, as compared with designing web pages such as those noted
by Kreijins (in Fadel and Dyson 2007).
In addition, Bennet and Bennet (2008) also stressed on the emotional aspect of the learners and the
learning experience that they are going through. According to these researchers, “ideally good
eLearning would facilitate an experience with the learner that creates emotional tags, thus enhancing
the ability to learn from that experience”. From the literature, it is confirmed that a good design of the
online learning material is important to help students learn better.
4. Emotional experiences in web-based application
Several issues of concern exist with regard to the user’s emotional experience in the web-based
environment. Although the area of emotional aspects of the user experience is a neglected area of
research (Thuring and Mahlke 2007), it is now an area of concern within the Human Computer
Interaction (HCI) community.
Agarwal and Meyer (2009) and Agarwal and Prabaker (2009) demonstrated that usability metrics
alone (e.g., time on task, number of errors) cannot sufficiently measure the overall user experience of
the product or interface. According to their research, emotional responses may also play a significant
role in measuring the user’s experience, as well as in judging the usability of the product or interface.
Whilst some research on emotion and online learning exists, there is still a lack of research on the
design of the learning material in an online environment, which could capture or engage students’
attention and emotion. Most literature is based on the general emotional experience of the learners
during the online course. Little attention is placed particularly on the emotional response to the design
of the learning materials. Other related researches in this area such as by Zembylas, Theodorou and
Pavlakis (2008), focused on the general experience of the learners in the online environment and the
emotional experience of the students going through the online study. Zembylas’ research does not
specifically concentrate on the emotional response to the design of the learning material. Additionally,
MacFadden (2005) proposed an emotionally oriented model for web-based education based on a
constructivist approach. This model consists of four stages, namely; safety, challenge, new thinking,
and consolidation. This proposed model is for general emotions within web-based learning.
It is evident from the literature that even though emotion is now an emphasised issue in web-based
applications, there is still no specific model for emotional experience evoked from the design of the
learning material in an online environment.
5. Kansei Engineering
There are many definitions of Kansei. According to Professor Mitsuo Nagamachi, the founder of
Kansei Engineering (KE), Kansei “implies psychological feeling and needs in mind” (Nagamachi
2008). Kansei also “refers to the state of mind where knowledge, emotion and passion are
harmonized” (Nagamachi and Lokman 2011: 5). The idea of KE has been around since 1970. In KE,
the aim is to develop a product that people desire for or have deeply in mind. As described by
Lokman (2011), KE provides a systematic way of understanding the insights of user perceptions
toward artefacts via several physiological and psychological measurement methods. These insights
are then translated into the design characteristics of the artefact. KE is a very important philosophy,
as it is not only customer/user oriented, but it also promotes the value of emotion or aesthetics in the
product design, as perceived by the customer/user.
Many researchers use KE, but few do so in the field of education. The philosophy of KE has only quite
recently permeated into educational systems (Sandanayake and Madurapperuma 2009). Several
studies have linked KE to online learning. Tharangie et al (2008) used KE to enhance eLearning web
interfaces, focusing on the study of colour. On the other hand, Chaminda et al (2009) proposed and
implemented an interactive eLearning system using KE. Levels of knowledge and emotion were
analysed using biometrics signals to analyse the emotions, which were then mapped against the
knowledge and emotional levels of the Kansei model (Chaminda et al 2009). In another study,
Sandanayake and Madurapperuma proposed a conceptual model for eLearning using KE techniques.
Specifically, they used a software agent capable of recognising and responding to the learner’s
660

Fauziah Redzuan et al.
emotional state during a period of learning (Sandanayake and Madurapperuma 2009). Chen, Chuah
and Teh (2010) also attempted to use KE in their study on instructional design.
Many methods of KE exist, namely, KE Type I and Type II, Category Classification Method, KE
System, Virtual KE System, and many more. For the purpose of this research, we selected KE Type II
as being the method most suitable for this area of study, as identified by its research design.
6. Proposed model
This paper reports a brief description of the proposed research model. The chosen model is based on
the principles of the Aptum model, highlighted by Hasle (2006) and interaction design. The Aptum
model was chosen as it consists of elements that are strongly proposed for effective communication
and is related to emotion. The interaction design also is included in the model, as from the literature it
is an important element in learning. The Aptum model consists of the elements of the Orator (the
speaker or instructor), the Scena (the audience or the receiver), the Res (the content), the Verba (the
style or the form of presentation or interface), and the Situatio (the context). According to Hasle (2006)
the Aptum model emphasises the balance between the elements, to achieve the apt. The more the
balance, the better is the apt.
However, the Aptum model is a general model, and adaptation is needed in order to discover the
knowledge of how a specific combination of design elements affects emotional experience in
eLearning environment. To enable this, KE methodology was adopted, as it has been successfully
proven in discovering relations between emotion and design.
Therefore, we argue that, based on the principles of Aptum model and interaction design, there is not
one element that affects the positive emotional experience in learning, but rather a combination of
elements that results in a positive emotional experience of the learners. The important elements
extracted from the Aptum model are the interface, content, and context. For this experiment, the
emotional experience based on the combination of interface, content, and interaction design were
being evaluated.
As far as the authors are aware, no other researchers have yet applied the principles of the Aptum
model, interaction design, in combination with KE, to extract the design elements based on the
emotional experience of the learners in online learning. For interested readers, who would like to read
in detail about the proposed model, please refer to Redzuan et al (in press).
7. Methodology
To proceed with the research, we took several steps based on the KE technique. The KE Type II
method based on (Nagamachi and Lokman 2011), is followed.
Steps were adopted from the KE Type II (Phase 1), but were adapted specifically at the Extracting
Item/Category step. Figure 1 shows in detail all of the steps adopted in this study using the KE Type II
method. Section 7 (Methodology) and Section 8 (Analysis and Discussion) of the paper are related to
Figure 1.
7.1 Selecting survey target
During the survey target selection phase, researchers began with the general problem, identified from
the literature and then focused on the specific problem of the design of the learning material for
courses in an eLearning environment. A short survey was administered to undergraduate university
students in Information Technology and Network, at the Faculty of Computer and Mathematical
Sciences to which the first author of this paper is affiliated. The survey was conducted in order to
confirm that the problem in the design of learning materials is still relevant. Initially, 45 students were
selected and surveyed to ascertain their opinions on which of five courses needed support for
eLearning. The results are shown in Table 1.
A comparison was made between the two groups. Both groups valued eLearning as important to
course code ITS472 (Database course) so that they would be able to study at their own pace and
time.
661

Fauziah Redzuan et al.
A second survey was subsequently administered, in order to understand the state of emotion
experienced by students, particularly in response to the online course on the i-Learn system. The i-
Learn system is an eLearning environment developed to allow lecturers to share online learning
material with students in the first author’s university. In this survey, 106 students were given several
open-ended questions related to the current online learning system that they were experiencing.
Analysis of the second surveys findings revealed that, even though many students responded that
they were satisfied, at the same time, many online learning materials in their courses still evoked
negative emotions.
Phase 1
Phase 2
Selecting Survey
Target
Extracting Low Level
Kansei Words
Primary Evaluation
Experiment
Statistical Analysis
of Kansei Words
Extracting High
Level Kansei Words
Secondary Evaluation
Experiment
Configuration
of SD Scale
(1)
Configuration
of SD Scale
(2)
Preparation of
Evaluation Target (1)
Preparation of
Evaluation Target (2)
Extracting
Item/Category
Figure 1: Research methodology steps using KE Type II method adapted from (Nagamachi and
Lokman 2011: 50)
Table 1: Students’ feedback regarding the importance of eLearning for the specific courses offered
IT Students (30) Network Students (15)
Course Code Numbers Course Code Numbers
ITS572 18/30 ITT5 50
10/15
ITS472 16/30 ITS472 9/15
ITS410 13/30 ITT450 9/15
ITS521 10/30 ITT460 8/15
ITS544 9/30 ITT510 7/15
7.2 Extracting low-level Kansei words
In this step, 511 Kansei words considered to be related to online learning, as well as the psychology
of learning, were collected from 83 journals and proceedings. The optimum number of Kansei words,
according to Nagamachi and Lokman (2011), is between 600 and 800, but 300 or 400 words are also
sufficient to complete the primary evaluation experiment. Kansei words were usually related to users’
662

Fauziah Redzuan et al.
emotions and opinions. They include adjectives, nouns, verbs, and sentences (Nagamachi 2011).
Kansei words are best understood in their adjective form (Nagamachi and Lokman 2011). Therefore,
all the 511 Kansei words were checked properly for their adjective and noun forms, and then
submitted
to the expert for validation.
The first expert, Expert (A) is a linguistic with 20 years experience in the field. Expert (A) validated and
checked whether all of the words were in their proper adjective and noun form. Expert (A) also
suggested several more words related to learning and emotions. The second expert, Expert (B) is an
educational psychologist who has been involved in the area for more than 10 years. Expert (B)
validated the Kansei words and judged whether each was related to online learning or general
learning. The third expert, Expert (C) is a team of eLearning designers and instructional designers
who have been involved in this area for more than four years. From the designers’ team, most of the
negative words were eliminated and suggested as unsuitable for eLearning. This was justified by the
designers’ expertise in the area that suggest eLearning to induce positive emotions to the user.
However, the designers also agreed that sometimes negative emotion could be generated from
eLearning materials. Therefore, negative emotion is also a valid construct to measure designs that
may evoke a negative emotional
experience. Finally, from all of the experts’ validations, a set of 478
Kansei words were concluded.
7.3 Configuration of SD scale
The Semantic Differential (SD) scale was then developed for the 478 Kansei words. According to
Nagamachi and Lokman (2011), a 5-level rating has been widely used for SD scales. They suggested
avoiding too many levels, as more levels make respondents more confused. In addition, according to
Nagamachi (in Lokman and Nagamachi 2010), the 5-degree scale is better than other scales, such as
the 7-, 9- or 11-degree scales, as it is best suited to the style of human judgment. Therefore, we used
a 5-level rating for this research.
7.4 Primary evaluation experiment
The primary evaluation experiment was conducted using respondents comprising 36 undergraduate
students in the area of Information Technology. The selection of respondents was based on the
subject (i.e., the students or the respondents) selection procedure. A purposive sampling technique
was used to select the students. Selection criteria included firstly, that the students were studying in
the area of Information Technology; secondly, that they were currently enrolled or had taken the
database course, and thirdly, that they had the ability to express their emotions. Initially, around 100
students took a short survey to test their ability to express their emotions. Following that test, only 55
students were selected and invited to complete the primary evaluation
experiment. Of these, only 36
students
accepted the invitation and completed the experiment.
As for the specimens (i.e., the online database courses), ten specimens were selected based on the
criteria that the design elements, such as colour, typography, layout, content, and interaction of each
differed from one another. The specimens are available online. Each specimen was assigned a
number
(Specimen No.) by the researcher.
The primary evaluation experiment conducted in a computer laboratory, took 6.5 hours. There were
breaks every two hours to help respondents to refresh their mind. Each specimen was evaluated
using the 478 Kansei words. Each respondent was assigned a number (Subject No.) by the
researcher and a computer with an Internet connection was used to do the experiment. The
respondents interacted with the actual specimens online and the suggested time for the actual
evaluation for each specimen was between 20 to 30 minutes. In addition, to minimize the effects of
fatigue, boredom, or bias, the 478 Kansei words were arranged into two different versions. To further
help the respondents verify the meaning of the Kansei words, Malay translations
were provided, even
though the medium of the instructions of the respondents’ university is English.
8. Analysis and discussion
8.1 Statistical analysis
Some data cleaning was performed on the collected data, and the average evaluation values were
calculated. The average evaluation data was then analysed using JMP software, in order to
663

Fauziah Redzuan et al.
understand the correlation of Kansei words, as well as to calculate factor analysis, in order to
understand the significant factors that were formed from the evaluation results. In addition, based on
the
average score, specimens that fitted best to Kansei were identified.
Figure 2 shows some examples of the average evaluation results based on the 35 subjects’
evaluations. Correlation analyses, based on a pairwise correlation method were performed on the
data, in order to understand the strength of the correlation between two or more Kansei words.
Correlation Coefficient Analysis (CCA) is the measure of strength between variables. CCA is
important to identify smaller, but precise sets of correlated Kansei words, from the larger set of
emotion words (Lokman and Ibrahim
2010). Figure 3 depicts the correlation matrix of the Kansei
words with their correlation values.
Subject No
Specimen No
Average Data 2
KW
1 2 3 4 5 6 7 8 9 10
1 Relaxed 3.028571 2.685714 4.142857 3.457143 3.647059 3.857143 3.457143 3.8 2.828571 3.942857
2 Intimidated 2.285714 2.742857 2.441176 2.828571 2.852941 2.885714 2.8 3.2 2.885714 3.257143
3 Happy 2.657143 2.742857 4.085714 3.171429 3.676471 3.514286 3.342857 3.771429 2.885714 3.485714
4 Merry 2.314286 2.514286 3.914286 3.114286 3.294118 3.142857 3.142857 3.6 2.657143 3.457143
5 Unpleasant 2.742857 2.705882 2.257143 2.657143 2.441176 2.457143 2.771429 2.6 3 2.771429
6 Leisurely 2.857143 2.828571 3.742857 3.028571 3.333333 3.428571 3.4 3.342857 3.114286 3.314286
7 Soothing 3.085714 3.085714 3.529412 2.914286 2.970588 3.085714 3.2 3.114286 3.142857 3.228571
8 Unsafe 2.371429 2.514286 2.085714 2.257143 2.176471 2.428571 2.371429 2.2 2.628571 2.428571
9 Sensational 2.514286 2.857143 3.571429 3 3.235294 3.257143 3.028571 3.628571 2.714286 3.257143
10 Surprised 2.542857 2.714286 3.285714 2.771429 2.852941 3.085714 3.257143 3.514286 2.742857 3.028571
11 Unwise 2.571429 2.342857 2.228571 2.514286 2.235294 2.457143 2.205882 2.485714 2.885714 2.428571
12 Heavy 3.171429 3.147059 2 2.8 2.705882 2.828571 3 3 3.342857 2.828571
13 Learnable 4.352941 3.828571 4.285714 3.942857 4.264706 4.314286 3.971429 4.228571 3.714286 3.971429
14 Solemn 3.764706 3.114286 2.676471 3.371429 3.058824 3.114286 3 3.114286 2.857143 3
15 Touched 2.323529 2.514286 3 2.942857 2.823529 2.857143 2.914286 3.314286 2.857143 3
16 Restless 2.882353 2.8 2.542857 2.828571 2.852941 2.685714 2.828571 3.028571 2.6 2.657143
17 Proud 3 2.914286 3.8 3.057143 3.294118 3.228571 3.342857 3.542857 2.885714 3.4
18 Mad 2.058824 2.428571 1.714286 2.142857 2.029412 2 2.142857 2.057143 2.514286 2.085714
19 Pleasant 2.939394 3.257143 3.8 3.342857 3.529412 3.2 3.285714 3.514286 3.171429 3.257143
20 Unwilling 2.606061 2.657143 1.914286 2.714286 2.617647 2.428571 2.628571 2.314286 2.914286 2.342857
21 Neutral 3.617647 3.085714 3.514286 3.514286 3.470588 3.771429 3.828571 3.742857 3.542857 3.714286
22 Irritated 2.647059 3.028571 2.2 2.742857 2.205882 2.371429 2.514286 2.6 3.028571 2.457143
23 Curious 3.441176 3.485714 3.514286 3.514286 3.794118 3.4 3.371429 3.285714 3.428571 3.485714
24 Despairing 2.764706 2.771429 2.428571 2.771429 2.588235 2.742857 2.771429 2.771429 3.114286 2.771429
25 Competent 3.176471 3.257143 3.457143 3.314286 3.323529 3.371429 3.314286 3.142857 3.314286 3.485714
26 Unique 2.647059 2.971429 3.971429 2.8 3.205882 3.514286 3.4 4.085714 2.8 3.542857
27 Involved 3.411765 3.257143 3.971429 3.457143 3.647059 3.542857 3.6 3.6 3.057143 3.657143
28 Confused 2.970588 3.114286 2.114286 2.885714 2.088235 2.257143 2.857143 2.685714 2.828571 2.571429
29 Inefficient 2.617647 2.857143 2.205882 2.6 1.911765 2.2 2.685714 2.342857 3.057143 2.114286
30 Inconvenient 2.588235 2.885714 2 2.628571 2 2.114286 2.685714 2.514286 2.828571 2.171429
31 Unresponsive 2.441176 2.828571 2 2.714286 2.176471 2.257143 2.485714 2.314286 3.028571 2.4
32 Desirable 2.941176 3.342857 3.882353 3.142857 3.588235 3.457143 3.371429 3.314286 3.2 3.457143
33 Perfect 2.735294 2.857143 3.735294 3.2 3.823529 3.657143 3.257143 3.742857 2.685714 3.714286
34 Uncooperative 2.588235 2.6 2.058824 2.571429 2.323529 2.090909 2.4 2.485714 3 2.257143
35 Ready 3.588235 3.285714 3.852941 3.428571 3.823529 3.714286 3.371429 3.8 3 3.771429
36 Unconscious 2.382353 2.6 2.176471 2.714286 2.5 2.4 2.628571 2.342857 3.028571 2.542857
Figure 2: Example of the calculated average values for each specimen based on the Kansei words
A correlation value above 0.8 was considered strong; therefore, some strong correlations did exist
between specific Kansei words. This detail is important to understand, in order to extract the highlevel
Kansei words, or in other words, to identify the most significant Kansei words. Words in the
correlation matrix with a correlation value of 0.8 and above were extracted and further grouped
together, based on each of the Kansei words. This finding serves as the basis for identifying the highlevel
Kansei words necessary to reduce the number of Kansei words from 478 to between 50 and
around 100 words, for the secondary experiment, as was suggested by (Nagamachi and Lokman
2011).
664

Fauziah Redzuan et al.
Relaxed IntimidatedHappy Merry Unpleasant Leisurely Soothing Unsafe Sensationa Surprised Unwise Heavy Learnable Solemn Touched
Relaxed 1 0.277989 0.919003 0.913603 ‐0.688922184 0.855807 0.420139 ‐0.68612 0.856116 0.717382 ‐0.47207 ‐0.77504 0.567165 ‐0.3697 0.673719
Intimidated 0.277989 1 0.280317 0.365044 0.232271392 0.159982 ‐0.21995 0.158378 0.43857 0.421073 0.10845 0.203714 ‐0.32602 ‐0.31614 0.68356
Happy 0.919003 0.280317 1 0.970428 ‐0.741837874 0.92362 0.471634 ‐0.74222 0.946186 0.807961 ‐0.52832 ‐0.79873 0.478218 ‐0.59406 0.76010
Merry 0.913603 0.365044 0.970428 1 ‐0.63848158 0.881699 0.49791 ‐0.70881 0.944758 0.818477 ‐0.48556 ‐0.78703 0.328836 ‐0.6115 0.83185
Unpleasant ‐0.68892 0.232271 ‐0.74184 ‐0.63848 1 ‐0.62453 ‐0.305 0.798943 ‐0.68654 ‐0.41398 0.653881 0.856979 ‐0.72939 0.228529 ‐0.22499
Leisurely 0.855807 0.159982 0.92362 0.881699 ‐0.624530168 1 0.644742 ‐0.54919 0.807851 0.790641 ‐0.45967 ‐0.75443 0.395862 ‐0.70251 0.67153
Soothing 0.420139 ‐0.21995 0.471634 0.49791 ‐0.305000046 0.644742 1 ‐0.21631 0.401793 0.490385 ‐0.26803 ‐0.5811 0.131816 ‐0.6162 0.2292
Unsafe ‐0.68612 0.158378 ‐0.74222 ‐0.70881 0.798943014 ‐0.54919 ‐0.21631 1 ‐0.66727 ‐0.48051 0.602464 0.77435 ‐0.6617 0.072983 ‐0.4032
Sensational 0.856116 0.43857 0.946186 0.944758 ‐0.686537646 0.807851 0.401793 ‐0.66727 1 0.863711 ‐0.48777 ‐0.68619 0.389566 ‐0.56568 0.818278
Surprised 0.717382 0.421073 0.807961 0.818477 ‐0.413975128 0.790641 0.490385 ‐0.48051 0.863711 1 ‐0.40956 ‐0.44243 0.272106 ‐0.54417 0.828878
Unwise ‐0.47207 0.10845 ‐0.52832 ‐0.48556 0.653880984 ‐0.45967 ‐0.26803 0.602464 ‐0.48777 ‐0.40956 1 0.609961 ‐0.37405 0.22565 ‐0.0962
Heavy ‐0.77504 0.203714 ‐0.79873 ‐0.78703 0.856979334 ‐0.75443 ‐0.5811 0.77435 ‐0.68619 ‐0.44243 0.609961 1 ‐0.47921 0.45027 ‐0.3555
Learnable 0.567165 ‐0.32602 0.478218 0.328836 ‐0.729386979 0.395862 0.131816 ‐0.6617 0.389566 0.272106 ‐0.37405 ‐0.47921 1 0.268167 ‐0.0029
Solemn ‐0.3697 ‐0.31614 ‐0.59406 ‐0.6115 0.228528517 ‐0.70251 ‐0.6162 0.072983 ‐0.56568 ‐0.54417 0.22565 0.45027 0.268167 1 ‐0.5696
Touched 0.673719 0.683563 0.760106 0.831852 ‐0.224990089 0.671537 0.22927 ‐0.40321 0.818278 0.828878 ‐0.09626 ‐0.35553 ‐0.00293 ‐0.56965
Restless ‐0.16029 0.148854 ‐0.14259 ‐0.16803 0.077906585 ‐0.37851 ‐0.59832 ‐0.24013 ‐0.01654 0.076461 ‐0.11315 0.392862 0.233591 0.599819 0.013889
Proud 0.885369 0.129905 0.927823 0.925256 ‐0.681181685 0.881997 0.667222 ‐0.75339 0.880761 0.837473 ‐0.58371 ‐0.79508 0.517417 ‐0.5054 0.658228
Mad ‐0.84963 0.234159 ‐0.75882 ‐0.70189 0.832847362 ‐0.70407 ‐0.45768 0.832156 ‐0.63962 ‐0.49963 0.570719 0.842079 ‐0.82655 0.082251 ‐0.29074
Pleasant 0.629381 0.100715 0.849727 0.849513 ‐0.714626213 0.723715 0.461728 ‐0.76136 0.826238 0.63234 ‐0.54603 ‐0.80494 0.229567 ‐0.66257 0.63449
Unwilling ‐0.79733 0.054228 ‐0.77514 ‐0.77199 0.745317547 ‐0.72599 ‐0.73518 0.65477 ‐0.77547 ‐0.65938 0.514784 0.816035 ‐0.5862 0.371597 ‐0.4130
Neutral 0.549048 0.290537 0.373532 0.364045 0.030873029 0.492378 0.138929 ‐0.13073 0.282113 0.532207 0.047996 ‐0.03322 0.344631 0.014601 0.4709
Irritated ‐0.86053 0.042983 ‐0.80729 ‐0.71072 0.774419672 ‐0.78131 ‐0.34278 0.725547 ‐0.64159 ‐0.48324 0.643877 0.764573 ‐0.75116 0.229688 ‐0.3187
Curious 0.057499 ‐0.16949 0.132086 0.062295 ‐0.347859938 0.018285 ‐0.21356 ‐0.30167 ‐0.02269 ‐0.40962 ‐0.33777 ‐0.34753 0.114331 ‐0.06929 ‐0.24004
Despairing ‐0.64137 0.3248 ‐0.65855 ‐0.5968 0.908218601 ‐0.54151 ‐0.38911 0.836539 ‐0.57982 ‐0.34429 0.82704 0.867866 ‐0.67325 0.146022 ‐0.0948
Competent 0.521993 0.132205 0.438541 0.490882 ‐0.271125066 0.565743 0.499659 ‐0.04938 0.314399 0.136668 ‐0.26959 ‐0.60274 ‐0.1038 ‐0.604 0.22519
Unique 0.800092 0.376524 0.877162 0.870511 ‐0.573255585 0.822538 0.584127 ‐0.54017 0.937051 0.943056 ‐0.46373 ‐0.58422 0.399078 ‐0.57311 0.75420
Involved 0.890088 ‐0.04486 0.855295 0.836568 ‐0.794170197 0.789216 0.539138 ‐0.83092 0.76348 0.638081 ‐0.74249 ‐0.87892 0.639748 ‐0.30138 0.435599
Confused ‐0.77513 ‐0.0616 ‐0.82347 ‐0.70539 0.769562789 ‐0.80692 ‐0.33191 0.567627 ‐0.67438 ‐0.41104 0.399499 0.745058 ‐0.58595 0.48827 ‐0.391
Inefficient ‐0.83556 ‐0.22284 ‐0.76285 ‐0.68804 0.732304501 ‐0.60856 ‐0.12138 0.663885 ‐0.69282 ‐0.39847 0.551359 0.65285 ‐0.69281 0.146928 ‐0.3579
Inconvenient ‐0.84464 ‐0.05521 ‐0.78061 ‐0.69101 0.771028301 ‐0.7259 ‐0.31996 0.604501 ‐0.64716 ‐0.3551 0.46481 0.765184 ‐0.68128 0.294999 ‐0.3061
Unresponsive ‐0.83834 0.109319 ‐0.80006 ‐0.69858 0.844823365 ‐0.73129 ‐0.41274 0.784536 ‐0.69835 ‐0.55204 0.650542 0.761681 ‐0.86232 0.155078 ‐0.2930
Desirable 0.669402 0.079044 0.818594 0.773812 ‐0.723191521 0.829049 0.627146 ‐0.49734 0.749299 0.55762 ‐0.63994 ‐0.82064 0.249574 ‐0.7961 0.43266
Perfect 0.922006 0.434428 0.925741 0.888976 ‐0.715446817 0.784706 0.236625 ‐0.67541 0.909856 0.704073 ‐0.57491 ‐0.68487 0.525621 ‐0.41295 0.67689
Uncooperative ‐0.83859 0.002975 ‐0.73949 ‐0.67963 0.817064561 ‐0.71487 ‐0.41637 0.587716 ‐0.68253 ‐0.52316 0.734637 0.780916 ‐0.6546 0.251541 ‐0.2659
Ready 0.833299 0.084437 0.745731 0.682108 ‐0.777142457 0.571952 0.243202 ‐0.7602 0.717838 0.484618 ‐0.55577 ‐0.67179 0.821616 ‐0.02223 0.33174
Unconscious ‐0.63574 0.264601 ‐0.58039 ‐0.50032 0.805373529 ‐0.47223 ‐0.42258 0.71043 ‐0.58121 ‐0.46367 0.594868 0.647677 ‐0.86025 ‐0.0532 ‐0.0991
Preoccupied 0.671904 0.282972 0.502478 0.479952 ‐0.327323564 0.634529 0.437552 ‐0.0415 0.493757 0.565184 ‐0.35573 ‐0.35403 0.331293 ‐0.31563 0.322378
Anxious ‐0.04323 0.332347 ‐0.03909 0.044142 0.414664685 ‐0.0373 0.270919 0.166513 0.123043 0.434026 0.285348 0.367518 ‐0.1119 0.002163 0.31722
Oblivious 0.010878 0.742183 0.08576 0.216914 0.494979007 0.100163 0.032915 0.349927 0.129424 0.133396 0.312483 0.194555 ‐0.70351 ‐0.54614 0.509879
Unexplainable ‐0.80612 0.158315 ‐0.83721 ‐0.76097 0.923303052 ‐0.71193 ‐0.43855 0.898419 ‐0.77336 ‐0.5771 0.610202 0.853134 ‐0.80186 0.214284 ‐0.3767
Just 0.40681 0.414965 0.454512 0.483408 ‐0.4414735 0.09974 ‐0.30912 ‐0.59549 0.537132 0.161019 ‐0.36183 ‐0.35945 0.199695 ‐0.00995 0.39025
Uncomfortable ‐0.9329 ‐0.19241 ‐0.90546 ‐0.88994 0.754108265 ‐0.78545 ‐0.4863 0.755336 ‐0.85513 ‐0.6328 0.566164 0.824053 ‐0.60333 0.379863 ‐0.5520
Creative 0.88934 0.448969 0.931366 0.948854 ‐0.632749646 0.813647 0.483289 ‐0.63341 0.981818 0.873532 ‐0.49969 ‐0.68223 0.390635 ‐0.54499 0.797854
Dumb ‐0.87923 ‐0.02415 ‐0.67705 ‐0.65822 0.712843933 ‐0.61048 ‐0.20405 0.648544 ‐0.58703 ‐0.39743 0.504704 0.737109 ‐0.66826 0.008138 ‐0.3260
T bl d 0 88572 0 40544 0 8967 0 82784 0 632736557 0 8667 0 2244 0 548407 0 82504 0 7334 0 504841 0 598554 0 48618 0 473498 0 68318
Figure 3: Partial view of the correlation matrix results of the Kansei words
8.2 Extracting high-level Kansei words
In a KE study, one of the most important analyses is multivariate analysis. According to Nagamachi
and Lokman (2011), “multivariate analysis is a technique used to clarify the impact of variables that
affect the characteristic values as a phenomenon”. Multivariate analysis is useful for identifying factors
that might lead to certain outcomes. In this study, correlation coefficient analysis and factor analysis
are performed to understand the data better.
Based on the correlation analysis, we identified groups of highly correlated Kansei words. For
example, the Kansei word ‘Enchanting’ is highly correlated with the words ‘Fun’, ‘Merry’, ‘Happy’ and
‘Wonderful’, with correlation values of more than 0.971.The Kansei word ‘Interested’ is highly
correlated with the words ‘Efficient’, ‘Satisfied’, ‘Admirable’, and ‘Understandable’, with correlation
values of more than 0.953. The correlation analysis revealed that not only was positive emotional
keywords highly associated with each other, but that negative emotional keywords were as well. For
example, ‘Unhappy’ is highly correlated with ‘Inactive’, ‘Lonely’, ‘Upset’, ‘Lost Interest’, ‘Tired’, and
‘Frustrated’, with values of more than 0.952. ‘Mad’ is highly correlated with ‘Unresponsive’,
‘Unexplainable’, ‘Discardable’, ‘Faulty’, ‘Disappointed’, and ‘Stupid’, with correlation values of more
than 0.908.
Some of the dimensions for positive emotions included ‘Enchanting’, ‘Interested’, ‘Concentrated’,
‘Harmonious’, and ‘Stimulating’, and those for negative emotions included ‘Unhappy’, ‘Mad’, and
‘Hated’. It can be concluded that the emotional responses from the perspective of the students
towards the online database courses, can be represented by the 130 significant emotional
dimensions. Therefore, it can be concluded that the design of the online database course should fall
within these dimensions.
Figure 4 depicts the partial results of the performed factor analysis. The factor analysis was performed
using varimax rotation. From this first experiment, shown in Table 2, we see that Factor 1 explains
47% of the data, and the accumulated percentage, to explain the data from Factor 1 to Factor 4 is
77%. Factors 5 to 8 contribute smaller percentages, of approximately 4 to 5% for each of the factors,
665

Fauziah Redzuan et al.
to explain the data. Looking in detail at Figure 4, Factor 1, Factor 3, and Factor 5, consist of positive
emotional responses. However, Factors 2, 4, 6, and 7, consist of negative emotional responses. From
these findings, we can conclude that the specimens used in this experiment evoke positive, as well as
negative emotions. We would like to explore further, which of the specimens evoked positive emotions
and which of the specimens evoked negative emotions.
Table 2: Variance analysis
Factor Variance Percent Cum Percent
Factor 1 225.271 47.128 47.128
Factor 2 77.507 16.215 63.343
Factor 3 38.082 7.967 71.310
Factor 4 28.629 5.989 77.299
Factor 5 25.111 5.253 82.552
Factor 6 22.709 4.751 87.303
Factor 7 21.243 4.444 91.747
Factor 8 20.535 4.296 96.043
Factor 9 18.913 3.957 100.000
Factor 10 0.000 0.000 100.000
Row Factor 1 Row Factor 2 Row Factor 3 Row Factor 4 Row Factor 5 Row Factor 6 R
Enchanting 0.98488107 Furious 0.956395 Overwhelming 0.8203534 Time Consuming 0.954044013 Responsible 0.87841228 Hysterical 0.834266138 D
Fun 0.984120404 Irrational 0.946843 Expert 0.797115466 Anxious 0.937249876 Neutral 0.692149833 Jittery 0.790303391 D
Gay 0.983947942 Indignant 0.928412 Frenzied 0.767302271 Pathetic 0.797806291 Compliant 0.653910994 Ebullient 0.721031853 S
Happy 0.981556799 Anomalous 0.913394 Professional 0.746310898 Deceptive 0.727372572 Preoccupied 0.605250808 Longing 0.675167366 S
Lovable 0.979917393 Enraged 0.901949 Challenging 0.73245983 Scared 0.663154037 Narrow 0.594036061 Triumphant 0.652038054 U
Interesting 0.978678142 Impetuous 0.880886 Feeble-Minded 0.725517832 Insensitive 0.647008061 Emphatic 0.566388662 Greedy 0.590046375 C
Joyful 0.978480037 Shocked 0.867651 Mature 0.714109204 Distracted 0.60633301 Concentrated 0.538987537 Simple 0.561065677 A
Wonderful 0.976800965 Ambivalent 0.856998 Restless 0.663129242 Unresolved 0.563311822 Authentic 0.53265586 Unadulterated 0.533338315 N
Calm 0.976674315 Melancholic 0.855691 Wealthy 0.621129915 Alarmed 0.559887379 Sympathetic 0.528968428 Placid 0.526828717 D
Merry 0.975463459 Hostile 0.841331 Jealous 0.580731704 Dramatic 0.55567322 Light 0.528592194 Sympathetic 0.524859121 U
Enjoyable 0.974353959 Contradictory 0.839706 Bewildered 0.569265777 Disgusted 0.543325333 Bright 0.512471894 Ingenious 0.52135029 U
Delighted 0.97414408 Humiliated 0.83092 Inconsolable 0.566227513 Guilty 0.529889045 Actual 0.511138413 Envious 0.520576619 J
Youthful 0.971088339 Distressed 0.821598 Arbitrary 0.564773492 Comprehensible 0.521364736 Sober 0.49658674 Dignified 0.511870441 I
Amused 0.965431088 Avaricious 0.807801 Just 0.55494958 Unenlightened 0.505149486 Learnable 0.478227907 Isolated 0.465398531 L
Glad 0.964130141 Resentful 0.800197 Actual 0.549978307 Wide Awake 0.504420578 Ingenious 0.472511964 Neutral 0.453423059 Y
Creative 0.964115453 Insulting 0.787234 Integrated 0.542836668 Agitated 0.500672657 Insightful 0.471260729 Perceivable 0.435047631 T
Sensational 0.961800244 Grieving 0.787105 Flowing 0.537153735 Optimistic 0.500309376 Simple 0.461699782 Sorrowful 0.429286494 G
Fascinated 0.960100098 Terrified 0.77193 Content 0.536452052 Ingenious 0.499610253 Wandering 0.458460021 Epiphanic 0.423423747 S
Nice 0.959305568 Vulnerable 0.764008 Stable 0.534509947 Isolated 0.498500946 Altruistic 0.454634773 Vigorous 0.415107241 Q
Aroused 0.957065025 Dreadful 0.756239 Self-Regulated 0.532120488 Unthinking 0.494847871 Threatening 0.454222238 Competent 0.409836121 S
Proud 0.955757294 Chuckle 0.755121 Valuable 0.516134982 Impulsive 0.491697841 Expected 0.45041581 Threatening 0.398788884 S
Entertaining 0.955226079 Mournful 0.749116 Promising 0.515832994 Restless 0.485100503 Easy to Read 0.43981112 Avaricious 0.392544545 I
Excited 0.953399646 Intimidated 0.741233 Resolved 0.50742822 Confused 0.470306466 Thinking 0.43624208 Easy to Read 0.389298767 S
Active 0.951207053 Indifferent 0.732188 Trustable 0.50454291 Altruistic 0.46704963 Intense 0.4320209 Puzzled 0.383484589 P
Serene 0.948914514 Oblivious 0.731889 Solemn 0.501301357 Pressured 0.459956102 Skilled 0.408992956 Oblivious 0.383275801 D
Playful 0.948057396 Contemptible 0.725011 Important 0.493795618 Graceful 0.444447975 Justifiable 0.398327892 Self-Pitying 0.380922015 S
Admirable 0.941842833 Wistful 0.721744 Good 0.491080114 Apprehensive 0.440777763 Appreciative 0.389115107 Pervasive 0.377493499 S
Friendly 0.93852647 Regretful 0.717201 Revolted 0.49027899 Embarrassed 0.428440812 Autonomous 0.386568404 Dispositional 0.375225652 P
Easy 0.934881948 Arbitrary 0.715643 Pleasurable 0.484269961 Grouchy 0.427818921 Appropriate 0.385842901 Romantic 0.370910447 R
Captivating 0.934628462 Aggravating 0.701663 Well 0.479789935 Struggling 0.422910047 Wise 0.378522168 Sincere 0.368070647 H
Feeling 0.931627352 Misconception 0.697312 Self-Confident 0.474999146 Pained 0.412318765 Helpful 0.373398949 Irresponsible 0.34441702 I
Figure 4: Partial view of the factor analysis results
Table 3: Partial view of the Kansei words with the highest specimen’s average
Positive Emotions
Negative
Emotions
Kansei words Highest Average Specimen No Kansei words Highest Average Specimen No Kansei words Highest Average Specimen No
Enchanting 3.5 3 Interested 4.05 3 Professional 4.22 5
Fun 4.08 3 Efficient 3.97 5 Mature 3.94 5
Merry 3.91 3 Satisfied 4 3 Content 4.25 5
Happy 4.08 3 Admirable 3.85 3 Expert 4.2 5
Wonderful 4 3 Understandable 4.17 5 Valuable 4.34 5
Lovable 3.34 3 Preferable 4.17 5
Creative 4.28 8 Likeable 4.14 5
Sensational 3.62 8 Great 3.94 8
Amused 3.58 3 Cool 4.08 8
Delighted 3.6 3 Courageous 3.88 3,5
Enjoyable 4.17 3
Youthful 3.48 3
Joyful 3.8 3
Captivating 3.94 8
Nice 4.11 3
Mad 2.51 9 Unhappy 3.31 9 Heavy 3.34 9
Unexplainable 2.91 9 Inactive 3.08 9 Stressed 3.4 9
Discardable 3 9 Lonely 2.94 9
Faulty 2.71 9 Upset 2.77 9
Disappointed 3.11 9 Lost Interest 3.68 9
Stupid 2.28 9 Tired 3.48 9
Rejected 3.2 9 Frustrated 3 9
Sad 2.48 9 Annoyed 3.2 9
Dissatisfied 3.05 9
666

Fauziah Redzuan et al.
The average data calculated previously were examined in detail. From the correlation analysis results,
some of the words were carefully chosen from each dimension to examine the specimens further.
Table 3 shows the selected Kansei words and the highest average value from the specimens. It is
evident from Table 3 that most of the positive emotions are related to Specimen Nos. 3, 8, and 5. In
contrast, the negative emotions are consistently related to Specimen No. 9. Further examination of
the specimens (as shown in Table 4) reveals the design elements of the identified specimens.
Table 4: Brief design elements based on the selected specimens
Specimen No. Design Dimension Design Elements
3
Interface Simple, not crowded, colour: white background with some
orange colour in the title, simple and organized menu on the
left
Interaction Short quiz/challenge, interaction with learner, learner gives
some input, gets instant feedback on their input
Content Short notes, precise, concise, some pictures, animation,
learner able to explore
5
Interface Simple, colour: white background with green colour in the logo
and some words, simple menu on the left
8
9
Interaction Short exercise, learner can try the exercise, gets instant
results and feedback on the exercise
Content Short notes with examples
Interface Simple, related to example given
Interaction Walkthrough examples
Content Showing pointer and has audio
Interface Simple, colour: white background with blue text, picture of
instructor, no menu, next and back button/links
Interaction No learner interaction
Content Give examples with wordy explanations
It is interesting to note, that a simple interface with some interaction with the learners, as well as short
notes, evokes positive emotions in the learners. However, a simple interface with no learner
interaction and with lengthy explanations evokes negative emotions.
9. Conclusion
The aim of this study was to identify the design elements of learning material in an online course that
could evoke emotional responses from the students. Based on the principles extracted from the
Aptum model, interaction design and using the KE Type II method, the researchers pursued this study
according to the steps outlined in the KE Type II method. The experimental procedure involved ten
online database courses to be used as specimens and 478 Kansei words, with 36 respondents or
subjects. Statistical analysis was performed using correlation and factor analysis, in order to
understand the data better. The results show that there are 130 significant emotional dimensions for
the design of an online database course. The design elements of the specimens for positive and
negative emotional experiences were also identified.
This study reveals that it is possible to measure the emotional experience of the learners when
interacting with online learning material, and to investigate the design elements that evoked emotional
experiences of the learners in an online environment. Further study is to be performed on the second
experiment, to investigate the detail design elements and its combination that contributes to the
evoked emotions, especially on the positive emotional experience of the learners.
References
Agarwal, A. and Meyer, A. (2009) "Beyond Usability: Evaluating Emotional Response as an Integral Part of the
User Experience", Paper read at the 27th International Conference on Human Factors in Computing
Systems (CHI 2009), Boston, Massachusetts, ACM.
Agarwal, A. and Prabaker, M. (2009) “Building on the Usability Study: Two Explorations on How to Better
Understand an Interface”, in Jacko, J.A. (ed.) Human-Computer Interaction, Part I, Lecture Notes in
Computer Science (LNCS 5610), Berlin Heidelberg: Springer.
Bennet, A. and Bennet, D. (2008) "ELearning as Energetic Learning", VINE: The Journal of Information and
Knowledge Management Systems, Vol. 38, No. 2, pp 206-220.
Capota, K., van Hout, M. and van der Geest, T. (2007) "Measuring the Emotional Impact of Websites: A Study on
Combining a Dimensional and Discrete Emotion Approach in Measuring Visual Appeal of University
667

Fauziah Redzuan et al.
Websites", Paper read at the International Conference on Designing Pleasurable Products and Interfaces.
Helsinki, Finland, August. ACM.
Chaffar, S. and Frasson, C. (2005) "The Emotional Conditions of Learning", American Association for Artificial
Intelligence.
Chaminda, H.T., Basnayake, A., Madurapperuma, A. and Osano, M. (2009) "An Interactive ELearning System
using Kansei Engineering", Paper read at the International Conference on Biometrics and Kansei
Engineering, IEEE.
Chen, C.J., Chuah, K.M. and Teh, C.S. (2010) “Bridging the Emotional Divide in Instructional Design: A Kansei
Perspective”, in Iskander M. et al (ed.) Technological Developments in Education and Automation: Springer
Clapper, T.C. (2010) "Beyond Knowles: What Those Conducting Simulation Need to Know About Adult Learning
Theory", Clinical Simulation in Nursing, Vol. 6, pp e7-e14.
Domagk, S., Schwartz, R.N. and Plass, J.L. (2010) "Interactivity in Multimedia Learning: An Integrated Model",
Computers in Human Behavior, Vol. 26, pp 1024-1033.
de Jong, T. (2009) “Affective Processes in Learning”, Explorations in Learning and the Brain: Springer.
Fadel, L.M. and Dyson, M.C. (2007) "Enhancing Interactivity in an Online Learning Environment", Paper read at
the 11th IFIP TC 13 International Conference (INTERACT), Rio de Janeiro, Brazil, September.
Fisher, E.A. and Wright V.H. (2010) "Improving Online Course Design Through Usability Testing", MERLOT
Journal of Online Learning and Teaching, Vol. 6, No. 4, pp 228-245.
Hasle, P.F.V. (2006) “The Persuasive Expansion - Rhetoric, Information Architecture and Conceptual Structure”
in Scharfe, H., Hitzler, P. and Ohrstrom P. (ed.), Lecture Notes in Computer Science: Springer.
Kampov-Polevoi, J. (2010) "Framework for Analysis of Online Course Design", ASIS&T Annual Meeting.
Pittsburgh, P.A, U.S.A.
Linnenbrink-Garcia, L. and Pekrun, R. (2011) "Students' Emotions and Academic Engagement: Introduction to
the Special Issue", Contemporary Educational Psychology, Vol. 36, pp 1-3.
Liu, F. (2007) "Personalized Learning Using Adapted Content Modality Design for Science Students", Paper
read at ECCE Conference, London, UK. August.
Lokman, A.M. (2011) “Kansei/Affective Engineering and Web Design”, in Nagamachi, M. (ed.) Kansei/Affective
Engineering, Boca Raton, Florida: CRC Press.
Lokman, A.M. and Nagamachi, M. (2010) Kansei Engineering A Beginners Perspective, University Publication
Centre (UPENA), Seri Kembangan, Selangor.
Lokman, A.M. and Mior, I.E.N. (2010) "The Kansei Semantic Space in Children's Clothing", IEEE.
MacFadden, R.J. (2005) "Souls on Ice: Incorporating Emotion in Web-Based Education", Journal of Technology
in Human Services, Vol. 23, No. 1/2, pp 79-98.
Nagamachi, M. (2008) "Perspectives and the New Trend of Kansei/Affective Engineering", The TQM Journal,
Vol. 20, No. 4, pp 290-298.
Nagamachi, M. (2011) “Kansei/Affective Engineering and History of Kansei/Affective Engineering in the World”, in
Nagamachi, M. (ed.) Kansei/Affective Engineering, Boca Raton, Florida, U.S.A: CRC Press Taylor &
Francis Group.
Nagamachi, M. and Lokman, A.M. (2011) Innovations of Kansei Engineering, CRC Press Taylor & Francis
Group, Boca Raton, Florida, U.S.A.
O'Brien, H.L. and Toms E.G. (2008) "What is User Engagement? A Conceptual Framework for Defining User
Engagement with Technology", Journal of the American Society for Information Science and Technology,
Vol. 59, No.6, pp 938-955.
Rager, K.B. (2009) "I Feel, Therefore I Learn: The Role of Emotion in Self-Directed Learning", New Horizons in
Adult Education and Human Resource Development, Vol. 23, No. 2, pp 22-33.
Redzuan, F., Lokman, A.M., Othman, Z.A. and Abdullah, S. (in press) “Kansei Design Model for Engagement in
Online Learning: A Proposed Model”, Paper to appear in The International Conference on Informatics
Engineering & Information Science (ICIEIS). Springer.
Sandanayake, T.C. and Madurapperuma A.P. (2009) "Conceptual Model for ELearning Systems using Kansei
Engineering Techniques", Paper read at the International Conference on Biometrics and Kansei
Engineering. IEEE.
Stephenson, J.E., Morris, W.B., Tempest, H.G., Griffin, D.K., Mileham, A. and Payne, A.M. (2007) "The Use of an
ELearning Constructivist Solution in Workplace Learning", Paper read at ECCE Conference, London, UK.
August.
Tharangie, K.G.D, Irfan, C.M.A, Marasinghe, C.A and Yamada, K. (2008) "Kansei Engineering Assessing System
to Enhance the Usability in ELearning Web Interfaces: Color Basis", Paper read at the Workshop Proc. 16th
Inter. Conf. on Computers in Education (ICCE), Workshop on Testing and Assessment, Taipe, Taiwan,
October.
Thuring, M. and Mahlke, S. (2007) "Usability, Aesthetics and Emotions in Human-Technology Interaction",
International Journal of Psychology, Vol. 42, No. 4, pp 253-264.
Um, E.R., Song, H. and Plass J. (2007) "The Effect of Positive Emotions on Multimedia Learning", Paper read at
the World Conference on Educational Multimedia, Hypermedia and Telecommunications (EDMEDIA) 2007,
Vancouver, Canada. AACE.
van Schaik, P. and Ling, J. (2008) "Modelling User Experience with Web Sites: Usability, Hedonic Value, Beauty
and Goodness", Interacting with Computers, Vol. 20, pp 419-432.
van Schaik, P. and Ling, J. (2009) "The Role of Context in Perceptions of the Aesthetics of Web Pages Over
Time", International Journal of Human - Computer Studies, Vol. 67, pp 79-89.
668

Fauziah Redzuan et al.
Vrasidas, C. (2004) "Issues of Pedagogy and Design in ELearning Systems", Paper read at the ACM Symposium
on Applied Computing (SAC2004), Nicosia, Cyprus. ACM.
Wang, C-Y., Ke S-Y., Chuang, H-C, Tseng, H-Y and Chen, G-D (2010) "ELearning System Design with Humor
and Empathy Interaction by Virtual Human to Improve Students' Learning", Paper read at the 18th
International Conference on Computers in Education, Putrajaya, Malaysia. Asia-Pacific Society for
Computers in Education.
Yang, Y. and Cornelius, L.F. (2004) "Students' Perceptions Towards the Quality of Online Education: A
Qualitative Approach", Association for Educational Communication and Technology, Vol. 27, pp 861-877.
Zembylas, M., Theodorou, M. and Pavlakis, A. (2008) "The Role of Emotions in the Experience of Online
Learning: Challenges and Opportunities", Educational Media International, Vol. 45, No. 2, pp 107-117.
669

Changing Teacher Beliefs Through ICT: Comparing a
Blended and Online Teacher Training Program
Bart Rienties 1 , Simon Lygo-Baker 1 , Natasa Brouwer 2 and Danielle Townsend 3
1
University of Surrey, Guildford, UK
2
Universiteit van Amsterdam, Amsterdam, The Netherlands
3
Maastricht University, Maastricht, The Netherlands
b.rienties@surrey.ac.uk
N.Brouwer-Zupancic@uva.nl
s.lygo-baker@surrey.ac.uk
d.townsend@maastrichtuniversity.nl
Abstract: Teachers’ beliefs towards learning, teaching and ICT may have a strong impact on how learning
opportunities are designed and implemented. However, research has shown that providing effective opportunities
for teachers to develop skills and competence with ICT tools in order that they effectively redesign their learning
environments are not straightforward. In this study we examined two teacher training programmes, both
multidisciplinary, that aimed to use and develop understanding of learning and teaching as well as ICT as it
related to practice of teachers. In Study 1, 74 teachers from eight higher education institutes in the Netherlands
participated in an online teacher training module. In Study 2, 31 teachers from one higher education institute in
the UK participated in a blended approach. Data were gathered by using the Teacher Beliefs and Intentions (TBI)
instrument of Norton et al. (2005), in a pre-post test design, in order to measure changes in the participants’
beliefs and intentions towards knowledge transmissions and learning facilitation. The results indicate that the
teacher beliefs and intentions in the online programme have not substantially changed during the module,
although the beliefs towards knowledge transmission were lower in the post-test. In contrast, participants in the
blended programme have significantly increased their beliefs and intention towards learning facilitation. Future
research should assess the long-term impact of blended and online teacher training programmes on students’
learning experiences.
Keywords: teacher beliefs, teacher intentions, comparison of teacher training programs, blended vs. online
learning
1. Introduction
Given recent and wide-spread governments cuts across Europe, academic scholars in higher
education are under increasing pressure to be productive, effective and cost-efficient members of
society (Adcroft, Teckman & Willis 2010; Marginson 2006). That is, scholars are expected to be
excellent researchers, who successfully apply for research grant-funding and deliver high-quality
teaching (Adcroft, Teckman & Willis 2010). As pressure builds, cutting corners in teaching may seem
to be an attractive solution to gain additional time for research, particularly in institutions where
research is perceived of as developing greater prestige for an individual (Marginson 2006; Kinchin,
Lygo-Baker & Hay 2008). At the same time, in both the Netherlands and UK, most new lecturers are
obliged to follow some form of formal professional development training in order to enhance their
teaching skills.
In the last ten years three major developments in higher education have added to the complexity of
the role and increased the (perceived) workload of, and pressures on, academic scholars within their
teaching. First, the type of students entering higher education has significantly changed in the last ten
years. Due to increased mobility the student population has a more diverse cultural and socio-cultural
background (Jindal-Snape 2010; Rienties et al. 2011a). In addition, several researchers have found
that secondary education does not provide the knowledge and skills that universities expect (Brouwer
et al. 2009; Tempelaar, Rienties & Giesbers 2009). As a consequence, teachers need to be able to
adapt and adjust their teaching to address a wider diversity of learners’ needs (Jindal-Snape 2010).
A second important development that challenges the focus on cost-efficiency in higher education is
that research evidence has shown that traditional teacher-centred forms of education, such as
lectures, do not provide an optimal learning experience for all types of learners (Biggs & Tang 2007;
Nicholls 2001). In student-focussed teaching, the role of the teacher changes from a teacher-centred
approach or knowledge transmission orientation to a student-centred or learning facilitation orientation
(Kember & Gow 1994; Biggs & Tang 2007). The implication is that teachers need to have a greater
range of strategies available to them to meet the demands of a diverse group of learners.
670

Bart Rienties et al.
A third important development in higher education is the increased possibilities brought by Information
Communication Technology (ICT) that if used effectively can be a powerful learning experience for
learners (Tempelaar, Rienties & Giesbers 2009; Brouwer et al. 2009). For example, ICT tools like
discussion forums, Wikis, or web-videoconferencing can provide a rich and valuable learning
experience for students to acquire knowledge and transferable skills (Giesbers et al. 2009). A
consequence may be that traditional methods become questioned as expectations change within the
student population in particular (Garrison & Vaughan 2008). In addition, it requires academic staff to
engage with potentially new learning environments and adds to the complexity faced.
Despite an increased understanding of how students learn (Ferla, Valcke & Schuyten 2009; Fyrenius,
Wirell & Silén 2007) and recognition of the effectiveness of student-centred learning facilitated
through ICT, challenges remain. Many innovations in higher education, and ICT in particular, have not
delivered the fundamental changes in higher education that many teachers and researchers hoped for
(Rienties et al. 2011c; Resta & Laferrière 2007; Mishra & Koehler 2005). Recent research has
highlighted that the application of ICT in education does not necessarily lead to improved learning
experiences for students or enhanced learning processes, study performance or retention (Giesbers
et al. 2009; Valcke & Martens 2006). This has been attributed to a lack of organisational embedding
of innovation and ICT in particular (Resta & Laferrière 2007; Rienties et al. 2011c), a lack of
understanding of the essential parameters for effective teaching with ICT (Rienties et al. 2011c;
Mishra & Koehler 2005; Valcke & Martens 2006), and finally a lack of appropriate teachers’ training to
effectively design and implement powerful learning and teaching experiences for students (Lawless &
Pellegrino 2007; Löfström & Nevgi 2008).
Several researchers have suggested that higher education institutions (HEI) should provide adequate
training and support for teachers in order to increase their awareness of the complex interplay
between technology, pedagogy and the cognitive content in their disciplines (Lawless & Pellegrino
2007; Löfström & Nevgi 2008; Rienties & Townsend 2011). In particular, it is important that training
provided to teachers is embedded into their daily practice (Löfström & Nevgi 2008; Lawless &
Pellegrino 2007).
However, research has shown that providing effective training opportunities for teachers to learn how
to effectively redesign education by incorporating ICT is not straightforward (Lawless & Pellegrino
2007). Therefore, in this paper we investigate the impacts of two teacher training programmes
specifically designed to change teachers’ beliefs and intentions towards teaching. In Study 1, 74
teachers from five higher educational institutes in the Netherlands participated in an online module
specifically considering how teachers can integrate a technological pedagogical content knowledge
model (TPACK: Koehler & Mishra 2010; Mishra & Koehler 2005) into their daily practice. In Study 2,
31 teachers from a UK HEI participated in a blended setting, which was based on a broader
educational science perspective, bringing process and content learning together using ICT as a
method to share and exchange knowledge and expertise.
2. Designing a powerful teacher training learning environment
2.1 Teachers’ beliefs and attitudes towards teaching and learning
It is believed that teachers who have a more student-focussed approach to teaching are more likely to
achieve conceptual change amongst students, while teachers who have a more teacher-focused
approach to teaching are more likely to aim at the transmission of information and knowledge to
students (Prosser & Trigwell 1999; Norton et al. 2005). Previous research has found that teachers
adopting a more student-focussed approach to teaching are more likely to stimulate students to adopt
a deep rather than a surface approach to their learning (Gow & Kember 1993; Prosser & Trigwell
1999). In addition, Kember and Gow (1994) and Prosser and Trigwell (1999) found that the ‘culture’ of
an institution, the complex set of factors that bring together the usual sets of actions in each
institution, influences how teachers teach. Institutions where the culture is believed to value teaching,
provide teachers with (relatively) small class sizes, and provide teachers with control over what and
how their topic is taught induce a more student-focussed approach to teaching and learning
(Chickering & Gamson 1987; Prosser et al. 2003; Prosser & Trigwell 1999).
While some believe that institutional culture is not identifiable or useful as a notion, because it is not
readily described (Kogan 1999), others have suggested that it may have a strong influence on how
teachers teach (McNay 1995). Certainly it has been argued that within each institution there are
671

Bart Rienties et al.
substantial differences in how teaching occurs (Norton et al. 2005). Teachers’ beliefs, and how these
impact on action as a consequence, can have a strong impact on how learning opportunities are
designed and implemented (Kember & Gow 1994). Based upon an inventory of 170 teaching staff,
Gow and Kember (1993) distilled five conceptions of teaching which could be located on a continuum
from a totally teacher-centred, content-orientated conception of teaching to a totally student-centred
and learning-orientated conception of teaching. The full continuum reads: teaching as imparting
information; teaching as transmitting structured knowledge; teaching as an interaction between the
teacher and the student; teaching as facilitating understanding on the part of the student; and
teaching as bringing about conceptual change and intellectual development in the student.
Norton et al. (2005) note that there is an apparent ambiguity in “approaches to teaching” used by
Kember and Gow (1994), as teachers’ beliefs towards teaching are not necessarily the same as their
intentions. In a review of literature on teachers’ actions, beliefs and intentions, Norton et al. (2005)
found that teachers have both “ideal” conceptions and “working” conceptions of teaching. Based upon
an adjusted version of Gow and Kember’s inventory to approaches to teaching, 556 respondents from
four UK universities across three broad academic disciplines (arts, science and social science) were
compared in relation to their beliefs and intentions (Norton et al. 2005). Teachers in that particular
study were found to have significantly different beliefs and intentions, indicating that their own ideal
conceptions of teaching differed from those played out in practice. Across the four institutions and
three disciplinary areas these teachers had relatively similar beliefs towards teaching, but significantly
differed in their intentions to teach, in particular interactive teaching, training for jobs and motivating
students (Norton et al. 2005). This implies that while teachers in general have a similar belief of what
good teaching should be, their intentions are substantially different.
While a large number of studies have argued that formal training can enhance teachers’
understanding of their practice (Prosser et al. 2003), no significant relationships were found with
formal training in the study by Norton et al. (2005), which backs up previous research by Gibbs and
Coffey (2004). Therefore, in line with previous research, Norton et al. (2005) argue that genuine
development of teachers’ approaches to teaching come from addressing their underlying conceptions
of teaching and learning (Norton et al. 2005; Prosser & Trigwell 1999; Trigwell & Prosser 2004). An
increasing number of researchers have argued that formal training of teachers should be embedded
in their daily practice, in particular when referring to the integration of technology into teaching
(Löfström & Nevgi 2008; Lawless & Pellegrino 2007). However, to our knowledge only a limited
number of studies have addressed how this integration of daily practice into formal teacher education
can be effectively established. Much of the focus has been upon developing opportunities for
conceptual change through some form of reflective practice (Young 2008). So, rather than engaging
directly within practice, teachers are asked to look at past actions and how these could be modified
within future actions. Our interest was in considering whether ICT could help to harness a more
reflexive approach which enabled teachers to consider how they acted, why and the intentions implied
or enacted as they occurred.
2.2 Comparison of conceptual framework of two programmes
2.2.1 Study 1 Conceptual framework of online programme: MARCH ET
In Study 1, 74 teachers from eight higher educational institutes in the Netherlands volunteered to
participate in an online teacher training program. As a primary learning objective, teachers were
expected to develop a redesign of their own teaching module (Rienties et al. 2011b). With all teachers
an intake interview was taken, whereby particular attention was focussed on challenges that teachers
experienced in their classroom. Afterwards, teachers had to implement the redesign into their
teaching practice as well as completing the module. Participants from a range of academic disciplines
were mixed into groups of 4-7 participants and worked for a period of 12 weeks, with a workload of 25
hours. On average, participants met each other online during four web-videoconferences of one hour.
In addition, participants were able to contact each other and share knowledge and expertise using
discussion forums. Participants did not meet each other face-to-face because the travel time between
sites would have been significant. In line with recommendations of Lawless and Pellegrino (2007) and
Biggs and Tang (2007), teachers were actively encouraged to reflect on how their teaching practice
could be redesigned based upon the topics discussed in the module. Furthermore, participants were
actively encouraged to critically discuss and reflect upon the redesign within their group. The
sequence of the learning steps was as follows:
672

Bart Rienties et al.
Step 1. Orientation on the subject, ICT tools and the teacher’s practice
This step had three purposes: 1) to provide participants with a basic understanding of the main
subject matter (e.g. how can teachers facilitate learners working together in discussion forums); 2) to
discuss the advantages and disadvantages of ICT tools used; 3) to provide participants a first-hand
experience of the discussed tools.
Step 2. Make a relevant technological and pedagogical choice for the teacher’s practice
In the second step, participants explored the information about the tools and were encouraged to
elaborate what would be the most effective ICT tool(s) and pedagogical approach(es) for their
respective educational setting. This selection was discussed with their peers during one of the webvideoconference
meetings.
Step 3. Redesigning the teacher’s practice
The third step required participants to apply the issues discussed in the module to the context of their
own teaching practice. Participants started to redesign their module, changed their educational setting
and used the selected ICT tool(s). While participants were redesigning their own teaching practice,
they had to critically reflect on the alignments between content, pedagogy and technology.
Furthermore, they received feedback from other participants in their group and provided feedback on
the designs of other participants.
Step 4. Reflection on the teacher’s learning process
At the end of the training programme, participants reflected on their own learning process and their
experiences in the module and looked back to the situation when they started. After this step, the
group process was finished.
Step 5. Implementation of the redesign into practice and evaluation
Participants implemented their redesigned lesson(s) in their teaching practice and evaluated the
effects on learning by students within six months after completing the MARCHET module.
2.2.2 Study 2 Conceptual framework of blended programme: Grad Cert
In Study 2, 31 teachers from one university in the UK participated in a blended teacher training
program consisting of four separate modules. The entire programme takes 18 months to complete,
but the focus of module 1, the “Theory and practice of Teaching” is more or less comparable to Study
1 if one includes the hours participants in MARCH ET spend on redesigning their educational design
outside the formal training. As a primary learning objective teachers were expected to develop greater
understanding of their role as a teacher within the learning environment (Biggs & Tang 2007).
Participants from four broad academic disciplines were mixed into groups of 4-5 participants and
worked for a period of 4 months, with an estimated workload of 150 hours. However, the majority of
hours were self-study, as only five face-to-face meetings of two-three hours were arranged during the
module. At the same time, discussion forums and emails were used to share information and
knowledge in between face-to-face meetings. The sequence of the five learning steps in the module
was as follows:
Step 1. Orientation and introduction to learning and teaching
This step provided participants with a general introduction to the module based around underpinning
theory upon which the learning was to be based. It also provided an introduction to the online aspect
of the programme and the rationale for this. After this session the large group was divided into groups
of 4-5 participants.
Step 2. Micro teach
The second step required each participant to present a short teaching session on a subject of their
choice to their peers in the small group. Before and after the micro teach, participants were expected
to reflect on their own and their peer’s micro teach in the discussion forum. The micro teach was
673

Bart Rienties et al.
videoed for reference and response by the teacher. In addition, discussions were held after each
session within the group and reflections posted on line for further online conversation with the view of
these helping form the content for future sessions.
Step 3. Reflection on micro teach
Step 3 involved drawing out of the online discussions the questions that had been prompted within the
online environment to stimulate further debate and to seek practical solutions and reflect on the
supporting theory to assist future practice as this was developed by each teacher.
Step 4. Analysis of teaching practice and choose common theme/problem area
As discussions focussed upon particular topics, such as ‘motivation’ or ‘assessment’, these were
further explored online and then in small group sessions with the aim that each group would formulate
a structure to explain their learning at the final stage. Discussions and planning continued in face to
face meetings and online between participants.
Step 5. Group presentation of understanding of practice
Finally, each small group put forward a presentation which aimed to demonstrate how understanding
of learning and teaching was evolving and how they were able to apply this and then analyse the
impacts and consequences of the actions taken.
2.3 Research questions and hypotheses
Based upon our theoretical framework, we first determined the teacher beliefs and attitudes towards
learning, teaching and ICT of participants at the beginning of each training programme. Afterwards,
we compared the extent to which the teacher beliefs and attitudes towards learning, teaching and ICT
had changed by the end of the training programme. Therefore, the following research questions were
formulated:
What are the teacher beliefs and intentions towards teaching and learning at the start of the two
teacher training programs?
To what extent did the teacher training programs lead to a change in teachers’ beliefs and
intentions to effectively implement learning, teaching and ICT in practice?
3. Method
3.1 Participants
In Study 1, 74 teachers from eight institutes in the Netherlands participated in at least one MARCH ET
module in the period April 2010 – June 2011. The participants were divided into twelve groups,
whereby the average group size was 7.55 (SD = 3.85). The average age of the participants was 43.04
(SD = 10.16), while the average years of teacher experience was 7.27 (SD = 7.29). Overall, 64% of
the teachers were male.
In Study 2, 31 teachers from one university in the UK participated in a module between January –
June 2011, of which 28 returned the pre-test questionnaire. The participants were divided into six
groups, whereby the average group size was 4.66 (SD = 0.51). The average age of the participants
was 33.52 (SD = 6.46), while the average years of teacher experience was 3.04 (SD = 2.99). Overall,
28.5% of the teachers were male. Given that the average age of participants in Study 1 was ten years
higher and relatively more men participated, we used pair-wise matching of participants in Study 1 to
Study 2 on gender and age, leading to 28 participants in Study 1 with an average age of 33.28 (SD =
5.03) and 50% of the teachers were male.
3.2 Pre- and post-test of Teacher Beliefs and Intentions (TBI)
In order to measure the any change in views of the teacher after the intervention of the teacher
training, the Teacher Beliefs and Intentions (TBI) instrument of Norton et al. (2005) was used, which
consists of 20 belief items and 20 intention items. In the questionnaire, a distinction is made between
learning facilitation and knowledge transmission. Within learning facilitation, five factors are identified,
namely: problem solving, interactive education, facilitative education, pastoral care, and motivating
674

Bart Rienties et al.
students. For knowledge transmission three factors are identified, namely training for jobs, imparting
information, and knowledge of subject. Cronbach alphas on the nine scales ranged from .408-.627,
indicating reasonable reliability.
4. Results
In Figure 1 and Figure 2 the pre- and post-participants’ beliefs and intentions measured of the two
studies are illustrated. In both studies, beliefs towards learning-facilitation orientation at the beginning
of the teacher training programmes are stronger than knowledge transmission, as is illustrated by
scores above 4.0 for the learning-facilitation orientation scales of problem-solving, interactive teaching
and motivating students. Training for jobs, imparting information and knowledge of the subject are
below 3.8. At the same time, teachers’ intentions are primarily directed towards problem solving,
motivating students, training for jobs and knowledge of the subject at the beginning of the training
programme.
Follow-up paired sample T-test analyses indicate that in both studies teachers have changed their
teaching beliefs and intentions when comparing pre-test and post-test scores. For Study 1, beliefs in
knowledge transmission are significantly reduced, while all other scales do not change significantly
based upon a 5% confidence interval. In Study 2, beliefs in problem solving, facilitative teaching, and
training for jobs have significantly increased. At the same time, teachers’ intentions for problem
solving and facilitative teaching have increased significantly.
Figure 1: Beliefs of learning facilitation and knowledge transmission
Figure 2: Intentions of learning facilitation and knowledge transmission
675

5. Discussion
Bart Rienties et al.
In this paper, we examine the impact blended and online teacher training programmes may have on
teachers’ beliefs and intentions when developed to specifically link theory and practice to learning and
teaching to the daily practice of teachers. Using the conceptual approaches to teaching framework
developed originally developed by Kember and Gow (1994) and adjusted by Norton et al. (2005), we
have compared the teacher beliefs and intentions of 56 teachers across two studies in a pre-post test
design. When comparing the pre- and post-test scores in the complete online teacher training
programme, the teacher beliefs and intentions have not substantially changed during the module,
although the beliefs towards knowledge transmission were lower in the post-test. In contrast, in the
blended teacher training programme teachers’ beliefs and intentions towards learning facilitation have
increased significantly, as well as relevance of providing students with relevant training for jobs. In
other words, a significant training effect was found for the blended programme, while for the online
programme only one out of 16 scales of teachers’ beliefs and intentions changed. However,
preliminary findings from semi-structured interviews conducted three months after the online
programme indicate that participants did change their teacher beliefs and intentions and used more
learning facilitation and integrated ICT into their teaching practice.
Do these results imply that teacher training programmes provided blended or face-to-face are more
effective than online programmes? Or does this also mean that the process of change needs more
time to complete? We urge researchers and training experts to be cautious about over-interpreting
our findings. First of all, although we pair matched participants on age and gender, the participants in
Study 2 are on average younger than Study 1 and have less teaching experience. As a result,
participants in the blended programme may be more open to new pedagogical approaches that are
more learner-oriented. Secondly, participants in Study 2 are involved for a substantial longer time
duration of 18 weeks, while participants in Study 1 have worked together for only twelve weeks. A
possible explanation of only a small change in teacher beliefs and intentions observed at the end of
the training programme in Study 1 may be explained by a lack of “incubation time”. Research on
training programmes has shown that the direct impact of training measured after the programme was
limited (Lawless, 2007). More research is needed to prove this. Finally, whether teachers are actually
providing a richer learning experience for their students also needs to be tested with further research.
Acknowledgments
This article was written in the context of the MARCH ET project (http://www.marchet.nl/) that is funded
by the SURF Foundation as well as in context of the Gradcert programme at University of Surrey.
References
Adcroft, A, Teckman, J & Willis, J 2010, 'Is higher education in the UK becoming more competitive?',
International Journal of Public Sector Management, vol. 23, no. 6, pp. 578-588.
Biggs, J & Tang, C 2007, Teaching for quality learning at University, 3 edn, Open University Press, Maidenhead.
Brouwer, N, Ekimova, L, Jasinska, M, Van Gastel, L & Virgailaite-Meckauskaite, E 2009, 'Enhancing
mathematics by online assessments, two cases of remedial education considered', Industry and Higher
Education, vol. 23, no. 4, pp. 277-284.
Chickering, AW & Gamson, AF 1987, Seven principles for good practice in undergraduate education, The
Johnson Foundation, Inc., Racine, WI.
Ferla, J, Valcke, M & Schuyten, G 2009, 'Student models of learning and their impact on study strategies',
Studies in Higher Education, vol. 34, no. 2, pp. 185-202.
Fyrenius, A, Wirell, S & Silén, C 2007, 'Student approaches to achieving understanding approaches to learning
revisited', Studies in Higher Education, vol. 32, no. 2, pp. 149 - 165.
Garrison, D & Vaughan, ND 2008, Blended learning in higher education: framework, principles and guidelines,
Wiley and Sons, San Francisco.
Gibbs, G & Coffey, M 2004, 'The Impact Of Training Of University Teachers on their Teaching Skills, their
Approach to Teaching and the Approach to Learning of their Students', Active Learning in Higher Education,
vol. 5, no. 1, pp. 87-100.
Giesbers, B, Rienties, B, Gijselaers, WH, Segers, M & Tempelaar, DT 2009, 'Social presence, webvideoconferencing
and learning in virtual teams', Industry and Higher Education, vol. 23, no. 4, pp. 301-310.
Gow, L & Kember, D 1993, 'Conceptions of teaching and their relationship to student learning', British Journal of
Educational Psychology, vol. 63, no. 1, pp. 20-23.
Jindal-Snape, D 2010, Educational transitions: Moving Stories from Around the World, Routledge.
Kember, D & Gow, L 1994, 'Orientations to Teaching and Their Effect on the Quality of Student Learning', The
Journal of Higher Education, vol. 65, no. 1, pp. 58-74.
Kinchin, I, Lygo-Baker, S & Hay, D 2008, 'Universities as centres of non-learning', Studies in Higher Education,
vol. 33, no. 1, pp. 89-103.
676

Bart Rienties et al.
Koehler, MJ & Mishra, P, Technological Pedagogical Content Knowledge (TPACK). Available from:
. [26 January 2011].
Kogan, M 1999, 'The Culture of Academe', Minerva, vol. 37, no. 1, pp. 63-74.
Lawless, KA & Pellegrino, JW 2007, 'Professional Development in Integrating Technology Into Teaching and
Learning: Knowns, Unknowns, and Ways to Pursue Better Questions and Answers', Review of Educational
Research, vol. 77, no. 4, pp. 575-614.
Löfström, E & Nevgi, A 2008, 'University teaching staffs' pedagogical awareness displayed through ICT-facilitated
teaching', Interactive Learning Environments, vol. 16, no. 2, pp. 101-116.
Marginson, S 2006, 'Dynamics of National and Global Competition in Higher Education', Higher Education, vol.
52, no. 1, pp. 1-39.
McNay, I 1995, 'From the collegial academy to corporate enterprise: the changing cultures of universities', in The
Changing University?, ed. T Schuller, Open University Press/SRHE, Buckingham, pp. 105-115.
Mishra, P & Koehler, MJ 2005, 'What happens when teachers design educational technology? The development
of technological pedagogical content knowledge', Journal of Educational Computing Research, vol. 32, no.
2, pp. 131-152.
Nicholls, G 2001, Developing Teaching and Learning in Higher Education, Routledge, London.
Norton, L, Richardson, T, Hartley, J, Newstead, S & Mayes, J 2005, 'Teachers’ beliefs and intentions concerning
teaching in higher education', Higher Education, vol. 50, no. 4, pp. 537-571.
Prosser, M, Ramsden, P, Trigwell, K & Martin, E 2003, 'Dissonance in Experience of Teaching and its Relation to
the Quality of Student Learning', Studies in Higher Education, vol. 28, no. 1, pp. 37 - 48.
Prosser, M & Trigwell, K 1999, Understanding Learning and Teaching: The Experience in Higher Education,
SRHE and Open University Press, Buckingham, UK.
Resta, P & Laferrière, T 2007, 'Technology in Support of Collaborative Learning', Educational Psychology
Review, vol. 19, no. 1, pp. 65-83.
Rienties, B, Beausaert, S, Grohnert, T, Niemantsverdriet, S & Kommers, P 2011a, 'Understanding academic
performance of international students: the role of ethnicity, academic and social integration', Higher
Education.
Rienties, B, Brouwer, N, Bohle Carbonell, K, Townsend, D, Rozendal, AP, Van der Loo, J & Dekker, P 2011b,
'Changing teacher beliefs and intentions towards ICT: can training make a difference?', in Proceedings of
the 18th EDINEB Conference From Innovation to Crème de la Crème Education!, eds B Rienties, S Reeb-
Gruber, P Daly & P Van den Bossche, FEBA ERD Press, Lyons, pp. 194 – 202.
Rienties, B, Kaper, W, Struyven, K, Tempelaar, DT, Van Gastel, L, Vrancken, S, Jasinska, M & Virgailaite-
Meckauskaite, E 2011c, 'A review of the role of Information Communication Technology and course design
in transitional education practices', Interactive Learning Environments.
Rienties, B & Townsend, D 2011, 'Integrating ICT in business education: using TPACK to reflect on two course
redesigns', in Advances in Business Education and Training, vol. 4, ed. P Van den Bossche, Springer,
Dordrecht, pp. XX-XX.
Tempelaar, DT, Rienties, B & Giesbers, B 2009, 'Who profits most from blended learning?', Industry and Higher
Education, vol. 23, no. 4, pp. 285-292.
Trigwell, K & Prosser, M 2004, 'Development and Use of the Approaches to Teaching Inventory', Educational
Psychology Review, vol. 16, no. 4, pp. 409-424.
Valcke, M & Martens, R 2006, 'The problem arena of researching computer supported collaborative learning:
Introduction to the special section', Computers & Education, vol. 46, no. 1, pp. 1-5.
Young, SF 2008, 'Theoretical frameworks and models of learning: tools for developing conceptions of teaching
and learning', International Journal for Academic Development, vol. 13, no. 1, pp. 41 - 49.
677

Moodle and Affective Computing: Knowing who´s on the
Other Side
Manuel Rodrigues 1,2 , Florentino Fdez-Riverola 2 and Paulo Novais 3
1
Informatics Group, Secondary School Martins Sarmento, Guimarães, Portugal
2
Informatics Department, University of Vigo, Ourense, Spain
3
Informatics Department, University of Minho, Braga, Portugal
mf.rodrigues@esmsarmentro.pt
pjon@di.uminho.pt
riverola@uvigo.es
Abstract: In traditional learning, teachers can easily get an insight into how their students work and learn, and
how they interact in the classroom. However, in online learning, it is more difficult for teachers to see how
individual students behave and learn, and very important, their mood to do it. Student’s emotions like selfesteem,
motivation, commitment, and others that are believed to be determinant in student’s performance can
not be ignored, as they are known (affective states and also learning styles) to greatly influence student´s
learning. This paper deals with the student’s behavioural and affective aspects in virtual learning environments to
enhance the students’ learning, gain and experience. The goal is to achieve a similar performance to a skilled
teacher that can modify the learning path and his teaching style according to the feedback signals provided by
the students - which include cognitive, emotional and motivational aspects. This can be done through the
recognition of students actual mood, and we propose a framework to implement and address such issues in
Moodle.
Keywords: affective computing, learning styles, eLearning, Moodle
1. Introduction
Since we’ve become teaching, providing knowledge to all that seek for it, with individualized support
whenever it’s needed (anytime, anywhere), when solicited, ideally even when that support isn’t asked
but the need for it is there, has been the ultimate goal search for everyone. In traditional learning,
teachers can easily get an insight into how their students work and learn, and how they interact in the
classroom. However, in online learning, especially when using systems like learning management
systems (LMSs), and others, for instance Moodle, it is more difficult for teachers to see how individual
students behave and learn, specially to characterize student’s emotions like self-esteem, motivation,
commitment, and others that are believed to be determinant in students performance. This paper
deals with the student’s behavioural and affective aspects in virtual learning environments, not always
noticed and taking into account. Behavioral aspects are understood in terms of motivational factors
such as confidence, effort, and independence for instance. They are important to infer the student´s
mood (interested/disinterested, satisfied/dissatisfied). As Moodle is one if not the most ELearning
platforms used worldwide, also in portuguese schools, it will be used experimentally to try and
develop a new affective module to address students’ affective states and learning styles. The goal as
stated, is to try to attend to students particularities, thus improving learning and success. We start in
section 2 by clarifying the need for such abilities in Moodle. Later in section 3, affective computing,
affective states and learning styles are briefly revised, in order to present in section 4 a framework for
addressing some of the problems identified. The framework presented is part of an ongoing larger
work that aims to develop a system to improve students’ learning success, when using Moodle.
2. eLearning and Moodle
Currently, education organizations can not exclude them selves from information society, and are
always confronted with new technological challenges. The student population comes from very
different social backgrounds, with different needs and expectations. Moreover, society is demanding
for more qualified technicians. Schools are, therefore, faced with a new technological paradigm, a
new kind of public and new demands from society.
Education organizations have tried to address these challenges by investing in organization,
management, market research, and in human and technological resources. New pedagogical tools,
such as eLearning platforms and Intelligent Tutoring Systems have been also subject of attention.
These investments are very expensive; schools cannot afford to have unsuccessful students. As a
consequence, the students’ careers must be closely followed. Schools should have devices to
evaluate their students’ learning state, i.e., they should possess means to keep their students’
678

Manuel Rodrigues et al.
descriptions up to date, that way being able to follow and diagnose, if not in real time, at least
periodically, the learning paths, to avoid failures as much as possible. Furthermore, the need to
supply the market with effectively qualified personnel favours these evaluations (Almeida et al, 2008).
This evaluation and following should be performed by teachers and psychologists, who access and
diagnose the learning paths of the students to detect symptoms of deviations and act accordingly.
However, this kind of expertise isn’t always available, and when it is, it’s insufficient to address all the
needs.
The need for some eLearning system that provides these features is then obvious. ELearning
systems are software programs that help and provide support to learning. They include personal
training systems, usually designed for a certain knowledge domain, known as Tutoring Systems
(VanLehn, 2006), as well as general learning management tools suitable to manage distinct types of
learning content, covering several knowledge domains. Pedagogical concerns when building such
systems aren’t always present, but some attempts have been made. In (Rodrigues et al, 2005) a
framework is proposed to mitigate some of these known problems.
Learning Management Systems/Course Management Systems (LMS/CMS) are domain independent,
general purpose programmes/platforms, which provide authoring, sequencing, and aggregation tools
that structure content to ease the learning process. MOODLE platform is an example of a LMS/CMS.
It is widely used and the number of installations is growing very rapidly. In fact Moodle is by far, the
most used eLearning platform in secondary schools in Portugal. Its usage has been widely
recommended and encouraged by the official education organizations (Valente, 2007).
2.1 The MOODLE Learning Management System
MOODLE (Modular Object-Oriented Dynamic Learning Environment) has a number of interactive
learning activity components like forums, chats, quizzes and assignments. In addition, MOODLE
includes a logging module to track users’ accesses and the activities and resources that have been
accessed. Administrators and teachers can extract reports from this data. Figure 1 shows a high level
view of the MOODLE modules.
Figure 1: MOODLE LMS modules
By having a modular type design, the platform can be enriched with different plug-ins, designed to
meet specific needs a specific set of users.
Whatever the case, LMS (Learning Management Systems) should have some sort of knowledge
about the students and about their learning processes. This knowledge, i.e., the beliefs the system
has about the students, is usually called the Student Model (SM). Without a SM a system would
simply behave the same way for all students. Additionally, this Student Model must be dynamically
upgraded to reflect students affective states, motivation, etc, to adapt not only to different students,
but also to the different states that a student has, when using such a platform.
LMS, such as Moodle, are very successful in e-education but they do not accommodate full fledged
adaptively (Graaf, 2006).
679

Manuel Rodrigues et al.
Moodle doesn’t provide any of the issues previously discussed, and the need for some module that
implements them it’s crucial to improve student’s success. In section 3, some of the important
aspects to consider, regarding students, their emotions and learning styles, in an eLearning platform
are subject to a brief analysis. Later we will present a framework to implement and address such
issues in Moodle.
3. Affective computing
Affective computing has emerged as a study area of Artificial Intelligence, whose objective is the
study of emotions and their application in computer systems. According to (Picard, 2000), studies how
computing systems can detect classify and respond to human emotions. Affective computing in
Human-Computer Interaction can be defined as “computing that relates to, arises from, or deliberately
influences emotion” (Picard, 2000).Various studies support that affect plays a critical role in learning
performance as it influences cognitive processes (LeDoux,1998).
“The extent to which emotional upsets can interfere with mental life is no news to teachers, students
who are angry, anxious, or depressed have difficulties in learning; people who are caught in these
states do not take information efficiently or deal with it very well”, (Goleman, 1995). However, the
relationship between learning and emotions is far from being that simple and linear.
Positive and negative affect states produce different kind of thinking and this might hold important
implications from educational and training perspective. A consistent theory of learning that integrates
effectively cognitive and emotional factors is strongly needed, (Kort, 2001).
A wide range of emotions occurs naturally in a real learning processes, from positive ones (joy,
satisfaction, etc.), to negative ones (frustration, sadness, confusion), to emotions more related to
interest, curiosity and surprise in front of a new topic. Emotion is characterized by "any agitation or
disturbance of mind, feeling, passion; any vehement or excited mental state. There are a hundred
emotions along with their combinations, variations and mutations. In fact, there are more subtleties of
emotions than the words we have to define them." Already affection means briefly "the whole realm of
emotions properly said, the feelings of emotions, sensory experiences, and especially the ability to be
able to get in touch with the sensations" (Bercht, 2001).
Emotions have a close relationship to education, because the affective state of the student directly
affects the motivation and aptitude to learn something. Thus, knowing the user’s affective state might
play an important role improving the effectiveness and efficacy of eLearning. The unawareness of
emotional states has been considered one of the core limitations of the traditional eLearning tools.
Skilled teachers modify the learning path and their teaching style according to students feedback
signals (which include cognitive, emotional and motivational aspects), eLearning platforms generally
don´t take into account these feedbacks signals resulting too rigid and weakened, as they perform the
same manner for all students.
A number of projects are currently being conducted in order to design eLearning platforms endowed
with affective computing capabilities, although very few or no commercial results are currently
available. (Balestra, 2005).
3.1 Affective states in eLearning
Most of the eLearning systems focus attention towards knowledge acquisition or cognitive processing.
When building such a system, affective states (such as motivation and emotion for instance), are
considered only in terms of how the content is structured and presented. To make learning efficient
and to deliver personalized content, adaptive systems are based on student’s goals models,
knowledge, and preferences. Thus, a student model that integrates the cognitive processes and
motivational states would lead to more efficient and personalized adaptation,(Cocea
2007).Transforming a non-affect sensitive eLearning system into a system that includes user’s
affective states requires the modelling of a cycle known as the affective loop. The affective loop
encompasses detection of a user’s affective states, appropriate actions selection for decision making,
and the synthesis of appropriate affective state by the system, (D'Mello, 2008).
As stated before, affection influences the learning performance and decision making. This means that
students who become caught in affective states such as anger or depression do not process and
680

Manuel Rodrigues et al.
absorb information efficiently. From this, it can be inferred that a user’s affective state has a major role
in improving the effectiveness of eLearning, (Weimin 2007).
“Emotion, mood and affective attitude are different things but strongly related and
influence each other. An emotion is “composed” by a facial expression, a feeling (the
conscious experience of the emotion) cognitive processing aimed at evaluating the
situation in terms of personal relevance, physiological change and action readiness. It’s a
short but intense episode, (for instance I feel and noticed I am happy when seeing an old
friend).In contrast, mood refers to the presence of moderate levels of affect. Mood is not
consciously attributed to a causal factor. I can feel frustrated for half a day not knowing
why. An affective attitude is an affective association coupled with a thing or person, an
emotion is an evaluation of a thing or person in terms of personal relevance”,
(Broekens,2010).
In this work, we will use the term mood in order to determinate or name a students particular state of
mind or emotion, that is, a particular inclination or disposition to learn something.
Numerous parameters can be used to describe students’ affective states, motivation and interest, for
instance. Confidence, effort and confusion are highlighted among the possible factors influencing a
student’s motivation (Qu, Wang, and Johnson 2005).
Also, the motivational model presented by (De Vicente and Pain, 2002) includes variables related to
trait (control, challenge, fantasy, and independence) and state (confidence, sensory interest, cognitive
interest, effort, and satisfaction).
In (Khan,2010a), 4 methods to infer student´s affective states are proposed , namely verbal approach,
where a questionnaire or self report instrument is presented to the student, nonverbal approach
where psycho-physiological instrument measures physical states, through the use of sensors,
intrusive approach trough the use of intrusive instruments to measure affective states, (although that
intrusive instruments influence a student’s normal affective state and may thus lead to
misinformation), and non-intrusive approach where the affective state is identified through interaction
with the system.
Another model, known as OCC, is frequently referred also as the standard cognitive appraisal model
that provides a clear and convincing structure of the eliciting conditions of emotions and the variables
that affect their intensities. This psychological model is popular among computer scientists that are
building systems that reason about or incorporate emotions, (Ortony, Clore, & Collins, 1990).
3.2 Learning styles
The idea that student’s learn differently is valued and probably had its origin with the ancient Greeks.
For many years, it has been noticed that some students prefer certain methods of learning more than
others. These dispositions, forms a student's unique learning preference that is, student learning style
,and aid educators in planning small-group and individualized instruction. (Grasha 1996), has defined
learning styles as, "personal qualities that influence a student's ability to acquire information, to
interact with peers and the teacher, and otherwise participate in learning experiences". There are
probably as many ways to "teach" as there are to learn.
Learning styles specify a student’s own way of learning. Someone that has a specific learning style
can have difficulties when submitted to another learning style (Felder, 1988). When the presenting
instruction style matches the student’s learning style, the process is maximized, that is, the student
learns more and better. Based on literature, we can establish that the consideration of learning styles
in a learning environment influences a student’s learning. In the present era, learning styles are being
investigated in order to incorporate them into adaptive online learning environments (Graf, 2006).
Adaptive online learning environments are ideal for generating learning style based instructional
material in large classes as they do not have the same limitations as human instructors that due to the
lack of resources and time are unable to focus on individual students. One popular learning style
inventory and one that is often used in distance learning research is the Kolb Learning Style Inventory
(LSI). Kolb's LSI measures student learning style preference in two bipolar dimensions, (Kolb, 1986).
681

Manuel Rodrigues et al.
Other several learning style theories exist, for instance, Honey and Mumford (Honey, 1982) and
Felder-Silverman learning style model (Felder, 1988). The later seems to be the most appropriate for
use in eLearning systems (Carver et al., 1999). Most other learning style models classify learners in
few groups, whereas FSLSM describes the learning style of a learner in more detail, distinguishing
between preferences on four dimensions. In (Graaf, 2006) a very interesting work is proposed to
automatically detect learning styles through student modelling.
Table 1: Learning styles adapted from (Shahida, 2008)
Learning Style Characteristics
Active Tries things out, works within a group, discusses and explains to others
Reflective Thinks before doing something, works alone
Sensing Learns from and memorizes facts, solves problems by well-established methods, patient
with details, works slower
Intuitive Discovers possibilities and relationships, innovative, easily grasp new concepts,
abstractions and mathematical formulation, works faster
Visual Learns from pictures, diagrams, flow charts, time lines, films, multimedia content and
demonstrations
Verbal Learns form written and spoken explanations
Sequential Learns and thinks in linear/sequential steps
Global Learns in large leaps, absorbing material almost randomly
Currently two approaches are used for identifying learning styles, namely the use of questionnaires
and the use of data from students’ behaviour and actions in an online course. Shute and Zapata-
Rivera (Shute, 2008), identify at least two problems associated with questionnaire based information.
Students may provide inaccurate data either purposefully,(for instance a desire to present themselves
in a more prominent way) or accidentally, due to not knowing their own characteristics. A second
problem is that when completing the questionnaire during the online learning process, it consumes
time, and students tend to provide invalid data in order to shortcut to contents quickly. As stated,
Felder and Silverman developed an Index of Learning Styles Questionnaire (ILS) that is widely used
to identify learning styles explicitly (table 1).
An approach based on the actions and behaviour of the students during their interaction with the
system for learning may be used. No additional effort is needed by students in these approaches in
order to obtain information about their learning styles. In fact learning styles are inferred by the
system from the student’s actions, being the information captured that way free from uncertainty. In
(Khan, 2010a) a concept for identifying learning styles and affective states, using different approaches
is proposed.
In the next section, a Framework is presented, to mitigate some of the problems elicited before,
regarding eLearning platforms’, namely Moodle.
4. The framework
In figure 2 we present a framework to address some of the problems identified through this work. The
goal is to obtain an external module, to be linked to Moodle platform, enabling the detection of
student’s affective states together with learning styles, in order to really know each student and
presenting contents accordingly. The affective module will be responsible for gathering all this
information, and derive students mood (referred in this work as students particular state of mind or
emotion, that is, a particular inclination or disposition to learn something), in order to present relevant
clues for a personalization and recommendation module (not detailed in this paper).
The affective module has two sub-modules, explicit assumption and implicit assumption, whose
function is to detect student’s mood, maintaining that information (actual and past) in the mood
database, that will be used by another sub module, affective adaptative agent to provide relevant
information to the platform and to the referred personalization module. This enables that actual
students mood information can be displayed in Moodle platform, and may be used to personalize
instruction according to the specific student, thus enabling Moodle to act differently to different
students, and also to act different to the same student, according to his/her past and present mood.
The sub-modules are explained next. We are not developing integrally all the modules from scratch,
various research has been done in areas as facial recognition, keyboard and mouse stress detection,
682

Manuel Rodrigues et al.
for instance, that can be used here. Many research in log analyses for student characterization is also
widely available.
Figure 2: The proposed framework
4.1 Explicit mood assumption
One (the easiest way but not the most accurate) form of knowing a student’s mood to achieve a
certain class is by posing explicit questions to the student. Surprisingly, this may not be the most
accurate way, not always the answers obtained reveal the accurate state of the student. However we
can still use questionnaires, as a way of gathering some useful information. An explicit mood
assumption agent could periodically pose some questions, preferably in a visual way, for the student
to upgrade his/her mood to the system. Several researches have been done to detect student mood
explicitly (Broekens et al,2010).
4.2 Implicit mood assumption
The aim of this sub-module is to monitor the interactions between the student and the system, in
order to infer the students’ mood, doing so without being intrusive, that is, without the student being
aware of the analysis being performed. Agent technology is used to monitor four key aspects: facial
analyses, mouse analyses, keyboard analyses and log analyses. As web cams tend to be widely
standard equipment in computers, the goal is to use it to try to infer emotions from the user. Mouse
movements can also predict the state of mind of the user, as well as keyboard entries. Finally,
analysing the past interactions of the student, through the logs files of Moodle, turns possible to infer
some of the information we are looking for.
4.3 Facial recognition
As stated, and as it is widely recognized from psychological theory, human emotions can be classified
into six archetypal emotions: surprise, fear, disgust, anger, happiness, and sadness. Facial motion
plays a major role in expressing these emotions. Several automatic emotion recognition systems have
explored the use of facial expressions to detect human affective states. (Cohen et al,2003),(Pantic,
Rothkrantz, 2000). The main idea is to extract affectively relevant features from an image, in order to
establish the student current emotions, features like mouth angle and face movements are used.
683

Manuel Rodrigues et al.
Doing this implicitly makes more difficult to deceive the system, as the student isn’t aware of the on
going analyses.
Keyboard and Mouse
The way a user types, may indicate his/her state of mind. Pressing hard and rapidly the keyboard
could mean an altered state, anger for instance, while taking too much time may mean sadness for
instance. The same occurs with mouse movements. A system that monitors users’ behaviour from
standard input devices, like the keyboard or the mouse is proposed by (Zimmerman et al. 2003).
Analyzed features include: the number of mouse clicks per minute, the average duration of mouse
clicks (from the button-down to the button-up event), the maximum, minimum and average mouse
speeds, the keystroke rate (strokes per second), the average duration of a keystroke (from the keydown
to the key-up event) and performance measurements. (George, Tsihrintzis et al., 2008) included
keyboard –stroke information in order to improve the accuracy of visual-facial emotion recognition.
Log agent
Moodle has an activity logger to register users’ accesses (i.e., user ID, IP and time of access) and the
activities and resources that have been accessed. From the log, Moodle is able to generate, for each
student, activity reports. In ( Khan 2010b), learning styles and affective states information are
gathered from students’ interactions in a web-based learning management system. The students’
behaviour on features that are commonly used in Moodle is analysed. Those commonly used features
include content objects, outlines, exercises, self assessment tests, examples, discussion forums for
assignment related queries, discussion/peer rating forums related to the content objects, and
assignments. Considering information from all these features, the students’ learning styles as well as
affective states can be identified using a rule- based approach.
In our work, we are using non-invasive techniques because we believe that’s the best way to do it.
We believe that more intrusive techniques, like body sensors, heart monitors, etc are not well
accepted by users. Another interesting point to refer is that using this kind of technologies (web cam,
mouse and keyboard analysis) makes our solution cheaper, versatile and virtually undetected by
users, making the inference from interactions more reliable.
5. Conclusions
Through out this paper, some major problems regarding ELearning platforms, namely Moodle where
identified. The importance of students’ individual characteristics, as the way they learn and the mood
in which they do it, outstands as a crucial learning factor. Nevertheless, as stated, ELearning
platforms don’t take into account these issues. The Framework proposed tries to solve them, by the
inclusion of an affective module in Moodle. The affective module proposed, tries to identify learning
styles and students affective states that are widely recognized to be of great importance for learning
success. The use of implicit methods to do this is emphasised, as the student doesn’t need to be
aware of the ongoing analyses, thus the probability of diseasing the system gets lower. Several
research has been done in these methods, facial emotion recognition, mouse and keyboard emotion
detection and also log analyses. Detecting affective states and learning styles will enable our
recommendation module to do more effective recommendation namely in terms of content
presentation. We hope to use the results of such research, in order to develop the affective module.
We’re not trying to develop from scratch such methods. Experimentation and testing will be done
through the implementation of a real system in public secondary school in Portugal. This is part of a
larger project as stated.
References
Almeida P., Novais P., Costa E., Rodrigues M., Neves J. (2008): Artificial Intelligence Tools for Student Learning
Assessment in Professional Schools, in Proceedings of the 7th European Conference on eLearning,
Cyprus, ISBN 978-1-906638-22-1, pp 17- 28,.
Balestra, M. 2005): I progetti di ricerca CRAFT nel VI Programma Quadro: MySelf. ELearning & Knowledge
Management.
Bercht, M. (2001) “Em direção a agentes pedagógicos com dimensões afetivas”. Tese de Doutorado (Doutorado
em Ciência da Computação)-Universidade Federal do Rio Grande do Sul, Porto Alegre.
Broekens, J, Jonker, C.M., Meyer, J.J.Ch, (2010): Affective negotiation support systems, Journal of ambient
Intelligence and smart Environments, vol2, n. 2
684

Manuel Rodrigues et al.
Carver, C.A., Howard, R.A. and Lane, W.D., (1999): Addressing different learning styles through course
hypermedia. IEEE Transactions on Education, Vol. 42, No. 1, pp. 33-38.
Cocea, M., Weibelzahl, S. (2007). Eliciting motivation knowledge from log files towards motivation diagnosis for
Adaptive Systems. User Modeling 2007 LNCS Springer Berlin / Heidelberg.
Cohen, I. ,et al. (2003):Facial expression recognition from video sequences: Temporal and static modeling.
Computer Vision and Image Understanding, 91(1- 2):160–187.
D'Mello, S. K., Craig, S.D., Witherspoon, A., McDaniel, B., and Graesser, A., (2008). Automatic Detection of
Learner’s Affect from Conversational Cues. Journal of User Modeling and User-Adapted Interaction, 18(1-
2), 45-80.
De Vicente, A. Pain, H. (2002): Informing the Detection of the Students' Motivational State: An Empirical Study,
Proceedings of the Sixth International Conference on Intelligent Tutoring Systems, vol 2363 of Lecture
Notes in Computer Science, pages 933-943
Felder, R.M. and Silverman, L.K., (1988): Learning and teaching styles in engineering education. Engineering
Education, Vol. 78, No. 7, pp. 674–681.
Gilbert, J.E., Han, C.Y. (1999): Adapting Instruction in Search of a Significant Difference, Journal of Network and
Computer Applications, 22(3), pp 149-160
Graf, S., Kinshuk. (2006): An Approach for Detecting Learning Styles in Learning Management Systems, in Sixth
IEEE International Conference on Advanced Learning Technologies. Kerkrade, Netherlands, pp. 161-163
Grasha, A. F. (1996). Teaching with style. Pittsburgh, PA: Alliance.
Graf, S. (2007): Adaptivity in Learning Management Systems Focusing on Learning Styles, Ph.D Thesis, Vienna
University of Technology
George A. Tsihrintzis, Maria Virvou, Efthymios Alepis, Ioanna-Ourania Stathopoulou, (2008). Towards Improving
Visual-Facial Emotion Recognition through Use of Complementary Keyboard-Stroke Pattern Information.
itng, pp.32-37, Fifth International Conference on Information Technology: New Generation.
Goleman, D. (1995): Emotional intelligence. Bantam Books, New York
Honey, P. and Mumford, A., (1982). The manual of learning styles, Peter Honey, Maidenhead.
Jonassen, D. H., Grabowski, B. L. (1993): Handbook of Individual Differences, Learning and Instruction.
Lawrence Hillsdale, NJ: Erlbaum Associates
Khan,F.A., et al (2010a): Identifying and Incorporating Affective States and Learning Styles in Web-based
Learning Management Systems, Interaction Design and Architecture(s) Journal - IxD&A, N. 9-10, pp. 85-
103
Khan,F.A., et al (2010b): Implementation of Affective States and Learning Styles Tactics in Web-based Learning
Management Systems, 2010 10th IEEE International Conference on Advanced Learning Technologies
Kort, B., Reilly, R. (2001): Analytical Models of Emotions, Learning and Relationships: Towardsan Affectsensitive
Cognitive Machine. MIT Media Lab Tech Report No 548
Kolb, D. A. (1986). Learning style inventory: Technical manual (Rev. ed.). Boston, MA: McBer.
LeDoux, J. (1998): The emotional brain: The mysterious underpinnings of emotional life. Weidenfeld & Nicholson,
London
Ortony, A., Clore, G.L., Collins, A. (1988): The Cognitive Structure of Emotions. Cambridge University Press,
Cambridge, UK
Pantic, M. Rothkrantz. L.J.M. (2000): Automatic analysis of facial expressions: the state of the art. IEEE Trans.
PAMI, 22(12): 1424–1445,
Picard, R. (2000) “Affective Computing”. The MIT Press, ISBN: 0262661152.
Picard, R. W., Papert, S., Bender, W., Blumberg, B., Breazeal, C., Cavallo, D., Machover,T., Resnick, M., Roy,
D., Strohecker, C. (2004): Affective Learning - A Manifesto. BT Technology Journal Vol. 22, 4 253-269
Qu, L., Wang, N., Johnson, W.Lewis. (2005): Using Learner Focus of Attention to Detect Learner Motivation
Factors, In Ardissono, L., Brna, P., Mitrovic, A.: User Modelling 2005, pp 70-73
Rodrigues, M., Novais , P. , Santos, M.F. (2005), Future Challenges in Intelligent Tutoring Systems – a
Framework, "Recent research developments in learning technologies : proceedings of the International
Conference on Multimedia and Information & Communication Technologies in Education, 3, [Cáceres],
2005" Badajoz : Formatex
Shahida M. Parvez and Glenn D. Blank, (2008): Individualizing Tutoring with Learning Style Based Feedback,
[Intelligent Tutoring Systems, 9 th International Conference, Montreal, Springer.
Shute, ,V. J., Zapata-Rivera, D. (2008):Adaptive technologies, In J. M. Spector, D. Merrill, J. van Merriënboer, &
M. Driscoll (Eds.), Handbook of Research on Educational Communications and Technology (3rd Edition)
(pp. 277-294). New York, NY: Lawrence Erlbaum Associates, Taylor & Francis Group
VanLehn K. (2006) “The behaviour of tutoring systems”, International Journal of Artificial Intelligence in
Education, Vol. 16, No. 3, pp. 227-265.
Valente, L. & Moreira, P. (2007). Moodle: moda, mania ou inovação na formação? –Testemunhos do Centro de
Competência da Universidade do Minho. In P. DIAS; C. V. FREITAS; B. SILVA; A. OSÓRIO & A. RAMOS
(orgs.), Conferência Internacional de Tecnologias de Informação e Comunicação na Educação –
Challenges 2007. Braga: pp. 781-790.
Weimin, X., Wenhong, X.(2007). ELearning Assistant System Based on Virtual Human Interaction Technology,
ICCS 2007, LNCS, Springer Berlin / Heidelberg.
Zimmermann, P., Guttormsen, S., Danuser, B., Gomez,P.(2003): Affective computing-a rationale for measuring
mood with mouse and keyboard. International journal of occupational safety and ergonomics: JOSE vol. 9,
issue 4, p. 539-551.
685

Using Google Applications to Facilitate an Effective
Students’ Collaboration in the Teaching of Informatics to
Students of Secondary Education
Eleni Rossiou and Erasmia Papadopoulou
Department of Applied Informatics, University of Macedonia, Thessaloniki,
Greece
rossiou@uom.gr,
p_eri_@hotmail.com
Abstract: This paper describes the results of an empirical research on the impact of eLearning in the students’
activities in respect of their educational record, their active attendance and their collaboration in the class at the
teaching of the Informatics course, in the Experimental School of Aristotle University, Thessaloniki, Greece. The
investigation of blended learning effectiveness, using online collaborative tools (as Google Applications: email,
groups, calendar, forum, chat, shared documents), concludes that blended learning as an
additional/complementary tool of teaching and learning is effective as much in students’ performance as in their
positive attitude to the use of new technologies in the educational process of the Informatics course. More
specifically the students’ attitudes before, during and after the use of technology as a supporting tool of
educational process is investigated. Additionally the research paper examines the way that students’ active
attendance in the educational process - with the exploitation of eLearning and techniques of collaborative
learning from distance- is encouraged,. The current research does not aim to compare the effectiveness of
exploitation of eLearning with the one without it, but it approaches students’ motivation and it gives an example of
good practical exploitation of ICT. An educational framework of blended learning implementation which uses
collaborative online tools (less important point the tool itself) to support a “web-based” class complementary to
face-to-face class facilitates an effective students’ collaboration and improves the efficiency of learning regarding
both students’ grades and their positive attitudes in using Information and Communication Technologies (ICT) in
the educational process.
Keywords: blended learning, collaborative learning, teaching Informatics, secondary education
1. Introduction
Digital natives behave differently since they are familiar with blogs, webcameras, mp3, movies,
avatars, project, digital photos, open systems, exploration etc. (Prensky 2004). But although students
of today are aware of digital applications and use them in their daily life, their usage seem often to be
on a non-educational purposes. However, there is a continual increase of knowledge and the
requirements of saving time impose a non-stop search and preference in flexible forms of education,
work and study (Papadakis et al. 2006). The educational institutes should adapt new strategies for
new models of education that promote life learning and embed the technology in educational process
(Hicks et al. 2001). ELearning, the exploitation of new technology and the Internet, has changed the
quality of education and facilitated the access to resources of information, services and need for cooperation
from distance. Hybrid, combinational and blended learning are terms that are used to
express the instructional model of teaching which combines eLearning with face-to-face (f2f) learning
in the classroom (Schank 2001). The web-based education is a new challenge and an opportunity for
teachers and students as well (Passerini & Granger 2000). Distance education and eLearning are not
impeded by the limitations of traditional education (limited time of teaching and learning, unavailability
of instructors for additional support for students). Furthermore, there exists powerful indicators that
collaborative methods of learning encourage the building of knowledge and deeper comprehension,
encourage the active learning and in-depth understanding of information since investment of
important intellectual effort is required by the students (Dillenbourg 1999).
The educational institutes of Secondary Education use ICT for their administrative works and sharing
resources. Higher Education constitutes the right place for implementation and exploitation of new
technologies and pedagogical methods that use them. Secondary Education is a prevented place on
that usage due to the technological infrastructure of schools and the limited teachers’ skills in the use
of such examples as blended learning and the computer-supported collaborative learning. But. there
is much research and a large number of programs that use blended learning and emphasize the
effective use of ICT, in learning and the support that is obtained by students aiming at developing
their performance (Hatziplis 2005).
686

Eleni Rossiou and Erasmia Papadopoulou
Passive attendance during the first half of the school year 2010-2011 in addition to the small rate of
students’ success at the tests and self-assessments, puzzled the teacher of the course of Informatics
of the Experimental School of Aristotle University, Thessaloniki, Greece and they led her to find a new
way to solve or improve the situation of this problem. The research question was: how can traditional
teaching be supported in order to increase both students’ active participation during the educational
process and their performance, taking into consideration the limitations due to students’ time
availability and the available teaching hours? The research was implemented with 14 year old
students, second class of High School (Gymnasio), because they had the possibility of using their
netbooks given to them by their school, in the context of the European project “Digital class”
(according to 88310/Γ2/22-07-09 document/decision by the Ministry of Education).
The literature review shaped the expectation of improved student performance with the use of
eLearning as a complementary tool of teaching Informatics in Secondary Education, as well as the
choice of appropriate web-based tools and their usage in order to develop students’ performance,
interaction and active participation during the educational process (Ali et al. 2009, Murphy & Lebans
2007, Callagher 2008, Passey et al. 2003).
Google Groups (http://groups.google.com) which constitute a free, online service that allows the
formation of discussion groups, were chosen to be used in the classroom and outside it as well. With
the “web-based class”, it was expected to increase students’ interaction with each other, extend
classroom discussion to online and promote students to conduct debates and exchange ideas.
Additionally, teacher could prepare classroom materials and online resources were available to
students with Google Docs application.
This paper focuses on investigating the effectiveness of blended learning with the use of Google
Groups and other Google applications as complementary tools in teaching and learning of Informatics
in Secondary Education.
The aims of the current research are to:
Record students’ opinions, attitudes and expectations regarding the exploitation of eLearning in
Secondary Education
Record and investigate possible changes in a) students’ attitudes and convictions towards the
use of blended learning and b) students’ active attendance in the educational process with the
implementation of collaborative learning techniques from distance with Google applications usage
(groups and shared documents),
Investigate the effectiveness of blended learning in Secondary education both in students’
performance and in their active participation in face-to-face classrooms and in virtual classrooms,
as well.
The research examines (and measures where it is feasible), the impact of the use of eLearning in the
students’ activities in respect of their educational record, their active attendance and their
collaboration in the class at the teaching of the Informatics course.
With the investigation of blended learning effectiveness in high school, the hypothesis of research will
be confirmed or rejected: blended learning as an additional/complementary tool of teaching and
learning is effective as much in students’ performance as in their positive attitude to the use of new
technologies in the educational process of the Informatics course.
More specifically, the research questions are: a) what are the students’ attitudes before, during and
after the use of technology as supporting tool of educational process, and b) how their active
attendance in the educational process with the exploitation of eLearning and techniques of
collaborative learning from distance, is encouraged. The current research does not aim to compare
the effectiveness of exploitation of eLearning with the one without it, but it approaches the way of
students’ motivation and it gives an example of good practical exploitation of ICT.
2. Google application in Informatics curriculum
The curriculum of the Informatics course, in the second class of Gymnasio includes 4 units: a) know a
computer as a united system which aims to introduce students in hardware, multimedia and computer
networks, b) communicate through a computer, which familiarizes students with files and folders and
687

Eleni Rossiou and Erasmia Papadopoulou
how to use “help” when they need it during computer use, c) use expression and communication tools
which aims to train students in web-searching, spreadsheets, graphs and presentations, and d) ICT
and professional activities.
Google Groups was used to support the sharing of information and effective communication. Also,
students could share their files, post ideas, and conduct discussions either on the web or via email. All
emails sent to the group were archived for future reference and were easily searchable. Also,
students could organize web resources as they were working on a team project, they were informed
by the teacher of announcements, new assignments, latest news and they could extend class
discussions online
Google Docs were used by students and teacher as well. Students could work on their files from any
computer, thus they could continue to work on the assignments they had started in the classroom,
they could receive feedback from teacher or collaborators of their files and they could publish their
final individual or team work project. Emphasis is given on the use of spreadsheet documents relating
to the third unit of the curriculum.
With Google Calendar teacher and students customized reminders and it helped them to stay on
schedule since any event was flagged to them by email. Google applications of email and chat were
used for asynchronous and synchronous communication and student- student/s and teacherstudent/s
interactions.
3. Research methodology
The steps that have been followed in planning the current research are (Cohen & Manion 1994): a) to
define the aims and decide the necessary information to be gathered, b) define the sample and
choose the research method c) design the survey, the structure of it, its form and the way that data
would be analyzed, d) test in pilot and improve the questionnaire e) deliver the questionnaire, collect
the data from interviews, and gather the observation data, f) code the data, order it in tables and
statistically analyze and g) write the research report.
The above research method was used because we were aiming at collecting data in real time and
gathering information that could be collected in various ways such as questionnaires filled in-situ and
structured or semi-structured interviews. Also, triangulation strategy was used for increasing the
validity of results and research findings.
In details,
The recording of students’ perceptions, attitudes, expectations and potential hesitations
regarding eLearning in Secondary Education have been achieved via a questionnaire at the
beginning of the research,
The recording and investigation of potential changes in students’ attitudes and convictions
towards the exploitation of blended learning using Google applications as well as the way that
students are encouraged to participate actively during the educational process through
collaborative learning techniques, have been achieved via the questionnaire delivered in-situ and
both interviews implemented and teacher’s/researcher’s observation,
The effectiveness of blended learning using Google applications in high school has been
investigated through interviews and students’ performance in written tests in addition to data
collected through direct observation of students’ active participation in both face-to-face class
lessons and web-based assessments.
3.1 Theoretical and Methodological framework
Researchers of the use of eLearning in educational process have not focused solely on the
effectiveness of educational content but also significantly on a) interaction between students, b)
collaborative activities, c) students’ encouragement of active participation, d) students’ profile,
knowledge background and skill in using new technologies and e) affordances of technological
environments (Govindasamy 2002, Oblinger 2005, Ebner et al. 2007, Hrastinski 2008, Alivisos &
Apostolos 2009).
688

Eleni Rossiou and Erasmia Papadopoulou
The questionnaire design, the structure of interviews and the topics of observation follow the
theoretical framework based on three main types/issues:
Participation
Educational content and educational methodology
Technological environment
In order to manage the above types we determined the following measurable criteria accordingly
For participation the criteria are: students’ motivation to be active during the web-based
networked activity, students’ expectation of it and its impact on the learning process
For the educational content and the educational methodology: the clarity of aims and objectives of
web-based activities, the degree of students’ satisfaction from the available educational material,
the possibilities of interaction and collaboration, the time available for discussion and the
importance of attendance from distance
For the evaluation of the technological environment the criteria are: the degree of students’ easy
access and use of the Google applications.
3.2 Data collection
The investigation of the exploitation of blended learning using the Google application as a
complementary educational tool in Secondary Education gathered information and data via both the
quantitative approach with a questionnaire used and the qualitative approach with students’
interviews. Apart from questionnaires and interviews, observation has been used which is referred to
in the information collection from the researchers themselves and focuses on their opinions and not
on students’ answers (Makrakis 1998, pp.298). The interview process took place after the return of all
questionnaires. Students participated voluntarily and the sample constituted of eight (8) students. The
interviews took place at the laboratory of Information Technology in the school, which was familiar to
the interviewed students and this place was agreed between teacher/researcher and students. The
duration of each interview was 25-30 minutes and the interviewer was the researcher herself.
The points of observation were:
The technological environment measuring: a) the degree of installing and handling the
environment, b) the equipment needs and c) the appearance of technical problems because of
the technological systems or the Internet access
The educational material evaluating a) the clarity of objectives in the educational material and the
degree of its contribution to achieving them, b) the usability of the educational material and the
possibility of its re-use and c) the facilitation of reflection, rethinking and expressing of questions
and different opinions
The teaching methodology measuring: a) the degree of students’ satisfaction from their
participation, b) the degree of offering collaborative activities and active students’ participation
and c) the degree of time available for discussion.
The first questionnaire was delivered at the beginning of the research (before using Google
applications in the educational process) and its aim was to explore students’ expectations from the
use of eLearning methods in Secondary Education. The ten (10) item questionnaire consisted of two
parts: a) student’s profile (sex, knowledge of foreign languages) and existing familiarization with new
technologies (frequency of computer use and possibility of Internet access), b) the use of ICT and the
possibility of supporting the educational process through Internet and Google applications to achieve
educational aims.
By delivery of the second questionnaire, students had already implemented web-based activities with
Google applications (Google Groups, Forums, Talk, Docs, Search, Diary, Translation). All students
participated in discussions in their group for particular topics (given by the teacher or posted by
themselves according to their needs), collaborated for particular assessments, searched for
information and translated it when necessary, recorded and edited their results using Google Docs,
chatted when it was necessary to review and rearrange their plans, were updated by Diary. The main
aim of the second questionnaire was to explore students’ motivation in participating, students’
expectations and experiences prior to that period use of blended learning in Secondary Education.
The questionnaire of 25 items consisted of four (4) categories of questions which refer to students’
689

Eleni Rossiou and Erasmia Papadopoulou
profile, motivations, expectations and effects of networked activities in educational process, and
evaluation of educational material, educational methodology and technological environment. The
statistical analysis has been done with the use of statistical package SPSS (v.10).
3.3 The sample
The research sample consisted of 51 students of the second class of the Experimental School of
Aristotle University, Thessaloniki who were attending the obligatory course of Informatics, one hour
per week in the class, during the school year 2010-2011. Sampling has been done with the method of
“convenience” or “accidental” sampling (Cohen & Manion 1994, pp.128-130). The size of the sample
is considered as satisfactory because a sample of 30 people is considered by many researchers of
the field as the minimum size when a statistical analysis is designed (Cohen & Manion 1994, pp. 131).
3.4 The reliability and validity of the research
The triangulation, the research technique that “many researchers accept but a minority use in praxis”,
increased the authors’ certainty about the validity of the research results (Cohen & Manion 1994,
pp.321). With the use of questionnaire and interviews the research attempts to answer the same
research questions with different way and aspect. Through questionnaires, it is aimed to collect as
many students’ objective opinions as possible and through statistical analysis to generalize the
conclusions( where it is feasible). The anonymity of students during the questionnaire filling process
strengthens the degree of students’ honesty and sincerity and it protects the results of the current
research from potential wrong answers or students’ instigated behaviors in the case where exists the
uncertainty of their evaluation from the teacher. With the qualitative approach and the interviews, the
triangulation aims to improve findings related to the reliability and validity of the research. Moreover,
we achieve a more detailed investigation of the research issue with the common participation of
teacher and students in an interactive relationship which leads to a more equitable interpretation of
the phenomena and perceptions. For the above reasons, we have designed the same main issues for
the questionnaires items and the interview questions. Thus, a double result is sought: a) the
agreement of the results from the two techniques and b) the awareness of unexpected results, after
the discussion with the students.
4. Results and findings
Two questionnaires were delivered to 51 students of the second class of the high school. The return
rate of the first one, the questionnaire of the detection of needs, was 46/51 and 48/51 for the second
one - the questionnaire on the investigation of students’ opinions related to their experience of
blended learning implemented using Google applications.
4.1 Needs and expectations
Almost all students (52,17% boys, 47,83% girls) state that they use their personal computer (97,8%)
and do have Internet at home (91,3%). This high percentage is not unexpected since these students
live in ICT days and also they were offered a laptop from school, via the European project “Digital
Class” of the Ministry of Education.
As far as it concerns students’ expectations regarding the support of the educational process with
ICT, more than ¾ (89,1%) expect teachers to complement their traditional teaching with use of
Information Technology, computers and Internet (Figure 1).
Regarding students’ support with web-based audio-visual communication with their classmates and
teachers, students state their expectation is for a frequency of weekly or twice a month and a 1-2 hour
duration (Figure 2). During the planning of the complementary web-based lessons, the planning team
of the current research took students’ expectations into consideration.
Students were asked to express their opinion about the degree of contribution of educational activities
in their learning. Most of them answered that the most important activity is the procedure of solving
their questions frequently and also, their active participation in educational activities face-to-face or
from distance (Figure 3).
690

Eleni Rossiou and Erasmia Papadopoulou
Figure 1: Students’ expectations from embedding ICT in learning
Figure 2: Students’ expectations regarding the frequency and duration of web-based communication
Figure 3: Students’ opinions for the contribution of complementary educational activities in learning
process
691

Eleni Rossiou and Erasmia Papadopoulou
As it relates to students’ expectations of their teachers during their revision study, one out of three
thinks that it is important
(37%) or very important (28,3%) to have the revision of the important
concepts from the teacher face-to-face or from distance. Only 4% believes that it is not important. A
small percentage considers it as important (23,9%) or very important (6,53%) to be given extra
educational material. On the other hand, most of the students (89,1%) feel the procedure of
answering questions frequently is very important (39,1%), important (34,8%) or moderately important
(15,2%). As far as it relates to the individual studying, almost 40% of the students believe that it is of
little importance or unimportant. It is worthwhile recognizing the students’ expectations about the
active participation and collaborative activities. Definitively, they consider active participation in faceto-face
or distance learning activities very important (39,1%) or important (34,5%) and only 10%
consider them as unimportant or of little importance. Similarly the importance of having group-working
and collaborative activities in students’ learning is considered very important (21,74%) or important
(52,17%) by them. The high percentage of expectation of using collaborative learning, are totally in
line with the opinion of Dewar & Sharp (2006) who suggest the exploitation of collaborative learning
so as to give the opportunity for reflection and structured way of exercise.
4.2 Students opinions after their experience with blended learning
Forty eight (48) students of second class of high school answered the second questionnaire (24 boys
and 24 girls).
Most of the students have ADSL Internet connection when being at home (Figure 4).
Figure 4: Students’ profile related to Internet access location
In this questionnaire it was clear that almost all students who participated in the use of web-based
activities, participated more than once (66,67%), and one out of
three students participated only once
(Figure 5). This result is very encouraging for the follow up of this research and also, as we saw from
the interviews, students who got involved because of curiosity wanted to continue this new method of
learning process.
Figure 5: Students’ profile related to the frequency of their participation in using Google applications
692

Eleni Rossiou and Erasmia Papadopoulou
Most of the students stated that the main reason for their willingness to participate is that it is a new
challenge for them to use Information Technology in studying, and 50% of them participated due to
their curiosity. Two of the interviewed students in a relative question said: “let’s not forget that
students of today are part of the high tech generation and have grown up having Internet in their daily
life.” Almost half of the students who have used Google applications and assessed web-based
assignments group justified their participation with the willingness to expand their knowledge, and the
58,3% due to the fact that they didn’t have the opportunity to solve their queries in the traditional face-
to-face limited time teaching. One out of three students joined in the Google Group in order to benefit
a more time and with more quality communication with the teacher. This fact is interesting since
someone in the interviews emphasized the significance of this communication and the expectation to
“have the IT teacher as a chief guide in the new experience trip of exploring this new world of new
technologies”. These latter results are in line with the results of the teacher’s observation sheet, and
both of them confirm different researches that come to the conclusion that new generations prefer to
socialize through Internet or with usage of new technologies.
As far as it concerns the frequency and the duration of the web-based activities, students’ opinions
didn’t change after their experience with blended learning since most of them prefer weekly (52,1%)
or twice per month (27,1%) frequency with less than 2 hours duration (Figure 6).
Figure 6: Students’ opinions about the duration and frequency of the web-based activities
The degree of satisfaction from the educational material is high or above average for more than 50%
of students and more than 80% felt comfortable enough to easily express their questions and solve
their queries (Figure 7).
Figure 7: Students’ opinions about the available educational material and the possibility of solving
queries during the web-based activities.
It is worth mentioning that almost all students consider it important to have the possibility of attending
the activity when at home (Figure 8).
693

Eleni Rossiou and Erasmia Papadopoulou
Figure 8: Students’ opinion on degree of importance of participating from home
Students’ opinions related to the time available for discussion and the implementation of collaborative
activities during the use of Google applications, i.e. Google Groups and shared Google Docs
accordingly, resulted in the need for more time for discussion and interaction with their classmates. In
detail, students stated that there was enough (39,58%) or too little (35,42%) time for discussion but
more than 50% of the students believe that there was sufficient time for collaborative activities (Figure
9). Later on, it was clarified in the discussion that their expectation is to be given the opportunity to
exchange opinions not only asynchronously
but synchronously, as well. This result was not expected
since we all know that students meet live in the traditional classroom, so we need to investigate this
more.
Figure 9: Students’ opinions about the evaluation of educational methodology as it concerns the time
sufficiency for discussion and collaborative activities
When it comes to the degree of difficulty of access to web-based environments
of activities and
configuration of their settings, as expected, students did not have any specific difficulty as they feel
comfortable in these types of environments (Figure 10). The ease of access was also confirmed from
the interviews and the teacher’s observation sheet.
Furthermore, as students explained in the interviews, although they did not know of the existence of
Google Groups and the management of shared Google Docs, they found it extremely interesting to
have the opportunity of collaborating in such a manner. Because of having this opportunity, they can
overcome different obstacles that they face when they have group activities also in the context of
other lessons.
5. Discussion-conclusions
If one makes a general review of the main results of the current research, the main hypothesis that
‘the blended learning as a complementary teaching and learning tool is effective in performance and
students’ positive attitude towards the use of collaborative web-based tools as Google applications,
during education as wel’ is confirmedl. Many researchers agree that traditional learning models in
distance learning processes with the use of ICT are not examples of success stories, but on the other
694

Eleni Rossiou and Erasmia Papadopoulou
hand, educational activities that demand students’ active participation, are recognized as factors that
lead to improvements in the effectiveness of teaching (Hedesting & Κeptelinin 2002, Laurillard 1993,
Hearnshaw 1999, Raptis & Rapti 2001).
Figure 10: Students’ opinions about the ease of access and configuration of settings in technological
environment
Students familiarized themselves very quickly with the new way of communication via the web-based
group environment and it is remarkable that their participation was more active than in the traditional
classroom. Since students do actively participate in the web based group, they show their positive
attitude in the use of new technologies by using Google applications in the context of parallel activities
in the class and outside of it. Furthermore, their opinions about the extra skills and capacities that
their teachers should have in order to implement blended learning activities are encouraged. These
skills are related to organizational management, encouragement and students’ motivation,
friendliness, but also the creation of a safe learning environment where students can freely express
their questions and queries and trace their a priori knowledge. By giving students the opportunity to
easily express their questions and discover their fullest potential by gaining new skills, in an
environment where they feel comfortable since they can easily access, the teacher can encourage
their active attendance in the educational process.
After the written test of the second semester, and comparing the students’ performance in this with
their performance in the written test of the first semester, there are strong indications that there was
positive impact in the comprehension and usage of different concepts of Informatics. But these
indicators are not evidence yet, but successive repetition of blended learning process with new
activities is required in order to confirm or reject its effectiveness in the learning level. However, the
main hypothesis of this research can be confirmed.
The results indicate that students improved their level of knowledge of the concepts of Informatics and
this encourages the usage of collaborative tools as an educational intervention of blended learning in
teaching Informatics in the Secondary Education. The level of students’ satisfaction from their
experience in participating in the web based activities was very high. Their positive attitude in the use
of blended learning seems to be a very promising educational activity in obtaining an improvement in
their future studies. It is optimistic to see that students tendencies may turn to collaborate and
willingly involved in web-based additional educational activities and use their digital familiarization in
academic level and not only for chatting, Facebook, blogging, downloading of movies and mp3,
avatars etc
695

Eleni Rossiou and Erasmia Papadopoulou
Having in mind that teachers always look for new ways of improving the educational process at
school, the results of this research make one more step in this direction and it is certain that web-
based groups and activities should be developed and expanded further in the learning process of
Secondary Education. Future research is proposed to explore students’ opinions on the
implementation of blended learning in other courses too.
References
Ali, W., Nor, H. Hamzah, A. and Alwi, H. (2009) “The conditions and level of ICT integration in Malaysian Smart
Schools”, International Journal of Education and Development using ICT, (IJEDICT), Vol. 5, No. 2, pp.21-
31.
Alivisos, S. and Apostolos Κ. (2009) “Pedagogically-Oriented Evaluation Criteria for Educational Web
Resources”, eLearning Papers, Vol. 17, December. Retrieved on 15.2.2011 from:
http://www.elearningeuropa.info/files/media/media21214.pdf
Callagher, Μ. (2008) How can Student Interactivity be enhanced through the use of a Blended Learning
Approach? Elearning Fellow, New Zealand Ministry of Education. Retrieved on 13.10.2011 from:
http://markcallagher.com/wp-content/uploads/2008/12/research-report-mark-callagher.pdf
Chitanana, L. (2010) “Students’ perceptions of the role of telecollaborative learning projects: a case of the global
teenager project at Mucheke high school in Zimbabwe”, International Journal of Instruction, Vol. 3, No. 1,
pp. 19-38. Retrieved on 13.05.2011 from: http://www.e-iji.net/dosyalar/iji_2010_1_2.pdf
Cohen, L. and Manion, L. (1994). Research Methods in Education. 4th Ed. Routlege, London and New York.
Dewar, B. and Sharp, C. (2006) “Using evidence: how action learning can support individual and organisational
learning through action research”, Educational Action Research, Vol. 14, No. 2, pp. 219-237.
Dillenbourg, P. (1999) CollaborativeLearning: Cognitive and Computational Approaches. Advances in Learning
and Instruction series, Pergamon, Elsevier, 1999.
Ebner, Μ. and Ηolzinger, Α. (2007) “Successful implementation of user-centered game based learning in higher
education: An example from civil engineering”, Computers & Education, Vol. 49, No. 3, pp. 873–890,
Elsevier.
Govindasamy, T. (2002) “Successful implementation of eLearning. Pedagogical considerations”, Internet and
Higher Education, Vol. 4, pp. 287–299.
Hatziplis, P. (2005) “Evaluation of ICT contribution in teaching economical subjects in secondary education”, In
Gialama A., Tzimopoulos N. and Hloridou, A. (Eds), Proceedings of 3rd Hellenic Conference in ICT, pp.
210-219 (in Greek).
Hedesting, U. and Kaptelinin, V. (2002) “Re-contextualization of teaching and learning in videoconference -
based environments:
an empirical study”, Proceedings of the Conference on Computer Support for
Collaborative Learning: Foundations for a CSCL Community 2002, pp.177-188. Boulder, Colorado, 7–11/1/
2002.
Hearnshaw, D. (2000) “Towards an objective approach to the evaluation of videoconferencing”, Innovations on
Education and Training International, Vol. 3, No. 37, pp. 210-217.
Hicks, M., Reid, I., & Rigmor, G. (2001) “Enhancing online teaching: designing responsive learning
environments”, The International Journal for Academic Development, Vol. 6, No. 2, pp. 143-151.
Hrastinski, S. (2008) Asynchronous and Synchronous ELearning.
EDUCAUSE Quarterly, 31 (4).
Laurillard, D. (1993).
Rethinking University Teaching: a framework for the effective use of educational technology,
London, Routledge.
Makrakis, V. (1998) “Evaluation of Open and Distance Education Systems”, In Vergidis D., Lionarakis, A.,
Lykoyrgiotis A., Makrakis V. and Matralis H. (Eds), Open and Distance Education, Vol. A, pp. 245-304,
Patra,
Greece: Hellenic Open University (In Greek).
Murphy, J., and Lebans, R. (2007) “Using Web 2.0 Tools in the Secondary School Classroom: Unexpected
Student Learning, Evolving Teacher Identity, and Emerging Pedagogical Issues”, International
Journal of
Technology in Teaching and Learning, Vol. 4, No.2, 134-147.
Oblinger, D. and Oblinger, J. (2005) Educating the net generation. EDUCAUSE. Retrieved on 1.2.2011 from
http://www.educause.edu/educatingthenetgen
Pass ey, D., Rogers, C., Machell, J., McHugh, G., and Allaway, D. (2004) The Μotivational Effect of ICT on
Pupils: Emerging Findings London: DfES. Retrieved on 15.2.2011 from
http://downloads01.smarttech.com/media/research/international_research/uk/lancaster_report.pdf
Papadakis, S., Paparrizos K. and Rossiou E. (2006) “Using Blended Learning in Traditional Face-to-Face
Instruction: A case study teaching Algorithms to undergraduate students”, Proceedings of E-Learn 2006
“The International Forum for Researchers, Developers, and Practitioners to Learn about the Best
Practices/Technology in Education, Government, Healthcare, and Business, pp.839-846. Waikiki Beach,
Honolulu, Oct. 13-17.
Passerini, K. and Granger M.J. (2000). “A Developmental
Model for Distance Learning Using the Internet”,
Computers and Education, Vol. 34, No. 1, January, pp. 1-15.
Rapti s, A. and Rapti, A. (2001) Learning and Teaching Information period, Raptis Edition, Athens (in Greek) .
Schank, R. C. (2001) Designing world class eLearning: How IBM, GE, Harvard Business School, and Columbia
University are succeeding at eLearning. New York: McGraw-Hill.
696

Training Methods and Tools: Could eLearning be a Viable
Solution to Solve SMEs Training Problems?
Andrée Roy
Université de Moncton, Moncton, Canada
andree.roy@umoncton.ca
Abstract: The purpose of this study is to identify the different training methods and tools used by larger
businesses and the problems associated with these methods and tools and to determine, through a multiple case
study, the extent to which small and medium-sized enterprises in Atlantic Canada use the same methods and
tools than the other businesses, and if so, if they face the same problems than other organizations in the use of
these methods. The purpose of the study is also to verify if the utilization of eLearning may solve some of the
training problems encountered by SMEs, and if so, why?
Keywords: eLearning, SME, training methods, training tools
1. Introduction
In a context of knowledge economy where knowledge is increasingly regarded as a capital and when
the integration of technology in training is a determining factor in business competitiveness (Industry
Canada, 2002; Quirion, 2000), SMEs should, in principle, use eLearning at least for the advantages
associated with this one in order to be more competitive.
Actors of economic development should also hope that SMEs are using eLearning to better equip
these companies, increase their competitiveness and boost economic development because,
according to Smith and Hayton (1999), the most successful economies are those with the best trained
individuals. Furthermore, it seems that SMEs who are the most successful are the ones who offer
more training to their employees (Smith and Hayton, 1999), this observation is also valid for larger
enterprises (Barbian, 2002).
It is therefore important to understand the training process of SMEs, in particular, the stages of the
selection of training methods and training tools, in order to be able to identify the nature of the
problems in these stages and determine if eLearning can be a solution to the training problems of
SMEs and increase the competitiveness of SMEs because capability development of small firms in
Atlantic Canada remains critical to economic prosperity as in other parts of the world (see for example
Matlay, 1999; ACOA, 2005; Jayawarna & al., 2007).
In Canada, particularly in Atlantic Canada, the contribution of SMEs towards a healthy economy has
been recognized. They are the fastest growing segment of the economy and are considered as the
foundation of economic development (APECA, 2005; Mittelstaedt et al., 2003). They represent the
majority of businesses and they also create the majority of jobs (APECA 1998, 2005). Yet, despite
their great contribution to the region’s economy, there are very few studies on them and even less on
the training methods and tools they use and this, notwithstanding a strategic and economic dimension
to have better trained employees since training is supposed to lead to better economic performance
of enterprises.
The purpose of this study is two-fold. After having identified the different training methods and tools
used by SMEs and the problems associated with these methods and tools, based on a survey of the
documentation on the issue, the first objective is to determine, through a case study, if Atlantic
Canada SMEs use the same methods and tools than the others, and if so, if Atlantic Canada SMEs
face the same problems than other organizations in the use of these methods and tools. The second
step is to verify if the utilization of eLearning may solve some of the problems encountered by SMEs,
and if so, why.
Thus, the remainder of the document is arranged as follows. Section 2 presents the method used for
the article. Section 3 presents a literature census on the training methods and tools used by
businesses to train their employees and the problems associated with these training methods and
tools. Section 4 presents, through a case study, the extent to which SMEs in Atlantic Canada use the
same methods and tools than larger businesses and if they face the same problems. It also
697

Andrée Roy
determines if eLearning may solve some of the training problems encountered by SMEs. The
conclusion and discussion will be included in section 5.
2. Method
Given the present state of knowledge on training in SMEs and eLearning, the method used for this
article includes a census of the literature on training and eLearning combined with a qualitative and
exploratory research approach, i.e. multiple case studies. The literature census covers more
specifically how businesses evaluate the results of the training they offer and the eLearning tools
available to do so. The case study method is well adapted in situations where theoretical propositions
are few and field experience is still limited (Yin 1994). A multiple-site case study allows one to
understand the particular context and evolution of each firm with regard to eLearning. Sixteen SMEs
located in the Atlantic Region of Canada were studied, that is, four in each of the provinces of New
Brunswick, Nova Scotia, Prince Edward Island and Newfoundland, selected to be sufficiently
successful (at least 10 years in business) and representative in terms of industry and size, for
theoretical generalization purposes. These manufacturing SMEs stem from various sectors, such as:
textile, oil and gas, pulp and paper and processed food sector. Following North American research
(Mittelstaedt, Harben and Ward 2003; Wolff and Pett 2000), a small enterprise (SE) is defined as
having 20 to 99 employees, whereas a medium-sized enterprise (ME) has 100 to 499.
Data were collected through semi-structured tape-recorded interviews, ranging approximately two
hours each, with the owner-manager or CEO and with the firm’s HR manager or manager responsible
for training. eLearning users were also interviewed in four cases. Interview transcripts were then
coded and analyzed following Miles and Huberman’s (1994) prescriptions with the assistance of the
Atlas.ti application. For reasons of confidentiality, fictitious names of individuals and firms participating
in the study were used. As presented in the research results section, these firms range in size from 60
to 485 employees and operate in industries whose technological intensity varies from low to high. All
export except for one firm (M). The SMEs were regrouped in four eLearning profiles of increasing
intensity, based on the extent of their awareness and use of eLearning (none, weak, average, strong).
3. Literature census
A business preoccupation is ensuring that all categories of employees possess current and up-to-date
knowledge and skills. Having well trained employees is also a question of business performance and
business survival. The recent technological advances, along with a reduction of their costs, allow
businesses to reconsider training more easily including the method and tools of offering this training.
3.1 Training methods
Training method generally refers to the method that will be used in order to train the employees and to
enable learning (Laflamme, 1999). The training method must be selected to ensure the training
offered is effective, efficient, profitable, and interesting (Hansen, 2006). The objective of a training
method being to facilitate the transmission of knowledge, know-how, and know-being (Laflamme,
1999). Training methods can be formal or informal (Kotey and Sheridan, 2004; Tanova and Nadiri,
2005). The informal methods are neither planned nor documented and are unstructured (Smith and
Hayton, 1999; Tanova and Nadiri, 2005). In contrast, formal methods are planned, structured, and
documented and are offered both inside and outside companies [(Shepherd and Ridnour, 1996;
Tanova and Nadiri, 2005). Table 1 summarizes the different training methods identified by various
researchers (Bazin, 1994; Fernandez, 1988; Laflamme, 1999; Meignant, 1997; Mucchielli, 1988;
Sonntag, 1994). The methods are grouped according to whether they are considered to be
affirmative, interrogative or active.
Table 1: Training methods
Affirmative methods Interrogative methods Active methods
Lecture
Presentation and discussion
Conferences and seminars
Job rotation
Coaching
Employment training
Exercices and tutorials
Mnemonic method
Computer-based training
Vestibule training (external)
698
Case studies
Role playing
Simulation and gaming
Learning by doing
External internship
Creativity method
In-basket
eLearning

Andrée Roy
Studies have been made to determine what training methods are considered the most relevant and
preferred by the companies. It appears that larger companies use more formal training methods than
SMEs (Tanova and Nadiri, 2005). Mentoring (one to one), conferences, seminars, and short lectures
delivered in a classroom format are the training methods deemed most relevant and preferred by
businesses. Some studies have also confirmed that the Internet is considered a very effective way of
delivering information to different learners, as much in SMEs than in larger companies (Bassi and Van
Buren, 1999b; Elbadri, 2001; Lawless, Allan and O’Dwyer, 2000; Smith and Hayton, 1999).
According to the study done by Lawless, Allan and O’Dwyer (2000), 93% of SMEs prefer the "one to
one" training, ie that is the training directly offered to them by a trainer and this exclusively.
Furthermore, 70% of SMEs prefer to receive training on demand rather than regularly scheduled.
The choice of the training method should be made according to the needs and objectives of training,
the learning style of individuals who will be trained, the resources available, and the learning
principles. An approach that involves a variety of methods (different types of learning), tools, and
examples has a better chance of success (Bassi and Van Buren, 1999a; Bassi, and Van Buren,
1999b; Elbadri, 2001; Garavaglia,1993) because, as mentioned by Fry (2001), companies are made
up of employees whose age and experience vary, and have different preferences regarding ways of
learning, including technology.
When the training method has been chosen, you must select the tool or tools that are going to be
used in the provision of the training course. The choice of tools plays an important role in the learning
outcomes.
3.2 Training tools
Training tools are aids to learning and support for training (Laflamme, 1999). They should be selected
based on criteria such as the level of action required of the trainees and the trainer, the level of
interaction between the trainees and between them and the trainer, the number of senses that we
wish to touch through training, the ease of producing and using the tool, the cost of production and
use of the tool (Mayo and Dubois, 1987), and the learning objectives pursued (Noyé and Piveteau,
1993).
The tools can be grouped into four broad categories: visual tools, auditory tools, audiovisual, and
interactive tools (Laflamme, 1999). Table 2 summarizes the various tools as they are visual, auditory,
audiovisual, and interactive.
Table 2: Training tools
Visual
tools
Blackboard
Overhead projector
Lecture notes and
explicative documents
Auditory
tools
Tape recorder
Telephone
Audiovisual
tools
Slide show
Film
Video tape recorder
Interactive
tools
Computer
Courseware
Simulator
Multimedia
4. Research results
The study shows that SMEs draw on a mixture of training methods in order to offer training to their
employees and to increase their productivity as well as their economic performance. The study also
shows that the majority of SMEs use eLearning as a training method but that some do not use it at all
as shown in Table 3. A detailed study of SMEs stated knowledge about eLearning and their use of it
enables us to qualify their level of use. This analysis also provides for categorizing SMEs into four
distinct profiles of eLearning users. There are SMEs that use eLearning a great deal (strong use),
those that use it quite a bit (average use), those that don’t use it much (weak use), and those that
don’t use it at all (non-existent use) as indicated in Table 3.
An even more in-depth analysis of these SMEs enables us to see what training methods and tools
they use and, the reason they use eLearning.
699

Andrée Roy
Table 3: Profiles of eLearning’s Utilization by SMEs 1
Size
Number of
employees
Profile I
STRONG
(C, D, K, L)
Profile II
average
(B, E, M, O)
Profile III
WEAK
(A, F, I, J)
Profile IV
non-existent
(G, H, N, P)
300 to 485 60 to 280 150 to 350 75 to 400
ELearning
Utilization strong average weak non-existent
4.1 Training methods used by SMEs and the reasons to use eLearning
SMEs in Atlantic Canada use a variety of training methods, as shown in the review of literature for
larger companies. They select methods according to their training needs and sometimes depending
on the training that is accessible to them. These methods include mainly lectures, learning by doing
as well as conferences, exercises and tutorials, formal presentation, computer-based training (CBT),
and eLearning as shown in Table 4. The amount of training offered as well as the number of training
methods used by SMEs, to ensure that the employees’ skills are up to date, seem to confirm the
hypothesis that SMEs see training as a way to be more successful and not as an expense.
Some SMEs, especially those that have a strong or average use, use the eLearning for all employee
groups. To this end, Karen says: « We use eLearning for all our employees: production workers as
well as the executives » (K:661-665). SMEs that have a “weak” use of eLearning, use it only for their
executive employees and office workers. An interesting fact to note, in the case of a “weak” use of
eLearning by SMEs, it is usually the employee who has chosen to develop his/her knowledge through
eLearning. Julien provides an example “Some employees have chosen eLearning, but it is usually on
an exception basis, to develop their work knowledge and it was suggested by the employee and not
the employer.”
The decision to use eLearning by the employees in order to develop their knowledge is linked to their
perceived benefits of eLearning and to its capacity to solve some training problems. Fiona provides
us with an example “the employee chose this method because she wanted to continue working; she
needs to work; she wanted to complete her bachelor’s degree while working at the same time”.
Denise gives us another example of the perceived benefits of eLearning by the employees “Since I
have a three-year-old daughter and that my work schedule is rather full, I can complete the work at
3:00 AM in my slippers. Honestly, I chose eLearning because of the flexibility”.
Table 4: Training methods Used by SMEs
METHODS SMEs
Profile I Profile II Profile III Profile IV
strong average weak non-existent
C D K L B E M O A F I J G H N P
Affirmative
lecture x x x x x x x x x x x x x 2 x x x
presentation/ discussion x x x x x
conferences / seminars x x x x x x
job rotation x x
“coaching” x x x x
exercises and tutorials
Interrogative
x x x x x x
Computer-based training (CBT) x x x x
1 Nota: In Table 3, a “strong” use means that the business regularly uses eLearning to train its employees. An “average” use
means that the business has developed at least two courses in eLearning format and that the production employees must take
these courses. A “weak” use means that only a few employees use it in the business and a “non-existent” use means that the
business does not use eLearning to train its employees and that they do not use it to develop their knowledge.
2 This method is not used by the SME but is used by suppliers when the staff and office workers are on training abroad.
700

METHODS SMEs
METHODS SMEs
Active
Andrée Roy
Profile I Profile II Profile III Profile IV
strong average weak non-existent
C D K L B E M O A F I J G H N P
case studies x x
role playing x x
simulation and gaming x x
“learning by doing” x x x x x x x x x x x x x x x x
“eLearning ” x x x x x x x x x x x x
Table 5 shows each of the perceived benefits of eLearning and its capacity to solve training problems
encountered by SMEs. The flexibility and accessibility as well as the cost are the main perceived
benefits, that the eLearning is used or not used by SMEs. As for the employees, flexibility and
accessibility, speed, privacy and autonomy, interactive feedback, cost, and ability to meet their
learning style are their perceived benefits of eLearning.
Table 5: Perceived benefits and capacity of elearning to solve training problems
Perceived benefits and PME
capacity of eLearning Profile I Profile II Profile III Profile IV
to solve training problems strong average weak non-existent
C D K L B E M O A F I J G H N P
Flexibility and accessibility x x x x x x x x x x x x x x
Modularity x x
Speed x x x x x x x x
Privacy and independence x x
Interactive feedback x x x
Cost x x x x x x x x x
Learning style x x x x x x
Evaluation x x x x x
Distribution of literature x x x
Consistent delivery x x x x
The reasons for SMEs to use eLearning are mainly related to the benefits associated with it and its
capacity to solve training problems. These benefits are: a 24 / 7 availability, a great flexibility, the
possibility to test acquired knowledge and to work with new or unknown materials, an increased
independence in the workplace, an increased privacy in learning, an adaptation to the individual’s
speed and needs, a reduction in training time, a faster upgrade and distribution of training materials,
an increased training speed, a reduction of time lost from work due to training, a reduction in travel by
plane, a reduction in training costs (training, meals, hotel, travel), a consistent delivery of course
content compared to a teacher, the possibility of personalizing learning, and the possibilities to assess
more easily the results of training.
The other perceived benefits of eLearning that could help SMEs to solve some of their training
problems are: the possibility of accessing training that would not be available otherwise, a fast
upgrade of the employees, a reduction of stress and nervousness associated with the training, and
the quality of courses available. The employees also see a specific benefit to eLearning, which is the
possibility of developing their knowledge and advancing their careers.
4.2 Training tools used by SMEs and the reasons to use eLearning
SMEs use a vast range of tools to train their employees. These tools are presented in Table 6 and are
grouped according to the four major categories of tools cited in the literature, that is, visual, auditory,
audiovisual, and interactive tools.
As it can be seen, the training tools used by SMEs are varied and chosen depending on their needs.
The tools used include the visual, audiovisual, and interactive tools. The auditory tools are used inside
the other tools such as slide shows (PowerPoint). The criteria used to select the training tools are
varied and are closely linked to the training method used. The ease of producing and using the tool,
701

Andrée Roy
the cost of production and use of the tool, the learning objectives pursued, the trainer’s level of
qualification, the need to provide training, the need to train in a short period of time a large number of
employees and to standardize the content of the training are some selection criteria of the training
tools for SMEs.
Table 6: Training tools used by SMEs
Tools SMEs
Profile I Profile II Profile III Profile IV
strong average weak non-existent
C D K L B E M O A F I J G H N P
Visual tools
blackboard x x x x x x x x x x x x x x x
overhead projector x x x x x x x x x x x x x x x
lecture notes x x x x x x x x x x x x x x x
explicative documents
Auditory tools
tape recorder
telephone
Audiovisual tools
x x x x x x x
slide show x x x x x x x x x x x x x x x
film (movie) x x x x
video tape recorder
Interactive tools
x x x x
computer x x x x x x x x x x x x x x x
courseware x x x x x
simulator x x
multimedia x x x x x x x x x x x x
eLearning x x x x x x x x x x x x
There is no difference between SMEs that use eLearning and those who do not use it in the selection
of training tools with the exception of tools directly related to eLearning, that is the multimedia tools.
The audiovisual tools which include, among others, slide shows, movies, videos, and video tape
recorders are used by SMEs. Slide shows, more particularly PowerPoint, are used by virtually all
SMEs. To this end, Denise said: « We use PowerPoint as a presentation tool. We incorporate videos
that we filmed in our PowerPoints » (D :523-526).
With respect to interactive tools, i.e. computers, courseware, simulators, and multimedia, they are
used by the majority of SMEs. Interestingly, despite the fact that all SMEs, with the exception of one,
use slide shows (PowerPoint) to train their employees, many do not mention the computer as a tool
used in the training when asked what tools they are using, it is implied for them. The same
phenomenon occurs for small businesses that have computers on the production floor. Moreover,
SMEs (12 out of 16) who use eLearning have mainly recourse to the interactive tools but also the
visual, auditory, and audiovisual, in order to touch various senses of their employees and to increase
their learning and skills. For this purpose, Denise says: « In the two courses currently being
developed there will be pictures, sound, interactivity ... to ensure that employees can demonstrate
their competence » (D:132-138). For its part, Omer said: « We've just installed on the production floor
an on-line course. If an employee is experiencing difficulties with a process or a machine, he can go
to one of the computers and view the course. The course is divided into tabs according to the
process, there is text, photos and videos » (O:665-674).
SMEs are using the tools associated with eLearning for various reasons. They find that it allows them
to use a greater variety of training tools within the same course. They also find that the tools
associated with eLearning allow them to adapt themselves to the various styles of learning of the
employees and thus to improve their learning and skills level. Karen gives us a good example, saying:
« We have online courses, as our courses on forklift and health and safety, all our production
employees have to follow and redo them occasionally in order to get certified or re-certified. [...]They
go on-line and do the courses. In the courses there is text, photos, sound, videos and at the end of
the course, they must do a test, the application notes them and lets them know if they succeeded or
not. […] We also have practical evaluations » (K:103-108:117-119:610-614).
702

5. Conclusion and discussion
Andrée Roy
The study shows that SMEs in Atlantic Canada use a variety of training methods, as larger
businesses and organizations, in order to provide adequate training to their employees and ensure
that all employees possess current and up-to-date knowledge and skills. They select training methods
according to their training needs and sometimes depending on the offer of training that is accessible
to them, especially for SMEs located in remote areas.
The training methods used include mainly lectures, learning by doing as well as conferences,
exercises and tutorials, formal presentation, computer-based training (CBT), and eLearning. SMEs
offer training to their employees in order to increase their productivity as well as their economic
performance.
In general, SMEs have a very good knowledge of how eLearning can help their business to solve
some of the training problems they encounter and at the same time help them to become more
productive and competitive than traditional training. They are aware that the reduction in total training
costs and time related to eLearning, along with its flexibility and accessibility (24 / 7), allow an
organization to solve some training problems and at the same time be more productive and
competitive. The modularity of the courses, the possibility for the employee to learn at his or her own
pace, a faster upgrade and distribution of training materials, an increased training speed, a reduction
of the time lost from work due to training, a reduction in travel by plane, the privacy associated with
eLearning, and the possibilities to assess more easily the results of training are other characteristics
of eLearning that solve some of the training problems of SMEs. Even SMEs who do not use
eLearning believe that it would allow them to solve some of their training problems and be more
productive and competitive.
A vast range of visual, audiovisual and interactive tools are used by SMEs to train their employees.
Auditory tools are used inside other tools such as slide shows. They are using the tools associated
with eLearning for various reasons. They find that it allows them to use a greater variety of training
tools within the same course and therefore makes the training more adaptable to the various learning
styles and capabilities of employees and solve some of the training problems, such as the lost of
interest of the employees toward the course, sometimes encountered in traditional training.
Therefore, it seems that, based on the case study, ELearning offers SMEs the possibility to solve
some of their training problems and contributes to the economic performance of SMEs in providing
better trained employees.
References
APECA. (1998). État de la petite entreprise et de l'entrepreneurship dans la région de l'Atlantique 1998. APECA.
Direction générale des politiques et des programmes.
APECA. (2005). État de la petite entreprise et de l'entrepreneurship dans la région de l'Atlantique 2005. Moncton:
Direction générale des politiques et des programmes.
Barbian, J. (2002). Training top 100: AmeriCredit. Training, 39(3), 42-60.
Bassi, L.J. and Van Buren, M.E. (1999a). Sharpening the leading edge. Training & Development, 53(1), 23-33.
Bassi, L.J. and Van Buren, M.E. (1999b). The 1999 ASTD state of the industry report. Training & Development,
53(2), 3-27.
Elbadri, A.N.A. (2001). Training practices of Polish companies: An appraisal and agenda for improvement.
Journal of European Industrial Training, 25(2/3/4), 69-79.
Fry, K. (2001). ELearning markets and providers: Some issues and prospects. Education & Training, 43(4/5),
233-239.
Garavaglia, P.L. (1993). How to ensure transfer of training. Training & Development, 47(10), 63-68.
Hansen, C. (2006). Shopping for eLearning. The Journal of Corporate Accounting & Finance, 12(5), 19-22.
Industrie Canada (2002, septembre). L'industrie canadienne des services d’enseignement et de formation: vue
d’ensemble des industries de service. Saisie le 04 octobre 2006, de
http://strategis.ic.gc.ca/epic/internet/incet-ecf.nsf/ fr/ok01758f.html.
Jayawarna, D., Macpherson, A. and Wilson, A. (2007), Training commitment and performance in manufacturing
SMEs; Incidence, intensity and approaches, Journal of Small Business and Enterprise Development, 14 (2),
698-720.
Kotey, B. and Sheridan, A. (2004). Changing HMR practices with firm growth. Journal of Small Business and
Enterprise Development, 11(4), 474-485.
Laflamme, R. (1999). La formation en entreprise: nécessité ou contrainte? Québec: Les Presses de l’Université
Laval.
703

Andrée Roy
Lawless, N., Allan, J. and O’Dwyer, M. (2000). Face-to-face or distance training: Two different approaches to
motivate SMEs to learn. Education & Training, 42(4/5), 308-316.
Mayo, G.D. and Dubois, P.H. (1987). The complete book of training: Theory, principles and techniques. San
Diego, CA: University Associates Inc.
Matlay, H. (1999). Vocational education and training in Britain: A small business perspective. Education &
Training, 41(1), 6-13.
Mittelstaedt, J.D., Harben, G.N. and Ward, W.A. (2003), How small is too small? Firm size as a barrier to
exporting from the United States, Journal of Small Business Management, 41 (1), 68-84.
Noyé, D. and Piveteau, J. (1993). Guide pratique du formateur. Paris: Insep Éditions.
Quirion, C. (2000). Les multiples facettes de la formation à distance. Info-Tech Magazine, 21, 14-17.
Shepherd, C.D. and Ridnour, R.E. (1996). A comparison of the sales management training practices of smaller
and larger organizations. The Journal of Business & Industrial Marketing, 11(2), 37-46.
Smith, A. and Hayton, G. (1999). What drives enterprise training? Evidence from Australia. The International
Journal of Human Resource Management, 10(2), 251-272.
Tanova, C. and Nadiri, H. (2005). Recruitment and training policies and practices: The case of Turkey as an EU
candidate. Journal of European Industrial Training, 29(8/9), 694-711.
Wolff, J.A. and Pett, T.L. (2000), Internationalization of small firms: An examination of export competitive
patterns, firm size, and export performance, Journal of Small Business Management, 38 (2), 34-47.
Yin, R.K. (1994), Case study research: Design and methods, 2nd Edition, Thousand Oaks, California: Sage
Publications.
704

Using Blended Learning to Develop Critical Reading Skills
Zuzana Šaffková
Technical University of Liberec, Liberec, Czech Republic
zuzana.saffkova@tul.cz
Abstract: Teacher education programs expect student teachers to succeed in a wide range of academic tasks
that, apart from content knowledge, require effective application of their cognitive strategies as well as their
academic abilities. In order to help the students meet these requirements, a great emphasis should be placed,
among others, on their ability to read critically. However, students usually come to university with limited
experience in academic reading, which reflects in their failure in academic tasks and at worst, consequent
withdrawal from the university. Therefore, the introductory Study Skills course for the first year EFL students was
modified so that it offered students an extended exposure to the reading tasks and activities through blended
learning. An on-line module that supplemented in-class sessions was designed so that it complied with the
principles of scaffolding instruction that helps students, by offering enough support, apply skills and strategies
independently. This also aimed to “facilitate motivational development in reading” and assured “that the students
are not lost or ill-guided” (McNamara, 2007). For tutors, this scheme helped intensively assist the students´
learning and thus foster comprehension monitoring. The effects of the course were examined and the following
research questions were addressed: What are common students´ deficiencies in critical reading skills? Does the
computer-mediated instruction help students understand more in-depth subject matter? What further adjustments
of the course will have to be done? The article presents an insight into the EFL teacher trainees´ common
reading problems, reports on the effectiveness of the on-line environment as judged by the students, and
addresses desirable measures in further research.
Keywords: blended learning, critical reading, reading competence, scaffolding
1. Introduction
The students who enroll in the EFL teacher-training program of the Faculty of Science, Humanities
and Education at the Technical University of Liberec are required just from their first participation in
different courses, seminars and lectures, to demonstrate their ability to cope with a range of tasks that
demand their intellectual and academic maturity. One of the prerequisites necessary for their
successful encounter with a variety of academic tasks is the ability to read critically. However, the
students´ capabilities to use their analytical and evaluative mental processes are generally considered
by the teachers insufficient and superficial. This is mainly manifested in culture studies courses when
the students are asked to read a variety of literary texts, understand and interpret them; as well as in
writing classes, when the students have to organize their thoughts in a logical, persuasive text
supported by proper evidence. In both these types of activities, critical comprehension, interpretation
and evaluation are the most important skills that are imperative in an academic context. The common
deficiencies indentified by the teachers, and also generally mentioned in literature (Carrell, 2000;
Hellekjær, 2009; Hermida, 2009, etc.) refer mainly to the students´ poor ability to identify the authors'
major points – especially the implied ones, to distinguish facts and opinions, to spot arguments, and
to make judgments. All these problems multiply in second language reading since even skilled L1
readers fail to monitor their reading in L2 successfully until they have achieved a certain threshold of
proficiency in the new language, as Alderson supports: “in second language reading, knowledge of L2
is a more important factor than first language reading abilities” (2005, p.23). Thus, two essential
factors underlie the students´ reading competence: language knowledge and knowledge of reading.
Another reason, often hidden within these noticeable demonstrations, is the fact that the reading
comprehension instruction even at the university level is limited to checking or testing the students´
reading skills through comprehension question exercises with little demand on higher-level thinking
skills. It is assumed that the students have already acquired the skills and strategies needed for
approaching a text critically.
2. Critical reading
The ability to read critically is usually understood by the teachers as the ability to comprehend a text,
which means “extracting the required information from it as efficiently as possible” (Grellet 1981, p.3).
Even if the ability to derive meaning from written text accurately and efficiently is a vital skill, it is also
necessary to evaluate the information and ideas detected from the text and then decide what to
accept and believe, which is a “a level of understanding that entails distinguishing fact from opinion;
recognizing an author´s intent, attitude or bias; drawing inferences; and making critical judgments”
(Adams and Patterson 2008, p.141). This level of critical understanding that goes beyond literal
comprehension is crucial for effective reading. Wallace adds that “critical readers do not just
705

Zuzana Šaffková
comment metacognitively, showing awareness of the cognitive strategies they make use of, but also
metacritically” (2003, pp.42-43). This means that the readers examine their personal responses and
attitudes to the text, keeping a certain ´critical distance´ on their interpretations. An opportunity for the
students to reflect on and assess their own learning procedures and results helps them become more
aware of the strategies they use and how to modify them.
3. Building critical reading through blended learning
In order to develop EFL undergraduate students´ critical literacy skills at a required level, it is
necessary to offer the students more than just basic literacy skills but encourage them to become
critical consumers of the information through multiple modes of learning. One way to achieve this is
through extending a conventional EFL reading classroom by an on-line environment. This mode of
blended learning that combines face-to-face and on-line learning “has enormous potential to
transform the nature of the educational experience with the use of direct and mediated
communication and the rethinking of educational approach” (Garrison and Vaughan, 2008). First of
all, e-learning enables interaction with content, with instructor and with classmates, and thus it is an
opportunity for students´ active participation, sound responses and in-depth reflection (Stephenson,
2002; Bonk and Graham, 2006). Interactivity also encourages students to become actively engaged
with what they are reading. Engaged readers are then intrinsically motivated and thus want to
understand the text, enjoy their work and have confidence in their reading abilities. Moreover, since
the students can monitor their own learning by monitoring their responses to exercises, pace and
timing, overall direction and assessment of performance, they are gradually encouraged to read
independently yet under a highly structured support. Next, a variety of tasks that computer mediated
instruction offers makes the learning formats more explicit for the students, which creates a kind of
security and increases the students´ achievement in reading. As Grabe confirms, “explicit L2 reading
instruction is necessary and important” (2009, p.150). On-line environment also suits diverse learners
with different learning preferences, expectations, experiences and cultural backgrounds, by providing
specialized instruction (Kamil, Manning and Walberg, 2002). Finally, since students are more involved
in learning by the necessity to integrate knowledge into practice, for which they have to consider
choices and alternatives and reflect on their action, it can be assumed that online learners tend to
obtain skills that correspond to the ability to think critically.
4. The project
Objectives of the investigation:
The above mentioned findings from literature demonstrate the scope of investigation out of which
three important issues arise in connection with developing first-year EFL university students´ critical
reading skills: the students´ reading experience, their L2 proficiency, and the delivery of reading
instruction. The main goal of the investigation was to diagnose the students´ common insufficiencies
in employing their critical reading skills and through a supportive framework to help them gradually
build their reading competence. To optimize the students learning, scaffolded instruction „enabling
students to develop, move to the ´next step´ level; never doing for them anything they are capable of
doing for themselves with a little support“ (Nuttall, 2005, 36) was used. Specifically, the investigation
tried to answer the following questions:
Is scaffolded instruction effective for building reading competence?
Can blended learning become an effective environment for gradual assistance and support for
building the students´ reading competence?
What further adjustments of the course should be made and what further measures in research
will have to be addressed?
Subjects:
The students in the first year of the Bachelor's degree program study two major subjects, one of which
is the English language. In the winter semester of 2010, 112 students registered for the Introduction to
Study Skills course. The course aims at three interconnected areas of skills that enable students to
experience and develop learning and academic skills appropriate for successful tertiary study. First of
all, the course helps students gain skills in study and time management strategies, then it also
focuses on the development of writing skills, and finally, it provides the students specifically with the
opportunity to experience and build academic reading skills. As the students start working on Moodle
from the beginning of their studies in different courses, they are already familiar with this platform.
706

Methodology:
Zuzana Šaffková
Since the main aim of the investigation was improvement in general, action research as a recurring
process of problem diagnosis, action intervention, and reflective learning was applied. Other features
of the study also complied with action research, such as: natural environment in which the research
took place; both quantitative and qualitative data collected and described in detail; focus on process
rather than product; application of inductive analysis; and the utilization of findings for practice (Craig,
2009; McNiff and Whitehead, 2002). Descriptive statistics, as the common analysis in second
language acquisition research was used and the main aim was to record the frequencies which
provide “information on the performance of the subjects” and help to “obtain insights into and
understanding of the data and the results” (Seliger and Shohamy, 1989, pp. 211-212). First, the data
collected from the students´ responses to tasks were analyzed quantitatively by percentage counts for
future decision purposes. Then the open-ended items, which were used to support the quantitative
data, were examined qualitatively for discussion purposes. The semester-long trial implementation
consisted of three stages:
Phase 1: diagnosis of the students´ reading skills and abilities
Phase 2: modeling and guided practice in critical reading
Phase 3: evaluation of the effects of the reading intervention
Procedure and instruments:
Phase 1:
In order to gain an insight into the students´ general ability to comprehend a text, a diagnostic on-line
task at the beginning of the course was carried out (Figure 1). The students were required to read an
article about the benefits of education and identified the main idea (Q1) and the intent of the article as
well as selected facts (Q2) by means of multiple-choice questions. Then open questions were used to
check whether the students understand implicit ideas and details (Q3). Finally, they had to guess the
meanings of particular expressions from the context and explain them using their own words (Q4).
The task helped to detect both basic and higher-level cognitive reading skills as a fundamental
precondition for initiating training in metacognitive abilities.
Q1.1 Aim of the article
Q1.2 Intent of the article
Q2: Specific details
Q3.1 Interpretation of irony
Q3.2 Finding reference in the text
Q3.3 Meaning of the author’s conclusion
Q4: Meanings of words
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Correct
Imprecise*
Incorrect
Figure 1: First year students reading profiles (N=112)
*In Q2 the students could choose from more than one correct answer and in Q4 they had to provide
more than one precise reply. If the students had more than half of the responses incorrect, their
overall result was considered “imprecise”.
In order to motivate the students to approach the reading tasks responsibly and to make them work
with the text independently, they were informed of the significance of the on-line reading practice for
their success at school and, in addition, their responses were not marked. Instead, a scale of
“accepted – accepted with reservation – not accepted”, which is a scheme that is offered by the
707

Zuzana Šaffková
Moodle application, was used. Only those students who deliberately skipped some questions, or
submitted only a partially elaborated task obtained “not-accepted” evaluation, which happened only
exceptionally. The students´ reactions to the introductory text showed that the students were able to
comprehend a text if the question was explicit enough and required an answer that was easily
detectable in the text. The majority of the students were able to identify the intent of the text (Q1.2),
however, when they had to specify the aim of the article, only 66 students answered correctly. This
might have been ascribed to the fact that the answer to question Q1.2 was implied by the examples
and the tone of the article, while question Q1.1 required to explore the text at a deeper level in order
to confirm or put aside any tentative thoughts that the text prompted. The same was proved when the
students had to find specific information (Q2), most of which required in-depth analysis of the text.
Only 56 students out of 112 succeeded in this task, 45 students were able to spot some of the details
and for 11 students a close examination of the text was a problem. Q3 required conscious inference
to identify implicit ideas and also the ability to articulate them. Open questions (Q3.1 and Q3.2)
allowed the students greater latitude in formulating their answers; moreover, the students could fall
back on their own experiences with the theme while answering these questions, which certainly
contributed to the higher success rate (71% and 88% of the students replied correctly). On the other
hand, when the students were required to recognize analogies and generalize (Q3.2), the percentage
of attainment decreased (46%). In the last task (Q4) the students demonstrated their difficulties with
inferring the meaning of selected phrases from the context, which could be ascribed to their limitations
of vocabulary – especially phrasal verbs, a context analysis that was too superficial, and
exceptionally, the inability to express the meaning in L2. The analysis of the students reactions to the
text showed that the majority of the students demonstrated typical characteristics of nonproficient
readers (Block, 1986), i.e. inability to integrate information, use general as well as personal
knowledge and association and respond in reflective mode. Moreover, very often they focused
exclusively on their own feelings and thoughts rather than the information encoded in the text
“directing their attention away from the text and toward themselves” (ibid.).
Phase 2:
Based on the results obtained from the introductory task, explicit in-class skills instruction and practice
followed. The scheme of the whole process of instructional delivery aimed to help the students “by
explaining fully what to do, why, how, and when; by modeling their own thinking processes; by
encouraging students to ask questions and discuss possible answers among themselves; and by
keeping students engaged in their reading by means of providing tasks that demand active
involvement” (Farstrup and Samuels 2002, p. 256). In order to guarantee full involvement in reading,
the students worked on a variety of tasks, quizzes and assignments in on-line lessons guided by
asynchronous communication with the teacher and accessible self-tests and materials, and supported
by face-to-face contact and immediate feedback. The whole procedure of the transition from
classroom guided learning through web-based work to modified classroom instruction is as follows:
Figure 2: Scaffolding instruction in teaching reading
The skills practised in five units within the course focused on a gradual transition from cognitive skills
that, as the diagnostic task proved, were manageable for most of the students, to metacognitive skills,
for which more time and practice was provided by on-line work.
The selection from both cognitive and metacognitive skills comprised:
Capturing the writer’s point of view that can help readers understand the purpose of the text,
which enables them to direct reading “to the most relevant material, and to save time by reading
more efficiently” (Cottrell 2005, p. 37).
Being aware of what the argument is and what it is not, which is an important step towards critical
analysis by enabling students “to categorise different types of materials … and distinguish what is
really relevant from other forms of information (ibid., p. 51).
708

Zuzana Šaffková
Reading between the lines as an ability to spot and analyse underlying assumptions in text.
Finding and evaluating sources of evidence in arguments and whether the line of reasoning is
ordered in a logical way.
Recognizing and interpreting specific context clues.
Understanding the writer’s point of view is an important comprehension skill for students and therefore
in Unit 1 (Figure 3) the students could practice and consolidate a variety of techniques that help build
up this skill. Consequently, through explicit in-class modeling and examples, and a consequent online
practice, they rehearsed making connections between key points in the passage, drawing
inferences and extracting concepts. In a self-test activity they were required to read a text about the
atomic attack on Hiroshima and prove that they were able to comprehend the text and analyze it (Q1)
so that they could understand the writer’s point of view (Q2).
Q1.1 Specific detail 1
Q1.2 Reason
Q1.3 Specific detail 2
Q1.4 Specific detail 3
Q1.5 Specific detail 4
Q2.1 The w riter´s tone
Q2.2 Figurative language
Q2.3 Word choice
Q2.4 Writer´s point of view
Figure 3: Capturing the writer’s point of view (N=112)
0% 20% 40% 60% 80% 100%
Correct
Incorrect
The assessment of the students´ first work in a self-regulated online learning environment brought two
different results. The percentage of the students who succeeded in identifying the writer’s main idea
and specific details that support it increased and reached a stable rate (86% on average for Q1.1-
1.5), compared to the diagnostic task (76% on average for Q1.1-2 and 50% on Q2). On the other
hand the students´ struggled when they had to describe the writer’s point of view, which is a skill that
requires “interpreting text by going outside it” and “understanding information when not explicitly
stated” (Grellet 1981, p. 5). On average only 60% of the students successfully answered all the
questions that required these skills (Q2.1-2.4), which was also an important signal for a consequent
face-to-face practice.
Understanding and isolating key information in a text in order to identify argument and distinguishing
argument from different types of messages was the aim of Unit 2 (Figure 4). Within in-class training,
the students were exposed to examples of descriptive and explanatory texts, summaries and
background information passages that usually surround an argument, and were expected to locate
and determine them in short extracts. In an on-line activity, they were to read an article about
extraterrestrial life and identify the conclusion (Q1) as well as the reasons that support it and then
decide which part of the text functions as an introduction, description, explanation, summary and
background information (Q2-6).
The skill of identifying a variety of methods the writer can use to develop his/her ideas was beyond
the students abilities. The main reason for a failure in this task was caused, as the students
mentioned in a successive in-class session, due to some interference from their writing classes,
where the function of a conclusion was characterized as the final word of a paragraph or a summary
statement that restates the topic sentence. Therefore the majority of the students considered the last
sentence of the extract a conclusion (Ql – 95%), which was not correct. This initial misunderstanding
contributed to the fact that once, being taken in the wrong direction, they could not properly analyse
709

Zuzana Šaffková
the other properties of a text. The source of success in spotting the introduction (Q2 – 81%) is
obvious for the same reasons. The outcome from this activity significantly influenced the mode of the
subsequent face-to-face training that offered the students more guidance in the aforementioned area.
Q1 Conclusion
Q1.1 Reasons
Q2 Introduction
Q3 Description
Q4 Explanation
Q5 Summary
Q6 Background information
0% 20% 40% 60% 80% 100%
Figure 4: Distinguishing argument from non-argument (N=106*)
Correct
Incorrect
*The lower number of students shows a gradual drop-out rate, typical of the first-year undergraduates.
Reading between the lines, as the main theme of Unit 3 (Figure 5), is an ability that requires using
inference and deduction in order to understand what is meant, as well as what is expressed explicitly.
In the in-class session, the attention was paid to intensive practice in recognizing narrative
perspective, picking up clues form text, evaluating false premises and disclosing latent messages
used to reinforce an argument. In the on-line component of the lesson, the students read a short
paragraph and had to detect three hidden assumptions that the text contained (Q1).
Q1 Hidden
assumptions
0% 20% 40% 60% 80% 100%
3 assumptions
2 assumption
1 assumption
None
Figure 5: Reading between the lines (N=106)
A relatively high number of the students who were able to disclose at least one hidden assumption
(69%) and formulate satisfactorily the latent bias, indicates an effective prior instruction that
contributed to these results. Moreover, for the students, this was a completely new area that attracted
their attention as they considered it both important and interesting. Nevertheless in the consecutive inclass
session more practice was devoted to exploring and evaluating underlying assumptions.
The ability to recognize valid arguments was a further step towards developing the students´ critical
reading skills in Unit 4 (Figure 6). In the face-to-face session the students discussed different types of
evidence that makes a sound argument and rehearsed finding and critically evaluating potential
sources of support. Then their task was to determine the conclusion (Q1), distinguish relevant pieces
of evidence in a text (Q2) from sections that were not valid for the main conclusion (Q3).
710

Q1 Conclusion
Q2.1 Relevant evidence 1
Q2.2 Relevant evidence 2
Q3.1 Irrelevant section 1
Q3.2 Irrelevant section 2
Zuzana Šaffková
0% 20% 40% 60% 80% 100%
Figure 6: Finding and evaluating sources of evidence (N=97*)
*The number of students decreased due to a continuing undergraduate drop-out.
Correct
Incorrect
The students´ answers implied their improvement in identifying the conclusion (69% compared to 5%
in Unit 2), which can be ascribed to in-class remedial training. They also distinguished relevant proof
from irrelevant evidence in sentences where these were explicitly addressed. On the other hand, the
students (93%) were not able to cope with a section that represented a useful piece of background
information, but did not provide any evidence to support the conclusion (Q3.2). As most of them
mentioned in class, they had also trouble understanding the message due to their lack of vocabulary.
The last in-class lesson (Unit 5) provided the students with an explicit instruction on different types of
context clues: contrast, restatement, example, and general knowledge clues (Flemming, 2006). The
practice in recognizing specific context clues was aimed to help the students find and understand
relationships to other words in the surrounding text and recognise multiple meanings of some words.
As students become more proficient in using context clues in a text, they develop more extensive
reading vocabulary, which obviously contributes to more effective reading on the whole. In an on-line
interactive multiple choice self-test (Figure 7) they could check whether they were able to
independently apply the strategies needed for identifying meanings of unfamiliar words presented and
practiced in a prior face-to-face session (Q1-15). In order to prevent the students from using
dictionaries, they had only an eight-minute time limit to choose a correct word from four options.
Q1-15
guessing
meaning from
context
0% 20% 40% 60% 80% 100%
12-15 correct answers
8-11 correct answers
4-7 correct answers
0-3 correct answers
Figure 7: Recognizing and interpreting context clues (N=97)
The results show that the students were able to recognize most of the expressions, even if the time
constraint might have contributed to the fact that only 13 students were able to answer correctly most
or all of the questions. Time pressure might have become a barrier especially for those students who
were more careful thinkers by their nature and needed more time to complete the tasks.
Phase 3:
711

Zuzana Šaffková
In the last phase of the investigation, a summative as well as a formative assessment of the students´
progress was administered. For the reasons of the university requirements the students took a paperand-pencil
test in order to earn a credit. However, they could also check whether they had mastered
some of the critical reading skills and how competently they were able to apply them. The content of
the test corresponded with all the items that were rehearsed both in the in-class and on-line settings.
82 out of 97 students who finished the whole course (85%) successfully completed the test at a
required level, i.e. they achieved 70% and more of the total points they could obtain. Apart from the
selected tasks that required from the students to employ their higher-level cognitive reading skills in
the post-course test, a few items can be compared with the diagnostic on-line test (Figure 8) and thus
confirm an apparent progress in the students´ reading competence. Identifying the writer’s main idea
and detecting specific details, both stated implicitly (Q1, Q2), and especially guessing the meaning of
words from the context (Q3) were the areas that markedly demonstrated improvement. Regarding
higher-level thinking skills, the majority of the students also succeeded in spotting hidden
assumptions (67%) even if their ability to explain the preconceptions was not always accurate.
100%
80%
60%
40%
20%
0%
Q1 Main idea Q2 Specific
details
Q3 Meanings of
words
Figure 8: Comparison of pre-test (N=112) and post-test (N=97) selected items
Pre-test
End-of-course test
For the purposes of the investigation, a post course attitude questionnaire was designed for the
students as the last voluntary on-line assignment. The questionnaire, which was anonymous and
which was offered to the students as a platform where they could express their opinions about the
whole procedure of the course, provided a supporting picture to the analysis of their results as well as
useful suggestions for possible improvements. In the questionnaire (Figure 9), the students evaluated
various aspects of the course according to a five-point scale; however, they could also add any
comments or recommendations that they had considered valuable for the further improvement of the
instruction.
The results from the questionnaire show that the students appreciated the content of the course,
either in its in-class and on-line components and were aware of the fact that their reading skills were
developed. 64% of the students (Figure 10) considered working on-line supportive (12% strongly
agreed, 52% agreed) and appreciated especially immediate feedback provided on Moodle as well as
remedial work in the in-class sessions.
A lot of students valued the possibility to improve their critical thinking skills and even requested more
time devoted to practicing reading skills and thinking in general. They also indicated that they had
become aware of some areas in which they had made progress as Figure 11 demonstrates.
On the other hand, many students (56%) complained about the texts that were too difficult as regards
demands on both L2 proficiency and higher-level thinking skills. As far as the on-line tasks are
concerned, the students said that the most difficult for them was to uncover hidden assumptions and
even if they were able to spot them, then they struggled to explain them in an understandable way in
L2.
712

poor
not so good
average
good
excellent
Zuzana Šaffková
0% 20% 40% 60% 80% 100%
Figure 9: Effectiveness of the course (N=90*)
On-line practice
In-class instruction
Content of the course
Usefulnes of the course
*Since the questionnaire was an anonymous and voluntary assignment, only 90 students completed
it.
Figure 10: Effectiveness of web-supported learning (N=90)
5. Conclusion
strongly agree
agree
neutral
disagree
strongly disagree
The purpose of this article was to provide some insights into the first-year EFL students´
insufficiencies in their critical reading competences. Then, based on the analysis of the results from
the continuous work on on-line assignments and the students´ comments in the questionnaire,
suggest possible ways and implications for the further improvement of the reading instruction.
Considering the research questions stated at the beginning of the investigation, it can be affirmed
that:
Scaffolded instruction realized through modeling and guided practice in contact lessons and
independent practice in an on-line training setting, proved to be effective for building the students´
reading competence. The results from the achievement test at the end of the course indicated
improvement even at the level of metacognition, especially the ability to activate the students´
713

Zuzana Šaffková
prior knowledge, the ability to construct meaning, interpret events and distinguish between what is
important and what is unimportant .
100%
80%
60%
40%
20%
0%
Identifying the
author’s main
idea
Extending
vocabulary
Better
orientation in
text
Ability to
analyze facts
quickly
Understanding
weak and
strong
arguments
Figure 11: Improvement in sub-skills identified by the students themselves (N=90)
Understanding
valid and
weak
evidence
Blended learning became an effective means of assuring independent practice of the strategies
that enabled the students to adapt and internalize strategic reading and thus to take more
responsibility for their learning. The e-learning environment then contributed to increasing the
volume of reading as well as its quality by an individualized feedback that helped learners
“complete the reflective cycle” (Macdonald 2008, p.131). It also facilitated systematic sequencing
of explicit instruction, guided practice and independent work so that instructional support and
intellectual challenge were properly balanced. Finally, the e-learning scheme proved to be an
excellent tool for action research since as a means of “on going progress monitoring” it provided
valuable information on the students´ levels of mastery of the various concepts being studied and
helped plan new lessons and reteach unclear concepts (Schumm 2006, p. 52).
The research considerably helped to specify concrete steps and procedures for a novel
framework of the course. First of all, considering the choice of texts, more attention should be
paid to suitability of content, exploitability, and especially to readability of texts (Nuttall, 2005).
Even if the first-year university students are familiar with necessary grammatical structures,
authentic texts frequently contain more complex constructions and less frequent vocabulary,
which may cause the students´ failure in analyzing such texts. Thus equal-level or below-level
rather interpersonal texts should be used at the beginning of the course, “moving later to a
concern with the quality of description or exposition within a text, involving attention to more
hidden parts of the grammar” (Wallace 2003, p.82). Then, a direct explanation and training in
responding to tasks should be included in reading instruction. The analysis of the students´
responses disclosed their inability to convey their ideas accurately yet using simple words.
Further research would be needed to collate the students´ results with their own approaches to
independent on-line work to find out to what extent their effort to work independently on-line, without
the tutor’s direct guidance influences critical reading performance. Then, other reading strategies,
mainly those that comply with metacognition should be introduced and investigated. Finally, other
potential for an e-learning environment, which primarily facilitates collaboration and interaction with
peers, should be exploited and researched.
Acknowledgment
This paper was supported by grant SGS 5842/2011, Technical University of Liberec.
References
Adams, R.W. and Patterson, B. (2008) Developing Reading Versatility. Thomson, Wadsworth, Boston.
Alderson, J. C. (2005) Assessing Reading. Cambridge University Press, Cambridge.
Block, E. (1986) “The Comprehension Strategies of Second Language Readers” TESOL Quarterly, September,
Vol 20, No. 3, pp. 463-494.
714

Zuzana Šaffková
Bonk, C. J. and Graham, C.R. (2006) The Handbook of Blended Learning. Global Perspectives, Local Designs.
John Wiley & Son, San Francisco.
Carrell, P.L., Devine, J. and Eskey, D.E. (2000) Interactive Approaches to Second Language Reading.
Cambridge University Press, Cambridge.
Cottrell, S.(2005) Critical Thinking Skills. Palgrave McMillan, New York.
Craig, D.V. (2009) Action Research Essentials. John Wiley & Son. San Francisco.
Farstrup, A.E. and Samuels, S.J. (2002) What Research Has to Say About Reading Instruction. International
Reading Association, USA.
Flemming, L. (2006) Reading for Thinking: Houghton Mifflin Company, New York and Boston.
Garrison R. D. and Vaughan, N. D. (2008) Blended Learning in Higher Education. Framework, Principles, and
Guidelines. John Wiley & Son, San Francisco.
Grabe, W. (2009) Reading in a Second Language. Cambridge University Press, Cambridge.
Grellet, F.(1981) Developing Reading Skills. Cambridge University Press, Cambridge.
Hellekjær G.O. (2009) “Academic English reading proficiency at the university level: A Norwegian case study”,
Reading in a Foreign Language. October, Vol 21, No. 2, pp. 198-122.
Hermida, J. (2009) “The Importance of Teaching Academic Reading Skills in First-Year University Courses”, The
International Journal of Research and Review. September, Vol 3, pp. 20-22.
Kamil, M., L., J. B.Manning and Walberg, H.J. eds., (2002) Successful Reading Instruction. A Volume in
Research in Educational Productivity. Information Age Publishing, USA.
Macdonald, J. (2008) Blended Learning and Online Tutoring. Gower Publishing Limited, Hampshire.
McNamara, D.S. ed., (2007) Reading Comprehension Strategies. Theories, Interventions, and
Technologies. Lawrence Erlbaum Associates, New Jersey.
McNiff, J. and Whitehead, J. (2002) Action Research: Principles and Practice. RoutledgeFalmer, London and
New York.
Nuttall, C. (2005) Teaching Reading Skills in a Foreign Language. Macmillan Publishers, Oxford.
Seliger, H.W. and Shohamy, E. (1989) Second Language Research Methods. Oxford University Press, Oxford.
Schumm, J. S. ed., (2006) Reading Assessment and Instruction for All Learners. The Gulford Press, New York.
Stephenson, J. (2002) Teaching & Learning Online. Pedagogies for New Technologies. RoutledgeFalmer,
London and New York.
Wallace, C. (2003) Critical Reading in Language Education. Palgrave McMillan, New York.
715

A Mobile aid Tool for Crafting Active Learning Experiences
Ahmed Salem
Faculty of Engineering, King Abdul Aziz University, Jeddah, KSA
azsalem@hotmail.com
Abstract: A successful mobile aid tool would prove valuable in the design of learning experiences for both the
student and the teacher. For the students, it motivates and challenges them through a hands-on interactive
process, and gives them the needed time, software, and learning experiences environment to construct their own
mental image of the information they are exposed to. Such a tool would allow the students to assess their mental
model representation both inside and outside the class and keep modifying it if needed, until it proves to be the
right model (as per Bloom's Taxonomy of Learning Domains). For the teacher, it helps them to create a
successful design of the course and is valuable both for initial design as well as for iterative improvements of the
initial design. For both the student and the teacher, it offers multi-interactions, and receiving immediate feedback.
In this paper we introduce a mobile aid tool for designing Active Learning experiences that fulfils these attributes.
This mobile tool is composed of a course information data bank, questions bank, interactive Smart-Quiz system
with automated self assessment technique that offers instant feedback and a communication system. It is applied
in a college level course that introduces undergraduates to Engineering Design process, and is delivered in
Active Learning format. J2ME is used to develop this mobile tool and it is targeting Java ready mobile wireless
sets that are highly common among the students nowadays. A study was conducted for two semesters, using
qualitative and quantitative methodologies for data collection and interpretation to measure the effect of this tool
on students’ attitudes and performance. Two control groups were selected each semester – one is using this tool
and the other isn’t – to the experiment. The study showed positive effects of this tool on both attitudes and
performance in favour of the first group.
Keywords: mobile learning, learning experience design, active learning, in class environment, out of class
learning, assessment
1. Introduction
One of the most daunting tasks in designing an active learning course is to motivate the students for
participation in the activities in and out of the classroom. Another difficulty is the assessment of their
work, how to make it transparent and enlightening? The third dilemma would be “how to make the
students feel and share the responsibility about their education through sharing of power with them?”
(Meta- cognition). The forth one is “how to interest the student to feel the ownership of the process of
education?”
Instructors and educators go very creative about these four obstacles and there are many proposals
in the literature on what to do. Examples for the first issue “Participation” are (Bonwell, C. C., & Eison,
J. A. (1991)), (Ames, C., & Archer, J. (1988)) and (Barry W. McNeill, Lynn Bellamy et. al. (2002)). For
the second concern “Assessment” examples are (Bean, J. C., & Peterson, D. (1998)), ((Andrews, J.
D. W. (1980)), (Anderson, R. S., & Speck, B. W., (1998)), and (Angelo, T. A., & Cross, K. P. (1993)).
Finally for the third and forth issues, “Share of Power”, and “ownership of the process” the examples
are (Maryellen Weimer, (2002)) and (Barry W. McNeill, Lynn Bellamy et. al. (2002)).
Although there are many great ideas in all these resources, one was always left with the feel of
needing a tool that can address all these four points and also could be automated and is reusable at
almost every class.
This paper presents such a Tool. It is designed to interest the students, provide a clear and instructive
assessment, make the students feel the share of power with their teacher and finally let them feel the
ownership of their education process.
We present here the design and implementation of this tool. We also present a comparative study that
was conducted over one year on the students at the college level studying an introductory
engineering design course. Two control groups were selected where the first is using this tool in and
the second is not. The study showed a clear improvement of both the attitude and performance of the
first group as detailed in this paper.
2. Course information data bank
The “introduction to engineering design” course material is composed of the course syllabus with
course objectives and grade distribution, handouts, assignment statements, assignments checklists,
716

Ahmed Salem
assignment dates and time tables, and first day materials (as proposed by Laurie Richlin, (2006)). All
materials are made available for the students to download on their Mobile sets for constant
consultation and to follow up the course’s activities time table. In addition, the course has two
assigned books, Smart Quiz system and an ever-growing quiz bank that the students contribute to.
The students are allowed to install the Smart Quiz system on their Mobile sets and can keep a copy of
their contribution to the quiz bank to practice timely.
3. Questions bank
3.1 Domains of learning
In 1950s, Bloom, B. S., & Krathwohl, D. R. published a taxonomy of cognitive educational objectives
that was composed only of two domains. However, Anderson, L. W., and Krathwohl, D. R. (2001)
stated the Bloom’s complete domains of learning as the following three domains: cognitive (about
knowing), affective (about attitudes and feelings), and psychomotor (about doing). Each domain is
composed of several levels of learning. Each level of learning is achieved through practicing a set of
action verbs (Barry W. McNeill,et. al (2002),. Bloom, Benjamin S et. al. (1956), and Bloom, B. S., &
Krathwohl, D. R.(1956)), by the learner. These domains have been developed in so many details
afterward with many different levels for each. The version of Bloom’s Taxonomy of learning domains
(BTLD) in this paper is a modified version of those offered in Anderson, L. W., & Krathwohl(2001),
Laurie Richlin (2006), and Salem, A. Z. (2011). The ten specific levels we use herein are: Access,
Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation, Receiving, Responding,
and Valuing. They are chosen to cover all the three domains. The action verbs for each level are
those adopted by Barry W. McNeill et. al. (2002) and Robert Kleinsasser (1996). The class activities
that help the learning performs these action verbs we adopt here are those proposed by Salem, A. Z.
(2011).
3.2 Design of the questions bank
The questions bank is built based on the reading material of the course. It targets the previously
mentioned BTLD levels. Every question in the quiz bank is represented by a data structure that allows
for the following data fields:
The question body: This is a text form, a fill in the spaces form, or a question statement related to
a figure that appears in a pop up window.
The available answer choices: This is a list of all possible choices other than the correct answer.
The system can pick up any number from three choices up to seven, according to the teacher
setting of the system.
The correct answer indicator: This is a pointer that refers to the right answer. It is dynamically
updated as the question possible answers list is shuffled upon initialization.
The question BTLD level: This is an earmark for every question to indicate what level of the BTLD
it addresses (A: for access, K: for knowledge, C: for comprehension... and so on). This allows the
teacher to select the level of the quiz. For instance, it could be a pure knowledge level quiz, a
comprehension level quiz, and so on, or any mixture of the levels as shown by Salem, A. Z.
(2011).
The chapter name: This field indicate the reading materials of the course that this question
belongs to.
The student question’s grade: The earned grade is stored in this field.
The date it was originated: This filed hold a history log of the original and updating dates of the
question modifications.
The source of the question: The question bank is initially constructed by the teacher. However,
the students are allowed to add to the bank, pending the teacher approval.
The author of the question: This could be the teacher, a student, or an author of the course
material.
ABET classification symbol: This is an indication of the ABET objective level the question
addresses.
717

Ahmed Salem
Question difficulty indicator: A statistics is made after every quiz to determine how many times
every question appeared and how many times it was solved correctly. The reciprocal ratio of
these two numbers is the difficulty indicator. One is the easiest question, and zero is the most
difficult one. This indicator is updated continuously every time the system is used.
The link to the associated figure with the question (or video clip), if any: This is a self explanatory.
The data structure representation of the question is made flexible through linked lists structure to
accommodate any further data fields addition that may be needed in the future.
4. Interactive Smart-Quiz system
The Smart-Quiz system (SQS) is developed in Java language J2ME version. This gives it the
advantage of being portable to any Java ready set. Those sets are commonly available to the
students nowadays such as laptops, mini e-machines, (i/A)Pads, and Java ready cell phones. Every
student is allowed to have the SQS on his/her set where they can use it to ready themselves for the
class activities and to use it in and outside the classroom. It is made available through an open
wireless communication channel with the instructor set. The SQS is offering a user-friendly graphic
user interface and starts by asking the students to insert and verify their personal information. In this
process, the students have to fill an electronic checklist of the quiz about checking their readiness and
allowing them to submit any bonus granting extra efforts such as research or journal on the quiz
subject. Screen shots of the SQS are shown in Figure 1 below.
Figure 1: Screen shots from the GUI of the SQS
718

Ahmed Salem
The SQS feeds from the question bank of the course on the course’s server. The server is
programmed to offer the quizzes through the SQS in the appropriate classes’ time slot according to
the course activities agenda, which is distributed to the students at the first day of the classes. The
student can run their local SQS on their local sets all the time for training purposes, however during
the class; they have to pair with the server to take the assigned quiz.
Once the SQS starts, it looks for an initialization file that is to be supplied by the teacher. If none
exists, it uses a default file. This file specifies the quiz allowable time, number of questions, number of
choices for every question, the number of figures associated with the questions and the required
BTLD for the quiz among many other quiz customization data as detailed in section 3 of this paper.
The SQS then reads from the quiz bank the designated quiz and its associated figures, and generates
a set of questions that both targets the predetermined BTLD (or a composition of them) and fulfils the
customization criteria according to the teacher setting. A random subset is generated from this set for
every student.
5. Automated self assessment technique
Assessment is an indispensible part of the design of the learning experience. It sheds light on how far
the students progressed for the teacher and provides them with the appropriate feedback on how
good was their performance (Fabry, V. J., et al. (1997)). In this tool, we also add an educative
component to it that informs the students instantly if his/her choice of answer was correct or not. The
right answer is marked in every question’s choices and the computer compares the student answer to
the correct one. The students would know the right answer after they provide their answer. The self
assessment mechanism that is installed in the quiz system is fully automated and is adjustable to
cater for the teacher goals (Wiggins, G. (1998)). Once the student answers the question, a verification
windows pops up with both the question and the student’s selected answer together, asking the
student to confirm her/his choice. If the student still hesitant they can opt to leave the verification step
and go back to think about the answer. If s/he is sure, they proceed with the verification and accept
their answer.
If they answer is correct, the grade is calculate automatically against the right answer and is reported
to the screen. Thus, the student has an instant feedback. If the answer is wrong, the program reports
to the screen also instantly. In this case, the program reports what was the right answers, so that the
student gets to know why what s/he thought is the right answers was wrong. This feedback is crucial
pedagogy for the students to help them rethinking the construction of the right representation of their
knowledge according to the constructivism theory (Chapman, D. W. (2000), and L. Dee Fink (2003)).
When the student finishes the quiz, a fully detailed report is written and the total grade it reported
automatically to the student’s screen and to the teacher on the system server. Also, an Excel work
sheet is initiated and populated automatically for the grades of all the students in the class. A
performance distribution curve is fitted to the histogram of students’ grads. This gives the teacher an
instant assessment of the class performance in the quiz.
6. How it works
The Active learning experiences considered here is designed according to the simple design model
(Michael, Joel A. and Modell, Harold I. (2003)) as illustrated in (Salem, A. Z. (2011)) were every
learning experience is composed of three phases: Input, Process, and Output regardless of its scale.
The entire Active Learning course is seen as such a model and every learning experience within it is
modeled as such also. Even if a learning experience does include several mini learning experiences,
each one of them is modeled also according to these three phases. Figure 2 (curtsey from Salem A.
Z., (2011)) illustrates this model. The input phase is for assessing the readiness of the students and
raises it if needed. The process phase is done mostly in class and the output phase is result (and
continuation) of the learning experience.
Consider a learning experience that is designed to help the learner to reach the Comprehension level.
It starts in the input phase by two steps. First, helping the students to achieve both the Access and
Knowledge levels and second, prepare them for the Comprehension level process that will take place
in the classroom. We focus here on the second step assuming that the first was successfully
completed. To prepare the student for the Comprehension phase we may suggest the following
design of the learning experience.
719

Ahmed Salem
Figure 2: The Simple learning experience design model
6.1 Pre-class reading
The students have to join a collective pre-class reading session. This would take place one or more
days before the class. In this session they set in teams and every team is given a jigsaw reading
exercise. Every member would have to read and thoroughly understand a small portion within a
certain time, and then educate his/her fellow team members on this portion. The same reading
material is given to every team in the session.
6.2 Knowledge and comprehension levels quiz composition.
Upon finishing of the jigsaw exercise on the assigned reading materials, the teacher would assess the
successfulness of this exercise by asking some members of every team to recap what they were
taught by their fellow teammates. If the answers are correct, then the exercise was successful. If not,
the teacher would have to push in the right direction, without giving the full answer yet, and let the
students try again to refine their jigsaw, until they get the right answers.
After the jigsaw exercise is deemed successful, every team member would have to generate (in
written quiz’s question format) two questions on the portion they red. They present their questions to
the team and then to the class to omit repeated questions among teams. Upon teacher approval of
the questions, the students pair their sets with the server to submit their questions. This reinforces the
students’ feel of both belonging and the share of power (Maryellen Weimer, (2002)).
The teacher would have now collected a new addition for the quiz’s bank. Then the students can get
this portion of the quiz bank on their sets to practice at their homes for the coming class.
6.3 Quiz training
After the class the students can spend as much time as they need to play around with the SQS that is
fed with their quiz prototype. While this could be a small portion of the quiz bank, it is fully
representing the material they have to be acquainted with. The students download a default version of
the quiz initialization file that can control their training and adjust the timing of the quiz.
Practicing with the SQS out of the class, gives the students the chance to accurately assess their
mental model representation of the knowledge they will be exposed to and modify it if needed.
6.4 Conducting the quiz
The students take the quiz as the first activity in the classroom. They have to pair their sets with the
teacher’s to get their encoded version of the questions. The SQS generates an encoded (can’t be
opened to see the answers) random set of questions for every student according to the teacher’s quiz
initialization file, while maintain the overall difficulty level in balance by using the question difficulty
indicator. For instance, if the teacher set the a certain level of difficulty for the quiz in the initialization
file, the SQS will make sure that the average of the question difficulty indicator for any set of
questions that is generated for a certain student almost matches this level.
720

Ahmed Salem
Once the students finish taking the quiz, the result is posted to their sets’ screens. If the student did
not get a satisfactory result, s/he would have to redo the training again and to take a makeup quiz
later on according to the class preset agenda. The makeup is graded at a lower grade than that of the
original exam. This helps the student to fill any gaps in their constructed knowledge representation,
while emphasizing the importance of getting the right answers at the first offering of the quiz.
6.5 Reading indicators
Once the students finish the quiz, the results is instantly made available to the teacher. An Excel
sheet at the teacher computer is automatically populated by the SQS with the students’ grades.
Figure 3 below shows the design of the full learning experience as per (Salem, A. Z. (2011)) and the
utilization of the SQS in both the Input and Process phases. For a two hours session at the
comprehension level in an engineering design course, the input phase is selected to have two hours
long activities, and the output phase is selected to have four hours long activities.
Figure 3: The input, process, and output phases of the learning experience
6.6 Applying or modifying the learning experience
According to the students’ performance, the teacher would know for sure how ready are they for the
Comprehension level activities designed for the remaining of the class time. If the results show they
are ready, then the activities can take place as designed. At the end of the class the students would
take another comprehension level quiz to show how much progress they have made. The result for
such quiz is shown in section 7 below.
If the results show they are not ready yet, a redesign of the class activities may be needed to raise the
students’ readiness before conducting the class’s original design.
7. Results
A study was conducted for two semesters, using qualitative and quantitative methodologies for data
collection and interpretation to measure the effect of this tool on students’ attitudes and performance.
Two control groups were selected each semester – one is using this tool to prepare for the class and
the other isn’t – to the experiment. Each group is 60 students strong per semester and are given the
same comprehension level quiz.
The results for the collective performance of the two semesters (120 students) for each group are
shown in Table 1 below. These results are of the end-of-class comprehension tests for both groups.
The grades are given in industry job performance evaluation form namely: Exceed Expectations (E),
Meet Expectation (M), Need Improvement (NI), No Credible Effort (NCE) and finally No Submittal
(NS). This categorization is shown in the left column in the table. At the right most column, is the
criteria for each category.
As can be seen from Figure 4, the study showed positive effects of this tool on both attitudes and
performance in favour of the first group. The results are leaned more towards the left of the chart
where they showed shear improvement in the grads.
721

Ahmed Salem
Table 1: The two groups’ performances in the comprehension level quiz
Grad
e
No. of
students No. of students
Using the
Not Using the
Tool %
Tool % Notes
E 14 0.12 2 0.02 100% or > 80% with extra work (journal)
M 58 0.48 38 0.32 80% or > 70% with extra work (journal)
NI 42 0.35 64 0.53 > 40% and

References
Ahmed Salem
Andrews, J. D. W. (1980, Fall/Winter). The verbal structure of teacher questions: Its impact on class discussion.
POD Quarterly, 2(3&4), 129–163.
Anderson, L. W., & Krathwohl, D. R. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision
of Bloom’s Taxonomy of Educational Objectives. New York: Addison Wesley Longman.
Anderson, R. S., & Speck, B. W. (Eds.). (1998, Summer). Changing the way we grade student performance:
Classroom assessment and the new learning paradigm. New Directions for teaching and learning, No. 74.
San Francisco: Jossey-Bass.
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd
ed.). San Francisco: Jossey-Bass.
Ames, C., & Archer, J. (1988). Achievement goals in the classroom: students’ learning strategies and motivation
processes. Journal of Educational Psychology, 80(3), 260e267.
Barry W. McNeill, Lynn Bellamy, and Veronica A. Burrows, Introduction to Engineering Design The Workbook,
Tenth Edition (2002). Technical Editors: Sallie Foster & Don Butler, College of Engineering and Applied
Sciences, Arizona State University.
Bean, J. C., & Peterson, D. (1998). Grading classroom participation. In R. S. Anderson & B. W. Speck (Eds.),
Changing the way we grade student performance: Classroom assessment and the new learning paradigm
(pp. 33–40). New Directions for Teaching and Learning, No. 74. San Francisco: Jossey-Bass.
Bloom, B. S., & Krathwohl, D. R. (1956). Taxonomy of Educational Objectives: The classification of educational
goals, by a committee of college and university examiners. Handbook I: Cognitive Domain. New York:
Longman Green.
Bonwell, C. C., & Eison, J. A. (1991). Active learning: Creating excitement in the classroom. ASHE-ERIC Higher
Education Report No. 1. Washington, DC: The George Washington University, School of Education and
Human Development.
Bonwell, C. C. 1992–93. Risky Business: Making Active Learning a Reality. Teaching Excellence, 4 (3): entire
issue. Available from POD Network in Higher Education, P.O. Box 9696, Ft. Collins CO 80525.
Fabry, V. J., Eisenbach, R., Curry, R. R., & Golich, V. L. (1997). Thank you for asking: Classroom Assessment
Techniques and students’ perceptions of learning. Journal on Excellence in College Teaching, 8(1), 3–21.
Grasha, A. F. (1972). Observations on relating teaching goals to student response styles and classroom
methods. American Psychologists, 27, 144–147.
Halpern, D., ed. (1994). Changing College Classrooms: New Teaching and Learning Strategies in an Increasingly
Complex World.
Michael, Joel A. & Modell, Harold I. (2003). Active Learning in Secondary and College Science Classrooms, A
Working Model for Helping the Learner to Learn. By Lawrence Erlbaum Associates, Inc. 10 Industrial
Avenue, Mahwah, NJ 07430.
Johnson, D. W., Johnson, R., & Smith, K. (1991). Cooperative learning: Increasing college faculty instructional
activity. ASHE-ERIC Higher Education Report No. 4. Washington, DC: George Washington University,
School of Education and Human Development.
Johnson, D. W., Johnson, R., & Smith, K. (1998). Active learning: Cooperation in the college classroom. Edina,
MN: Interaction Book Company.
Kleinsasser, Robert Bloom’s taxonomy of educational objectives, (1996), (School of Languages and Comparative
Cultural Studies, UQ). Teaching and Educational Development Institute, The University of Queensland,
Prepared by Geoff Isaacs, TEDI.
L. Dee Fink, Creating Significant Learning Experiences, An Integrated Approach to Designing College Courses.
(2003) by John Wiley & Sons, Inc.
Laurie Richlin, Blueprint for learning: Constructing College courses to facilitate, Assess, and document learning.
First Edition, 2006. Published by Stylus Publishing, LLC.
Maryellen Weimer, (2002), Learner-Centred Teaching Five Key Changes to Practice. Jossey-Bass.
Salem A. Z., A Design Helping Mechanism For Active Learning Courses, (2011), EDULEARN11, the 3rd annual
International Conference on Education and New Learning Technologies, Barcelona (Spain).
Wiggins, G. (1998). Educative Assessment: Designing Assessments to Inform and Improve Student
Performance. San Francisco: Jossey-Bass.
723

King-Sized eLearning - how Effective can an Online
Approach be for Large Module Groups?
Marie Sams, Mary Crossan and Kate Mottram
Coventry University, UK
edu083@coventry.ac.uk
m.crossan@coventry.ac.uk
k.mottram@coventry.ac.uk
Abstract: There are ever increasing challenges facing academics in introducing more innovative approaches to
learning, improving the student experience and ensuring resources are managed more efficiently. eLearning is
increasingly being used as a mechanism for the delivery and assessment of students, however this can present
its own set of challenges, particularly when delivering this approach to large groups of students. Additionally, it is
important on business programmes that students get used to and develop skills in online communications as
many will be required in employment to use email and web based technologies. Online discussion also provides
a platform for a truly student centred approach in exploring teamwork and collaboration (McLoughlin and Luca
2002). This paper explores the findings of a research study undertaken at a UK Higher Education institution
which delivered a new blended learning module (Management and Organisational Behaviour) and online
assessment to over 400 first year business undergraduates over 20 weeks. This included a range of technology
approaches such as online case studies, DVDs, discussion forums, voice boards, quizzes and mind maps. The
paper discusses the benefits and drawbacks of undertaking the programme as part of a group of over 400
students and will include recommendations that can be adopted across other institutions that are considering
online or blended learning to students within a given timeframe. The findings discuss a broad range of areas
including the importance of flexibility in online learning and promotion of independent learning, assessment, and
motivation to engage. It covers the first level of the research, in terms of perceptions of engaging with large
groups of students online; however has scope for further analysis of each of the themes discussed.
Keywords: eLearning, e-assessment, learning, large, groups, modules
1. Background
A new module entitled 'Management and Organisational Behaviour' (MOB) was developed ready for
delivery in 2010/11 to students on a range of business and applied management programmes at
Coventry University. Similar modules attracting student numbers of this scale would traditionally be
taught in face to face lecture/seminar style followed by either essay or exam type coursework.
At the time of developing MOB, The Centre for Excellence in Learning Enhancement at Coventry
University put a call out to fund a small number of research projects that were considering developing
an e-assessment approach within their teaching and learning strategy. It was believed this would be
an ideal opportunity to pilot a blended learning and e-assessment for a larger group of students that
would have previously not been possible due to lack of resources and technology support.
The focus of the research took the form of investigating:
The benefits and drawbacks of engaging online from the students perspective
The impact of online learning in large numbers
It was envisaged that by piloting this study, that best practice and lessons learned would be available
to share with the wider University and other educational establishments considering adopting similar
approaches.
2. Literature and development of the blended approach
A review of the literature provided the researchers with knowledge to help develop the online
approach based on current practices and methodologies.
Context
Recent changes in the higher education sector have seen an increase in student to staff ratios, and
additionally in seeking out new, innovative means of delivering modules to learners (Nicol 2007,
Roberts and Lund 2007). This has meant there have been less opportunity for formative feedback
724

Marie Sams et al.
throughout learning, and an emphasis on the value to students from shifting towards a peer learning
approach to amplify the richness of their experiences (Nicol 2007, Daly et al 2010).
As the new module was being developed, it was understood that formative feedback would provide
part of the process in engaging students on the weeks in which they were expected to participate
online. Initial feedback would be encouraged through peer engagament and the design of activities,
and then at later stages by the facilitators.
Logistics
The online arena gives great flexibility in the logistical delivery of programmes (Smith and Rupp 2004,
Owens and Price 2010), both for the lecturer and the learner. However, the development work and
online feedback to students in an online environment takes significant groundwork to ensure the
student has a worthwhile experience underpinned by the learning outcomes of the module.
Additionally, for larger module groups, the consideration of the number of online facilitators is key to
the student learning experience.
Nagel and Kotze (2010) indicated that different researchers have put the ideal size of an online class
anything from 12 to 30 students. At Coventry, each online group needed to be facilitated and in this
case students were put into groups of 25-30, due to the size of the module and numbers of teaching
staff allocated. The module was split across 20 weeks, with every other week delivered being ‘online
learning’. The first summative assessment developed was a multiple choice questionnaire and the
second was a summer written exam. Dermo (2009) suggests that students expect some form of eassessment
in their studies and also that students are capable of dealing with the complexity of the
learning.
It was the researchers intention to prepare students for the multiple choice e-assessment by training
them on using the platform for delivery (Moodle) and giving them examples of the types/format of
questions they might be asked. This was embedded into both face to face delivery and online content
and was quite flexible and adaptive, depending on the needs and abilities of the students. This
encouraged and developed a more student centred approach to e-assessment and eLearning.
Engagement and motivation
Motivation is a key driver in the engagement with any online or blended delivery programme. Key to
this is the consideration of feeding back to students on their progress. Pachler et al (2010) question
the usefulness of generic feedback to students (for example based on a certain grade range).
Somyurek (2009) also suggests that the learning needs to be customised to each learner based on
their diverse backgrounds and requirements, however it was felt that feedback to the online phase
test would incorporate an element of generic feedback that could be provided to students more
instantaneously, this allowed the students to have a more in depth detailed response to their
assessment, which could also be followed up with each student in face to face sessions.
As MOB was intended to run as a blended learning module consideration was given to how to ‘weave’
topics through face to face and online environments to provide some continuity and maintain interest.
Ellis et al (2009) gathered from their research into blended learning that students who performed
poorly face to face did not engage well online. This suggested that a greater emphasis on how
eLearning is introduced at the outset carries great importance to ensure students understand the
benefits of engaging in an online environment, not only from their lecturers but also from their peers.
The first week developed an online lecture for students to view before their first face to face
lecture/seminar.
Feedback from the facilitators would be implemented through discussion forums however would take
the form of questioning, probing and guiding to stimulate debate and discussion rather than students
hanging back and waiting a ‘model’ answer, to encourage deeper, self directed and peer learning.
It was important to the MOB team to attempt the generation of peer learning. Although eLearning is
increasing, it is more frequently used in smaller student groups. Mompo and Redoli (2010) suggest
that collaborative learning is especially beneficial to those being taught in large groups which
encourages self directed learning.
725

Marie Sams et al.
MOB was set up in a way in which the teaching team could pick up and action any issues by
discussing the online delivery informally at face to face teaching sessions to get a feel for how it was
being received and what students were getting from it. The main points of focus for the team were the
flexibility in changing resources online, encouraging debate and discussion amongst students,
facilitating student led learning and implementing an assessment reflective of the learning outcomes
of the module.
3. Research approach
The research has been undertaken adopting both quantitative and qualitative methods of data
collection. The rationale for this was to better understand and interpret the current perceptions and
feelings of those participating in the module.
Full ethical approval was granted, and an online survey was developed and piloted for students
following completion of the e-assessment, which was approximately ten weeks into the module. The
students did not have automatic access to their marks on completion of the e-assessment to minimise
bias of responses. The online survey was made available to 419 students. 286 completed the
questionnaire, over 68% response. It should however be noted that respondents were not required to
answer all questions, and could opt not to answer a survey question if they wished.
Please see table 1 to show the profile of responses.
Table 1: Profile of responses
Male 61%
Female 39%
Full time students 91%
Part time students 9%
Initial analysis of the questionnaire findings were thematically analysed and from this a focus group
schedule was drawn up to further investigate some of the themes that had arisen from the survey
results. Two focus groups ran with a total of 11 participants.
Due to the nature of the research, the need for reflexivity was acknowledged (McGhee et al 2007) to
minimise the risk of subjectivity. It was recognised that the researchers were part of the process and
ultimately brought previous experiences, beliefs and values to the study. To acknowledge this helped
to back track from initial reactions to maintain and validate an objective view.
4. Findings
Benefits and Drawbacks Perceived By Students
Throughout the research into MOB, quantitative data and qualitative comments from students have
been collected through both an online survey and focus groups. From this, several key themes
demonstrating benefits and drawbacks perceived by the students studying the module, have been
gathered and collated.
Benefits:
Flexibility and Ease of Use; both full and part time students in focus groups suggested that
flexibility was one of the major advantages to MOB, allowing them to study at a time and place
which was most convenient for them. Owens and Price (2010) found that lecturers also had
increased flexibility when considering eLearning, compared to more traditional teaching formats.
This is something that will be discussed later in the paper, but is an idea that is also reflected by
lecturers and practitioners involved in MOB.
Development of Transferable Skills; 63% of participants who responded indicated that studying a
module via blended learning would contribute to their employability skills, with just under half of
students stating that they had developed entirely new skills when using technology. Focus group
participants discussed the benefits of online engagement as a good way of managing self
directed learning (time management), and developed skills in online discourse.
Assessment; Overall it appears that students were satisfied with the type of assessment used
within MOB. Many commented that they found an online assessment less stressful than
traditional written exams. Comments also extended to suggest that students found the online
726

Marie Sams et al.
assessment more appropriate as they were being assessed on a wider variety of topics covered
within the module, and felt it reflected what had been delivered on the module well. Due to the
level of importance placed upon assessment by students and universities, this key benefit will be
discussed in greater depth further on.
Varied Approaches and Styles; As MOB followed a blended learning approach, a variety of
teaching and learning methodologies were used. This meant that the variety of learning styles
present in a cohort such as this could be better served, with activities throughout the academic
year which would have suited all four VARK learning styles, including video, audio clips, written
case studies and interactive discussions. Fleming (1995) suggests that some learning styles are
not sufficiently catered for within University teaching, this is something that the wide variety of
activity styles aimed to overcome within MOB. However, it was acknowledged that this could have
impacted student engagement depending on the topic, format and delivery style.
Benefit to students with English as a second language; Statistically our highest achieving scores
in the e-assessment of 93% was awarded to 3 students where English was not their first
language. Mapping this to written exam results the statistics were often reversed. International
students often find the essay approach to writing more difficult and this impacts on how effective
they are as a result. The ability to go back and reflect/review discussion postings also aided
student learning.
Drawbacks:
Reliability of the VLE; Clearly a reliance upon technological aspects of large institutions come with
an associated risk of reliability. A small number of students stated that “the system was down”
and that they were “unable to access materials.” However, when asked if the online approach had
caused difficulties to them in accessing material, the majority of students felt that this was not the
case.
Motivation and Confidence; During the research it has been discovered that despite present
students being part of the technology revolution, many students lacked the confidence to use the
VLE and its associated applications such as discussion forums. Confidence issues appeared to
stem from the concern of materials being posted online as being permanent. This led students to
fear posting in case they felt they had given the “wrong answer”.
Feedback and Feed-forward; In order to gain as many responses to the survey as possible and to
ensure that results did not impact the responses the marks were held back from being released
until all students had sat the assessment. Therefore the delay in most cases was up to 4 days,
still considerably quicker than most other forms of assessment in this cohorts study. As a result it
did mean that feedback and marks were not immediate and students failed to fully engage in the
opportunity for feedback. Feedback was presented in three ways, 1 generic feedback, 2 a chance
to review their assessment to view what they got right and what questions were incorrect, and
also 3 overall percentage marks. In the focus groups students felt that the levels of feedback were
good and helped them focus on areas of weakness for their exam. Positive comments had been
made in relation to the allowing viewing their submissions, this did not happen in other modules
where online assessment had featured.
Assessment
Over 98% of students surveyed felt the type of assessment reflected the types of knowledge gained
throughout the module which compares well with Dermo (2009) discussing the anticipations of
university levels students in regard to e-assessment, and that students feel that e-assessment can be
effective at assessing the complexities of university level curriculum. Motivation and design of
assessment appear to be interlinked. 77% of students surveyed felt the e assessment in the MOB
module was “better” than written tests and 19% experienced no difference in the new e assessment to
that of traditional types of written tests. Along with this three quarters of students surveyed have been
assessed online previously, with the most predominant form of previous e-assessment being online
quizzes. Formative assessment was weaved through this module with online tasks and VLE
discussions through online postings with the aim of engaging peer to peer interaction and ultimately
learning. One student from the focus group commented on the lack of motivation to fully engage with
the module and in particular to engage with the online elements as "there is no punishment if I don't
post, or indeed there is no reward or mark for posting, so why bother...?" A number of students
discussed in the focus groups the lack of confidence in posting and needing assurance from seeing
others posts prior to posting. Lecturers aimed to provide a safe environment to post by firstly not
727

Marie Sams et al.
attaching the new concept of peer to peer postings to grades that impact their final module mark but
also to encourage freedom to embrace and fully explore this new approach without feeling pressured
or restricted by marking impacts. However, some students also commented that being able to post
online gave them time for reflection and thought, and also material to go back to at a later stage to
revisit key learning points.
Motivation
Full time students and part time students appeared to have very different motivations on this module.
Part time students emerged as more driven and self motivated, which became apparent within the
focus groups. This is perhaps reflective of the increased level of survey participation and responses
by part time students. One mature student expressed the value gained in becoming a more self
directed learner as the majority of part time students were in full time work, and had different
motivational factors than those studying full time on the programme. The focus group stressed the
need for more feedback both formative and summative to feel truly engaged and see a tangible value
in contributing to the self directed online work and discussions. There is some secondary evidence
from Nicols (2007) who conducted similar research and found that students had more of a need for
formative assessment and feedback from academic staff in larger class sizes in order to feel
supported. This showed symmetry between both research projects and also the identification that
students demonstrated a high reliance on the lecturer and valued their feedback however failed to see
their or peer discussions/postings and feedback as valuable. One respondent commented that they
liked the online learning. ”Yes, but I don’t enjoy commenting on other people’s uploaded work” and
this was similarly echoed in the focus group feedback. “I did not feel confident in posting as I felt
exposed in case it was wrong also this lack of confidence contributed in making me reluctant to post
in response to others postings”. However, another student commented “I felt scared to comment on
other peoples work to start with, yes. However we are first year students, and I could see my
confidence growing throughout the module to comment on other work”.
Flexibility
Data gathered from responses to the online questionnaire illustrate one of the key findings of a
blended learning module, such as MOB with respect to flexibility. This is also reflected when
considering a practitioners perspective. Student’s traditional expectations are challenged and
paradigms shifted when learning is moved from the class room or lecture theatre to an online
medium; encouraging students to develop more flexible ways of studying and learning. Of the 279
students who responded to this question, 85% believed that online learning provided them with a
flexible approach to study, see figure 1 below. Allowing students greater flexibility in the way in which
they learn allows them to develop skills, which can be transferred to other aspects of life, such as
employment skills. This theme was also made apparent within the focus group discussions, by both
full and part time students; from this it was clear that this was one of the major advantages for
students studying a module such as MOB.
250
200
150
100
50
0
Flexibility
Agree No Opinion Disagree
Figure 1: Percentage of students and opinions regarding flexibility
728

Marie Sams et al.
The increased use of VLE software by universities and other educational establishments in recent
years has meant that blended learning has become more of an available option. The flexibility it
provides to both staff and students allows, if managed correctly, the academic debates and
discussions which were traditionally confined to a class room to continue in a more technological
form. In the focus group the students who approached their studies with the aim of gaining a deeper
level of understanding from topics delivered felt that although it appeared to be more work when
studying online, the reality that students had to become more self directed and focussed to post
enabled them to gather a deeper understanding and learning than that of face to face classes. It was
a common theme amongst the focus groups that online elements were more time consuming. Some
saw the benefits of this where others felt face to face were "better" or after probing further "easier".
As the academic year progressed MOB developed and adapted with the students needs and
capabilities. Due to advancements and adaptability of the VLE, new methods and tools could be
added into the module website, depending on the needs of the task in hand and the students ability
and confidence. This included mind-mapping add-ins and voice recordings, to upload their thoughts
on a given topic. The speed at which the technology used within eLearning such as the VLE can
adapt is key for flexibility.
MOB aimed to provide students with an innovative and flexible means to their studies. Uploading of
videoed lectures to aid revision and/or catching up of sessions missed, added to the flexibility felt.
This made students feel more supported, which allowed students the freedom to study at times that
were convenient to them. This flexible approach to study meant that issues with students not able to
attend classes due to work, poor time management, or other commitments, could still keep up to date
with the classes. The ability to engage with the material at practically any time did however lead to
some students becoming frustrated with the delay in peer responses to online postings, one survey
respondent commented " I found it hard at times to motivate and lack of feedback de-motivated peers
as well which meant that they didn't carry out the work and so felt little point in doing it myself". Similar
thoughts were discussed in the two focus groups also. Ellis, Ginns & Piggott (2009) saw a third of
their students sampled not seeing a value in the posts and participation of other students again
reflective of our study findings.
5. Recommendations
There are key areas to consider when implemented a blended learning and assessment approach to
large groups of students that have been covered in this paper.
It was clear that some students needed to further understand the benefits of learning in an online
environment right at the outset of the programme. Although some students had participated in online
learning previously, it is clear that experiences can be diverse depending on the platform, nature of
learning, tools adopted and skills/experience of the lecturers. Getting students face to face at the start
of the programme and giving them ownership and steer of the online arena may increase levels of
engagement. Furthermore, understanding the profile and types of learners within the cohort at the
outset delivers the potential to group students into particular learning groups where different
resources can be utilised.
Using learner ‘e-champions’, or student reps to help others see the value in online learning and the
potentials of what could be gained can potentially be beneficial. Ultimately this could lead to students
acting as facilitators themselves, which has been explored in another area of the University and has
proved successful not only in online engagement but also in students developing management and
leadership skills (Clouder et al 2010). By exploring the possibilities of online learning in the first year
of study may equip students with the skills to start to lead and facilitate groups as they progress
through their programme.
Links to assessment will also impact levels of choices felt to the degree of interaction applied. It is felt
from the research that this should increase both motivation and amount of posts increasing interaction
through the VLE, although there are concerns that this learning process may be hampered by
summative marking.
Although flexibility is the key benefit to any eLearning programme, considerations need to be made in
larger groups how discussion postings will be managed, especially when requirements are placed
upon modules to cover an indicative content. It is therefore important that modules such as MOB,
729

Marie Sams et al.
include time constrained tasks and activities that ensure students are able to receive feedback,
whether from peers or facilitators, before they begin any additional tasks, this allows the students to
better utilise the feedback to improve future work.
The level of flexibility should however be determined by the students. This research has found that
the more flexible the approach to study and activities, the more independent some of the students
became. This level of independence develops throughout the academic year as both the facilitators
and students became more creative in their approaches.
There will always be arguments for and against eLearning, however it is clear that as education
evolves technology will certainly not be taking a back seat. Students will increasingly expect to
participate in some form of online education at points within their study, and the key message is rather
than bombard them with fancy technologies and heavy discussion forums, that the focus is kept on
learning outcomes and developing skills within the learner to become more independent and self
directed. By placing the student at the centre of shaping their learning, both the facilitators and
students can find a more rewarding experience. If students are able to see the benefits of eLearning
and self directed study within those areas, it is likely that they will become the drivers eLearning, as
well as gaining rich experiences that are transferable into employment.
References
Clouder, D.L. Krumins, M. Davies, B (2010). Leadership online: student facilitated interprofessional learning.
Book chapter: Interprofessional eLearning and Collaborative Work: Practices and Technologies. IGI Global
Dermo, J. (2009) ‘E-Assessment and the Student Learning Experience: A Survey of Student Perceptions of E-
Assessment’. British Journal of Educational Technology 40 (2), 203-214
Ellis, R., Ginns, P., Piggott, L. (2009) ‘ELearning in higher education: some key aspects and their relationship to
approaches to study’. Higher Education Research and Development 28(3), 303-318
Fleming, N. (1995), I'm different; not dumb. Modes of presentation (VARK) in the tertiary classroom, in Zelmer,A.
(1995) Research and Development in Higher Education, Proceedings of the 1995 Annual Conference of the
Higher Education and Research Development Society of Australasia (HERDSA),HERDSA, Volume 18, pp.
308 -313
McGhee, G. Glenn, R and Atkinson, J (2007) Literature reviewing and reflexivity. Journal of Advanced Nursing
60, 334-342
McLoughlin, C and Luca, J. (2002). A learner centered approach to developing team skills through web-based
learning and assessment. British Journal of Educational Technology. 33 (5), 571-582.
Mompo, R., Redoli, J. (2010) ‘Some Internet-based strategies that help solve the problem of teaching large
groups of engineering students’. Innovations in Education and Teaching International 47 (1), 95–102
Nagel, L., Kotzé, T. (2010) ‘Supersizing eLearning: What a CoI survey reveals about teaching presence in a large
online class’ Internet and Higher Education 13, 45-51
Nicols, D. (2007) ‘Laying a foundation for lifelong learning: Case studies of e-assessment in large 1st-year
classes’. British Journal of Educational Technology 38 (4), 668-678
Owens, D. Price, L (2010) ‘Is eLearning replacing the traditional lecture?’ Education and Training 52 (2), 128-139
Pachler, N., Daly, C. Mor,Y., Mellar,H. (2010) ‘Formative E-Assessment: Practitioners Cases’. Computers and
Education 54, 715-721
Roberts, E. Lund, J. (2007) Exploring eLearning community in a global postgraduate programme. In the Sage
Handbook of eLearning research, ed. R Andrews and C. Haythornwaite, 487-503. London. Sage.
730

Designing Effective Online Group Discussions
Rowena Santiago 1 , Amy Leh 1 , and Minoru Nakayama 2
1
Science, Mathematics and Technology Education Department, California State
University, San Bernardino, USA
2
CRADLE, Tokyo Institute of Technology, Tokyo, Japan
rsantiag@csusb.edu
aleh@csusb.edu
nakayama@cradle.titech.ac.jp
Abstract: Together with lectures, class discussions are mainstays of university teaching. Instructors engage their
students in class discussions to further interpret, analyze, evaluate, infer, reflect on or explain what is being
learned in class, in other words, to engage students in high-level thinking. Instructors are now turning to
technology (e.g., GoogleDocs, Blackboard) to enhance group discussions and to overcome traditional constraints
especially in large classes. However, when converting from face-to-face to online format, group discussions,
which are primarily student-student interactions, are the most difficult activities to design and implement (as
compared to student-teacher or student-content interactions). Failure to provide effective group discussions in
online classes adversely impacts the quality of online teaching and student learning. It then becomes imperative
that faculty learn how to design online group discussions effectively so that they can provide students with
successful high-level thinking experiences and increase the level of student engagement. The first part of this
paper will present the following: the rationale for the use of a faculty learning community to help faculty become
better designers of online group discussion activities, the process that was involved in the structuring of the
faculty learning community, and the set of instructional design guidelines that the FLC generated. The second
part of the paper will present the implementation results, including the SoTL (scholarship of teaching and
learning) integration that was carried out by one of the members of the faculty learning community. This includes
the improvements made to the original assignment activity as a result of peer review, what worked or didn’t work,
as indicated by the results of a student survey, and reflections for the next re-design. The paper will conclude
with a reflective summary of how results from the use of faculty learning community and the integration of SoTL
can contribute to designing effective online group discussions.
Keywords: online teaching, online group discussions, faculty learning community, scholarship of teaching and
learning, high-level thinking skills, faculty development
1. Introduction
Together with lectures, class discussions are a mainstay of university teaching. It is during class
discussions that opportunities for various types of student engagement (student-instructor, studentcontent,
student-student) can take place. When done as a group activity, class discussions can
provide opportunities for creating student learning communities under the guidance of the instructor
(Colorado State University, 2010). For many students, it is an opportunity to compare one’s
understanding of concepts and content with those of peers and even the instructor’s (University of
Pittsburgh, 2007). Further, class discussions can be used by the instructor to provide some
scaffolding before students are assessed individually for learning performance. However, one major
reason that instructors engage their students in class discussion is to have them further interpret,
analyze, evaluate, infer, reflect on or explain what is being learned in class. In other words, it is an
important activity for developing high-level thinking skills.
Large classes often present a bigger challenge in facilitating class discussions for high-level thinking
or critical thinking (Illinois State University Center for Teaching, Learning & Technology, 2010). To
promote student engagement in large classes, students are often divided into smaller groups for
discussion. This activity often requires not only careful structuring and management by the instructor,
but a substantial amount of class time.
To enhance group discussions and to overcome traditional constraints, instructors are now turning to
technology (e.g., GoogleDocs, Blackboard) for online group discussions. Online group discussions
provide the same opportunities as face-to-face for enhancing high-level thinking skills, student
engagement and more. Asynchronous online group discussions, in particular, also have the added
advantage of providing that much needed time to first carefully think, to construct a meaningful
response, or to reflect on the contributions of other group members, before posting one’s own
contribution to the discussion.
731

Rowena Santiago et al.
However, when converting from face-to-face to online format, group discussions, which are primarily
student-student interactions, are the most difficult activities to design and implement (as compared to
student-teacher or student-content interactions). Failure to provide effective group discussions in
online classes adversely impacts the quality of online teaching and student learning. It then becomes
necessary that faculty learn how to design online group discussions effectively and that efforts at
professional development or elaboration of course design and implementation be provided (Abrami,
et. al., 2008) so that instructors, in turn, can provide students with successful high-level thinking
experiences and increase the level of student engagement.
2. Purpose of the paper
The first part of this paper will present how a faculty learning community was formed and structured to
help faculty become better online instructors by learning collaboratively how to design effective online
group discussions. The second part will present the results of SoTL integration by one of the
members of the faculty learning community as part of the implementation of the class assignment on
online group discussion that was designed based on the guidelines generated by the faculty learning
community. Part Two results will include the improvements made to the original assignment activity as
a result of peer review, the implementation in class, the results of the student survey and reflections
for the next re-design.
3. Faculty learning communities (FLC)
A faculty learning community (or FLC) consists of faculty members from different disciplines who form
a group, with the purpose of learning together to enhance teaching and student learning. The group
engages in a series of activities (or curriculum) within an extended period of time (usually one year).
The use and effectiveness of faculty learning communities as a strategy for improving teaching skills
have been reported in the literature (Beach and Cox, 2009, Desrochers, 2010, SBCTC, 2006).
According to Cox (2004, p.18) who is a recognized leader in this field, FLC’s serve a bigger purpose
and achieve better outcomes in terms of faculty development because “a member of an FLC on any
topic will not only learn about that, but over the course of the year will design and implement it in a
focus course, with many opportunities to reflect with other FLC participants on its effectiveness and
the assessment of resulting student learning and feedback”.
4. FLC structure and methodology
The faculty learning community that was formed for online group discussions consisted of six (6)
faculty members from six different disciplines. The group met once every 3-4 weeks, from October to
May. FLC members engaged in a two-part process: (1) as a group, faculty learned how to design an
effective online group discussion, and (2) then, the resulting instructional design for online group
discussion (in the form of a class assignment) was implemented in individual courses.
The overall structure of the faculty learning community was designed to produce the following
deliverables:
Product: Development of an individual lesson design (or re-design) that incorporated the best
practices or principles of effective online group discussions to facilitate high-level thinking skills
and student engagement for a specific course goal. This was accomplished by generating a set of
guidelines based on (a) readings and discussion of relevant references (b) working with two
consultants as a group and individually, and (c) participating in an actual online group discussion
using Blackboard’s discussion forum.
Practice: Implementation in the classroom
Assessment: Evaluation of the effectiveness of the lesson design (or re-design) by doing a survey
of student learning experience
Report and Reflection: submission of a 500 to 600-word report that summarized the project, plus
a reflection section on how this will be used in future online classes
Dissemination: Presentation of the project in a department meeting
The overall structure for FLC-Part 1 is as follows: after sharing and discussing their concerns and/or
initial experiences in using online group discussions for teaching, FLC members worked with two
consultants and a set of curriculum materials (journal articles, books on the topic) to generate a set of
guidelines that incorporated the best practices or principles of effective online group discussions.
732

Rowena Santiago et al.
These guidelines were then used to design or re-design one’s class assignment(s) for online group
discussion, after peer review of each assignment.
Part 2 involved the actual implementation of the guidelines and assignment activity in each member’s
course. Prior to implementation, FLC members reviewed each other’s assignments and provided peer
feedback for improvement or clarification. FLC members then revised their assignments accordingly
for implementation. After implementing the assignment, each instructor assessed the effectiveness of
the activity by surveying students on their online group discussion experience. Both oral and written
reports were also required. For the oral dissemination, each FLC member made a presentation of the
project at a department meeting. The written reports that were submitted included the following: the
implementation summary, the results of the student survey, and the instructor’s reflection on the
results.
5. Scholarship of Teaching and Learning (SoTL)
Two decades after Ernest Boyer presented the concept of scholarship of teaching in his influential
book, Scholarship Reconsidered: Priorities of the Professoriate (Carnegie Foundation, 1990),
scholarship of teaching and learning (or SoTL) has become a growing movement in higher education,
and has become part of faculty development programs provided by centers for teaching and learning.
SoTL definitions given in the literature (Bruff, D., 2007; McKinney, 2003) reflect the consensus that
SoTL involves the following: (1) a systematic process of addressing an issue in teaching and learning
(identifying an existing issue, exploring and analyzing solutions, implementing a solution, and
observing, measuring and documenting results and consequences), (2) reflection, (3) peer review,
and (4) public sharing or communication of results. It is within this context of SoTL that the
implementation of the FLC assignment project was conducted by one of the authors and was also the
instructor of the course in Instructional Technology. Specifically, the SoTL goal was to better serve her
students by improving her instruction through the use of systematic inquiry, peer review and selfreflection;
and to re-implement the project after revisions were made, based on the evaluation of the
effectiveness of the current project implementation. For the overall course design, the instructor
designed and implemented two major innovations in her class: (1) the use of online group discussion
and (2) the use of Blog and Discussion Board. The instructor collected and analyzed data in order to
evaluate the effectiveness of these innovative projects. However, in this paper, she reported only on
one of the projects: the use of online group discussion in instruction.
6. Results
6.1 Part One: Faculty learning community results
6.1.1 Guidelines generated for online group discussions
After doing the readings and discussions, the faculty learning community generated a list of guidelines
for online group discussions. The resulting guidelines addressed five main issues which the FLC
identified to be necessary for developing their class assignments for online group discussions. The
guidelines were generated with the understanding that the readings and references used, as well as
other individual materials on the topic, will provide the necessary details for each issue, thus allowing
every FLC member to pursue details that would apply to each one’s specific course and assignment.
The guidelines addressed the following:
The assignment should state the learning objective(s) for high-level thinking skills (e.g., critical
thinking)
The assignment should come with rubrics for evaluating the quality of responses given.
The assignment should be aligned with learning objectives and rubrics
The assignment should provide students with discussion guidelines that are clear and appropriate
for the online discussion model that the instructor is using, e.g., teacher-centered, student-led,
etc.
The assignment should address all necessary management issues: guidelines for various roles
(initiator, moderator, responders, summarizer, etc); defining the required response levels: one
level (student responds to teacher’s post), two levels (response to a level 1 response, etc.); and
logistics (e.g., dates for posting discussion question or materials, time frame for sending
responses, netiquette, and length of responses).
733

Rowena Santiago et al.
6.1.2 Assignments and rubrics developed for online group discussions
In response to the need for course re-design and for learning how to structure more effective online
group discussions that facilitate critical thinking skills and student engagement, assignments and
rubrics were developed for the following courses:
Evaluating and critiquing peer assignments/projects (Instructional Technology)
Enhancing argument in an online class, which is case-based and student interaction and active
learning are important (Communication Studies)
Structuring online group discussion for multi-country group projects and team work (International
Management)
Reviewing and assessing the curriculum and lesson plans (Technology Education)
Engaging students in practice problems and discussion of online video/audio lecture (Accounting
and Finance), and
Facilitating online interactions and collaboration when analyzing and explaining the Spanish
language (World Languages).
Based on these assignments developed by FLC members, the most immediate and relevant impact of
this FLC was going to be at the classroom level where the practice of promoting critical thinking
across the curriculum was going to be addressed.
Each assignment was reviewed by group members, using the guidelines that were generated earlier.
Some comments were posted on the discussion forum, but most discussion and review feedback took
place during the face-to-face meetings. Most of the rubrics were revised due to two main reasons: too
much emphasis on participation, and no/fewer criteria for assessing how critical thinking was reflected
in student responses during the online discussion.
Results of the implementation of one of these assignments (Instructional Technology) are presented
in the next section.
6.2 Part Two: Implementation and SOTL integration results
6.2.1 Course
The course that was used for this project was ETEC648: e-Learning Delivery and Evaluation. It was
one of the courses in a certificate program for students who wanted to become online instructors or
online training facilitators. ETEC648 was a completely online course where interaction (studentcontent,
student-student, student-instructor) played a critical role, and at the same time, it aimed to
develop high-level thinking skills among students by engaging them in the following online group
discussion activities: (1) students critiqued peers’ e-Learning delivery plan and materials for a “good
start” of e-Learning instruction; and (2) students explained and justified their viewpoints for accepting
and not accepting peer review suggestions.
6.2.2 Assignment
The original assignment that was designed for this class had a brief introduction on what the students
did the previous week. The brief introduction helped the students make a connection between the
previous assignment and the new assignment for which students needed to post their e-learning
delivery plans including their materials on how to conduct a “good start” for their online instruction
For the online group discussion, each student was asked to select two classmates and to critique their
delivery plans and relevant materials (for example, their technology checklists for a good start). To
make sure that everyone’s project would be critiqued, each student chose two assignments to review,
and then announced their choices by posting threads in the Blackboard discussion forum. For
example, to critique the assignments of Mary Smith and John Garcia, the student created a thread
with his/her name, after confirming that no other person had already posted his/her intention to
critique Mary’s and John’s work. This critique was considered as Level I Response, where every
student project (delivery plan and materials) was reviewed by a maximum of two peers. The critique
had to be within 500-800 words and it should cite the sources (or references) to support the critique
being given (for example, page 207 of Waterhouse’s book). Each critique consisted of two parts: first,
734

Rowena Santiago et al.
it pointed out the good points and strengths of the classmate’s project, and second, it gave
suggestions for improvement and explained how to make the projects better. These critiques were
used to help one’s peers refine their final projects.
After receiving the critiques given to one’s own project, each student responded to his/her reviewers’
critiques, first by thanking them for their critiques, and then letting them know if their suggestions were
to be accepted and included in the revision with appropriate explanations. This Level 2 Response had
to be within 200-400 words and supported by literature source(s). Each response consisted of two
parts: first, it pointed out which critique(s) from the reviewer were accepted and why, and second,
which critique(s) were not accepted and why.
To eliminate the possibility of instructor’s influence on the online discussion, instructor feedback was
given directly via individual email within two business days after the posting deadline. Students were
asked to follow netiquette rules and were allowed to continue the discussion with their peers beyond
what was required in the assignment. However, all discussions were ended by midnight of the
deadline date, during which time, the online forum was locked.
For assessment, grades for the “Online Discussion” activity were based on a rubric for: critical thinking
(Level I responses and Level II responses), organization, relevance of responses to course content,
timeliness and logistics, and writing mechanics.
6.2.3 Applying the FLC guidelines
As a result of the faculty learning community discussions, peer review, and the FLC guidelines
generated collectively on designing assignments for online group discussions that promote high-level
thinking skills, the strengths and weaknesses of the original assignment were identified, and changes
for improvement were made.
Strengths of the original assignment:
The assignment met the FLC guidelines: the learning objectives (critique individual plans and
materials when designing a “good start” for e-instruction) were clearly stated, the assignment
included a rubric, and clear explanation were given for the levels and length of responses,
timetable, netiquette and civility (saying thanks, addressing good points first).
The learning goals and the learning activity were aligned, i.e., the goal was for the students to
engage in critical thinking as applied to the subject matter to be learned for the session, and the
critique activity provided the means to carry out this learning goal.
Changes for improvement:
Rubric – There was a need for minor revisions to customize the rubric and align it better with the
learning goals and activity. One of the revisions was in the original criteria for evaluating the
quality of Critical Thinking Level I and II responses, which were as follows:
Outstanding (3 points): Presents the most important arguments (reasons and claims) pro and con.
Proficient (2 points): Presents relevant arguments (reasons and claims) pro and con.
Basic (1 point): Presents some arguments without reasons.
Below Expectation (0 point): Fails to present or hastily dismisses strong, relevant counter-arguments.
The text “presents the most important arguments (reasons and claims) pro and con” was revised to
read as “presents the most important strengths and good points, and provides clear suggestions for
improvement.” in order to align it better to the assigned learning activity (“provide critiques that
consisted of identifying the good points and strengths of the classmate’s project, and then giving
suggestions for improvement”).
Another revision to the rubric was the addition of criteria that would explicitly evaluate whether or not
the students posted the required Levels I and II responses. The range of the point system that was
added to the rubric was: from 3 points for “Provided both level I and II responses” to 0 points for “Did
not provide any Level I or Level II responses”.
Formatting for Readability – The original assignment was presented in a continuous text format,
and there was a need for improved formatting for better readability. One of the revisions made to
735

Rowena Santiago et al.
the original assignment was to include section headings and/or text boldface to improve
readability and to break the continuous text format.
6.2.4 Implementation
The assignment was implemented during the 5 th week of class (out of 11 weeks in a quarter system).
The class, which consisted of 11 students, had three weeks to complete the assignment and post
their Level I and II responses to the online group discussion. For the student survey and data
collection, student were given a two-week period (Weeks 8 and 9 of the quarter), plus another week
for reminding those who had not completed the survey to do so. Data analysis was done after finals
week and after grades were submitted.
6.2.5 Survey results
The survey had 15 questions and was administered right after the online discussion assignment.
Students selected SA (Strongly Agree), A (Agree), N (Neutral), D (Disagree) or SD (Strongly
Disagree) for each survey item. See Table 1 for survey items and data summary.
Table 1: Survey Items and Data Summary (in percentages)
ITEM
1. The objectives of the assignment were clear to me.
SA A N D SD
2. The connection between the objectives of Assignment 5 (online
31% 50% 19% 0% 0%
discussion) and the course objectives was clear to me.
3. I was clear on what I was going to learn from the online
discussion (critiquing my peers’ work and providing Level II
53% 33% 13% 0% 0%
response). 44% 44% 13% 0% 0%
4. My role (both as reviewer and as responder) and the nature of
my participation in the online discussion were clearly explained.
38% 31% 25% 6% 0%
5. The frequency and length of responses required were
appropriate and reasonable. 53% 27% 20% 0% 0%
6. The rubric clearly described how I would be evaluated in the
online discussion. 56% 25% 19% 0% 0%
7. The online discussion was conducted in a very collegial,
respectful and polite manner. 31% 19% 25% 19% 6%
8. The quality of my responses was better during online discussion
than in-class discussions (if the assignment would have been
conducted in class). 38% 44% 19% 0% 0%
9. I did more high-level learning (analysis and synthesis, reflective
thinking, critiquing, critical thinking, etc.) in the assignment as a
result of this online discussion activity. 27% 33% 33% 7% 0%
10. The online discussion gave me more opportunity to think and
ponder about my responses. 31% 50% 19% 0% 0%
11. I learned a lot from the responses given by my classmates
during the online discussion. 31% 56% 13% 0% 0%
12. My peers’ feedback helped me to produce a better project. 38% 31% 25% 6% 0%
13. I enjoyed participating in this online discussion. 38% 38% 25% 0% 0%
14. Given the opportunity, I will participate again in online group
discussions in future classes. 44% 38% 19% 0% 0%
15. If I would teach an online course, I would use online discussion
in my class. 31% 50% 19% 0% 0%
Eighty percent of the students selected “Strongly Agree” or “Agree” when responding to ten (Items 1,
2, 3, 5, 6, 8, 10, 11, 14, and 15) out of the 15 questions in the survey. Students indicated that (1) the
objectives of the assignment, (2) the connection between the objectives of the assignment and the
course objectives, (3) the evaluation (rubric), and (4) instructions on conducting the assignment, were
very clear. They also indicated that: the frequency and length of responses required for the
assignment were appropriate and reasonable; the quality of their responses was better during online
discussion than in-class discussions (if the assignment would have been conducted in class); the
online discussion gave them more opportunity to think and ponder about their responses. They felt
that they learned a lot from their peers, and given the opportunity, they will participate again in online
group discussions in future classes. They also would use online discussion in their classes if they
were to teach an online course.
736

Rowena Santiago et al.
Only one student selected “Disagree” on the following survey items:
My role (both as reviewer and as responder) and the nature of my participation in the online
discussion were clearly explained.
I did more higher-level learning (analysis and synthesis, reflective thinking, critiquing, critical
thinking, etc.) in the assignment as a result of this online discussion activity.
My peers’ feedback helped me to produce a better project.
Twenty-five percent of the responses indicated that the online discussion was not conducted in a very
collegial, respectful and polite manner.
The results indicate that, as a whole, the online discussion experience was positive although a few
students considered the experience to be not as positive as the others. Also, readers should be
cautious in interpreting these results because the class size was small and because some students
responded to the survey more than once. These results are expected to be most meaningful to this
instructor who will then apply them in the next cycle of re-designing her online group discussions.
6.2.6 Reflections
As part of the faculty learning community, instructors were required to submit a reflection essay on
their experience. This instructor wrote the following:
“I found that having students critique each other’s work could help develop and
demonstrate student critical thinking skills. The guidelines our FLC generated were very
helpful for designing and developing a well-organized/structured online group discussion.
According to my survey results, four responses indicated that the online discussion was
not conducted in a very collegial, respectful and polite manner. Unfortunately, students
did not provide comments on the issue. In the future, I plan to (1) explain to students how
to conduct online discussion in collegial, respectful and polite manner before they
actually start online discussion and (2) revise my survey by inserting comments on each
survey item. I feel that being part of a faculty learning community contributed to more
effective use of online group discussion and the development of students’ critical thinking
skills in my course. The guidelines and the rubric the FLC generated saved me much
time in the assignment development.”
7. Conclusion
The challenge of supporting faculty in designing effective online group discussions was addressed in
this project through the use of faculty learning community because of its collaborative and synergistic
features on learning together and doing peer reviews, and its built-in implementation and reflection
components. Faculty participants were able to generate a set of guidelines for online group
discussions (which were conducted as class assignments), and then used these FLC guidelines in the
design and revision of these assignments to improve the alignment of learning objectives with the
learning activity and with assessment (rubrics), and for better management of the online group
discussions (logistics, organization, netiquette). The FLC structure used in this project put equal
emphasis on learning and practice (implementation of the re-designed assignments in the classroom).
This reflects the focus that was placed on the instructional design of online group discussions, a
practice that is supported by Yang, Newby and Bill (2008, p.1585) whose study concluded that “with
cautious instructional design, active dialogue may be facilitated, critical thinking skills may be
advanced, and the barriers to distance learning may be diminished”. Although the FLC was conducted
for eight months, a longer period would have been more desirable to give participants additional
opportunities to apply what was learned and share results as a community. A more formal evaluation
of the impact of the faculty learning community, through survey and interview of participants will be
the next phase of this project.
The integration of SoTL in the implementation of the project was also presented in this paper. It was
conducted by one of the instructors, whose goal was to gather information on how well the
instructional design of her assignment helped achieve the learning objectives (high-level thinking
skills) and the successful management of online group discussions, and then, as part of the reflection
component of SoTL, use this information to improve the next re-design (or version) of the assignment.
It is the demonstration of this complete cycle of “systematic design-practice-gather information-reflectrevise
for improvement-share” that is considered as the other significant part of this project.
737

Rowena Santiago et al.
Together, the use of faculty learning communities for faculty development, and the integration of SoTL
for the continuous improvement of instructional design can contribute significantly to designing
effective online group discussions.
References
Abrami, P.D., Bernard, R.M., Borokhovski, E., Wade, A., Surkes, M.A., Tamim, R. and Zhang, D. (2008)
“Instructional Interventions Affecting Critical Thinking Skills and Dispositions: A Stage 1 Meta-Analysis”,
Review of Educational Research, Vol. 78, No. 4, pp. 1102–1134.
Bruff, D. (2007). “Scholarship of Teaching and Learning”, [online], Vanderbilt University,
http://cft.vanderbilt.edu/teaching-guides/reflecting/sotl/ .
Colorado State University (2010) “Teaching Guide: Leading Class Discussions”, [online], Colorado State
University, http://writing.colostate.edu/guides/teaching/discussions/.
Cox, M.D. (2004) “Introduction to Faculty Learning Communities”, New Directions for Teaching and Learning,
Vol. 2004, No. 97, pp. 5-23.
Desrochers, C.D. 2010 “Faculty Learning Communities as Catalysts for Implementing Successful Small-Group
Learning”, in J. Cooper and P. Robinson, Eds. Higher Education: Research and Practice. Stillwater, OK:
New Forums Press, pp. 1-17.
Illinois State University Center for Teaching, Learning & Technology (2010) “Class Discussion”, [online], Illinois
State University Center for Teaching, Learning & Technology,
http://www.cat.ilstu.edu/resources/teachTopics/classd.php.
McKinney, K. (2003). “What is the Scholarship of Teaching and Learning (SoTL) in Higher Education?”, [online],
Illinois State University, http://www.sotl.ilstu.edu/downloads/pdf/definesotl.pdf .
McKnight, C.B. (2000) “Teaching Critical Thinking through Online Discussions”, [online], Educause Quarterly,
http://net.educause.edu/ir/library/pdf/EQM0048.pdf.
SBCTC (2006) “Faculty Learning Communities”, [online], Washington State Board for Community and Technical
Colleges, http://www.sbctc.edu/college/_e-assessfaclearningcommunities.aspx.
University of Pittsburgh (2007) “Communication across the Curriculum”, [online], University of Pittsburgh,
http://xserve.umc.pitt.edu/development/cxc/site/instructor/class-discussions.html.
Yang, Y.C., Newby, T., and Bill, R. (2008) “Facilitating Interactions Through Structured Web-Based
Bulletin Boards: A Quasi-Experimental Study on Promoting Learners’ Critical Thinking Skills”, Computers &
Education, Vol. 50, No. 4, pp. 1572–1585.
738

The Game and the Alternating Roles of Learner/Teacher as
Facilitators of the Learning Process in Organizations
Vitor Santos and Luis Amaral
Universidade do Minho, Guimarães, Portugal
vsantos@dsi.uminho.pt
amaral@dsi.uminho.pt
Abstract: In this work a new strategy is proposed for online training of members of medium and large
organizations, based on the sharing and exchange of knowledge among them. The strategy is supported with the
creation of intra-enterprise online communities, where all members assume, simultaneously, the role of trainer
and trainee. The interaction in the community is achieved through a game, in which each participant challenges
others to learn what he has to teach in his domain area, sharing information and resources on matters that they
dominate and where are, at the same time, challenged to learn other subjects from different professional areas.
The adoption of this model could change the classic positioning of distanceLearning systems based on Internet
by giving a very significant role to the learning communities and to the use of games as a challenging way to
improve the level of expertise of the members of an organization, helping them to cooperate and to better
exchange information.
Keywords: learning organizations; learning communities; eLearning; gaming
1. Introduction
The company’s role in the development of individual skills is becoming universally regarded as
beneficial for both workers and companies.To perform this role, an important alternative to consider is
the use of information and communication technologies, especially those based on the Internet and
dedicated to the teaching (Santos 2000).
The use of these technologies is justified by the benefits that they can provide in terms of flexibility in
education and the low cost, allowing, in most cases, to access a bigger number of courses that
employees can perform in their job (Castells 2004).
Traditionally, the online training is based on a logic trainer /trainee where the roles of trainer and
trainees are quite distinguished. This online training strategy, which we call traditional, is not suitable
for an environment of sharing and exchange of knowledge between employees. Thus it is not ideal to
be used as a way to pass knowledge from the more experienced employees and experts to the
others.
However, it is possible through technology, to implement other than conventional approaches such as
the construction of communities of learning, where participants develop skills in a playful alternation of
roles between the learner and teacher allowing to share and exchange knowledge (Santos 2004,
2007).
In this article, we propose a strategy for online training of members of medium and large
organizations, based on sharing and knowledge exchange between these members.
This strategy will enable each employee to perform alternately the role of trainer and trainee,
participating in an intra-enterprise online community where, through a game, each participant
challenges others to learn what he has to teach in their area of professional expertise, sharing
information and resources on matters he dominates and where, at the same time, he is challenged to
learn from others’ professional matters.
2. Learning in organizations
"Knowledge" is one of the organizations’ main assets. The ability to maintain it and to increase it is a
big challenge. This challenge is based, essentially, in the qualification of human resources, in the
ability to retain and circulate the knowledge inside the organization and in creating conditions for
developing the employee skills.
739

Vitor Santos and Luis Amaral
To build a culture of collective learning, where the concepts of collaboration, sharing, learning and
knowledge are key words, a necessary condition for success is to engage all people in the
organization.
The organizations that aim to build structures and strategies to stimulate and increase the potential for
learning are called "Learning Organizations".
Senge defines Learning Organizations as organizations where people are focused on collective
learning and committed to results. According to this author, the organizations that learn are
"organizations where people continually expand their capacity to create the results they really want,
which stimulate new patterns of thoughts and comprehensive, collective aspiration to win freedom
and where people continually learn together " (Senge 1998).
To Friedman a learning organization is defined as an organization that "is a permanent process of
change, because workers are continually encouraged to perform modifications and adaptations. A
learning organization [...] focuses on the creation, acquisition and transmission of knowledge and the
adaptation of behavior in terms of knowledge" (Friedman 1999).
For an organization that wants to be a "Learning Organization", it is fundamental to stimulate an
environment that encourages learning. For that it is strategic to have a clear and fluid information
channel in the whole organizational structure and to ensure top management commitment.
The “Learning organizations” are particularly advantageous when they are inserted in turbulent and
unpredictable environments, as are the cases of IT markets. An environment that fosters learning,
facilitates the exchange and learning with the external environment, strengthens a receptive and nondefensive
behavior, and facilitates the recovery and retention of knowledge.
Searching for strategies that help grow the level of knowledge of an organization and motivate its
members to cooperate better; it consequently leads the organization to become more competitive and
resistant, therefore, of a growing interest.
3. Conceptual model
The model’s main objective is to enable the creation of online intra-business communities, where all
members perform, alternately, the role of trainee and trainer in a game where everyone can share
knowledge and learn from everyone else.
The interaction in the community is obtained through a game in which each participant challenges the
others to learn what he has to teach in his expertise areas. For that, each participant shares
information and resources on matters that they dominate, and, at the same time, challenged by the
learning materials provided by other members.
The learning resources are shared online by query where a resource classification area or areas is
proposed. For each learning resource an associated questionnaire to validate the learning
constructed by "trainees" is provided.
In this context, we understand learning resources as a set of texts, images, videos, webcasts, links,
references, tips, …, that helps one learn about a subject.
The contents and associated questionnaires validation is done after a query by one or more
individuals called "Gurus". The "Gurus" may be external or internal to the organization.
The validation ensures that the contents are correct on the technical point of view and that they
contain enough information to answer the questionnaires. In Table 1 an example of a "Guru" resource
validation grid is shown.
Table 1: Validation table example
Parameter
Content Correctness Y/N
Survey Suitability Y/N
740

Vitor Santos and Luis Amaral
The "Guru" also has the task of classifying the contents and the questionnaires for their relevance in
the educational and pedagogical aspects. This assessment grades the content. Table 2 presents an
example of "Guru" resources evaluation grid.
Table 2: Validation table example
Parameter Weight
Comprehensibility 25
Complexity 25
Applicability/Utilitty 25
Usability 15
Multimedia Learning objects 10
100
If the query is not validated then it is rejected. If accepted, then it is subject to a second assessment
by Human Resources or by the internal organizational entity that owns the people management in the
organization. This assessment aims to confirm if the proposed training area or areas and if it own
content are appropriate for the organization the learning objectives.
If the assessment is positive then the resources, its associated questionnaires and the score (result of
Guru’s assessment) are published. The obtained score became the content reference value (RV) in
the game.The goal of the game, for each and every individual and group is to “win” the "knowledge"
provided by the other community members and to transfer it to him and to the group where he
belongs, for example, as shown in Figure 1, a Department.
Figure 1: Platform generic representation
In our system the knowledge associated with each resource is identified by content reference value
RV (content) and is measured by points. The goal of every learner in this game is to raise points
individually and/or somatically to his parent group, increasing his Individual Score IS (learner), hence
increasing the Group Score GS (learner) he belongs to.
The added points reflect the application of the following formulas:
IS (learner)

Vitor Santos and Luis Amaral
Although not mandatory, the content application also gives points to increase the SCP - Score
Content Producer (content) and for the group, increasing the SGP - Score Group Producer (content).
Thus we have:
SCP (content)

Vitor Santos and Luis Amaral
Figure 3: Generic architecture
The services provided by the management platform can be grouped into 4 major functional groups:
Game, Content Management, User Management and Administration.
Table 3: Features
Functional Group Features
Game consulting resources, submission of answers to
questionnaires, social and recreational areas;
individual and collective rankings consultation
Content management resources submission (content and questionnaires);
Gurus and the Human Resources restricted access
area - available services for validation, acceptance,
classification and resources evaluation;
User management Learning community management - users
management: individual and group records, accounts
check, communication dispatch,...
Administration Update and site configuration, institutional
communication management.
To support the system, the technical architecture must ensure the availability of applications, both of
front-office and back-office. For that, we just need a typical and simple intranet architecture, with a
medium level of security, such as that is presented in Figure 4:
5. Game simulation
In order to have a better understanding about the success factors and the main conditions for the
model applicability, we carried out a simulation where we reproduced the model functional
architecture and operationalized a manual implementation of the interactions between different actors.
The simulation was run at Microsoft Portugal.
The simulation was designed to collect information on the functional viability and to understand what
would be the main difficulties to implement the model.
To support the simulation, we built a test portal named "Knowledge Game", mounted on SharePoint
Server 2007 server. The portal home page is shown in Figure 5.
743

Figure 4: Technical architecture
Vitor Santos and Luis Amaral
Figure 5: “Knowledge Game” Portal
The simulation took six people, organized into two teams, which were applied for content production
and, after, to participate in game.
At the end of the experience it was possible to conclude the model could help people inside
organizations to learn from the others workers, to share their own knowledge and, at the sometime, to
have challenge and fun.
We got some important learning with the experience: there is a strong dependence on the resources
timely availability. Therefore, to ensure success, there must be a strong commitment from top
management team, involving all employees to share knowledge and content production. It is also
noted the fact the rankings were calculated manually, and be available on the server with some delay,
negatively affected the participant enthusiasm and commitment to the game. This issue will most
likely be solved with a computer system that automates these calculations and make them available
immediately.
744

6. Conclusions and future work
Vitor Santos and Luis Amaral
With the proposed model, we aim to contribute for the use of information and communication
technologies based on the Internet as a vehicle for developing skills of individual members of
organizations, for they become more and more competitive and resistant.
This model is fully inserted in the learning organizations philosophy, stimulating an internal
environment that encourages learning. This happens by creating learning communities, where the
intra-corporate use of entertainment allows members to use one of the most powerful means of
learning and human development: the interpersonal support, minimizing one of the constraints of
online education, which is the absence of the human factor as learning intermediate.
The model requires participants to experiment the experience of knowledge production in a
community. The adoption of this model could change the classic positioning of distanceLearning
systems based on Internet by giving a very significant role to the learning communities and to the use
of games as a challenging away to improve the level of expertise of the members of an organization,
helping them to cooperate and to exchange information better. This may further allow the
improvement of inter-personal relations among organizations members.
In the future we intend to build systems that could implement the proposed architecture. There is also
the possibility of developing new intra-business learning community systems, based on the new ideas
that could arise from this proposal.
References
Castells, Manuel. A (2004) Galáxia Internet: Reflexões sobre Internet, Negócios e Sociedade. Fundação
Calouste Gulbenkian, Lisboa
Friedman, B. S.; Hatch, J. A.; Walker, D. M. (1999) Mehr-Wert durch Mitarbeiter. Wie sich Human Capital
gewinnen, steigern und halten lässt. Neuwied: Luchterhand.
Santos, Arnaldo (2000) Ensino à distância & Tecnologias de Informação – eLearning. FCA Editora, 2000.
Santos, Vitor; Tavares, Cristina. (2004) “O jogo como elemento facilitador da alternância de papeis
formando/formador em comunidades de aprendizagem” – eLES04 - Conferência eLearning no Ensino
Superior, Aveiro
Santos, Vitor (2008) "O Jogo e a Alternância de Papéis Formando/Formador em eLearning - Um novo modelo
facilitador do Processo de Aprendizagem nas Organizações” Tese de Mestrado em Tecnologias e
Sistemas de Informação, Universidade do Minho
Senge, P. M. (1998) A Quinta Disciplina: Arte, teoria e prática da organização de aprendizagem. São Paulo: Best
Seller
745

Implementing and Evaluating Problem-Based Virtual
Learning Scenarios
Maggi Savin-Baden, Cathy Tombs and Katherine Wimpenny
Coventry University, Coventry, UK
m.savinbaden@coventry.ac.uk
Abstract: This paper will describe a project adopting a pedagogical approach aimed at implementing and
evaluating a problem-based learning project in an immersive world. The project involved an iterative process of
testing scenarios using student feedback to improve upon the scenarios. This paper will also present findings
from the project that will argue that developing pedagogically driven scenarios may offer a new liquidity to
learning, and that combining technology with pedagogy can present mutual benefits.
Keywords: problem-based learning, virtual worlds, pedagogy
1. Introduction
Most research to date on learning in immersive worlds has been undertaken into students'
experiences of virtual learning environments (VLEs), and perspectives about what and how online
learning has been implemented. Immersive virtual worlds (IVWs) offer different textualities that are
increasingly ushering in new issues such as temporality and spatiality, becoming not just contested
but dynamic and intersected by one another. This paper suggests that there is a lack of pedagogical
underpinning relating to the use of virtual worlds in higher education, for example there are currently
few research papers that suggest why such worlds are being used or adopting social forms models of
education which would seem a better fit than the current behaviour models that largely operate across
most of the global higher education system. The paper presents the PREVIEW project (Problembased
Learning in Virtual Interactive Educational Worlds) that sought to combine pedagogy with
technology, which has been tested in health, medicine and social care and is now currently being
tested in education, physiotherapy and psychology. It is argued the current lack of pedagogical
underpinning has introduced a number of difficulties which might be overcome by using approaches
that readily combine pedagogy with technology, thereby shifting from the VLEs to IVWs. It should be
noted at the outset that this is one of a number of papers reporting on the PREVIEW project. This
article centres on the development and evaluation of the demonstrator. Further related papers include
Conradi et al ( 2009); Pulton et al (n 2009), Round et al ( 2009a) Round et al, (2009b) Beaumont et la
2011, forthcoming.
2. Background
Problem-based learning (PBL) was popularised in the 1980s, partly in response to the predominantly
content-driven transmission educative model of the time. It arose out of a desire to give students the
opportunity to apply practices and theoretical knowledge to problems or scenarios within the
professional or clinical setting, crucially in interactive collaboration with colleagues, thus replicating
features of the real-life context of application. It has become an increasingly influential approach in
curricula in a variety of settings, across a range of subject areas. The increasing adoption of problembased
learning (PBL) and the growth in online learning each reflect the shift away from teaching as a
means of transmitting information, towards supporting learning as a student-generated activity. To
date problem-based learning (PBL) has been seen as a relatively stable approach to learning,
delineated by particular characteristics and ways of operating. Most of the explanations of and
arguments for problem-based learning, thus far, have tended to focus on (or privilege) the cognitive
perspectives over the ontological position of the learner. However, facilitating this collaborative
approach to participation and learning is considerably more challenging in self-directed and distance
learning contexts, due to difficulties associated with effective discussion between geographically and
spatially disparate learners. However, linking PBL with IVWs brings other challenges, which are
evident in the IVW literature: there has been growth in the research into students' experiences of
virtual learning environments, discussion forums and perspectives in terms of what and how online
learning has been implemented. At the same time there is an increasing interest in the use of
immersive worlds for learning. One of the reasons for such interest appears to be a recognition that
for students in workplace or competency-led courses, learning through case-based scenarios is an
excellent method for acquiring sound knowledge and developing decision-making and problem
solving skills. Thus an increasing number of curricula are based on a particular variant of case based
learning: problem-based learning (PBL) which is an approach in which students work in teams to
746

Maggi Savin-Baden et al.
manage or solve a problem (Savin-Baden, 2000). Guided by a tutor they share their existing
knowledge and understanding relevant to the scenario, agreeing on what they need to learn and how
to carry it out. Medicine and Healthcare education have used this approach in the UK since the mid
1980s but there has been a shift in the last three years toward moving into online and immersive
spaces (Conradi et al 2009; Savin-Baden, 2007).
3. Informing literature
It could be argued, and increasingly is, that cyberspace has resulted in a sense of multiple identities
and disembodiment, or even different forms of embodiment. The sense of anonymity and the
assumption that this was what was understood through one’s words rather than one’s bodily
presence, is becoming increasingly unmasked through worlds such as Second Life. However, before
this is explored it is perhaps helpful to delineate current forms of PBL.
3.1 Face-to-face problem-based learning
Problem-based learning was an approach popularised by Barrows and Tamblyn (1980) following their
research into the reasoning abilities of medical students at McMaster Medical School in Canada. This
was because they found that students could learn content and skill, but when faced with a patient
could not apply their knowledge in the practical situation. Barrows and Tamblyn’s study and the
approach adopted at McMaster marked a clear move away from problem-solving learning in which
individual students answered a series of questions from information supplied by a lecturer. In this
early version of problem-based learning, certain key characteristics were essential. Students in small
teams would explore a problem situation and through this exploration were expected to examine the
gaps in their own knowledge and skills in order to decide what information they needed to acquire in
order to resolve or manage the situation with which they were presented. The ‘problems’, also termed
‘scenarios’ are central to student learning in each component of the curriculum (modules/units). The
lectures, seminars, workshops or laboratories support the inquiry process rather than transmitting
subject-based knowledge. Whether it is a module or a whole programme that is being designed, the
starting point should be a set of problem scenarios that enable students to become independent
inquirers and help them to see learning and knowledge as flexible entities. To date there has been
little in-depth discussion about the design of problem-based curricula. Instead the discussions have
tended to centre on what counts as problem-based learning, ways of implementing it and types of
problem-based learning.
3.2 Problem-based learning online
Problem-based learning online is defined here as students working in teams of four to six on a series
of problem scenarios that combine to make up a module or unit that may then form a programme.
Students are expected to work collaboratively to solve or manage the problem. Students will work in
real-time or asynchronously, but what is important is that they work together. Synchronous
collaboration tools are vital for the effective use of PBLonline because tools such as Chat, Shared
Whiteboards, Video conferencing and Group browsing are central to ensuring collaboration within the
problem-based learning team. Students may be working at a distance or on campus, but they will
begin by working out what they need to learn to engage with the problem situation. This may take
place through a shared whiteboard, conferring or an email discussion group. What is also important is
that students have both access to the objectives of the module and also the ability to negotiate their
own learning needs in the context of the given outcomes. Facilitation occurs through the tutor having
access to the ongoing discussions without necessarily participating in them. Tutors also plan real-time
sessions with the PBLonline team in order to engage with the discussion and facilitate the learning.
For students the shift to new forms of learning, different from the more traditional didactic approaches
they have experienced in school and further education, is often challenging. PBLonline introduces
students to two new elements of learning. This has an impact not only on the problem-based learning
and online learning but also on other forms of learning within the curriculum. There are few curricula
where problem-based learning is used as the only approach to learning, and increasingly students
have to manage not only the interplay of knowledge across modules but also different approaches to
learning. However, there are also issues about the reasons for using PBLonline in the first place. For
example, it is questionable as to whether there is value in using real-time PBLonline for students
undertaking the same programme at the same university, unless it is used because of long distances
between campus sites where students are using the same problem-based learning scenario. There
also needs to be questions asked about whether having asynchronous teams adds something
747

Maggi Savin-Baden et al.
different to PBLonline. Certainly, in distance education, across time zones and campus sites, this
would be useful and suit different students' lives and working practices. Yet this raises problems about
how cooperative and collaborative it is possible to be, in terms of sharing learning and ideas and
developing forms of learning that are genuinely dialogic in nature.
3.3 Problem-based learning in immersive virtual worlds
Learning in immersive virtual worlds (simulations and virtual worlds such as Second Life) could
become a central learning approach in many curricula, but the socio political impact of virtual world
learning on higher education remains under-researched. Much of the recent research into learning in
immersive virtual worlds centres around games and gaming and is largely underpinned by cognitive
learning theories that focus on linearity, problem-solving and the importance of attaining the ‘right
answer’ or game plan. Most research to date has been undertaken into students' experiences of
virtual learning environments, discussion forums and perspectives about what and how online
learning has been implemented. Although PBLonline combines problem-based and online learning, in
doing so it is recognised that students learn collaboratively through web-based materials including
text, simulations, videos and demonstrations. Resources such as chat rooms, message boards and
environments have been purpose-built for PBL; both synchronously and asynchronously, on campus
or at a distance. Practising skills within a virtual environment online offers advantages over learning
through real-life practice, in particular the exposure of learners to a wide range of scenarios (more
than they are likely to meet in a standard face-to-face programme) at a time and pace convenient to
the learner, together with consistent feedback. It offers learners the chance to make mistakes without
real-world repercussions. One such example is the PREVIEW project (Savin-Baden et al, 2008). This
project investigated, implemented and evaluated a user-focused approach to developing scenarios
and materials, linking the emerging technologies of virtual worlds with interactive PBL online, to create
immersive collaborative tutorials.
4. Objectives
This project investigated, implemented and evaluated a user-focused approach to developing
scenarios and materials, linking the emerging technologies of virtual worlds with interactive PBL
online, to create immersive collaborative tutorials. The project team, led by Coventry University and its
partner St George’s University of London, implemented and evaluated a user-focused approach to
developing problem-based learning environments and ‘good practice’ materials. This was achieved by
linking the emerging technologies of virtual worlds with interactive PBL online to create immersive,
collaborative tutorials in the virtual world Second Life (SL), which allows distance learners from the
geographically distant institutions to meet ‘in-world’ and collaborate around a case. This environment
differs radically from the VLE in that it draws on a primarily visual set of semiotic resources with each
participant having an online presence, or avatar, to aid their communication. The aims of the
PREVIEW project were to:
Deliver problem-based learning in Second Life,
Develop eight interactive PBL scenarios,
Guide development and evaluation alongside users,
Develop guidelines and best-practice for delivering PBL in immersive worlds
Share outputs and technology.
A variety of problem-based learning scenarios were developed within SL for distance-learning
students at the two institutions. The project was introduced to the part-time distance online MA in
Health and Social Care Management at Coventry University. The project was also implemented on
the second year of the three-year blended learning Paramedic Foundation Degree at St George’s
Universty of London. The PBL scenarios were categorised in two ways: information-driven scenarios,
and avatar-driven scenarios. Information-driven scenarios focus on internal virtual world content, such
as media technologies (video footage, images and audio) and objects within the virtual environment
that provide the user with written or spoken information. The scenarios also demonstrate external
content, such as web pages relevant to the situation.
Avatar-driven scenarios use non-player characters (NPCs) in two forms: as ‘chat bots’, where the
student interacts with the NPC to gather necessary information, or as avatars featured in Machinima
(3D videos filmed within a virtual environment) such as a pre-recorded discussion, play or critical
incident.
748

Maggi Savin-Baden et al.
The role of the students, as a collaborative exercise, is to gather as much information about the
situation and the disease as possible using a variety of information-driven methods before moving on
to an avatar-driven method. The students are required to interact with a ‘chat bot’ to distinguish what
their next actions should be.
5. Methods
The evaluation encompassed formative elements to inform the project team and summative elements
to establish the worth of what was achieved. The objectives of the evaluation were to:
Explore the impact of problem-based learning scenarios in 3D virtual worlds on learning
Assess the usability of the learning environments and user acceptance
Evaluate the effectiveness of feedback mechanisms and guidance materials
Offer an analytic account of the experience of the project from the perspective of all the key
stakeholders
Be responsive and flexible enough to capture unintended outcomes and unanticipated effects
Provide an overall summary of the project, highlighting strengths, weaknesses and areas of
development
Inform current and future developments, paying particular attention to their structures,
procedures, working practices, relationships and practices.
The evaluation was planned at the start of the project, informed by JISC guidelines (Glenaffric, 2007)
and illuminative evaluation, which is argued to take account of wider contexts than more traditional
evaluation and, is primarily concerned with description and interpretation rather than measurement
and prediction (Parlett and Dearden, 1977). Data were collected and analysed from three main
sources: internal and external project documentation; interviews with key respondents (project leader,
three subject matter experts who designed and facilitated scenarios, three learning technologists and
the technical developer) and finally evaluation activities involving students. Data were analysed
interpretively to examine the subtext of data and identify themes and patterns of response in relation
to the areas of enquiry. Findings were transformed into developmental models and practice materials.
New learning spaces and emerging technologies such as wikis and podcasts offer new possibilities in
terms of communication in distance learning, but also present limitations and barriers in terms of the
presentation of the self, meaningful synchronous interaction, and team-building. For these reasons,
caution must be exercised when making claims for their equivalence to the communicative modalities
of the face-to-face setting. When seeking to implement PBLonline, purpose-built educational virtual
learning environments (VLEs) such as Blackboard may also be limited and limiting. These digital
spaces (VLEs) have prompted concerns about both containment and exteriorisation in online
environments (Land, 2006) - containment is particularly evident in VLEs, inherent in their structuring
and management of learning. Whilst many learning technologists (for example, Beetham and Sharpe,
2007) have sought to account for academics’ reported difficulties with using VLEs by citing lack of
expertise or innovation, academics may in fact feel inhibited by a sense that the technology constrains
rather than enables pedagogy, leading to a situation in which creativity is limited by a misguided quest
for linearity and maintenance of control, prompted by the structuring force of the environment itself.
Lyotard highlights the power-freighted and restrictive potential of the digital environment when he
points out that “. . . knowledge and power are simply two sides of the same questions: who decides
what is knowledge, and who knows what needs to be decided? In the computer age, the question of
knowledge is now more than ever a question of government” (Lyotard, 1979).
An iterative process was used when implementing and evaluating the PBL scenarios. At several
stages throughout the project, testing of each scenario was undertaken, and the feedback from the
students’ experiences was analysed to improve on the scenarios. In practice testing took place over a
over a period of 9 months using the two categories of PBL scenarios (avatar driven and information
driven). Student evaluation data was collected from activities known as ‘Testing days’. The paramedic
scenarios were evaluated on three occasions during June and November 2008. In June participants
who trailed the scenarios were first year paramedic students (n=10) familiar with PBL and the
university virtual learning environment, Blackboard, but not Second Life. The first retesting, which was
carried out in a computer lab over 3 days in November involved four of the original ten participants
and one new student. A further test in November used a different opportunity sample of ten mixed first
749

Maggi Savin-Baden et al.
and second year paramedic students. The structure of tests consisted of SL orientation (1hr),
demonstration scenario (1hr) followed by group rotation around different scenarios (approximately 1hr
per scenario). This was followed by a paper questionnaire and focus group (1hr). Scenarios were
facilitated with groups of 2-4 students.
The health care management scenarios were tested on two occasions with 12 different volunteers
drawn from health-related professions, (not students on the target course) and almost all had no
experience of PBL. None had prior experience of SL. These activities generated considerable
volumes of data comprising in-world chat logs, video screen capture; video footage of the students
interacting with scenarios, post testing focus group responses (video-recorded) and questionnaires.
The scenarios were then reviewed further alongside students to ensure the feedback had been
beneficial to the project.
6. Findings
The findings in many ways were more positive than initially anticipated, but there were also a number
of challenges. The 4 themes that emerged from the data were Technological challenges,
Pedagogical design, Usability and Avatar identity, and Collaboration and Interactio.
6.1 Technological challenges
Feedback suggested that the information-driven scenarios did not work as well as those that were
avatar-driven. The scenarios were restructured slightly to compensate for the students’ comments
that they did not feel as immersed into the environment with information-driven scenarios. The
decision was made to design all the health care scenarios as avatar-driven to provide for a truly
immersive and realistic experience.
It was anticipated that the technological demands and initial lack of user friendliness of SL would be a
barrier to participation. Therefore as a precaution, for distance learners who may not have had the
technical capabilities to run SL, a ‘web application’ was developed. In fact the technology also had a
strong influence on the pedagogical model, as two tutors explained:
SL lends itself to individuals or pairs consolidation or decision making exercise …. . like
to see it as a standalone exercise without facilitation
I don’t feel it lends itself very well to a group (3-4)…– quite high boredom factor for those
not directly participating with NPC, … they were checking email, adjusting appearance –
so from facilitators point of view it is a good decision making exercise but not for what we
understand as traditional PBL session
Thus, the outcomes identified by students were not particular PBL learning issues, but other issues
such as poor group work or a team member not contributing significantly.
6.2 Pedagogical design
When the PREVIEW project underwent testing by staff and students, few access barriers were
reported, although this may become more of an issue with wider implementation of this approach.
However, students who were beginners to the SL environment needed more time than anticipated to
explore and experiment with the virtual world, and familiarise themselves with the new environment;
mock scenarios became an important strategy in this process.
“This is my greatest concern. In order to get the students close to a point where clinical
reasoning / learning is both valuable and the prominent area of concern. It seems to take
a large amount of effort to overcome the heavy interface of Second Life.”
“[Second Life] lends itself very strongly to creating a rich and valuable decision making
exercise.”
This suggests that a degree of initial strangeness and discomfort may have been experienced by the
participants, which is significant when considering that they would need a tolerable degree of
conformity with the visual /kinetic /semiotic resources of the world and their avatar identity, before
they could devote meaningful attention to group collaboration around a problem. One of the difficulties
with using problem-based learning, designing interaction learning in virtual worlds and developing
simulations is the ability to design and build effective complex and challenging scenarios. There is a
tendency to focus on knowledge and content coverage, rather than the way learning will be managed
750

Maggi Savin-Baden et al.
and the complexity of the problem scenarios. Schmidt and Moust (2000) suggested a taxonomy for
using problems in order to acquire different kinds of knowledge, rather than solving problems or
covering subject matter. The importance of the work undertaken by Schmidt and Moust (2000) is not
only the way they provide and explicate different problem types, but also their exploration of the way
in which the questions asked of students guide the types of knowledge in which students engage.
6.3 Usability and avatar identity
Designing learning in higher education has often focused on covering content and ensuring that
discipline-based pedagogies are adhered to. What these data appear to indicate is that the
experience of learning with and through an avatar differs between people, and invariably relates to
identity transitions and transformations in immersive virtual worlds. Students remarked:
“I got distracted when my avatar was sitting on the cupboard instead of what I wanted
‘her’ to do.”
“It does distract you when your avatar gets in the way. Just as I wanted to pick up
information she started flying and I got confused and it interrupted the experience since I
had to deal with the tech.”
I have to say that I wasn’t really paying a tremendous amount of attention to my avatar
in the actual scenario. I sat her down and then angled the camera so I couldn’t even see
her. I think I was in role – I tried to think like someone in that situation. I was just thinking
like that as ‘me’, not as my avatar.”
The sense of doing things differently, playing with learning, playing around and exploring were all
seen as advantages to problem-based learning in Second Life . Yet these advantages were often
seen by staff as troublesome in the sense that the learning boundaries were not necessary controlled
and managed by them, but by the students. Yet for students it was the opportunity to play, which
challenged the immutability of knowledge and the perception that learning was static and tutor
centred. Yet such liquidity in the learning also brought with it a sense of unease about the
provisionality of learning and identity in such spaces.
6.4 Collaboration and interaction
Preliminary results from the project indicate that SL held a great deal of potential for the development
and extension of PBL. Students seemed able to use their avatars to communicate, collaborate and
problem solve effectively.
I liked it! It’s more entertaining certainly! More fun. But I’m not sure that we’d have gotten
different results if we sat around a table with a bunch of papers chatting.
I liked the team collaboration aspect to it...I think it’s a different way of working out
solutions to problems. I liked it and it was fun!
The level of realism and immersion of the scenarios seemed to be enhanced by the virtual world
environment, including the option to use voice in addition to text-based communication, and students
reported that it felt like a more ‘authentic’ learning environment than PBL based in VLEs. Students
responded enthusiastically to the environment, interestingly tending to initially treat it as a ‘game’. This
(common) association of the look and feel of SL with online gaming may arguably be a limitation in
the educational setting - in that it could encourage individualism rather than collaboration, and may
simplify scenarios in which more nuanced critical engagement is required and no one clear solution is
available. However, it is likely to also be an advantage in that it may increase student enjoyment and
motivation via memorably novel forms of participation.
7. Discussion
This project developed an innovative approach to address problems faced by courses which wish to
use collaborative scenario-based learning as a tool for the learning of competency, but are restricted
in their opportunities for face to face learning. The approach took advantage of the new opportunities
offered by immersive virtual worlds which provide the authenticity of a simulated real-world
environment, and the open-ended nature of in-world activity. This may not be the first time that an
attempt has been made to develop immersive scenarios, however we believe this may be the first use
of PBL in immersive worlds in this way. Furthermore we believe this work goes some way to engaging
with the taxonomy suggested by Schmidt and Moust (2000). A particular strength of Second Life as a
learning environment is that it provides a variety of communication tools which are particularly
751

Maggi Savin-Baden et al.
important for PBL. Furthermore to date problem-based learning has been seen as a relatively stable
approach to learning, delineated by particular characteristics. Using PBL in Second Life embraces
issues connected with complex curriculum design and the need for complex PBL scenarios to be
developed. All the planned scenarios were delivered, and significant changes were made during
development to take most advantage of Second Life’s strengths. Students appreciated the value of
Second Life as a collaborative environment, but also viewed such practice-based simulations as
valuable for individual work. An interesting consequence of the richness and authenticity of the
Second Life scenarios is the large amount of detail provided, much more than is usual in paper-based
face-to face PBL sessions. Second Life can provide a more authentic learner environment than
classroom based PBL and therefore changes the dynamic of facilitation, but at this stage it is not clear
how this impacts on the way the scenario is used and facilitated. Savin-Baden and Wilkie (2006, p xxi)
points out that facilitation of PBL is itself a source of concern for many teachers and that there are
differences and tensions to be resolved between online and face to face facilitation.
However, there were also technical considerations, such as the relatively high specification
computers/high bandwidth which are required, and the interface is not as intuitive as might be hoped.
Interface complexity can provide memory overload. Furthermore, it is essential to prepare users
through structured, context-related orientation sessions prior to use as a learning tool. Yet the userguided
development process adopted by PREVIEW, involving the whole multi-disciplinary
development team and students from the target course worked effectively in highlighting strengths
and weaknesses in many aspects of the scenarios.
Given the success of PREVIEW as a demonstrator, it is essential to build on these results to promote
the embedding of scenarios in courses in terms of:
Further development and research to develop models and understanding of good practice
in areas such as scenario design in Second Life/ MUVEs,
Exploration of technology reuse and repurposing,
Locating mechanisms to improve usability
The development of PBL facilitation practices for Second Life/ MUVEs
Key issues for effective PBL:
What it means to learn in SL (Minocha & Reeves, 2010; Kemp & Haycock, 2008)
Student preparation
Usability and access issues
Collaboration & interaction
Individual and collective identity work (Warburton, 2009)
Pedagogical design
Authenticity of the environment
Facilitation (Savin-Baden, 2006)
What are students learning? (Boardman, 2009)
8. Conclusion
This project has a user-centred approach and has provided a strong pedagogical underpinning to the
use of virtual worlds in higher education. Developing open source pedagogically driven PBL scenarios
such as these may offer a new liquidity to learning, combining technology with pedagogy in ways that
are mutually beneficial not only in distance education, but also as a means to enrich the face-to-face
learning environment. However, these environments must be examined not only in terms of the new
freedoms they may afford, but also in recognition of their intermittently strange and ‘troubling’ nature,
which may in itself provide potential for creativity (Bayne, 2006). Such a vision however, will require
that we stop seeing the curriculum as a predictable, ordered and manageable space, but instead review
it as an important site of transformation characterised by risk and uncertainty.
References
Barrows, H.S. & Tamblyn, R.M. (1980). Problem-based Learning, an approach to Medical Education. (New York,
Springer).
752

Maggi Savin-Baden et al.
Bayne, S. (2006) Temptation, trash and trust: the authorship and authority of digital texts. E-Learning, 3(1) 16-26.
Beetham, H. & Sharpe, R. (eds) (2007) Rethinking Pedagogy for the digital age: designing and delivering elearning.
(London, Routledge).
Beaumont, C. Savin-Baden, M., Conradi, E., Poulton, T. (2011) Evaluating a Second Life PBL demonstrator
project: what can we learn? Interactive Learning Environments 20 (in press)
Boardman, K. (2009) Dreams into [virtual] reality Paper presented at ALT-C In dreams begins responsibility" -
choice, evidence, and change Manchester, UK, 8-10 September
Conradi, E., Kavia, S., Burden, D., Rice, D., Woodham, L., Beaumont , C., Savin-Baden, M. and Poulton., T.
(2009) Virtual patients in Virtual World: Training paramedic students. Medical Teacher 31 (8) 713 – 720
Glenaffric Ltd (2007). Six Steps to Effective Evaluation: A handbook for programme and project managers.
Retrieved from
http://www.jisc.ac.uk/media/documents/programmes/elearningcapital/evaluationhandbook.pdf
Kemp, J. W. & Haycock, K. (2008). Immersive learning environments in parallel universes: Learning through
Second Life. School Libraries Worldwide, 14(2), 89-97. Retrieved from
http://asselindoiron.pbwiki.com/f/RevisedKEMPhaycock.Sept08.pdf
Land, R. (2006) Paradigms Lost: academic practice and exteriorising technologies E-Learning, 3, (1) 100-110.
Lyotard, J.-F. (1979) The postmodern condition: a report on knowledge. (Manchester, Manchester University
Press).
Minocha, S. and Reeves, A.J. (2010). Interaction Design and Usability of Learning Spaces in 3D Multi-user
Virtual Worlds in Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts,
Katre, D.; Orngreen, R.; Yammiyavar, P.; Clemmensen, T. (Eds.), ISBN: 978-3-642-11761-9, Springer, pp.
157-167.
Parlett, M and Dearden, G. (eds.) (1977) Introduction to Illuminative Evaluation: Studies in Higher Education.
California, Pacific Soundings Press.
Poulton,T. Conradi, E. Kavia, S. Round,J. & Sean Hilton (2009) The replacement of ‘paper’ cases by interactive
online virtual patients in problem-based learning Medical Teacher 31 (8) 752–758.
Round,J., .Conradi, E. & Poulton, T. (2009a) Improving assessment with virtual patients Medical Teacher 31 (8)
759–763.
Round,J., .Conradi, E. and Poulton , T. 2009b) Training staff to create simple interactive virtual patients: the
impact on a medical and healthca Medical Teacher 31 (8) 764–769.
Savin-Baden, M. (2000) Problem-based Learning in Higher Education: Untold Stories, SRHE/Open University
Press, Buckingham.
Savin-Baden, M. (2006) The challenge of using problem-based learning online, in M. Savin-Baden and K. Wilkie
(eds) Problem-based Learning Online. Maidenhead: McGraw Hill.
Savin-Baden, M (2007) A Practical Guide to Problem-based Learning Online. (London, Routledge).
Savin-Baden, M and Wilkie, K. Introduction (2006) in M. Savin-Baden, & K. Wilkie (eds). Problem-based Learning
Online (Maidenhead: Open University Press)
Savin-Baden, M. Beaumont, C., Poulton, T and Conradi, E. (2008) Students’ experiences of learning in
immersive world environments ReLIVE Conference, Open University 22-23 rd November
Schmidt, H.G. & Moust, J. (2000) Towards a taxonomy of problems used in problem-based learning curricula,
Journal on Excellence in College Teaching, 11(2/3): 57-72.
Warburton S. (2009). Second Life in higher education: Assessing the potential for and the barriers to deploying
virtual worlds in learning and teaching, British Journal of Educational Technology, 40 (3) 414–426.
753

The Evolution of eLearning Platform TESYS User
Preferences During the Training Processes
Adriana Schiopoiu Burlea 1 , Amelia Badica 2 and Carmen Radu 2
1
Management Marketing Department, University of Craiova, Romania
2
Statistics and Informatics Department, University of Craiova, Romania
adriana_burlea@yahoo.com
ameliabd@yahoo.com
carmenrozana@yahoo.com
Abstract: The use of eLearning platform TESYS during continuous training programmes started in 2005, when
the first training project PHARE was initiated. Therefore, a study conducted in 2005-2007 highlighted that the
requirements of platform TESYS users were influenced by their personal characteristics rather than the
characteristics of the eLearning platform. In the period 2007-2010, eLearning platform TESYS has been
continuously improved in order to meet both the needs of the user and the demands of the teachers. Taking into
account both, user requirements and technological developments, the eLearning platform TESYS has been
improved and, consequently, our study aims to analyze how users perceive technological improvements of the
platform during a new training process conducted in 2009-2011. In the course of the research, we followed the
correlation between the experienced users and the first-time learners on eLearning platform TESYS. Therefore,
our quantitative and qualitative research reveals that users’ expectations and needs referring to the eLearning
platform TESYS are inhomogeneous and unpredictable. The relationship between learners-professors is very
important in the knowledge management process. The course structure is a very important qualitative variable
that contributes to the satisfaction of the learners and the quality of the eLearning platform Tesys cannot supply
the content of the chapters. The practical implications of our research are important both for the users of the
eLearning system that need different levels of competences in using eLearning platform, and for the technical
team which deals with the management of the eLearning platforms in order to understand the needs of the
learners and to adapt the technological characteristics of the platform to these requirements to obtain high
results. The study indicates that the role of the human factor in the eLearning process is prominent and users
perceive any improvement of the eLearning platform as necessary and even mandatory for a proper functioning.
This article will represent the framework for our future research, its goal being the study of the relation between
the satisfaction of the eLearning platform TESYS users and the results obtained in exams by the students that
answered our questionnaires. Based on the grades we will determine whether there is a direct correlation
between the results obtained in the exams, the content of the courses and the usefulness of the eLearning
platform.
Keywords: eLearning platform TESYS, knowledge, users satisfaction, usability
1. Introduction
The eLearning education system increasingly gains more ground against classical education and it is
considered as one of the systems that use the most expensive technology tools (Salinas, 2008). The
advantages and disadvantages of this system were and are studied by a number of scholars, but
these approaches have a unilateral character:
Some approaches watch the technical features of the eLearning platforms, without considering
the user satisfaction (Mahdizadeh et al. 2008), and
Others consider mainly the human factor, decreasing the importance of the eLearning system
technical characteristics as a whole (Burlea Schiopoiu, 2008; Nakayama et al. 2009).
Arguments for and against unilateral approaches are supported on the one hand by the large number
of eLearning platforms on the market (Moodle, Blackboard, eCollege, DataTel, Web Advisor, Tesys,
Caroline, ClickClass, Clivir, DazzlerMax, Docebo, e-ducativa Virtual Campus, eLeaP LMS, eStudy,
IBT Server, Interact), and on the other hand, by the increasingly homogeneous preferences of the
eLearning system users, both students and teachers (Bolliger & Wasilik, 2009; Craig et al. 2008).
Many studies reporting investigations of student perspectives on using eLearning platform (McGill &
Hobbs, 2008; Nakayama & Yamamoto, 2011) were conducted, but the number of studies
investigating the perspectives of teachers remains limited in number (Jones & Jones, 2005; Palmer &
Holt, 2009).
754

Adriana Schiopoiu Burlea et al.
Our article is a second part of the research conducted in 2005-2007, which aimed at evaluating how
the eLearning platform TESYS contributes to learning development activities (Burlea Schiopoiu,
2008).
During 2005-2007, 722 students enrolled at the courses in eLearning system and only 412 students
completed.
The high rate of abandonment of the courses was a motivating factor for our research, aiming at that
time, to detect whether this drop was due to problems related to the use of eLearning platform or to
problems given by human factor.
In 2008 (Burlea Schiopoiu, 2008, pp. 273-274), the research concluded that platform Tesys should
have certain characteristics:
It has to be rapid - on one hand, to prevent the user from abandoning the eLearning activity in
case the computer does not respond very quickly and, on the other hand, in order not to make out
of the lack of free time for studying the main reason of abandon,
It has to be simple – so that the user, irrespective of the level of knowledge in the informatics
field, should be able to manage the system and stimulate the growth of the user’s responsibility
for his or her own learning in the release of personal potential,
To allow the development of the users’ creativity – by a strategic use of the connection
between personal experience and technological performance of the Tesys platform, and
To have a high motivational level in order to attract people of the age categories between 21-35
years who had abandoned the course in a high proportion – 74.19 per cent of the total number of
people who abandoned the course.
2. The improvement of the TESYS eLearning platform
The eLearning platform TESYS was funded in 2005 by the European Union project named “Training
program via eLearning in the fields of economics and informatics for developing the labour force from
the Oltenia region” (Burdescu et al., 2006, p. 315, Burlea Schiopoiu, 2008, p. 271).
During 2005-2007, the eLearning platform TESYS had the following structure: three sections, each of
them consisting of four disciplines; each discipline had a professor as a responsible; there were
almost 2500 questions for all disciplines available on the platform, containing the following elements:
Support applications for the communication between students and their professors and for online
and off-line training;
Applications for student evaluation at exams, for creating, updating and using databases with
questions and problems and for statistical analysis of the candidates’ results and of the test
construction (difficulty, structure).
During 2007-2009, the eLearning platform Tesys has undergone continuous improvements, taking
into account both the recommendations of students and teachers who have used this platform as well
as the evolution of eLearning systems worldwide. Thus, Burdescu et al., 2010 (p. 37) mentioned an
eLearning process improvement that “is designed to run at chapter level. This means a discipline
needs to be partitioned into chapters. The chapter needs to be assigned a concept map, which may
consist of about 20 concepts. Each concept is assigned to a document and a set of quiz questions.
Each concept and each quiz has a weight, depending on its importance in the hierarchy. It is
supposed that the platform runs for some time such that there is a history of performed activities for a
large number of learners”.
Along with the technological improvements of the e-leaning platform TESYS, a three-dimensional
correlation between the initial knowledge level of the student: time constraints, technical and cultural
background of the student were also accomplished.
The relationship between the technical characteristics of the eLearning platform and the personal
characteristics of the users has been identified as the most important determinant for students to
share knowledge through Tesys. Based on this, our hypotheses are:
Hypothesis 1: The usability of the eLearning platform TESYS is positively related to user level
(beginner or advanced)
755

Adriana Schiopoiu Burlea et al.
Hypothesis 2: Course quality on eLearning platform TESYS is positively related to user satisfaction
Hypothesis 3: Course quality on eLearning platform TESYS is positively related to exam
performances
Hypothesis 4: The knowledge acquired by students are positively related to the usability of the
eLearning platform TESYS
3. Main explanatory variables
Dependent variable. The respondents had to answer a question asking whether they are used for
the first time an eLearning platform to acquire and evaluate knowledge. The dependent variable
consists of a dummy variable, which indicates if the respondent is a new or an advanced eLearning
platform user. The data showed that 232 respondents (63.2%) of the students were using an
eLearning platform for the first time. The hypotheses predict that some aspects of the eLearning
platform influence the student’s decision to start using Tesys.
To test the hypotheses adequately we used two subgroups of the original sample: beginner and
advanced users (see table 3 and table 4). According to the hypotheses, the specified conditions
should have an impact on the student’s decision whether to make use of Tesys or not. The reported
significance levels are based on one-tailed tests.
Independent variables. The respondents had to answer the following questions (five-point Likert
scales) which were grouped by four independent variables:
1. The usability of the eLearning platform TESYS (UFT)
UFT1: Did you learn easily how to use the eLearning platform TESYS?
UFT2: How do you evaluate the access to courses through eLearning platform TESYS?
2. The eLearning platform user satisfaction (US)
US1: Do you think that student satisfaction in the process of acquiring new knowledge was an
important goal for the team that designed and implemented the eLearning platform TESYS?
US2: Have any of your recommendations been considered by the project team so far?
3. Course quality on eLearning platform Tesys (TQ)
TQ1: Were your expectations regarding course curricula developed through eLearning platform
TESYS met?
TQ2: How do you evaluate the eLearning platform TESYS course contents?
4. Knowledge management (acquiring and sharing knowledge; knowledge evaluation) – (KM)
KM1: Is the evaluation system through eLearning platform TESYS good enough?
KM2: How do you rate the communication between you and teachers in the eLearning system?
KM3: Do you consider that the acquisition of new knowledge through eLearning platform TESYS was
beneficial for you?
KM4: Is there a correlation between exam performances and the reliability of eLearning platform
TESYS?
KM5: Did you manage to capitalize the knowledge acquired through eLearning platform TESYS?
756

4. Methodology
Adriana Schiopoiu Burlea et al.
This study tests the four hypotheses using data obtained with the help of an email questionnaire sent
to a multi-stage random sample of EDU-Antreprenor students. The data were collected from January
2011 until April 2011.
A total of 367 out of 458 students returned a usable questionnaire (~80.13%).
As required by research ethics procedures, the surveys were voluntary and students were given the
opportunity to remain anonymous or to mention their identity.
The processing of the questionnaires was done with SPSS 16.0.
The respondents were asked to indicate their opinion related to the use of the eLearning platform,
using a five-point Likert scale (ranging from 1 = strongly disagree to 5 = strongly agree). A rating of 1
represented low satisfaction, while a rating of 5 represented high satisfaction.
Of the 458 questionnaires distributed, 367 completed and usable questionnaires were returned,
representing a response rate of 80.1%.
Table 1 lists a summary of the response rate and demographic information for the respondents in
2009 and 2010. The effective response rate was 80.14% in 2009, and 80.50% in 2010. A range of
demographic information was available for the overall respondents, as well as collected part of the
survey, including gender and age range. This permitted a comparison between the respondent
sample and the overall staff population based on these demographic dimensions, as presented in
Table 1.
Table1: Basic characteristics of respondents (n = 367)
Basic characteristics Frequency Percentage
2009 2010 2009 2010
Gender
Male 64 61 37 31
Female 109 133 63 69
Total 173 194 100 100
Age range
20-24 45 51 26 26
25-29 58 41 34 21
30-34 31 32 18 16
35-39 19 30 11 15
40-44 12 26 7 14
45-49 6 9 3 5
50 + years 2 5 1 3
Total 173 194 100 100
Table 2 provides a summary of the response rate of beginner and advanced students. This permitted
a comparison between the two student categories.
Table 2: Response rates
User level Response rate (%)
Beginners - 135 respondents 79.88%
Advanced – 232 respondents 86.57 %
Although the obtained response rates were comparatively high, they were not unexpected for an
online voluntary survey and the generally good match between the sample and population
demographic characteristics and the relatively large absolute numbers of respondents, for both
beginner and advanced students in 2009 and in 2010. It suggests that we can have some confidence
in drawing more general inferences from the respondent data.
757

Adriana Schiopoiu Burlea et al.
Table 3 provides a summary of the mean responses and standard deviation for advanced students
from 2009 and 2010. Based on a t-test of differences in mean ratings from advanced students
between 2009 and 2010, Table 3 indicates where the corresponding ratings were significantly
different between 2009 and 2010 and provides an indication of the level of significance.
Table 3: Summary of measurement scales for advanced users
Items
TOTAL USERS Advanced users
(n=232)
Mean Std. Mean Std.
Deviation
Deviation
A. ELearning platform User Level: beginner and advanced – UL
UL1. Is this the first time you use an eLearning platform to
acquire and evaluate knowledge?
1,37 ,483
- -
B. The usability of the eLearning platform TESYS - UFT Cronbach's Alpha = Cronbach's Alpha =
,678
,518
UFT1. Did you learn easily how to use the eLearning
platform TESYS?
4,34 ,747 4,66 ,511
UFT2. How do you evaluate the access to courses through
eLearning platform TESYS?
4,50 ,548 4,69 ,471
C. The eLearning platform user satisfaction - US Cronbach's Alpha = Cronbach's Alpha =
US1. Do you think that student satisfaction in the process of
,878
,851
acquiring new knowledge was an important goal for the
team that designed and implemented the eLearning platform
TESYS?
4,50 ,548 4,67 ,499
US2. Have any of your recommendations been considered
by the project team so far?
4,48 ,581 4,66 ,535
D. Course quality on TESYS eLearning platform -TQ Cronbach's Alpha = Cronbach's Alpha =
,787
,741
TQ1. Were your expectations regarding course curricula
developed through eLearning platform TESYS met?
4,47 ,571 4,65 ,522
TQ2. How do you evaluate the eLearning platform TESYS
course contents?
4,43 ,549 4,58 ,503
E. Knowledge management (acquiring and sharing Cronbach's Alpha = Cronbach's Alpha =
knowledge; knowledge evaluation) - KM
,784
,708
KM1. Is the evaluation system through eLearning platform
TESYS good enough?
4,48 ,562 4,66 ,510
KM2. How do you rate the communication between you and
teachers in the eLearning system?
4,00 ,743 4,19 ,695
KM3. Do you consider that the acquisition of new knowledge
through eLearning platform TESYS was beneficial for you?
4,46 ,566 4,62 ,521
KM4. Is there a correlation between exam performances and
the reliability of TESYS eLearning platform?
4,06 ,746 4,19 ,728
KM5. Did you manage to capitalize the knowledge acquired
through TESYS eLearning platform?
4,42 ,617 4,55 ,615
Table 4 provides a summary of the mean responses and standard deviation for beginner students in
2009 and 2010. Based on a t-test of differences in mean ratings from beginner students between
2009 and 2010, Table 4 indicates where the corresponding ratings were significantly different
between 2009 and 2010 and provides an indication of the level of significance.
Table 4: Summary of measurement scales for beginner users
Items
TOTAL USERS Beginner users
(n=135)
Mean Std. Mean Std.
Deviation
Deviation
A. ELearning platform User Level: beginner and advanced - UL
UL1. Is this the first time you use an eLearning platform to
acquire and evaluate knowledge?
1,37 ,483
- -
B. The usability of the eLearning platform TESYS - UFT Cronbach's Alpha = Cronbach's Alpha
,678
=,568
UFT1. Did you learn easily how to use the eLearning
platform TESYS?
4,34 ,747 3,79 ,774
UFT2. How do you evaluate the access to courses through
eLearning platform TESYS?
4,50 ,548 4,18 ,516
758

Adriana Schiopoiu Burlea et al.
C. The eLearning platform user satisfaction - US Cronbach's Alpha =
,878
US1. Do you think that student satisfaction in the process of
acquiring new knowledge was an important goal for the
team that designed and implemented eLearning platform
TESYS?
Items TOTAL
USERS
Cronbach's Alpha =
,845
4,50 ,548 4,20 ,501
Beginner
users
(n=135)
Items TOTAL
USERS
D. Course quality on eLearning platform TESYS -TQ Cronbach's Alpha = Cronbach's Alpha =
,787
,747
TQ1. Were your expectations regarding course curricula
developed through eLearning platform TESYS met?
4,47 ,571 4,16 ,521
TQ2. How do you evaluate the eLearning platform TESYS
course contents?
4,43 ,549 4,18 ,531
E. Knowledge management (acquiring and sharing Cronbach's Alpha = Cronbach's Alpha =
knowledge; knowledge evaluation) - KM
,784
,777
KM1. Is the evaluation system through eLearning platform
TESYS good enough?
4,48 ,562 4,16 ,507
KM2. How do you rate the communication between you and
teachers in the eLearning system?
KM3. Do you consider that the acquisition of new
4,00 ,743 3,67 ,711
knowledge through eLearning platform TESYS was
beneficial for you?
4,46 ,566 4,19 ,539
KM4. Is there a correlation between exam performances
and the reliability of TESYS eLearning platform?
4,06 ,746 3,84 ,725
KM5. Did you manage to capitalize the knowledge acquired
through TESYS eLearning platform?
4,42 ,617 4,20 ,557
The table 5 shows the Pearson Correlations between variables.
Table 5: Pearson Correlation between variables
UL1 UFT1 US1 US2 UFT2 KM1 TQ1 KM2 TQ2 KM3 KM4 KM5
UL1 1,000 -,558 -,413 -,413 -,455 -,427 -,409 -,335 -,356 -,365 -,231 -,272
UFT1 -,558 1,000 ,538 ,490 ,538 ,468 ,428 ,335 ,369 ,353 ,202 ,243
US1 -,413 ,538 1,000 ,783 ,704 ,739 ,653 ,416 ,565 ,526 ,278 ,434
US2 -,413 ,490 ,783 1,000 ,745 ,683 ,700 ,361 ,550 ,540 ,271 ,477
UFT2 -,455 ,538 ,704 ,745 1,000 ,815 ,728 ,336 ,590 ,540 ,270 ,399
KM1 -,427 ,468 ,739 ,683 ,815 1,000 ,783 ,419 ,658 ,584 ,281 ,485
TQ1 -,409 ,428 ,653 ,700 ,728 ,783 1,000 ,425 ,650 ,544 ,258 ,472
KM2 -,335 ,335 ,416 ,361 ,336 ,419 ,425 1,000 ,436 ,357 ,325 ,274
TQ2 -,356 ,369 ,565 ,550 ,590 ,658 ,650 ,436 1,000 ,804 ,394 ,551
KM3 -,365 ,353 ,526 ,540 ,540 ,584 ,544 ,357 ,804 1,000 ,494 ,717
KM4 -,231 ,202 ,278 ,271 ,270 ,281 ,258 ,325 ,394 ,494 1,000 ,489
KM5 -,272 ,243 ,434 ,477 ,399 ,485 ,472 ,274 ,551 ,717 ,489 1,000
The levels of the 12 items are used to produce acceptable reliability estimation, and the empirical
results support their content validity, and discriminated validity.
The results proved that the course content (TQ2) was very important for students as well as the fact
that student satisfaction in the process of acquiring new knowledge was an important goal for the
team that designed and implemented the eLearning platform TESYS (US1). The students consider
that the usability of the Tesys is high because they can perform the activities accurately and
completely without frustration.
The students consider that the access to courses through eLearning platform TESYS is very good
(UFT2), but they see a low correlation between exam performances and the reliability of TESYS
eLearning platform (KM4).
759

Adriana Schiopoiu Burlea et al.
Even the students that used for the first time an eLearning platform (UL1) considered that they
learned easily to use Tesys (UFT1).
The relationship between students and project team (US2) was well appreciated by the students and
the correlations between student satisfaction in the process of acquiring new knowledge (US1) and
this relationship has a value of 0.783.
The students’ expectations regarding course curricula developed through eLearning platform TESYS
(TQ1) are positively related to the access of the courses (UFT2 – value between items: 0.728) and to
the evaluation system through eLearning platform TESYS (KM1 – value between items: 0.783).
The communication process between student and teacher (KM2) is positively related to the course
content (TQ2 – value between items: 0. 436)
The acquisition of new knowledge through eLearning platform TESYS was beneficial for students
(KM3) and they related this performance to the course content (TQ2 – value between items: 0.804).
The students succeeded to capitalize the knowledge acquired through eLearning platform TESYS
(KM5) and they consider that the acquisition of new knowledge through eLearning platform TESYS
was beneficial for them (KM3 - value between items: 0.717).
Most important, training students how to use TESYS led them to the high level of satisfaction in the
process of using TESYS, and the course curricula eased the students’ access to the knowledge
quickly, efficiently, and with positive results.
Some students that use mobile phone devices for uploading assignments or participating in
discussions considered to have the most difficulty with platform regarding their attempts to upload
work.
The results are significant and indicate many positive relationships between technical characteristics
of the eLearning platform, the personal characteristics of the users, courses quality on eLearning
platform TESYS, and exam performances.
Therefore, the results support all four hypotheses.
Theoretical and practical contributions of our research consist of the fact that, despite effective usage
of eLearning platform Tesys is often hindered by technological characteristics. With our research, the
TESYS team has a quick grasp on which enablers are critical for Tesys usage.
Theoretically, this study supports the argument for the inclusion of eLearning platform characteristics
and functionalities within the technical perspective and within the users’ expectations and needs.
To enhance the fit between technology and user satisfaction, Tesys should be made versatile by
providing a range of functions to match the individual tasks and to be made open to the users, no
matter how competent they are. Moreover, set of well-designed courses is necessary, to heighten
self-efficacy of the students, no matter if they are beginner or advanced in using an eLearning
platform. It is very important for the students to acquire useful knowledge free, quickly, and have
resting time at home. The eLearning platform TESYS is very good for teachers and students, and is
so friendly.
In summary, this study helps the team that administrates an eLearning platform to set up policies and
to take corrective actions that would make users not only willing to use this platform, but also to enjoy
doing so. When the user spends less time for administration of the Tesys, the user has more time for
interactive engagement for increased learning. In terms of fostering creative expression, the students
find eLearning platform Tesys largely lacking.
5. Conclusions
In this paper, we have reported findings from two studies realised in the period 2005-2007 and 2010-
2011, concerning the improvement of the eLearning platform TESYS and the user’s satisfaction.
760

Adriana Schiopoiu Burlea et al.
First, we demonstrate that users apprehend the improvement of the eLearning platform TESYS in
period 2009-2010 – they consider there is a direct connection between the reliability and quality of
platform TESYS and the knowledge they gained.
Second, the users’ satisfaction requires quality assessments of communication and interaction
between student – student, student - teacher, teacher - administrator, and student – administrator.
Third, we demonstrate the relationships between student satisfaction in the process of acquiring new
knowledge through eLearning platform TESYS and the usability of the platform.
The two categories of users, both beginners, who used platform TESYS for the first time and the
advanced, who have been previously using the eLearning platform, were more sensitive to the quality
of the curricula content in the eLearning environment.
In conclusion, our research confirms that the eLearning platform TESYS was improved, taking into
account student recommendations during 2005-2007 as well as those from students during 2009-
2010, improvements highlighted by their positive answers.
References
Bolliger, D. U., Wasilik, O. (2009) “Factors influencing faculty satisfaction with online teaching and learning in
higher education”, Distance Education, 30(1), pp. 103-116.
Burlea Schiopoiu, A. (2008) „The Complexity of an eLearning System: A Paradigm for the Human Factor”, in The
Inter-Networked World: ISD Theory, Practice and Education. Vol. 2, pp. 267-278, Springer-Verlag: New
York.
Burdescu, D.D., Mihaescu, CR., Ionascu, C., Logofatu, B. (2010) „Support System for eLearning Environment
Based on Learning”, in Proceedings of The Fourth International Conference on Research Challenges in
Information Science (RCIS), pp. 37-42, Nice, France.
Burdescu, D.D., Mihaescu, M.C. (2006) “Tesys: eLearning Application Built on a Web Platform”, in
Proceedings of International Joint Conference on e-Business and Telecommunications, pp. 315-
318, Setubal, Portugal. INSTICC Press.
Craig, A., Goold, A., Coldwell, J., Mustard, J. (2008) "Perceptions of Roles and Responsibilities inonline
Learning: A Case Study", Interdisciplinary Journal of ELearning and Learning Object, Vol.4, pp. 205-223
Jones, G. H., Jones, B. H. (2005) “A comparison of teacher and student attitudes concerning use and
effectiveness of web-based course management software”. Educational Technology & Society, 8(2), pp.
125-135.
Mahdizadeh, H., Biemans, H., Mulder, M. (2008) “Determining factors of the use of eLearning environments by
university teachers”, Computers & Education, 51(1), pp. 142-154.
McGill, T. J., Hobbs, V. J. (2008) “How students and instructors using a virtual learning environment perceive the
fit between technology and task”. Journal of Computer Assisted Learning, 24(3), pp. 191-202.
Nakayama, M., Yamamoto, H. & Santiago, R. (2009) “Relationship between Learners' Characteristics and
Learning Behavior of Japanese Students in Blended Learning Environment: A Three-Year Study”,
Proceedings of ICEL 2009, pp.377-385, Academic Conferences.
Nakayama M., Yamamoto H. (2011) „Assessing Student Transitions in an Online Learning Environment”,
Electronic Journal of eLearning, Volume 9 Issue 1, pp.75-86.
Palmer S., Holt D. (2009) “Staff and student perceptions of an online learning environment: Difference and
development”, Australasian Journal of Educational Technology, 25(3), pp. 366-381.
Salinas, M. F. (2008) “From Dewey to Gates: A model to integrate psycho-educational principles in the selection
and use of instructional technology”, Computers & Education, 50(3), p. 652-660.
761

Teachers’ Skills set for Personal Learning Environments
Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
Institute of Business Administration Karachi, Pakistan
zashaikh@iba.edu.pk
skhoja@iba.edu.pk
Abstract: This study provides an in-depth investigation of the issues that may impact the critical role anticipated
from today’s teachers to perform while using their personalized learning approaches and hence, in guiding students
develop their personal learning environments (PLEs). Using modified Delphi method (Kenis, 1995; Webler,
1991), the researchers worked with 34 experienced PLE stakeholders taken from the areas of research, teaching
and practice from different geographic regions of the world to discover teachers’ skills and competencies for
PLEs. A questionnaire instrument was developed through analysis of the existing literature with the goal of providing
a recognized foundation to the participants based on the previous work. An exhaustive list containing
teachers’ 60 personalized learning skills was developed. In the three-round process of this study, participants
consented on teachers’ five core PLE competencies, found earlier by Shaikh and Khoja (2011a), and formulated
a skills set for each competency. Based on the findings, the researchers argue that teachers need to become
involved with the relevant skill sets; they will then develop and become entrenched in these new fields of learning.
Keywords: personalized learning, personal learning environments, teachers’ PLE skills, e-learning, Delphi study
1. Introduction
Personalization, as discussed by Burgos, Tattersall, and Koper (2006), is the adaptation of the learning
process and the environment with three key stakeholders: the learner, the teacher, and the set of
rules. Personalized learning considers each student as an individual learner, recognizing the fact that
every student has his or her own learning style and face different challenges in executing this process
(The Personalized Learning Foundation, 2011). This type of learning can be supported by enhancing
students’ learning processes through PLEs, a learner-centric approach to instruction that translates
the principles of personalized learning into actual practice (Shaikh and Khoja, 2011(a); Attwell, 2009).
According to Lubensky (2006), a PLE is a “facility for an individual to access, aggregate, configure
and manipulate digital artifacts of [his] ongoing learning experience”. PLE provides learners a platform
to make good use of knowledge gained by their interactions with peers, mentors and teachers (Downes,
2010; Peña-López, 2010; Wilson, 2008). Drexler (2010) and Väljataga & Laanpere (2010) argue
that an effective PLE requires teachers to be open to new pedagogical possibilities and ideas, help
learners construct their personalized learning plans, and allow learners to pace their learning according
to their needs and challenges to achieve desired goals.
The latest findings in educational research confirm teachers’ critical role of guiding learners to create
and manage their PLEs by using their personalized learning skills (Shaikh and Khoja, 2011(b);
WikiEducator, 2010; Global Teacher, 2010). Powell, Tindal, and Millwood (2008) argue that teachers
can create a powerful motivational and creative force in learners by emancipating learners and permitting
personalization in their pedagogies. However, till today the literature has not qualitatively and
empirically examined teachers’ personalized learning skills, competencies, and knowledge (Minocha,
et al, 2011; McLoughlin and Lee, 2010; Powell, Tindal, and Millwood, 2008). It can also be argued
that most of the characteristics and competencies of teachers are not documented. Hence it could be
ascertained that such work will have a huge impact and potential in the research fraternity.
This study provides an in-depth investigation of the issues that may impact the critical role anticipated
from today’s teachers to perform while using their personalized learning approaches and hence, in
guiding students to develop their personal learning environments (PLEs). The study worked on two
objectives. The first objective was to identify teachers’ skills and knowledge necessary for providing
proper guidance for personalized learning. The second was to find the changes in teachers’ traditional
competencies required by the increased availability and use of PLEs and changes in pedagogy which
favour a more constructivist and less teacher-centric approach. For the purpose of this study, competency
or skill has been defined as “an area of knowledge or skill which is critical to the production of
key outputs” (McLagan and McCullough, 1983, p.7).
Using modified Delphi method (Kenis, 1995; Webler, 1991), the researchers worked with 34 experienced
PLE stakeholders taken from the areas of research, teaching and practice from different geo-
762

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
graphic regions of the world to discover teachers’ skills and competencies for PLEs. Data from the
three rounds of study were organized and compared with descriptive statistics (interquartile range
mean and ranking).
An agreed and well-formed synopsis was developed by the consensus of the participants for debate.
The study lasted for three rounds, until it reached on a consensus. Personal perspectives were
brought forward indicating perception of importance and consent of the statements as stated in the
questionnaire. Aggregated results based on the survey were developed as concluding reports. Special
care was taken to select the Delphi’s group members. Experts were selected from the international
community. At the start of this study, a total of 36 PLE experts accepted to participate, however,
2 could not respond to final round questionnaire in time, hence totaling to 34 participants. Of the 34
participants, 19 (56 percent) were male and 15 (44 percent) were female.
The review of the existing literature for the design of series of three rounds questionnaire covered five
areas: (a) an analysis of studies that highlighted the need for faculty development in personalized
learning environments, (b) an analysis of the identified personalized learning competencies and skills
noted in the literature, (c) competencies and skills anticipated from today’s teachers to familiarize
learners with personalized learning, (4) competencies and skills anticipated from today’s teachers to
guide learners how to design a personal space for learning, and (5) discussion of the importance of
personalized learning competencies and skills identification for teachers’ training and development.
The Round I Questionnaire (RIQ) included 60 personalized learning skills. Researchers found teacher
competency proposals suggested by Alvarez et al (2009), Trilling (2008) and Williams (2003) also
appropriate for personalized learning methodology. As developed by Shaikh and Khoja (2011a), the
teachers’ personalized learning skills were sought for five core competencies, which were: Planning
and Design (P&D), Instruction and Learning (I&L), Communication and Interaction (C&I), Management
and Administration (M&A), and Use of Technology (UoT).
2. Methodology
The study begins with an examination of teachers’ generic competencies and skills for face-to-face,
distance, online, virtual and personalized learning methods of instruction. It continues with identifying
the personalized learning skills and knowledge that teachers need to have for PLEs. The researchers
found Delphi study a useful methodology to structure the group activity in order to determine teachers’
skills set for PLEs, as practiced by previous competency studies (Williams, 2003; Piskurich and
Sanders, 1998; Rothwell and Cookson, 1997; McLagan, 1989).
Delphi Method is an exercise that has been used in research for gathering opinion of a group of geographically
dispersed experts through multiple rounds of questionnaire to reach a consensus without
holding a group meeting in order to deal with a complex problem (Ziglio, 1996). As suggested by
Kenis (1995) and Webler (1991), this PLE Delphi study employs a modified Delphi version that gets
expert opinion from group members on prescribed set of closed-ended and open-ended questions. At
the same time, gives freedom of agreement or disagreement from issues discussed in series of questionnaires
and allows addition of any further issues
Delphi technique accompanies several features that add rigor in research, such as: selection of geographically
dispersed experts, small number of group members, multiple round survey technique, anonymity
of group members, reporting of previous round responses to individual members, and use of
statistical tests to reflect consensus development (Shaikh and Khoja (b), 2011; Turoff and Hiltz, 1996;
Linstone and Turoff, 1975). As noted by Williams (1996), Delphi technique also neglects the drawbacks
of face to face discussions, such as: “the bandwagon tendency; deference to the most dominant
member of the discussion group; the vulnerability to manipulation; and the reticence on the part
of individuals to change their minds in front of others”. Isaac and Michael (1995) argue that Delphi is
not always aimed at achieving a strong consensus and to force a quick compromise, however, its
main objective is to produce a critical assessment and discussion to increase problem understanding
and information generation. Delphi also respects views of each individual and does not discourage
minority views to move toward group consensus.
The criteria that guided the selection of expert group for this Delphi was a multistep process carefully
devised to emerge the pool of recognized experts of PLE domain. The criteria for expert selection
used in this study were that the expert (a) must be a known PLE futuristic of at least one of four cate-
763

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
gories viz. the teacher, the researcher at doctoral level, PLE designer/developer, and PLE practitioner/e-learning
expert, (b) must have some publication track record and/or research contributions, (d)
must have at least one year of domain experience, and (d) must be willing to participate at voluntarily
basis.
At the start of this study, a total of 36 experienced PLE stakeholders accepted to participate, however,
two could not respond to the final round. Hence the Delphi group was comprised of 34 members who
participated in all three rounds. Of the 34 participants, eight had an academic background of teaching,
six had a researching background as PLE researchers at doctoral level, six participants identified
themselves as PLE designers/developers, and 14 were PLE practitioners/elearning experts working
as staff members or executives in commercial enterprises. Participants included both males and females.
There were 34 panelists, out of which 19 (56 percent) were male and 15 (44 percent) were
female. There were six panelists (two males and six females) with doctorate and/or more advanced
level qualifications. The time took to complete this three-round study was one and one-half months
during the summer of 2011.
The RIQ consisted of an exhaustive set of 60 skills and knowledge derived from the literature on
teachers’ personalized learning skills set. Experts were asked to rate the skills on criticality level on
assigned five-point likert-type scale. They were also given the opportunity to provide suggestions in
description, in terms of deletion, addition, comment, or recommendation of any of the skills and also to
suggest additional skills. Before attempting for Round II Questionnaire (RIIQ), the Delphi participants
were presented a summary of the changes generated in skills from Round I responses.
The RIIQ required participants to select one or more competencies, in which, as they perceive, did
each of the teachers’ skills or tasks fall. A set of skills that received participants’ 50 or more votes for
each competency in Round II responses were ranked on the group mean value in descending order.
For Round III Questionnaire (RIIIQ), the experts were instructed to rate the skills on desirability scale.
They were requested to select the highly desirable skills that according to their perception a teacher
must possess.
3. Data analysis
This Delphi’s RIQ was aimed at formulating a personalized learning skills set of teachers for each
competency required for students’ PLE process. Data analysis for this round was carried out using
asymmetric lambda to develop the table to allow participants to select one or more competencies.
Sample of data analysis of questions and the choices for responses to participants in RIQ has been
shown in Table 1.
Table 1: Showing a skill in round I questionnaire
Teachers’ Personalized Learning Skills Teachers’ PLE Competencies
Define participative and social learning activities that support indi-
P&D I&L C&I M&A UoT
vidual learners’ personal life goals and needs.
Table 1 shows one of 60 personalized learning skills that were identified through literature review.
During RIQ, participants were instructed to select one or more core competencies to be as a required
skill of that competency. The categories are: Planning and Design (P&D), Instruction and Learning
(I&L), Communication and Interaction (C&I), Management and Administration (M&A), and Use of
Technology (UoT). In Round I responses, participants evaluated teachers’ skills and competencies
critically and developed a preliminary list of personalized learning skills against each competency in
response to how to develop professional competencies as per their responsibilities. Some new skills
such as: defining and designing personalized learning processes and activities for learners, engaging
students in group processes of inquiry, enabling self direction, knowledge building and autonomy of
students, and acquainting students’ with basic technology skills, were suggested. Few skills that they
consented as not important were asked to be deleted for the consequent rounds.
The RIIQ was developed after the analysis of RIQ responses. This round is used to develop consensus
or achieve at least some level of stability in the diverse opinions offered by participants. The aim
was to further develop group consensus on this revised skills set. During this round, participants were
asked to re-evaluate each skill according to the perceived importance (number of times participants
selected it) group placed on each skill in RIQ, as shown in Table 2. They were instructed based on
their selection of skills during this round, a skill that receives fifty or more (>= 50) percent of partici-
764

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
pants’ votes for a particular competency will be considered as a skill of that competency for upcoming
rounds.
Table 2: Showing a skill in round II questionnaire
Teachers’ Personalized Learning Skills
Define participative and social learning activities that support indi-
Teachers’ PLE Competencies
vidual learners’ personal life goals and needs. (P&D=19, I&L=9,
C&I=3, M&A=4,UoT=6)
P&D I&L C&I M&A UoT
The Round III Questionnaire (RIIIQ) was the culminating questionnaire of this study. The main objective
of this round was to achieve a high level of consensus among participants, and thus, the questionnaire
was specifically customized for each participant. Data analysis to develop this questionnaire
was carried out through ranking of skills for each competency (as per 50 percent formula of Round II).
Participants were instructed to select the highly desirable skills that a teacher must possess, as
shown in Table 3.
Table 3: Showing a skill in round iii questionnaire
Teachers’ Personalized Learning Skills Highly Desirable
Define participative and social learning activities that support individual learners’
Yes No
personal life goals and needs.
As with similar Delphi studies, the 25 to 75 percent interquartile range (IQR) was considered suitable
to determine the level of consensus on each skill. For this study, consensus was defined as being
achieved when an IQR of 1.0 or less is attained for most of the skills. In general, the Delphi rounds
are terminated when moderate to high convergence (e.g., IQR

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
Overall, the participants reached to a high consensus and formulated a broad list of 73 skills for five
competencies (see Table 4). In terms of the desirability weight, there were a number of skills with very
high consensus. Sixteen skills were identified for teachers planning and design competency. The
highly desirable skills include: (a) defining and designing personalized learning processes and activities
for learners, (b) course design that trust in students’ self-managing ability of learning, (c) harnessing
outside learning resources, (d) adapting the curriculum that includes learner focused forms of
feedback and assessment, and (e) guiding students in setting their PLEs. Teachers’ instruction and
learning competency received the largest skills set: (a) embracing learner-centered but providerdriven
approach to education, (b) fostering students’ self-actualized skills of learning, (c) managing
non-formal and informal learning of students, (d) having knowledge of collaborative, connective, active,
constructive, reflective aspects of learning, (e) understanding of how humans learn, and (f) providing
students an adaptable and flexible learner and task scaffolding, were considered to be the
highly desirable skills.
Table 4: Teachers’ skills set for personal learning environments
766

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
Table 4 shows 18 skills for communication and interaction competency. The most desirable skills of
this list include: (a) allowing learners to define, create and shape their own learning content, (b) encouraging
peer learning, social links and participation, (c) promoting teamwork among learners, (d)
engaging students in group processes of inquiry, and (e) building confidence and communication
skills of their students. Management and administration competency received 9 distinct skills set. The
most important skills include: (a) maintaining and managing learning environments within and outside
the classrooms, (b) promoting accessibility for learners, (c) providing learning guidance, evaluation,
feedback and advice to students, (d) enabling self direction, knowledge building and autonomy of students,
and (e) assessing efficacy of the current learning process and learning services.
Towards the students’ area, acquainting students’ with basic technology skills becomes more important
for the use of technology competency, such as: facilitating students to become prosumers of
knowledge, ideas and artifacts, helping students see the value and benefits of new technologies and
empower him to adapt to the changes, supporting applications of learning, learning communities,
groups of learners and networks, facilitating information presentation, and creating online interactive
content for learning.
767

6. Conclusion and recommendations
Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
This study provides teachers’ personalized learning skills ranked on highly desirable scale. The results
suggest that apart from teachers’ skills, nature and complexity of their tasks are the key issues
while guiding students forming a PLE. The importance of the skills and competencies may vary depending
on the education environment particularly related to the personalized learning model being
implemented. It was concluded that teachers do not necessarily require excelling in all the skills proposed
by this study, however, they need to become involved with the relevant skills set and show
their willingness to change; they will then develop and become entrenched in these new fields of
learning. Additionally, it is also concluded from the results of this study that all these skills should be
considered in the curriculum for staffing and training of teachers.
The recommendations that could be made based on the results of this research are: (a) teachers
need to become involved with the skills; they will then develop and become entrenched, (b) teachers
who adopt personalized learning in their pedagogies and practices may be rewarded at institutional
and management level, (c) courses that emphasize on forming students’ PLEs may be offered to students
so often, (d) teacher training programs may incorporate the skills set identified by this study as
a framework, and (e) teachers are offered to learn in environments similar to those we expect our students
to learn in.
Further research might identify the level of knowledge, skills, and attitude needed for different types of
learning pedagogies and for the different instructional delivery models. Furthermore, research is also
needed to identify the knowledge, skills, and attitude making up these personalized learning skills and
competencies of teachers. More specific teacher training programs could be offered that target the
perceived knowledge, skills, and attitude required of these skills set. In addition, research that determines
the level of skill mastery needed for the different competencies may also be carried out. Due to
the rapid changes in educational and communication technologies, it is recommended that repeated
studies are required that identify teachers’ skills for PLEs.
Acknowledgements
The researchers would like to extend their gratitude to the participants’ of this Delphi study who have
made this project possible.
References
Alvarez, I. Guasch, T. and Espasa, A. (2009) “University Teacher Roles and Competencies in Online Learning
Environments: A Theoretical Analysis of Teaching and Learning Practices”, European Journal of Teacher
Education, Vol 32, No. 3, pp 321-336.
Arthur, E. (2009) “Experience the digital education revolution”, [online]
http://www.convergemag.com/edtech/Experience-the-Digital-Education-Revolution.html?page=3 (accessed
on 10/05/11).
Attwell, G. (2009) “Personal Learning Environments: The future of education”, [online], SlideCast,
http://www.pontydysgu.org/2009/01/personal-learning-environments-the-slidecast/ (accessed on 15/05/11).
Burgos, D., Tattersall, C., and Koper, R. (2006) “How to Represent Adaptation in eLearning with IMS Learning
Design”, Educational Technology Expertise Centre (OTEC). Heerlen: The Open University of the Netherlands.
Downes, S. (2010) “The Role of Educator in a PLE World”, [online], Stephen s Web,
http://www.downes.ca/post/54312 (accessed on 15/10/2010).
Kenis, D. G. A. (1995) “Improving group decisions: Designing and tes ting techniques for group decision support
systems applying Delphi”, Doctoral Dissertation, the University of Utrecht, The Netherlands.
Drexler, W. (2010) “The Networked Student Model for Construction of Personal Learning Environments: Balancing
Teacher Control and Student Autonomy”, Australasian Journal of Educational Technology, Vol 26, No.
3, pp 369-385.
Global Teacher, (2010) “A Community for Victorian Teachers and their Students”, [online],
http://global2.vic.edu.au/ (accessed on 15/11/2010).
20-34.
Isaac, S., and W. Michael (1995) “Handbook in Research and Evaluation”, 3 rd ed. San Diego, CA: Educational
and Industrial Testing Services.
Lubensky, R. (2006) “The present and future of Personal Learning Environments”, [online] Blog Entry,
http://www.deliberations.com.au/2006/12/present-and-future-of-personal-learning.html (accessed on
10/11/10).
McLagan, P. (1989) “Models for HRD practice”, Alexandria, VA: American Society for Training and Development
768

Zaffar Ahmed Shaikh and Shakeel Ahmed Khoja
McLagan, P., and McCullough, R. (1983) “Models for excellence: The conclusions and recommendations of the
ASTD training and development competency study”, Alexandria, VA: American Society for Training and Development
McLoughlin, C. and Lee, M. J. W. (2010) “Personalised and Self-regulated Learning in the Web 2.0 Era: International
Exemplars of Innovative Pedagogy using Social Software”, Australasian Journal of Educational Technology,
Vol 26, No. 1, pp 28-43.
Minocha, S., Schroeder, A. and Schneider, C. (2011) “Role of the Educator in Social Software Initiatives in Further
and Higher Education: A Conceptualisation and Research Agenda”, British Journal of Educational
Technology, Vol 42, Wiley Online Library, doi: 10.1111/j.1467-8535.2010.01131.x
Peña-López, I. (2010) “Personal Learning Environments: Blurring the Edges of Formal and Informal Learning”,
[online], Working Paper http://ictlogy.net/20101105-personal-learning-environments-blurring-the-edges-offormal-and-informal-learning-an-experiment-with-anthologize/
(accessed on 10/10/2010).
Powell, S., Tindal, I., and Millwood, R. (2008) “Personalized Learning and the Ultraversity Experience”, Interactive
Learning Environments, Vol 16, No. 1, pp 63–81.
Rothwell, W., and Cookson, P. (1997) “Beyond instruction: Comprehensive program planning for business and
education”. San Francisco: Jossey-Bass.
Shaikh, Zaffar A. and Khoja, Shakeel A. (2011a) “Role of Teacher in Personal Learning Environments”, Paper
read at The PLE Conference 2011, 11-13 July, 2011, University of Southampton, Southampton,
http://journal.webscience.org/568/
Shaikh, Z. A. and Khoja, S. A. (2011b) “Role of ICT in Shaping the Future of Pakistani Higher Education System”,
Turkish Online Journal of Educational Technology, Vol. 10, No. 1, pp 149 – 161
The Personalized Learning Foundation (2011) “It's Time for Learning to Get Personal”, [online],
http://personalizedlearningfoundation.org/id3.html (accessed 01/03/2011).
Trilling, B. (2008) “21st Century Learning, the New Balance”, in Proceedings of International Conference on
Teaching and Learning with Technology, Singapore
Turoff, M., and S. Hiltz. (1996) “Computer based Delphi processes”, In: ‘Gazing into the oracle: The Delphi
method and its application to social policy and public health’, ed. M. Adler and E. Ziglio, 56–88. London:
Kingsley.
Väljataga, T. and Laanpere, M. (2010) “Learner control and personal learning environment: a challenge for instructional
design”, Interactive Learning Environments, Vol 18, No.3, pp 277-291.
Webler, T., Levine, D., Rakel, H., and Renn, O. (1991) “A Novel Approach to Reducing Uncertainty: The Group
Delphi”, Technological Forcasting and social change, Vol 39, No. 3, pp 253-263.
WikiEducator, (2010) “Learning Objects, Personal Learning Environments, Study Guides”, [online]
http://wikieducator.org/User:Vtaylor/Learning_objects,_personal_learning_environments,_study_guides (accessed
on 15/11/10).
Williams, P. E. (2003) “Roles and Competences for Distance Education Programs in Higher Institutions”, American
Journal of Distance Education, Vol. 17, No. 1, pp 45–57.
Wilson, S. (2008) “Patterns of Personal Learning Environments”, Interactive Learning Environments, Vol 16, No.
1, pp 17-34.
Ziglio, E. 1996. The Delphi method and its contribution to decision-making. In Gazing into the oracle: The Delphi
method and its application tosocial policy and public health, ed. M. Adler and E. Ziglio, 3–33. London:
Kingsley.
769

Bridging the Feedback Divide Utilising Inclusive
Technologies
Angela Shapiro and Aidan Johnston
Glasgow Caledonian University, Glasgow, UK
A.Shapiro@gcu.ac.uk
A.Johnston@gcu.ac.uk
Abstract: The aim of this action based research project is to enhance the student learning experience through
the production of vidcasts that focus on the conventions of academic writing in Glasgow Caledonian University’s
School of Building and Natural Environment (BNE). Vidcasts (using a blend of narrated audio and on screen
images and slides) aim to meet students’ individual academic writing requirements. Four vidcasts have been
produced at this stage; the first level focuses on an introduction to academic writing explaining and illustrating
how to write a paragraph at micro level and how to apply the conventions of referencing. The second level
considers how to plan, construct and write a report including a detailed breakdown of appropriate material for
each section of the report. Level three concentrates on features of group reports whilst level four is aimed at
executing a complex report applying a case study format. Each vidcast has links to additional support materials
that are available on the web. The objectives were to produce models of good practice on the issues of academic
writing conventions at four distinct levels. The vidcasts have been placed on the virtual learning environment
(VLE), Blackboard, and feedback was requested from student representatives studying across the programmes
in BNE. We are also exploring the effectiveness of three different approaches that have been applied to obtain
feedback; these included the use of online feedback on the Effective Learning Service’s website through Google
analytics and Poll Everywhere, an interactive electronic voting tool and tracking students’ use via the university
VLE. An integral aspect of this project is the involvement of students and BNE staff in the evaluation of the
different approaches. We recognise that the feedback, like the vidcasts, should be context based and
acknowledge that one size does not fit all. Thus, we acknowledge that the feedback mechanisms will need to be
inclusive and tailored to different ability levels as well as for different cohorts of students. An interesting outcome
from this project has been that despite students noting the importance of receiving timely feedback from staff;
they appear unwilling to complete surveys giving feedback formally, although willing to express their views
anecdotally. One reason could be that students are constantly being asked to complete evaluation from individual
end of module forms to National Student Survey leading to survey fatigue. This suggests that using immediate
feedback formats such as Poll Everywhere has a role in encouraging students to respond although it is
acknowledged that there could be insufficient time for students to reflect.
Keywords: academic writing conventions, feedback, vidcasts, student learning
1. Introduction
The aim of Glasgow Caledonian University’s Caledonian Scholars and Associates initiative’s based
research project is to enhance the student learning experience through the production of innovative
feedback practices by the medium of 'vidcasts' in Glasgow Caledonian University’s (GCU) School of
Building and Natural Environment (BNE). This project commenced in October 2010 and is aligned to
GCU's Learning, Teaching and Assessment Strategy (2008-2015). The project was funded by GCU’s
Caledonian Scholar and Caledonian Associate Initiative, the purpose of which is ‘… to contribute to a
distributive leadership model of innovation in learning and teaching across the University’ (Caledonian
Academy, 2010). By supporting staff in this initiative the intention is ‘…to contribute to the
enhancement of learning and teaching practice and the quality of the student experience’ (Caledonian
Academy, 2010). The focus was directed towards meeting Objective 1 of GCU's Learning, Teaching
and Assessment Strategy (2008-2015), a summary of which is outlined as follows: ‘ …To equip
students with the knowledge, skills and attributes to operate as flexible, independent lifelong learners.
This will be through the application of incorporating innovative approaches to teaching and learning.’
(Glasgow Caledonian University Quality Office, 2008)
As Cohen et al., (2007) reiterate, action based research is an approach that enables the practitioners
to pose questions, carry out actions and justify activities. Once evidence has been gathered, in
collaboration with other educators in the work place, there follows the process of reflection enabling
social transformation and a critique of practice. All takes place against the theoretical background of
action based research (Kemmis & McTaggart, 1992, cited in Cohen et al., 2007).
The original objectives of the project were to produce and evaluate models of good practice on the
issues of academic writing conventions. Within higher education, students are required to engage with
Advanced Academic Literacy (AAL) and as such, write at an accepted complex level as required by
higher education. Indeed, Tardy (2005, p326) observed that ‘… students need to learn ways of
770

Angela Shapiro and Aidan Johnston
thinking about, interacting with, and constructing knowledge of disciplinary communities and content’.
All has to occur within the discipline of the individual academic department within the higher education
institute; this methodology is generally defined as being an academic literacies approach (Lea &
Street, 2010).
2. The effective learning service
In Glasgow Caledonian University, the Effective Learning Service (ELS) first commenced in 2000. At
that time, academic support was regarded as being necessary in order to develop the personal and
study skills learners needed to meet the demands of Higher Education (Gerrard et al., 2005). This
could be argued as being the traditional study skills method, a deficit approach (Lea & Street, 2010).
However, since its conception, the ELS approach has developed to one that recognises that
irrespective of the student’s background and knowledge, entering or re-entering higher education is a
transitional process which is unique to each individual. This has underpinned the development of the
ELS in GCU, such that it aims to develop the academic literacies that are required in higher education
(Kern & Schultz, 2005). This approach is one that that has been integrated on an institution-wide
basis, is inclusive and student focused, emphasising that all students can improve learning, rather
than a remedial few (McAllister & Shapiro, 2004). Moreover, this developmental approach recognises
that learners derive from different backgrounds, have different motivations and learning needs (Illeris,
2006). Thus, the pedagogical approach that the ELS offers is one that seeks to encourage the
students to take on responsibility for their own learning in that they identify what is important, rather
than being directed by the lecturer (Illeris, 2006).
We believe that students at GCU are comfortable with accessing technology, indeed, over the
academic year 2009/2010, over 13,000 visitors accessed the Effective Learning Service (ELS)
website, of whom 17% accessed the online guides (Google Analytics, 2011). At the same time, over
5,000 students attended over 200 workshops that were run by the ELS over the academic year
2009/2010. However, although the students attending the workshops indicated their usefulness, in
feedback many commented that they would also have liked to have the opportunity to re-access the
material at a later date. Moreover, every workshop also generates students requesting individual
follow up appointments or requests for additional support material. This suggested that using vidcasts
in Higher Education would be helpful for students in their academic journey and aid their progression
towards independent learning.
3. Using inclusive technologies
A Vidcast or vodcast contains audio and images, either moving or fixed (Traxler, 2008). It is widely
acknowledged that students have different learning styles (visual, reflective, auditory, and
kinaesthetic) but at times some of these learning styles are not accommodated fully in Higher
Education, although Fox and Ronkowski (1997) acknowledge that some lecturers accommodate
these differences in their teaching methods. This project will use vidcasts to incorporate the students’
preferred pedagogical preferences in an online environment. Podcasting, for example, will clearly
appeal to auditory learners; with the addition of visual prompts the vidcast will also support dyslexic
students in retaining the information (Edirisingha et al., 2008). Essentially by students having to
concentrate and apply active listening when using podcasts for educational pursuits, this means that
they are cognitively involved rather than merely listening, as one does to music.
In addition, users of vidcasts with visual impairments will be able to select to listen solely to the audio
element. Students can also choose when and where they wish to engage with the vidcasts and this
approach enables students to revisit the material at their own pace and can download the material on
to their mobile device (Informal Mobile Podcasting and Learning Adaptation (IMPALA) 2006; Gribbins,
2007). Moreover, students at university have a wealth of experience, irrespective of age and if the
learning is purposeful, then the adult learner is more likely to become engaged in the process (Rogers
& Uddin, 2005).
4. Investigation stage
Two workshops approximately 45 minutes in length were recorded using a fixed video camera and
subsequently digitised and placed on the ELS website along with a description of the workshop. Initial
feedback from students suggested this approach offered little value as it was not easy to find the
relevant information as they had to scroll though 45 minutes of video to find the relevant part. Further
771

Angela Shapiro and Aidan Johnston
feedback also suggested it was difficult to see the PowerPoint slides in the background and that
seeing the presenter offered little value. (Figure 1).
Figure 1: Project journey
5. Evaluation of first year of funded project
The initial evaluation approach that was selected, asked users to evaluate the usefulness of the two
completed vidcasts on essay writing and writing final year reports through feedback from the Survey
Monkey on-line questionnaire service. Producing vidcasts to enhance students’ learning is only a
worthwhile activity if students use the vidcasts. It was important, therefore, to know what students
thought about this type of delivery and whether they felt more engaged in the student learning
experience. Accordingly, the questionnaire was placed on the ELS website in November 2009.
However, feedback was slow and only 50 questionnaires were completed. Nevertheless, the
responses in the main were positive, and users (both students and staff) gave recommendations for
improvement. At the same time statistics were collected from Google Analytics which indicated that
over 1,000 users had accessed the vidcasts, which clearly indicated that they were being utilised.
The respondents to the Survey Monkey online survey stated that they preferred the vidcast to be
delivered in shorter sections; that they wanted brief summaries of the content and suggested
improvements in navigation. These changes were subsequently actioned; shorter chapters consisting
of no more than five minutes of content were added along with detailed chapter descriptions.
Improvements were made to address the navigation issues by upgrading the web video player to
allow users to control navigation. In response from feedback from post-graduate level students, an
additional vidcast on writing literature reviews was produced. (Figure 1).
6. Evaluation of second year of funded project - the four new vidcasts
In January 2011, an additional four vidcasts were placed within GCU’s institutional virtual learning
environment (VLE). This new approach was selected to allow us to target this particular academic
school (BNE) and engage the students in their own learning environment rather than placing the
content on the ELS website, which was used previously.
The four new vidcasts are also set within the national context of exploring new assessment and
feedback practices, the purpose of which is to enhance the student experience, and more specifically
to support the development of self-regulated learners (Nicol & Macfarlane-Dick, 2006). We wanted to
discover the following:
Can increased dialogue make for more effective feedback as perceived by our students?
772

Angela Shapiro and Aidan Johnston
Can the use of new learning technologies enhance such dialogue and thereby support the
academic literacies model through an online process. (Figure 1).
7. Conventions of academic writing and feedback
Primarily the foundation for placing the second year aspect of the Scholar and Associate Project was
based on Nicol and Macfarlane-Dick's (2006) principles which focused on the importance of giving
students appropriate and timely feedback that encourages and helps the students learn and progress.
Feedback is essential to student learning. Providing feedback in different media reinforces the
message and gives students a more personal learning experience. The first principle recommends
helping students to clarify the criteria for good performance (outcomes, performance criteria and
expected standards). Our response was to produce exemplars demonstrating acceptable academic
writing conventions using the vidcasts for students studying in the School of Built and Natural
Environment at academic levels 1-4 (Figure 2).
Chapters
Related
Links
Mobile
version
download
Web video
player
Figure 2: Extract from introduction to academic writing vidcast.
The first level vidcast focuses on an introduction to academic writing conventions by explaining and
illustrating how to write a paragraph at micro level including the conventions of referencing. Animated
sequences highlighted the various errors throughout the paragraph. This approach blends the
narration and the visual representation shown on screen to emphasise the point to the viewer. The
second level vidcast explains how to plan, construct and write a report including a detailed breakdown
of appropriate material for each section of the report. The level three vidcast concentrates on features
of group reports and examines some of the issues that may occur when individuals are working
together on one task. The level four vidcast is aimed at post-graduate students and demonstrates
how to execute a complex report, applying a case study format. Each vidcast has related links to
additional reputable support materials that are available on the internet. (figure 2). The next principle
influenced by Nicol & Macfarlane-Dick (2006) indicated the importance of facilitating the development
of self assessment (reflection) in learning. We decided to send emails requesting feedback from
selected students and staff in the School of BNE and carry out feedback using Poll Everywhere with
the first year cohort following the use of the first level vidcast. Poll Everywhere is an interactive
audience response system that uses mobile phones, twitter, and the web. The responses are
displayed in real-time. The polls can be multiple choice based questions or open-ended questions that
can be created to create conversations with the participants. The advantage of Poll Everywhere is
that immediate feedback may be obtained. We thought that since a high percentage of the students in
BNE first level were young males, that they would have a mobile phone and be comfortable with using
technology.
773

Angela Shapiro and Aidan Johnston
Figure 3 illustrates an example of results from a sample poll using Poll Everywhere. Students were
shown a vidcast as part of their lecture and then asked questions to test their understanding about
conventions of academic writing. However each group of students revealed an unforeseen result in
that students preferred to either vocalise their thoughts, or put their hand up rather than using their
mobile. This may have been for several reasons; firstly, students had to use a long number to send
their results back (Figure 3) and secondly, some students were concerned that responding would
incur cost (despite the fact that most had call plans with free texts). The lecturer took time to become
familiar with using Poll Everywhere and in the initial polls students could guess the correct answer as
the results were revealed as student responses were sent. In the later polls, the lecturer learnt to hide
the results so they did not appear instantaneously. Despite these challenges, students enjoyed the
interaction; however, as this took place very quickly after the learning experience, there was no time
for reflection by the learner.
Response
options
Poll question
Response
methods:
SMS, Twitter
or Web
Respons
e results
Figure 3: Sample student response using Poll Everywhere
Following this, we distributed a brief questionnaire requesting comments from selected staff and
students in the school of BNE. To date, although we know the vidcasts are being used regularly,
based on Google analytics web tracking (1583 people accessed the vidcasts in 2010), only a handful
of BNE staff and students have responded to the email. However anecdotal feedback has been
heartening with users stating that they found the vidcasts easy to follow and liked the fact they had
the option of being able to navigate through the chapters. The issue of survey fatigue has been
suggested in studies, with respondents in these studies stating that reasons for non completion
included: too many surveys, lack of time and that many of the surveys appear to be irrelevant (Porter
et al., 2005). Other research highlighted that the culture of the department may affect the level of
responses in conjunction with the personality and attributes of the respondent (Porter & Whitcomb,
2005).
8. Conclusion
The results indicated that students liked to respond using their mobile phones, but some were
unwilling to engage with the feedback process, either because of security or cost, although
interestingly students were happy to respond by putting their ‘hands up’ when asked the same
questions that were on the Poll Everywhere slide. This suggested to us that students do not feel
uncomfortable responding in front of their peers. However, it is acknowledged that this approach of
feedback does not allow time for reflection. Additionally, Clouder (1998) suggests that ‘… students are
not all at similar cognitive levels; therefore their evaluations can only be a product of their
expectations for their level of cognitive development.’ Indeed in a study by Arthur (2009) it was noted
that some lecturers believed that they are the professionals and thus they are the only ones who are
sufficiently competent to evaluate lecturers’ work. We would disagree but recognise that care needs
774

Angela Shapiro and Aidan Johnston
to be taken when distributing questionnaires. However we believe that while students and staff may
not formally respond to questionnaires, by accessing the site repeatedly and spending time using the
vidcasts and the related links this partially indicates student and staff approval.
9. Recommendations for future
We wish to further provide opportunities to reduce the divide between current and desired
performance. We intend to reproduce the vidcasts in smaller sections and place them on the new
Effective Learning Service’s website. The material will be accessible for all students and staff at
Glasgow Caledonian University. We also wish to carry out further research on employing electronic
feedback methods in order to enhance feedback mechanisms from staff and students. We believe
that Poll Everywhere has potential to be applied in a variety of ways throughout our institution. We
intend to liaise and collaborate with staff institutionally to facilitate the provision of innovative feedback
mechanisms.
References
Arthur, L. (2009) “From performativity to professionalism: lecturers' responses to student feedback', Teaching in
Higher Education, Vol 14: No.4, pp 441 – 454.
Cohen, L. Manion, L. and Morrison, K. (2007) Research Methods in Education, Routledge, London: Sixth ed.
Clouder, L. (1998), ``Getting the `Right Answers': student evaluation as a reflection of intellectual development?'',
Teaching in Higher Education, Vol. 3, No 2, pp 185-95.
Crowther, L. (1998) “Getting the ‘Right Answers’: student evaluation as a reflection of intellectual development?”
Teaching in Higher Education, Vol.3, No2.pp185-195.
Edirisingha, P. Salmon, G. and Nie, M. (2008) “Developing pedagogical podcasts”. In: G. Salmon and
P Edirisingha, eds. Podcasting for Learning in Universities, Maidenhed: Open University Press,
pp 154-168.
Fox, R L. and Ronkowski, S. A. (1997). “Learning Styles of Political Science Students.” PS: Political Science and
Politics Vol 30. No 4.pp 732-37.
Gerrard, C. Tweedie, S. and McVey, D. (2005) “Embedding effective learning skills in the curriculum: case
studies and interventions” Paper presented at Centre for Research in Lifelong Learning Glasgow, Glasgow
Caledonian University, June.
Glasgow Caledonian University. (2008). Learning, Teaching and Assessment Strategy 2008-2015. Available:
http://www.gcu.ac.uk/quality/strategy/ltas.html . Last accessed 10th January 2011.
Glasgow Caledonian University. (2010). Overview of the Caledonian Scholars and Associates Initiative 2010.
Available: http://www.academy.gcal.ac.uk/professional/sanda.html. Last accessed 1 May 2011.
Google Inc.. (2011). Google Analytics Statistics - Effective Learning Service Vidcast Page.. Available:
http://www.google.com/analytics. Last accessed 15 May 2011.
Gribbins, M (2007) “The Perceived Usefulness of Podcasting in Higher Education: A Survey of
Students’ Attitudes and Intention to Use” [online] Proceedings of the Second Midwest United
States Association for Information Systems, Springfield, IL. http://aisel.aisnet.org/mwais2007/6, May.
Illeris, K. (2006) ‘What is special about adult learning?’ In: Sutherland, P and Crowther, J. eds. Lifelong Learning.
London, Routledge, 15-24.
Informal Mobile Podcasting And Learning Adaptation (2006) IMPALA Project Website. Available:
http://www.le.ac.uk/impala/index.html Last accessed 10 December 2010.
Kern, R and Schultz JM. (2005) “Beyond Orality: Investigating Literacy and the Literary in Second and Foreign
Language Instruction” The Modern Language Journal, Vol. 89, No. 3, Special Issue: Methodology,
Epistemology, and Ethics in Instructed SLA Research, pp. 381-392.
Lea, M. R. and Street, B, V.(2010) “The "Academic Literacies" Model: Theory and Applications” Theory Into
Practice, Vol. 45, No 4, pp368-377.
McAllister, C. and Shapiro, A. (2004) “Developing learners at Glasgow Caledonian University: the Effective
Learning Service response”, Paper presented at Forum for the advancement of Continuing Education
Conference, July 2004.
Nicol, D. J. and Macfarlane-Dick (2006) “Formative assessment and self-regulated learning: A model and seven
principles of good feedback practice” Studies in Higher Education, vol 31, no 2,pp 199-218
Porter, S. R. and Whitcomb, M.E. (2005) “ Non- response in student surveys: The role of demographics,
engagement and personality”. Research in Higher Education. Vol 46. No. 2 pp 128-150.
Porter, S. R., Whitcomb, M. E. and Weitzer, W. H. (2004), “Multiple surveys of students and survey fatigue.” New
Directions for Institutional Research, 2004: pp 63–73.
Rogers, A. and Uddin, M. A. “Adult Learning Theory and the Provision of Literary Classes in the Context of
Developing Societies”, in Street B (ed), Literacies Across Educational Contexts, pp235-260 Caslon
Publishing, Philadelphia.
Tardy, C.M (2005) “Its like a story’: Rhetorical knowledge development in advanced academic literacy.” Journal of
English for Academic Purposes Vol 4, no 4 pp 325-38.
Traxler, J. (2008) 'Podcasting in Context', in Salmon G and Edirisingha P (Eds), Podcasting for Learning in
Universities, pp 12-19. Open University Press, Maidenhead.
775

Post-Academic Masters Course in Management of
Transfusion Medicine: Why the Difference in Access to the
eLearning Between Countries?
Cees Th. Smit Sibinga
ID Consulting for International Development of Transfusion Medicine (IDTM),
University of Groningen, The Netherlands
c.sibinga@planet.nl
Abstract: Health care includes supportive services such as laboratory, radiology and blood transfusion. Blood
safety and sustainability of the blood supply is increasingly organized on a WHO advocated nationally supported
principle where regional blood procurement centres supply hospitals. To manage such regional or national blood
supply centres, leadership development is paramount. Since 2004, WHO has initiated a specific post-academic
Masters course focused on management of Transfusion Medicine (MMTM). This MMTM course is largely based
on eLearning (distance learning principle). Approach – Since the eLearning course became operational in 2006
there have been registered two dozens of qualified fellows from a variety of developing countries and a few more
advanced countries. Fellows are provided personal electronic instructions how to access once all criteria
including acceptance of a proper post-academic dissertation proposal have been met. The access codes are
personal and not public and need to be archived during the course to guarantee a continued access over time.
Fellows were followed during the eLearning period and their progress monitored and evaluated. Specific attention
was given to the ease of handling the e-environment and its related e-technicalities. Fundamental are access to
internet, uninterrupted power supply, consistence of use of a computer (PC or laptop) and computer literacy.
Results –Most of the fellows come from developing parts of the world, predominantly sub-Saharan Africa. There
are distinct differences in country infrastructure and e-environment. Although academically qualified (in-country
University diploma’s) a majority is not familiar with e-technology and computer handling other than some internet
exploring and basic office functions like Word and PPT. Accessibility awareness is not really developed. Major
obstacles encountered are – inconsistent internet access due to supplier problems; unreliable power supply due
to poor and incompetent infrastructure; virus contamination of lap tops and PCs due to frequent uncontrolled use
of memory sticks in internet café’s and through friends and relatives; poor and not maintained firewall conditions;
mediocre computer literacy, particularly when skills beyond basic office functions are required; lap top and PC
breakdown due to uncontrolled working conditions and improper working environments; Conclusions – As fellows
are scattered around the world, and live in developing societies with a limited e- and ICT teaching infrastructure,
it would be appropriate to include in the eLearning package an instructive e-module on how to handle and
manage the eLearning tools, how to manage day-to-day problems of access and downloading, as well as reaccess
for e.g. e-exams and access to new modules in the course. Such instructive e-module would contribute to
a better accessibility awareness leading to a more easy and customer friendly e-access.
Keywords: postgraduate masters, eLearning, evaluation
1. Introduction
Globally, blood safety and sustainability of the blood supply is increasingly organized on a WHO [1]
and WHA [2] advocated nationally supported principle where regional blood procurement centres
supply hospitals. The provision of safe and efficacious blood components is a supportive service.
Health care includes such supportive services e.g. laboratory, radiology and blood transfusion. These
supportive services are an essential part of the health care system and need a careful and balanced
integration in the health care system. As they operate in the demand-supply chain, proper and
intellectually well-equipped leadership is needed to adequately manage these services. The
internationally advocated organizational structure is based on the principles of product liability
(procurement of blood – collection, processing and testing, storage and distribution) and consumer
rights protection (rational use of the products produced and supplied by the blood procurement
organization or blood centres).
Despite all international efforts advocated by WHO through a series of WHA resolutions, most parts of
the world the service offered by blood centres is still supply driven. These parts of the world are
characterized by the UN classification of Low and Medium Human Development Index (L-HDI and M-
HDI) and consist of over 60% of the countries where more than 80% of the global population lives and
roughly 20% of the global blood supply is being consumed [3]. In fact the actual clinical needs should
be met in a demand driven way to avoid shortages and logistic problems of supply and allow the
development of rational and appropriate use of supportive haemotherapy.
776

Cees Th. Smit Sibinga
To manage such regional or national blood supply centres, leadership development is paramount.
Since 2000, WHO has initiated a specific post-academic Masters course focused on management of
Transfusion Medicine (MMTM). This initiative has been materialized through the creation of a special
Academic Institute for International Development of Transfusion Medicine (IDTM) at the University of
Groningen (www.idtm.org). The contents of the post-academic modular Masters course is built up of
three clusters – two theoretical clusters with 9 e-modules or e-books and one combined cluster of a
practical (observation) real time exposure and a theoretical part with 5 face to face tutorial modules.
Following validation of both the electronic architecture (e-Academy and e-library) and the postacademic
level of the e-book contents, the MMTM course became operational by 2006. [4].
2. Methods
Since the post-academic eLearning course became operational in 2006 there have been registered
two dozens of qualified fellows from a variety of developing countries, largely Africa and a few more
advanced countries such as Singapore. Entrance criteria are – academically qualified (completed a
university education, MSc), affinity to and experience in the field of transfusion medicine, formal
support or recommendation from regional or national blood supply organisation, motivation and
leadership capacity and last but not least having paid the MMTM course tuition fee. Once all criteria
including acceptance of a proper post-academic dissertation proposal have been met, fellows are
provided personal electronic instructions how to access. The access codes are personal and not
public and need to be archived during the course to guarantee a continued access over time.
Monitoring and Evaluation - Those who were provided access were followed during the eLearning
period and their progress monitored and evaluated. Specific attention is given to the ease of handling
the e-environment (internet access) and its related e-technicalities (hardware and software).
Fundamental are access to internet, uninterrupted power supply, consistence of use of a computer
(PC or laptop) and computer literacy.
The encountered problems were documented and analysed for commonalities and opportunities for
improvement of the customer or user friendliness of the programme.
3. Results
Most of the fellows come from developing parts of the worlds, predominantly sub-Saharan Africa.
There are distinct differences in country infrastructure and e-environment which contribute to the
access and operational continuity of an eLearning based programme. Although academically qualified
(in-country University diploma’s) a majority is not familiar with e-technology and computer handling
other than some internet exploring and basic office functions like Word and PPT. Besides we
experienced differences in personal attitudes and culture once the eLearning has started [5].
Hardware used is not always guaranteed and in many a situation of second or even more hand
nature. The eLearning programme is not public but highly individualised with an electronic match
between the master system and e-academy in the Netherlands and the personal laptop or PC at
home. Changes in hardware need a renewal of the match to allow continuation of the access.
Accessibility awareness is not really developed [5] and major obstacles encountered over the past
five years are –
Inconsistent internet access due to supplier problems;
Unreliable power supply due to poor and incompetent infrastructure;
Virus contamination of laptops and PCs due to frequent uncontrolled use of memory sticks in
internet café’s and through friends and relatives;
Poor and not maintained firewall conditions;
Mediocre computer literacy, particularly when skills beyond basic office functions are required;
Laptop and PC breakdown due to uncontrolled working conditions and improper working
environments;
In different countries different grades of eLearning environmental and climate conditions were
observed, but these are essentially variations of the same theme. One element needs to be added
based on the observations, and that is the personal discipline of the fellows, caused by a variety of
conditions e.g. limited career perspectives, micro-economical situation (family income) and paucity of
academic interest and motivation.
777

4. Discussion and conclusion
Cees Th. Smit Sibinga
At the turn of the 20st century the need for an educational development programme for potential
managers and leadership in blood transfusion was recognized by WHO. The question asked was how
to create a specific curriculum that would provide both knowledge and managerial/leadership skills in
blood transfusion and would provide access to such potential all over the world without the need for a
longer absence from the base in the home country. Most countries were and are still in a transition
and have initiated developments at national level based on the WHO recommendations and the WHO
Essential Health Technology Department of Blood Safety Strategic Plan 2000-2003 [6]. In such
situation identified and appointed competent leadership needs to be on the spot and not so much
send abroad for a period of 2 to 3 years.
The solution is in distance learning [7] and in particular in eLearning for the acquisition of the
necessary knowledge load, to be completed by a relatively short period of six months for real time
exposure to the practice of management and leadership in an appropriate and developed transfusion
medicine environment [4]. Such environments are to be found in the more advanced world. During the
same period a series of additional management related tutorials then could be provided through an
academic teaching institute in relation to the real time exposure.
4.1 Expectations
The MMTM eLearning curriculum has two clusters, one teaching the essentials of management
(organisation and structure, legislative and regulatory aspects, human resource management and the
finance and economical aspect) and a second cluster that focuses on the management of a larger
national blood supply structure (societal interface, clinical interface, process management and quality
management). In principle the modules would allow a 12 months study period [4]. However, due to a
series of conditions it so far has turned out to be substantially longer. The full course is concluded
with a thesis at post-academic level of contents and leads to the formal University of Groningen postacademic
Masters title MMTM – Master of Management of Transfusion Medicine.
The reasons analysed in this and the previous study [5] differ between countries and fellows.
Where the personal experiences with the eLearning part of the course relate largely to the level of
academic background, intellectual comprehension and attitude, there are more conditions that are
related to the societal infrastructure and its functioning.
4.2 Differences observed
The differences observed over the five year period can be classified into differences in academic level
of the fellows [5] and differences in environment and climate needed for a smooth and problem free
operation and performance of the eLearning part of the MMTM course.
4.2.1 Differences in academic level
As mentioned a prime entrance criteria is an academic level education to be evidenced with a
University Masters of Science diploma and the period over which the academic education has been
followed. All fellows have met this criteria. However, during the eLearning period is was observed that
the levels differ. Most of the fellows (e.g. Eritrea, Uganda, Tanzania, Zambia, but also Singapore)
have a difficulty with the post-academic level of the module contents and this is particularly
demonstrated in the level of comprehension of the modules on organisation and structure (module 1)
and on finance and economy (module 5). Also the module 8 on process management showed some
difficulties of comprehension at intellectual level. This was more prominent with those from the African
continent than with the fellow from Singapore.
Besides, the differences in levels come very clearly to live with the design of the thesis project, the
project proposal and the applied scientific work to be done for the thesis. Some are acceptably
familiar, but others show quite some difficulty in the design and writing of the proposal (e.g. Uganda,
Zambia, Tanzania). Apparently the academic education in these countries does not prepare for an
academic level attitude and curiosity, and does not foresee in basic intellectual training for scientific
work to be done.
778

Cees Th. Smit Sibinga
4.2.2 Differences in e-environment and e-climate
Despite the presence and availability of modern communication technology, there are differences in
accessibility of particularly internet due to inconsistent and unreliable providers of internet services.
Some countries (e.g. Eritrea) allow only one supplier, controlled through the authorities, other have
more local suppliers. As the services depend on power supply, this factor becomes a key in the
access to internet. With the exception of Singapore, the countries where most of the fellows live (Sub-
Sahara Africa and the Caribbean) experience regular and unpredictable power supply problems.
These power supply problems also are unpredictably long, sometimes even a full day or more, which
happens particularly in the Tropical raining Season. We have observed a variety of virus
contaminations which have affected the software and therefore the operations and performance of the
eLearning programme. Most of these contaminations are picked up when the laptop is used in
internet cafés or through exchange of USB memory sticks and diskettes. Most of these have been
observed in Uganda and Zambia, but also in Eritrea and Tanzania. Fellows lack the knowledge and
discipline about virus contamination prevention and elimination. USB memory sticks are just
exchanged without scanning for contamination. Most of the fellows do not have a fire wall system on
their laptop of PC. If so, there are scarcely updates done and regular scanning does not belong to the
cyber culture of most of the fellows, particularly in Africa and the Caribbean. Given the diversity of
questions asked to the help desk, there is a limited computer literacy with most of the fellows. The
laptop and/or PC have become part of modern life, but without sufficient knowledge of even the
basics of information technology, leaving them with poor to absent literacy of how to handle basic
computer programmes, e-mail and electronic documents. Another observation is in the quality and
operations of the hardware used. Most common laptops and PC are bought on the market without
reference and second or even more hand. Poor quality batteries, limited memory capacity and
technical failures, including breakdowns and crashes. Absence of repair and maintenance services
aggravate the situation as do limited working space. In one situation a laptop was placed in front of an
open window. When a sudden rain shower took place the laptop was soaked and evidently no longer
usable. In another situation fellows who had registered and accepted into the course, suddenly
claimed the right to be given for free a laptop and appropriate software.
4.3 Conclusion and recommendation
ELearning has become a common approach in teaching and training in many parts of the world.
However, there are still limitations of which some are difficult to influence and eliminate, because they
are an integral part of less developed societies in different parts of the world. As fellows are scattered
around the world, and live largely in developing societies with a limited e- and ICT teaching
infrastructure and culture, it would be appropriate to include in the eLearning package an instructive
e-module on how to handle and manage the eLearning tools, how to manage day-to-day problems of
access and downloading, as well as re-access for e.g. e-exams and access to new modules in the
course. Such instructive e-module would contribute to a better accessibility awareness leading to a
more easy and customer friendly e-access. It would also be appropriate to stimulate the development
of essential computer literacy among this group of academics that were not exposed during their
academic education at the respective universities or medical schools.
References
Blood Safety. Aide Mémoire for National Blood Programmes. WHO/BCT/02.03. Geneva, 2002.
Blood Safety and Clinical Technology. Strategy 2000-2003. WHO/BCT/01.01 Geneva, 2001.
Safe Blood and Blood Products: Establishing a distance learning programme in blood safety: a guide for
programme coordinators. WHO/BCT Geneva 2009.
Smit Sibinga CTh, de Gunst R. E-academy for international development of transfusion medicine – a unique
institution. Proceedings of the 7 th European Conference on eLearning, Remenyi D. ed. Academic Publ. Ltd,
Reading UK, 2008; volume 2:479-84.
Smit Sibinga CTh. Post-academic Masters in Management of Transfusion Medicine (MMTM); an evaluation of
the eLearning part of the course. Proceedings of the 9 th European Conference on eLearning, Paula
Escudeiro ed. Academic Publ. Ltd, Reading UK, 2010; volume 2:826-28.
WHA63.12 – Availability, safety and quality of blood products. WHA Resolution, Geneva 2010.
WHO/EHT/04.09 Global Database on Blood Safety Report 2001-2002. Geneva 2004.
779

Engagement With Students in ‘Middle Ground’: A Flexible
Learning Environment Allowing Simultaneous Access to
Social Networking Sites and Formal Academic Space
Anne Smith and Sonya Campbell
Glasgow Caledonian University, UK
anne.smith@gcu.ac.uk
sonya.campbell@gcu.ac.uk
Abstract: Content: The twenty first century student demands more from universities in terms of engagement that
is flexible, accessible and immediate. This means Universities revisiting their engagement agenda at a time when
financial constraints can least afford expensive technologies and resource dependent engagement solutions.
Solutions are likely to be varied however they must fundamentally deliver what students expect in terms of
engagement. Engagement requires a partnership between academe and student body, but often this relationship
is a tension between what universities want to deliver, and what students expect to receive. This complex
environment of constraint, tension and expectation means that solutions will be tested by both parties on those
variables. In pursuit of solutions it is presumed that there could be a ‘middle ground’ that would be acceptable to
both parties. The aim of this paper is to present the concept of ‘middle ground’ engagement, where parties
engage using a simple, cost effective and easily accessible communication tool. Middle ground is an emerging
concept informed by results from a study of student communication and interaction. It enables freedom of
movement for the user to communicate, engage and participate with others. The tool tested in the study is not a
formal learning space such as a VLE, or a branded social space such as facebook, but rather a flexible
environment allowing simultaneous access to social networking sites and formal academic space. The
subsequent challenge is to shape and roll out a communication tool that is ‘middle ground’.
Keywords: engagement, participation, formal/informal learning, social learning, collaborative learning, social
interaction
1. Introduction
It is hard to avoid the influence that Web 2.0. and social software has within the 21 st century (The
Department of Education, 2010, ESRC, 2011). The above terms are stated with regularity in all
spheres of our lives, and whether we choose to engage with them or not, they are now a major
feature of our world. To understand their impact, we must first understand what they actually mean.
The term Web 2.0, attributed to Tim O’Reilly in 2005, defined Web 2.0 as the network platform,
spanning all connecting devices; web 2.0 applications are those that will make the most of intrinsic
advantages of that platform. Social software on the other hand is seen as one of the applications
working with, and from the platform. Parameswaran and Whinston (2007) define social software as
“applications and services that facilitate collective action and social interaction online with rich
exchange of multi-media information and evolution of aggregate knowledge.”
Within Higher Education (HE) there has been a growing awareness that these technological advances
are having an impact on the way we teach and learn (Jones, Blackley, Ftizgibbon and Chew, 2010;
Selwyn, 2007). This paper will focus on understanding student’s experiences and perceptions of
social software in regard to its importance to their engagement with learning. Its aim is to provide
evidence to support the assertion that the boundaries between learning and social usage are
expanding, and that there is not, in fact, a need to make clear definitions between formal and informal
usage for social software to be used in an academic setting (Margaryan and Littlejohn, 2008). This
paper will review the theories and concepts that underpin understanding of Web 2.0 in learning
environments thus locating the emerging consensus. A methodology will present the methods
employed in the study before discussing the findings and the emergent concept. The paper concludes
with a clear trajectory for future research into this emergent area of study; space between formal and
informal learning.
2. Literature review
As a population of new learners are born and raised within this setting, the ‘net generation’ appears to
be distinct from previous generations in their abilities, expectations and motivations (Oblinger and
Oblinger, 2005; Prensky, 2001). Specifically, Prensky, (2001) considers the current generation to be
‘digital natives’ and the previous generations to be ‘digital immigrants’. Within this distinction is the
780

Anne Smith and Sonya Campbell
difference in the way learners will engage and move within the sphere of Web 2.0, the assumption
being that the natives will find it easier to adopt and use these technologies than the immigrants.
With an awareness that learner’s themselves are changing, there is a growing need to understand
how learning and teaching should change to support their needs (Schroeder et al 2010; Skiba and
Barton, 2006; Williams and Chinn, 2009). It appears traditional models which use ‘The TTT’ approach
(talk, text, and test) are not valued by the net generation (Oblinger and Oblinger, 2005). In contrast
the 21 st century learner is more likely to prefer group working rather than isolation and teaching
approaches that recognise the impact of social interaction and collaboration on learning appear to
more closely meet their needs (Beldarrain, 2006; Lave and Wenger, 1991). Students who are the net
generation exhibit behaviours that desire constant connectivity, convenience, collaboration and
information rich access (Paul, 2001; Weiler, 2004). This behaviour is embedded in the business world
and according to Friedrich, Peterson, and Koster, (2011) will impact on the future business
environment, how we do business and how the economics of business will be supported by
heightened levels of connectivity.
As we attempt to gain adoption over the emergent technologies inspired by Web.2.0 concepts, there
is a desire in pedagogical circles to understand and identify relevant themes and rationales in relation
to their impact on learning and teaching (Conole, de Laat, Dillon and Darby, 2007; Minocha, 2009a).
Although research into the impact of social software and Web 2.0 is increasing, it is still very much in
its infancy (Selwyn, 2007). There is consensus however in the evidence of tensions, functionality and
applicability linked to its uses and applications within Higher Education (Cole, 2008, Margaryan and
Littlejohn, 2008; Williams and Chinn, 2009). Further commonality is emerging through research into
the use of social software in relation to collaborative learning opportunities without the need for face to
face contact (Belderrain, 2006). In a review of the role of social software in education, Minocha
(2009a) highlighted the view that although the tools provided opportunities for group learning, the
need to share and collaborate brought with it additional responsibility and workload which some
students found inflexible and forced. Other areas of student concern raised were usability and the
distinction between privacy and public nature.
In summary, Minocha (2009a) argues that as there is an increasing adoption of social networking
tools, this has led to a need to investigate the reasons for adoption of the tools and the benefits and
challenges the students face. Barnes, Marrateo and Pixy Ferris (2007), cited in Williams and Chinn
(2009) found that students chose to use different tools to deliver a variety of different ends, and were
often multi-tasking within various roles at any given time. Within this research it was found that the
ability to relate to prior experiences led to positive interactions in learning environments. As students
already possessed certain levels of technology skills, potentially acquired whilst using the tools for
social means, as ‘digital natives’ they could then use these skills as a platform (Prensky, 2001).
Although it has been shown that students benefit in terms of technology skills from using tools in an
informal setting, there is research that points to their reluctance to use informal tools in which to
formally learn (Madge et al 2009, Jones et al 2010). Research conducted by Madge et al (2009) found
that 91% stated they never used Facebook (FB) for communication with university staff, and only 10%
used FB throughout the year to discuss academic work. There is some dubiety in the results which
shows that 46% used FB informally to discuss academic work on a daily/weekly basis, and 53% were
positive about FB being used in a formal but administrative way to support learning. The ways stated
were social/peer led academic support, possible revision opportunities and notification of changes to
lecture times. Their conclusion recognised that although FB was an important tool, caution needed to
be taken in the ways it was used as it was apparent that students felt this was their area for social,
rather academic purposes.
In a recent study, Jones et al (2010) reviewed multiple case studies pertaining to the student
experience of social software from an educational perspective. They describe the perceived
differences as “Learning is a painful process where as social life is pleasure to many students.” There
was a perceived conflict in using social software as students felt it was important to separate life and
study spaces. However, rather than insist that two distinct spaces be created for students, Jones et al
created the ‘Continuum of socio-learning divide’ see Figure 1. They argued that although there were
separate domains within the continuum, at any given point these might overlap to address the needs
of both the institution and individual leaning preferences.
781

Anne Smith and Sonya Campbell
x y
completely separation selectively separation/ completely combination
of life & studying combination of life & studying of life & studying
studying
Figure 1: Continuum of socio-learning divide”
It is within this area of selective separation that this study looks to achieve a better understanding of
learner preference and usage. The study attempts to identify and explore what is “middle ground”. In
summary the literature shows that although there has been some adoption in formal contexts of Web
2.0 there continues to be a high level of social networking sites being used in learning contexts. The
literature shows that a level of confusion still exists in relation to the questions ‘how do students want
to connect during their engagement with university life?’ and ‘what tools would they prefer to use?’.
Universities are challenged with the need to provide contemporary learning solutions to a generation
with high levels of connectivity, and high levels of choice. This study captures evidence of that need
as well as a solution that might contribute towards the levels and type of connectivity desired. The
following section details the methodology applied in the study.
3. Methodology
Mixed methods were applied in this study, specifically, an action research approach over a 4 week
period which involved testing an application supplied by ‘Youthwire’ in conjunction with semi
structured questionnaires investigating communication and interaction (Borrego, Douglas, Amelink,
2009; Bryant, 2007; Hohenthal, 2006). Mixed methods have been criticised essentially because of
contradictions created in terms of philosophies and approaches relating to analytical validity (Borrego
et al, 2009). In this case however, mixed methods were considered highly appropriate in order to
triangulate the results (Borrego et al 2009). The students, n=8, were organised into two groups of 4
and all were asked to complete a semi structured questionnaire on communication and interaction at
university. These students were enrolled on a 4 th year undergraduate BA (Hons) Management,
Technology and Enterprise programme. The nature of their programme meant that they were
technology users but not specialists. Except for one student all were under twenty five years of age.
Students joined a 4 week block of testing using a communication tool supplied by ‘Youthwire’. A
second group of students n=13, were then recruited for a repeat block of testing. Unfortunately group
2 were less engaged and the results were unsatisfactory due to coincidental timing of assessment
deadlines. Nonetheless, sufficient data for the purposes of the study was generated from group 1
enabling the study to progress. During the test with group 1, students met every week in labs to use
the communication tool in a group project environment. The following section presents and discusses
the findings from group I.
4. Findings and discussion
This section will firstly introduce the communication tool before providing an overview of the action
research findings. The section proceeds with an insight to current usage by students of social
networking before capturing student voices through statements of their experiences and thoughts
towards social media and formal learning; these sections are structured to capture perspectives on
student led connectivity and then VLE led connectivity. Finally, the findings and discussion inform the
conceptualisation of ‘middle ground’.
4.1 The communication tool
The communication tool being tested to inform the conceptualisation of middle ground was provided
under licence by ‘Youthwire’. The tool can be installed on PC’s and appears as a visual on lab
screens in the university. Visually, the tool appears on the home screen as a rectangle in the top
corner of the screen with app style function buttons. The ‘apps’ offer an array of university services as
well as access to FB and similar sites. Students can use this tool by registering on line. The research
team were using this product to test a new ‘app’ style function for ‘middle ground’. Table 1 below
presents findings from the action research tests. During the 4 week period student needs were
recorded through discussions with staff and observations. Discussions with students and developers
at ‘Youthwire’ created potential solutions.
782

Table 1: Action research findings
Anne Smith and Sonya Campbell
Action Research Findings
The student brief was to undertake a small research project that involved collecting information on a new
business opportunity for a recycling company. They were instructed to collect data in a group and share their
findings using the new ‘app’ style function provided by ‘Youthwire’. They were required to do this in a lab
environment meeting once a week for two hours over a four week period.
Student Needs Solutions
Students want intuitive tools that look like social software. A simpler way of registering and accessing
friends (i.e. the group) was requested by
students.
Students use chat functions continuously whilst working in Switch between Chat ‘app’ and Share ‘app’
labs with each other.
Students share snippets of information such as tables, videos,
images, websites and so on rather that completed documents
and larger swathes of work
Students will not use a tool that does not offer immediate
solutions i.e. any bugs in a system will disengage individuals.
Students want control over who they work with, what they
share and with whom they share.
Students while working on one assignment will switch directly
into another; coursework multi tasking.
Students surfing the net identify relevant information for a
variety of coursework resulting in them wanting to share with
multiple individuals on separate group pieces.
Multi tasking options required on main
screen. Refresh issues to be resolved
Design required to function more intuitively
Selection and filtering required on selection
of group members that can include
academics
Document multitasking required
Share options required for multi groups and
multi shares of information to single users.
In summary, students desire a ‘middle ground’ communication tool that controls who they share with,
extensive multi-tasking features for sharing and selection and communication filtering. Literature
provides a consensus that student’s desire high level connectivity but the applications and tools are
contested as is the space in which they occupy (Madge et al 2009, Jones et al 2010). The following
sections present and discuss the results from the semi structured questionnaires; first social
networking usage by students is established, second student led connectivity and third academic VLE
space and connectivity.
4.2 Social networking usage
Students were asked about their social networking usage and the majority demonstrated that they
accessed a site at least ‘once a day’. There were other responses that indicated their usage to be
more than once a day. Students in the study clearly recognised social websites as being designed for
social purposes, stating that they used them ‘to keep up-to-date and in touch with friends and family’.
Students were also aware of the interaction that the platforms offered and commented that they often
were able to ‘interact with friends and family’ (Madge et al 2009). However, it is also notable from
Table 2 that students had specific ‘likes’ about social networking and were prepared to concede
‘dislikes’.
Table 2: Social networking - student likes and dislikes
Likes Dislikes
Easy to access
Easy to use
Multiple uses – upload photos, play games, create
events etc
User friendly
Free communication tool
Convenience
Extended networks – keeping in touch
Security issues
Confidentiality issues
Usability issues - slow
Difficult to manage who sees what
These initial comments capture the net generation views and the consensus that they desire
convenience, connectivity, sharing and interaction (Paul, 2001; Weiler, 2004). These initial responses
show that social interaction using social tools is embedded in everyday life of the sample and the
trends for internet and social website usage depict continuing growth for the future (Mori, 2007).
783

4.3 Student led connectivity
Anne Smith and Sonya Campbell
Once the overall understanding of usage had been established the views sought required a
contextual change. The student sample was asked to consider social networking, interaction and
communication in the university and learning context; where social ties might be new and or
developed through common ground at university either through study or social commonalities. The
space for networking begins to expand as students appreciate the unique mix of friendships and
learning. Students were asked if they ever used their personal networking sites to facilitate any group
communication about coursework. Replies included:
‘Yes, for group work and to get advice on homework from other students.’ ‘Yes to discuss
coursework or course content, to arrange meetings, to share ideas, to catch up on work
missed in class.’
The social confusion with formal learning space becomes evident. Students do use social networking
sites for group work and communication. Interaction is important to the net generation and the
cognitive processes are being mixed with communication tools resulting in what might be considered
a confused and dynamic state between social and formal learning (Minocha 2009).
Given that boundaries are being stretched, expanded and crossed, and that industry demands
collaborative working practices (Cole, 2009; Engestrom, 2008; Margaryan and Littlejohn 2008;
Williams and Chinn, 2009), students were asked to comment on what they considered to be the ‘best’
ways to communicate and interact at university.
‘Something like Facebook where you can restrict certain people seeing what you are
doing and you can only add the people who you wish to add. This would be great as we
could all communicate about coursework without thinking what others would think of our
work’
‘Also students can discuss their group work on Facebook and I see a lot of students
logged onto their Facebook account while doing their work but then some students may
not be happy with this idea as they wouldn’t wish to add some people whom they don’t
know.’
Other solutions were more pragmatic:
‘I think the best way to contact someone is through maybe email or text message’
In summary, students show a need for connectivity to support learning and interaction. The study
group clearly see the benefits of social networking sites as providing the connectivity solution, but
equally the provision through existing social media contains several compromises involving personal
space. The next section considers the routes to connectivity through academically driven Virtual
Learning Environments or VLE’s.
4.4 Academic VLE led connectivity
The idea that there is a need for ‘middle ground’ in student learning is supported through responses to
questions about formal and academic led University spaces such as Virtual Learning Environments
(VLE’s). Students were asked about their usage:
‘..only when necessary, so very little’.
Clearly a VLE formal space is perceived to offer an unfavourable space.
‘Don’t like the fact that there is so much going on in it. Would rather just receive e-mails
from lecturers and have an online library. Sometimes it runs quite slowly’,
The functionality of VLE’s is also a negative for students.
‘The layout is something I don’t like and also how I have problems accessing modules I
should be attached to’.
The VLE is designed to be academic led, but this is a different usage from the concept of student led
connectivity. Students were asked about connectivity with academic staff. One student responded:
‘if the lecturer can see and contribute then they can ensure that work is on track. Also,
this ensures the lecturer can see that equal contribution is being made or that students
are recognised for the contribution which they do make’.
784

Anne Smith and Sonya Campbell
The opportunity to ‘free-ride,’ the lack of activity on the part of other students has been expressed as
a concern in other recent studies. (Minocha, 2009a, Minocha, 2009b). This view of being connected
with an academic through VLE or otherwise was supported by others:
‘I don’t mind as they could possibly direct us in the right way if we are doing something
wrong’ and ‘it will be helpful when groups are having difficulties, a lecturer would help
keep on track of contributions’.
Students were also able to address the sharing and contribution requirements on formal group
learning tasks and a student/academic connectivity was important for equality and fairness:
‘The fact that they can see who is and isn’t pulling their weight..’
Nonetheless, there were negative views about connectivity with academics:
‘Don’t really like the idea of the lecturers “listening in” to our discussions’ and ‘May feel
like my every action is being watched’.
In summary, students are seeking opportunities to chat and share but with selected stakeholders only.
Unfortunately the existing dislike of academic VLE led approaches move student learning towards
social media usage and away from an academically crafted learning space which should provide
value.
4.5 Conceptualising middle ground
It appears that connectivity should be a solution that allows self management on the part of the
student in conjunction with value laden academic learning requirements. This is perhaps ‘middle
ground,’ connectivity using a communication tool to support a mix of academic learning; intellectual,
cognitive and social interaction with selected peers and academics. This concept is distinguished from
the social space supported by social networking sites that is to ‘interact with friends and family.’
‘(its) Not that important as there are benefits for the lecturer being able to see the work,
and if there was something that I didn’t want the lecturer to see then I would use another
communication tool to discuss this’.
‘It would be preferred as a group communication but then again having lecturers seeing
our work could also be an advantage’.
‘One member of the group can post a comment which everyone can then see and
comment on, and also because it is online, group work can then be done at any time
without needing to meet up outside of class time’ and ‘It means that we can all talk
together at once without having to arrange any meetings and we can be clear about what
we want each other to do because there is not limit to how long the conversation can go
on for’.
‘……….students may not be as willing to contribute to discussions if they know that the
lecturer is watching’
‘……..it would be about the assignment and not a personal space.’
The requirements for student connectivity in academic contexts, creates the concept of ‘middle
ground’ see Figure 2. This requirement includes communication for group working remotely, sharing
images & videos, documents and links whilst chatting (Belderrain, 2006). Middle ground is depicted in
the diagram below showing a space that exists between formal leaning and social networking.
Essentially, ‘middle ground’ has been explored through results from an action research study and
student comments retrieved from semi structured questionnaires. The overall narrative has been rich
enough to reveal a particular view of communication and interaction in university and is shaping what
might be solutions to the contested space between social and formal learning. The step forward using
results from the action research test is to shape a tool that has the capability to provide the desired
level of student connectivity in this space. As yet academic views on ‘middle ground’ have to be
gathered and usage tested from an academic perspective. The authors accept that this current
limitation means that ‘middle ground’ can only be achieved where there is a true nexus between
student needs and those required by academics for effective learning.
785

Figure 2: Conceptualising ‘middle ground’
5. Conclusion
Anne Smith and Sonya Campbell
The paper has focussed on a contested space between formal and informal learning. The view is that
this space is not best serviced by either social networking sites or VLE’s and that there is a ‘middle
ground’. The paper has approached the problem firstly by attempting to better understand the 21 st
century student, their learning and connectivity needs. Supported by literature there is consensus that
student connectivity is highly important and levels are only likely to increase with demand for more
convenience and more accessibility. The tensions arise when more formal academic activities seep
into social spaces and personal networking sites. Students recognise this as being a problem.
Therefore this paper concludes with the concept of a student perspective on what ‘middle ground’ is
and a specification for requirements of a new tool that is able to meet the connectivity requirements in
‘middle ground’ space; the requirement essentially includes communication for remote academic
group working, sharing images & videos, documents and links whilst chatting (Belderrain, 2006).
Future developments in this study will involve the rollout to a larger community of students, of a low
cost communication tool designed to support connectivity in ‘middle ground’. Thereafter academic
views will be captured in a test study in order to develop a better understanding of how academics
embed the tool in formal learning contexts.
Acknowledgements
‘Youthwire’ is an application owned by Youth Media (UK) and we wish to acknowledge their
collaborative efforts during this study. ‘Youthwire’ are currently developing the application and retain
ownership of all product specification.
"Campus Service Manager (CSM) and its associated software applications including all copyright and
other intellectual property rights therein is the sole and exclusive property of Youth Media (UK)
Limited and is used under licence by Glasgow Caledonian University.”
Authors would like to thank Dr Keith Halcro and Dr Peter Duncan for various contributions during the
study.
References
Beldarrain, Y. (2006), "Distance Education Trends: Integrating New Technologies to Foster Student Interaction
and Collaboration", Distance Education, Vol. 27, No. 2, pp. 139-153.
Blackey, H., Jones, N., Chew, E. and Fitzgibbon, K. (2010) "Get Out of MySpace", Computers & Education, Vol.
54, No. 3, pp. 776-782.
Borrego, M., Douglas, E. and Amelink, C. (2009) “Quantitative, Qualitative, and Mixed Research
Methods in Engineering Education”, Journal of Engineering Education, Vol. 98, No. 1, pp.53–66.
Bryant, P. (2007) "Self-regulation and Decision Heuristics in Entrepreneurial Opportunity Evaluation and
Exploitation", Management Decision, Vol. 45, No. 4, pp.732 – 748.
Cole, M. (2008) "Using Wiki Technology to Support Student Engagement: Lessons from the Trenches",
Computers & Education, Vol. 52, No. 1, pp. 141-146.
786

Anne Smith and Sonya Campbell
Conole, G., de Laat, M., Dillon, T. and Darby, J. (2007) "'Disruptive Technologies', 'Pedagogical Innovation':
What's New? Findings from an In-depth Study of Students' Use and Perception of Technology", Computers
& Education, Vol. 50, No. 2, pp. 511-524.
Engestrom, Y. (2008) From Teams to Knots: Activity–Theoretical Studies of Collaboration and Learning at Work,
Cambridge University Press.
ESRC (2011) Shaping Society, accessed 4 th April 2011, (online), http://www.esrc.ac.uk/news-andevents/news/14950/working-together-to-deliver-our-strategic-goals.aspx
Friedrich, R., Peterson, M. and Koster, A. (2011) “The Rise of Generation C How to Prepare for the Connected
Generation’s Transformation of the Consumer and Business Landscape”, Strategy and Business, Booz &
Company Inc., New York, Issue 62, Spring.
Hohenthal, J. (2006) “Integrating Qualitative and Quantitative Methods in Research on International
Entrepreneurship”, Journal of International Entrepreneurship, Vol. 4, No. 4, pp. 175-190.
Jones, N., Blackey, H., Fitzgibbon, K. and Chew, E. (2010) "Get out of MySpace", Computers & Education, Vol.
54, No. 3, pp. 776-782.
Lave, J. and Wenger, E. (1991) Situated Learning. Cambridge: Cambridge University Press.
Madge, C., Hooley, T., Wellens, J. and Meek, J. (2009) "Facebook, Social Integration and Informal Learning at
University: 'It is More for Socialising and Talking to Friends About Work than for Actually Doing Work'",
Learning, Media and Technology, Vol. 34, No. 2, pp. 141-155.
Margaryan, A. and Littlejohn, A. (2008) “Are Digital Natives a Myth or Reality?: Students’ use of Technologies for
Learning” Caledonian Academy, Glasgow Caledonian University, UK. This is a final draft, shared under
Creative Commons Attribution-Non-Commercial-Share Alike 3.0 Unported Licence December 11, 2008
Mori, I. ( 2007) Student expectations Survey 200. Coventry: Joint Information Systems Committee. Retrieved
August 15 th , 2011, from http://www.jisc.ac.uk/media/documents/publications/studentexpectations.pdf
Minocha, S. (2009a) "Role of Social Software Tools in Education: A Literature Review", Education & Training,
Vol. 51, No. 56, pp. 353-369.
Minocha, S. (2009b), "A Case Study-based Investigation of Students' Experiences with Social Software Tools",
New Review of Hypermedia and Multimedia, Vol. 15, No. 3, pp. 245-265.
Oblinger, D. and Oblinger, J. (2005) “Is it Age or IT: First Steps Towards Understanding the Net Generation” in
Oblinger and Oblinger (Eds.), Educating the Net Generation (section 2.3). Boulder, CO: EDUCAUSE.
Retrieved August 15, 2011, from http://www.educause.edu/educatingthenetgen.
O'Reilly, T. (2005) What is web 2.0? Retrieved August 15 2011, from http://oreilly.com/web2/archive/what-is-web-
20.html.
Paul, P. (2001) “Getting Inside Gen Y”, American Demographics, Vol. 23, No. 9, pp.42-49.
Parameswaran, M. and Whinston, A. (2007) "Social Computing: An Overview," Communications of the
Association for Information Systems: Vol. 19, Article 37.
Prensky, M. (2001) “Digital Natives, Digital Immigrants”, On the Horizon, NCB University Press, Vol. 9, No. 5,
October, retrieved PDF http://www.albertomattiacci.it/docs/did/Digital_Natives_Digital_Immigrants.pdf.
Schroeder, A., Schneider, C. and Minocha, S. (2010) "The Strengths, Weaknesses, Opportunities and Threats of
Using Social Software in Higher and Further Education Teaching and Learning", Journal of Computer
Assisted Learning, Vol. 26, No. 3, pp. 159-174.
Selwyn, N. (2008) "An Investigation of Differences in Undergraduates' Academic Use of the Internet", Active
Learning in Higher Education, Vol. 9, No. 1, pp. 11-22.
Skiba, D. and Barton, A. (2006) "Adapting Your Teaching to Accommodate the Net Generation of Learners",
Online Journal of Issues in Nursing, Vol. 11, No. 2, Manuscript 4.
Weiler, A. (2004) “Information-seeking Behaviour in Generation Y Students: Motivation, Critical Thinking and
Learning”, The Journal of Academic Librarianship, Vol. 31, No. 1 pp. 46-53.
Williams, J. and Chinn, S.J. (2009) "Using Web 2.0 to Support the Active Learning Experience", Journal of
Information Systems Education, Vol. 20, No. 2, pp. 165-174.
The Department of Education (2010), The Importance of Teaching, The Schools White Paper, accessed 11 th
March (online) http://www.education.gov.uk/publications/eOrderingDownload/CM-7980.pdf
787

The Learning Management System as a Social Mediator: A
Story With a Happy Ending
Dina Soeiro, António Dias de Figueiredo and Joaquim Armando Gomes
Ferreira
University of Coimbra, Coimbra, Portugal
soeiro@dei.uc.pt
adf@dei.uc.pt
jferreira@fpce.uc.pt
Abstract: Learning is social and mediated, as argued by social constructivism. When learning groups gather very
different people, this difference can be challenging. We had an Interpersonal Relationships course, a large class,
around fifty students working collaboratively in groups where students from different degrees, academic years
and ages, some of them deaf, tried, and to some extent were able, to communicate. We analyze this example of
how diversity can be an asset and Moodle can act as a mediator. We were carrying out a participatory action
research project within a blended learning environment supported by Moodle to develop collaborative and
personal pedagogical strategies to improve the inclusion and engagement of higher education students in their
own learning and evaluation. We have used content analysis of the online discussions held by the students,
reflective descriptions of the classes, the students’ e-portfolios, and interviews with the students. The paper
describes, from this project, the challenges and potential of using Moodle in a learning context where deaf
students interact with hearing students and illustrates how Moodle can facilitate inclusion and the participation of
the students.
Keywords: blended-learning, diversity, higher education, inclusion, participation, social constructivism
1. Introduction
Inclusion and the participation of the students is a central issue in the present and future of higher
education. Within a participatory action research project, we explore collaborative and personal
pedagogical participatory strategies to improve inclusion and the engagement of higher education
students in their own learning and evaluation in a blended learning environment supported by Moodle.
Using the students’ discourse, we will present a real case that shows how different people can
interact and learn while using a learning management system in a b-learning context. We intend to
critically analyze this example of how diversity can be an asset and Moodle can act a mediator. To
this end, we will resort to content analysis of online discussions, reflective descriptions of the classes,
students’ e-portfolios, and interviews with the students.
About three hundred and eighty students have been involved in the study, which developed
throughout the academic years 2008/09, 2009/10 and 2010/11, covering a heterogeneous population
from the first to the senior year, taking nine different subjects, in twelve degrees, at the Polytechnic
College where we teach. This population ranged from young full-time students to mature students
working full-time, some of them deaf, and covered a diversity that illustrates the richness of new adult
publics in higher education and creates new challenges in the academic contexts. As Light and Cox
(2001) explain, for students who have just joined the university, the academic situation is typically new
and strange, and its languages and practices frequently unfamiliar. Their encounter with higher
education and learning is not simply a cognitive or intellectual grappling with new ideas, concepts and
frameworks, but also a personal and emotional engagement with the new situation. Applying
complexity theory to this reflection, we share with Davis and Summara (2010) the perspective that we
need to understand learning events in terms of co-participation, co-emergence, and co-implication,
and see classrooms as knowledge spaces producing networks rather that contexts that are teacher or
learner-centered. New media contribute to enhance these changes in the role of teachers and
students. In the past, the students used to be mere consumers, but now they are increasingly
becoming producers, in the sense that their participation is facilitated and encouraged in blendedlearning
environments.
2. Methodology
We have followed a participatory action research approach, essentially qualitative, based on content
analysis carried out on the online discussions, reflective descriptions and videos of the classes,
students’ e-portfolios, and interviews with the students.
788

Dina Soeiro et al.
The protocol for the semi-structured interviews was tested previously with a few students. For the
deaf students, besides the pre-test, we have enlisted the support of a sign language interpreter, the
same professional who worked with the classes. The interview protocol integrated and adapted the
questions according to the development of the action research cycles. The topics and issues of the
interview were the first category framework for content analysis, which was enriched with the
emerging categories that resulted from a comparative analysis of the data (Bogdan and Biklen 1994,
Strauss and Corbin 1998, Richards 2005, Cohen, Manion and Morrison 2007, Creswell 2008).
In agreement with our qualitative intention, we have used “purposeful sampling” (Creswell 2008, p.
214). After a preliminary exploratory content analysis, we have intentionally selected for deep analysis
the materials that we felt significant for the study. This decision was inevitable given the huge volume
of data obtained. To facilitate and support the analysis we have resorted to NVivo.
Figure 1 illustrates the major initial interpretative categories of the content analysis tree.
Figure 1: Major initial interpretative categories of the content analysis tree
To strengthen validity, we have relied on the diversity of the participants and contexts, the duration of
the study, and an attitude of critical reflection, as well as on the triangulation of multiple methods,
multiple data, multiple sources, and multiple theories. With the same aim, we have carried out a
review and verification of the written information and shared interpretations with the participants.
Besides this, we have resorted to a research critical friend (Messner and Rauch 1995), who will
conduct an external audit (Creswell 2008) after we complete the whole process of data collection, at
the end of this academic year.
3. Inclusion and participation of the students through Moodle
Learning is social and mediated, as argued by social constructivism (Vygotsky 1978, 1997).
Participation, engagement, and collaboration are, on the other hand, processes that characterize
learning communities (Wenger 1998). With this in mind, we have invited the students to participate
democratically in the management of the courses. Departing from the skills, goals, and contents
originally established for the course syllabus, the students have been invited to build and develop
shared and negotiated learning projects. This included defining collaboratively the learning and
evaluation activities, strategies, processes, and products, while negotiating the corresponding
deadlines.
Each student clarified in his learning contract his intended participation in the development of the
projects, and his portfolio described and reflected about his learning process, the collective learning
process, and the quality of the course. When learning groups are heterogeneous, gathering very
different people, this difference can be very challenging, as the following true story, spoken in the
words of their protagonists, tells us.
We had an Interpersonal Relationships course, a large class, around fifty students working
collaboratively in groups where students from different degrees, academic years, and ages, some of
them deaf, tried, and to some extent were able, to communicate. We give the word to the students.
Interview, 2008/9, 1, CDM, RI, Francisco, 25 years – “The dynamics was good, we were
always helping each other (…) this seemed to be deliberate, hearing students had to
communicate with deaf students without knowing sign language. In my group there were
two hearers and four deaf.”
It was deliberate, but it wasn’t easy, as we can illustrate with the following case of two very different
students who tried, struggled and succeeded.
789

Dina Soeiro et al.
He was older, old enough to be her grandpa. “Sir”, as she called him! He was the leader of an
association for the deaf. In fact, “he is deaf, but he doesn’t listen!” she said. “Everything has to be as
he wants!” They had to work together, but they didn’t listen to each other. “It was not because he is
deaf, but because he doesn’t listen with his heart!” she cried frustrated. And he became worried about
her.
Away from the crowd, from the noise of the class, the learning management system, Moodle, was the
neutral space where it was easier to communicate, more thoughtfully.
Interview, 2008/9, 1, LGP, RI, Ana, 19 years – “The course was interesting, but I didn’t
know we were going to have deaf colleagues in the class. The thing that struck me most
was the argument with Belchior. We think we are doing good things, but the other person
thinks we are doing everything wrong. I was really upset. (…) Crying in the train. (…)
Then I went home, thinking about the situation. How I’m going to explain my point of
view? Then I realized I could use Moodle, send him a message. I wrote him, and I was
lucky. (…) We solved the problem in Moodle. (…) In the heat of the moment we are
dealing with emotions. We are anxious, we have the boundary between deaf and hearer.
In Moodle, it’s different. We sit, think, and think again before we write a message, and
the other does the same thing. Things were clarified, everything turned ok. We created a
bond. I call him grandpa, he is the class grandpa. Now we communicate well. I’m the
only one with this relationship with him. This happened to me. It was terrible at the time,
but now it is very positive, because we have a bond. Now it is funny, we laugh about it. I
grew up.”
Interview, 2008/9, 1.º, LGP, RI, Belchior, 58 years – “I loved to work with her, really. In
the beginning she had inflexible ideas. I punched the table. I was wrong to be that
impulsive. She left crying.”
Ana explains in her portfolio:
Porftolio, 2008/9, 1, LGP, RI, Ana, 19 years – “There was a conflict in my learning group.
It was a consequence of two different ways of communication, it was very intense. For
me, a hearer, the world of the deaf was different and I wasn’t used to it. That reflected on
the frustrated attempted to try to enter it. It is a friendship that started badly, but now it
gives me great joy to think of all we have overcome. We ended up knowing each other,
gradually understanding our worlds. It was very difficult but it was worth it. I take from this
very important memories and lessons for life, for my growth as a person.”
Belchior also learned with the situation. In his portfolio he reflected about his leadership of the group,
based on his conviction that because he was more experienced and old, things needed to be done as
he though they should be. And at first he didn’t feel right the decision of the group to work with shared
leadership.
Portfolio, 2008/9, 1, LGP, RI, Belchior, 58 years – “The stubbornness and inexperience
of youth will lead to failure. Because I’m more experienced and old, I could impose a
rigorous way of doing things. But the group chose shared leadership, so the youngest
have to learn, as I learned, with their own mistakes. It’s the best way to learn”.
In the Moodle forum, talking about interpersonal communication, Belchior wrote: “here I confess my
guilt, I admit to have lost my self control. To communicate is not just to talk. The attitude and
understanding allied to a affirmative dialog, and above all, to be able to “listen”, are vital conditions to
communicate.”
He realized the “girl” had an important contribution to share and that perhaps her ideas were not to be
rejected. She acknowledged that his life experience was useful, but he also heard that his dominating
and paternalistic attitude would not work.
After a constructive dialogue, they used shared leadership as a way to overcome the conflict and
difficulties. With this group decision, they made a pact to make an effort to communicate assertively
and to respect each other ideas. Their commitment was visible and fruitful.
Report work group, 2008/9, 1, LGP, RI, Ana, Belchior… - “The group has been led by
Belchior, because he is the oldest and more experienced of the group. We discussed
and worked using Moodle. That was an innovation for us. We have decided that for the
790

Dina Soeiro et al.
future the leadership will be shared, to include everyone and to be coherent with the
course goals. (…) Good communication is possible. If we want, we can.”
Interview, 2008/9, 1, LGP, RI, Belchior, 58 years – “I am 58 years old, but all the ideas
she shared with me, I accepted. There was communication. I learned a lot from a girl
aged 19. Among young people, I feel young like them. They treat me like one of them. In
certain things, I must be the youngest of them all. They may have 20 years, they can be
my grandchildren. Last year they called me grandpa.”
The interaction between young and older students was important for mutual learning and to share
energy. But in this case, the age was not the issue. The emotions, the previous experiences and the
expectations the students had played an important role.
Interview, 2010, 2, 3, ASE ARP and EST, Carla, 46 years – “I was afraid of failure. It is a
bit difficult, it involves lots of feelings. (…) In the first year, people are gaining confidence
again, after those years without being a student. (…) I was afraid, at the beginning of the
course, to work with young students. I felt I wouldn’t be accepted. (…) They could think I
had outdated ideas. I was completely wrong! I felt they always liked to work with me. I
never felt rejected, on the contrary, there were situations where they invited me to work
with them. So they believed I had skills and knowledge. Once, one of the colleagues
asked me my age, I told her. She said: “It’s the age of my mother. I can’t imagine my
mother doing this work with me”. (…) My life context is different, but I could understand
them. I felt I was a mother figure (…) I was wanted in the class. I was not a strange
element, which was my fear: I was loved. They wanted me to meet their mothers. They
admired me because I was, at this age, studying in higher education, it was even an
encouragement to their parents.”
Moodle forum, 2008/9, 1, LGP, RI, Ana, 19 years – Our class is very heterogeneous, not
to mention the age differences that bring different mindsets and perspectives of life.
The differences were opportunities to explore and work with the students, starting from their
acknowledgment of their own goals.
Moodle forum, 2008/9, 1, LGP, RI, Ana, 19 years – I’m afraid I’m not a person who is
very much at ease to expose what I feel and want. I prefer someone to take the first step,
because I fear the reactions of others to my opinions (sometimes, even, stutter). (…)
What I need to accomplish in this course is to be able to communicate mine, fight for
them, but also respect the others’ opinions, because I’m stubborn.”
She accomplished her goal, because she was the one who took the first step to solve the conflict, and
Belchior acknowledged that and praises her.
Moodle forum, 2008/9, 1, LGP, RI, Belchior, 58 years – “Dear Ana. I’m thinking about
your words here in the forum. Congratulations, you are gradually coming to understand
how a deaf person feels in daily life. One of the things I learned at the university of life,
about interpersonal relationships, is that constructive dialog and good will can solve
everything.”
Moodle forum, 2008/9, 1, LGP, RI, Belchior, 58 years –“ I have followed your evolution
since the first class. You have developed an autonomy that you didn’t have in the first
days. I’m pleased to see that you can defend your opinion, fight for it when you think it is
fair and consistent. It’s been a pleasure in my life experience to see you grow. Keep it up,
I will help you.”
Ana replies: “Dear Belchior, you don’t know how happy I am with your message! The
beginning was hard, different mentality, life experiences! It is important to communicate
here. I hope to learn more with you and in the future laugh about our beginning.
Respectful kisses.”
Moodle forum, 2008/9, 1, LGP, RI, Belchior, 58 years – “I already laugh a lot thinking of
the early days. I’m used to these confrontations in daily life. I talk, listen to the others’
opinions, but in the end I do what I had in mind since the beginning! Ehehehe! But your
enthusiasm, joy, participation in the group is changing my mind, and I now can leave to
you most of the responsibility and leadership: so you can learn more, because you are
the future and I’m a relic. And to see the results, I think it’s worth: congratulations. I leave
richer in terms of maturity, with a different vision about interpersonal relationships,
791

Dina Soeiro et al.
specially about the interaction between two distant and different worlds, but
simultaneously, so close and similar.”
The participation of the students creates problems because conflict is inherent to democracy. But
open dialog and conflict took students to their “zone of proximal development” (Vygotsky 1978), with
the students managing their conflicts, with no direct action from the teacher. The teacher only opened
space for dialog, online or in person. Interestingly, the conflicts at the collective level were usually
discussed and solved in classroom, while the conflicts with one or two students were overcome with
messages in Moodle, to avoid confrontation face-to-face.
Most of the deaf students were not comfortable participating in Moodle, because of their poor writing
skills. They did not want to expose themselves to teachers and colleagues (whether deaf or listeners).
They would rather communicate only by sign language. To answer this need, teachers should
improve the use of video, but not indulging in doing without the essential exercise of writing.
We cannot avoid the exercise of writing because it is essential in our life and in our personal
relationships. We could not drop it. We had to include it in the evaluation, even more because they
had difficulties in that respect. Even inside the deaf community, as they call themselves, there are the
born deaf, the deaf who became deaf, the deaf who use a device to hear. A differentiation exists
between them. Those who write correctly are those who could hear at a time in their lives, particularly
if they learned to read and write before they became deaf. The others who cannot write well don’t
want to expose their writing.
Despite this difficulties, students recognize the utility that Moodle has to communication and learning
in a heterogeneous group.
Porftolio, 2008/9, 1, LGP, RI, Ana, 19 years – With easy access, Moodle greatly
facilitated communication between colleagues and access to information on interesting
topics promoting discussion. For me and for many colleagues this was a revolutionary
suggestion. It was a new experience, where I obtained useful information that enabled a
cultural enrichment and promoted greater acceptance of the others’ opinions, allowing
good communication between everyone. In fact, our space has eliminated
communication barriers between students, because in Moodle we could communicate in
various ways. It was not just a course, it was a dynamics, using new technologies for
personal development.”
Porftolio, 2008/9, 1, LGP, RI, Ana, 19 years – “For those who felt uneasy about the world
of silence, I learned a lot from the development of the activities and dialogues that have
been achieved through Moodle.”
Moodle forum, 2008/9, 1, LGP, RI, Belchior, 58 years – “Through Moodle, deaf and
hearers maintained a positive dialogue that they didn’t have in the face-to-face classes.”
Flexibility, autonomy and sense of community have been promoted by blended-learning strategies. In
Moodle they had their own space, without the need to rush, so they had the time to reflect,
communicate, discuss: time that was insufficient in face-to-face classes. There were too many
students in a two-hour class, per week, to work this subject. Diana, another working student,
confessed in the Moodle forum: “I wish it was Thursday. I long for our class.” While they were waiting
for the class, they could be at Moodle, and they liked it.
There is a sense of belonging to the learning community, especially in the working students, that is
fostered by participation through the Moodle. The role of technology is important to promote
democratic participation and, as the working student Carla says: “we are connected to the class and
the teacher”.
4. Conclusions
Rather than to prove anything, this particular narrative is intended for the reader to learn and get
inspired. As Friesen (2008) argues, the knowledge that can be derived from a particular narrative for
research and learning in eLearning is situated, practical, and in some ways, personal. Research into
the affective and experiential aspects of eLearning is growing (Pachler and Daly 2011) and the
qualitative approach can be useful to research this highly complex issues.
792

Dina Soeiro et al.
The paper describes the challenges and potential of the use of Moodle as a learning management
system in a learning context where deaf students interact with hearing students and young and older
students try to learn together. This study illustrates how Moodle, as a learning management system,
can facilitate inclusion and the participation of the students in a democratic context. It is also important
and effective for working students. On the other hand, deaf students who did not want to expose
themselves could benefit much more from adequate pedagogical strategies and tools. The students
worked together, managed to find ways to communicate and, through that effort, they learned and
developed themselves. Despite the difficulty of the process, the story has a happy ending. Everyone
lives learning through life and understands the potential of learning together.
References
Bogdan, R. and Biklen, S. (1994) Investigação Qualitativa em Educação: Uma Introdução à Teoria e aos
Métodos, Porto Editora, Porto.
Cohen, L., Manion, L. and Morrison, K. (2007) Research Methods in Education. New York: Routledge.
Creswell, J. (2008) Educational research. Planning, conducting and evaluating quantitative and qualitative
research, Pearson International Edition, New Jersey.
Davis, B. and Sumara, D. (2010) ‘If things were simple…’: complexity in education. Journal of Evaluation in
Clinical Practice, [online], http://brentdaviscalgary.appspot.com/articles.htm.
Friesen, N. (2008) “Chronicles of change: The narrative turn and eLearning research”, ELearning and Digital
Media, Vol 5, no. 3, pp 297-309.
Light, G. and Cox, R. (2001) Learning and Teaching in Higher Education: the reflective professional, Sage,
London.
Messner, E. and Rauch, F. (1995) “Dilemmas of facilitating action research”, Educational Action Research, Vol 3,
no. 1, pp 41-53.
Pachler, N. and Daly, C. (2011) Key Issues in eLearning: Research and Practice, Continuum, London.
Richards, L. (2005) Handling qualitative data: a practical guide, Sage, London.
Vygostky, L. (1978) Mind in Society: Development of higher psychological processes, Harvard University Press,
Cambridge.
Vygotky, L. (1997) Educational Psychology, St. Lucie Press, Boca Raton.
Wenger, E. (1998) Communities of Practice: Learning, Meaning and Identity, Cambridge University Press, New
York.
793

Can the Medium Extend the Message? Using Technology
to Support and Enhance Feedback Practices
Mekala Soosay
Leeds Metropolitan University, Leeds, UK
m.soosay@leedsmet.ac.uk
Abstract: The research reported in this paper investigates the use of technology-supported feedback that
enhances student understanding/learning. According to the UK National Student Survey (NSS) which gives final
year undergraduates the chance to reflect on their course and to have their views heard, students are markedly
less positive about feedback on their assessments than about other aspects of their learning experience. Thus,
there is strong rationale to provide more effective feedback that enhances understanding and learning by
exploiting the potential benefits that Technology Enhanced Learning (TEL) has to offer, whilst reducing the
burden on tutors as being the sole providers of feedback (Sadler 2010). These issues are addressed in this
research, which is a small-scale evaluation project using a mixture of quantitative and qualitative methods,
exploring the role of technology in the process of giving and receiving feedback on the final year BSc. (Hons)
Computing programme at Leeds Metropolitan University (Leeds Met). The primary data is obtained from a mixed
method approach using questionnaires and interviews. The research findings suggest that although tutors apply
a variety of feedback mechanisms dependent upon the nature of the learning, teaching and assessment (LTA)
design of their modules, students do not hold a uniform view of what effective feedback means and how it could
be used to enhance their understanding and learning. It is also found that students perceive feedback as being
useful when it is mediated through a guided dialogic process where a common consensus is more likely to be
arrived at. It is interesting to note that characteristics of effective feedback in face-to-face delivery are not
diminished in blended learning delivery. The results have implications for increasing awareness in students on
how to recognise what constitutes feedback, as well as how to use it. The results also support the evidence that
when technologies that maximise dialogue and learning as shared discussion of tasks are used appropriately, the
emphasis shifts from delivering instruction to producing learning. Further, the findings propose that in supporting
time-starved tutors who are under pressure to provide effective feedback, the pedagogic opportunities that
technology affords can be suitably harnessed for collaborative learning, in particular self and peer feedback
provision. Within Higher Education (HE) curricula, this translates as an opportunity to promote self-regulation
through independence and personal ownership of learning, increasing students’ ability to self-assess and selfcorrect,
skills which form the hallmarks of undergraduate education. The paper concludes with recommendations
to influence existing feedback practices through technology-supported activities to benefit students’ learning
experiences.
Keywords: technology-enhanced feedback, exemplars, peer feedback, self-regulation
1. Introduction
Ensuring that all students receive helpful and timely feedback is one of the top priorities in Leeds
Met’s LTA Strategy for 2008-12. In committing to improve the NSS scores, opportunities for ongoing
formative feedback are provided at regular intervals throughout the academic year. The institution
also encourages a greater use of TEL to support students. Driven by a combination of academic
ownership and enthusiasm, and the growing demand and expectations of students, TEL is embedded
in all modules through the institutional Blackboard Virtual Learning Environment (VLE), known as Xstream.
It has been recognised that staff development activities contextualised to subject area
requirements are more appropriate in providing targeted feedback. As Knight and Yorke (2003) argue,
this is critical to successful knowledge acquisition and subsequent employability amongst students.
This paper explores the current perception on feedback within the context of UK Higher Education, in
particular the final year BSc. (Hons) Computing programme where students have experienced
feedback throughout the past 3 years in a blended-learning environment. The following research
questions are addressed in this paper:
How can technology enhance or replace the methods of feedback?
What are the preferred methods of giving and receiving feedback amongst staff and students
respectively?
What is the relationship between students’ preferences amongst different methods for providing
feedback and their effectiveness in enhancing student understanding/learning?
794

2. Literature review
Mekala Soosay
In keeping within the scope of this evaluation study, the literature reviewed was ascertained by
looking at the current feedback practices within the BSc. (Hons) Computing programme, and how it is
generally received and used by students, in relation to the three research questions. It involved
exploring the processes, tools and resources affecting the provision and assimilation of effective
feedback, in terms of closing the gap between the perceptions of tutors and students in the final year.
Over the past 10 years, tutors are responding to greater student numbers and lower staff-student
ratios (Bolton 2011), and this has increased the burden on tutors with respect to effective feedback
provision. The appraisal also revealed ways in which technology-enhanced feedback can be
effectively used to support self-regulation and peer feedback appropriate for the academic level of
students, relieving tutors as the sole provider of feedback.
Bangert-Drowns, Kulick and Morgan (1991) assert that the most useful type of feedback is the
provision of specific corrective advice about errors and improvement, as it encourages students’
attention to focus on the task rather than being fixated on getting the ‘right answer’. Further, Fritz et al.
(2000) demonstrate that this process can promote acquisition of new information. Sadler (1981), Boud
(1995) and Yorke (2003) advocate using a method of explicitly demonstrating what constitutes good
quality work in the subject area, known as ‘exemplars’. Sadler (2010) argues that when students
actively participate in learning activities using exemplars to allow them to make evaluative judgements
about their own work and the work of their peers, they engage in role-play similar to that of their
tutors. These arguments build a strong case for self and peer feedback, mediated by technology. In
addition, they have implications on the second and third research question, as to whether students
inevitably know which method(s) of feedback work best for them in terms of enhancing their
understanding or learning of a subject.
In relation to the first research question, Mayes and Fowler (in Martin and Madigan 2006) argue how
technology can provide affordances for students to observe a variety of interactions between tutor and
students. Participation in asynchronous and synchronous discussions is more enduring than face-toface
interactions, and educational technology affords ways in which feedback is provided between
tutor(s) and students. Garrison and Anderson (2003) expand that tutors need to set the climate of
portraying a strong teaching presence, ranging from being a facilitator to a direct instructor or neutrally
as a moderator. The process establishes credibility and shifts responsibility to students by assuming a
teaching presence, and becoming more self-directed. Here, the influential 5-stage model by Salmon
(2004) offers a conceptual framework for guiding student learning progression as they participate
collaboratively online and construct their learning. Additionally, tutors require skills in reconciling
different perspectives of contributors to achieve the learning outcomes.
Feedback using atypical forms of media such as audio or video can potentially cater for different
learning styles possessed by students. When taken further, web conferencing tools promoting
synchronous audio and video-based interactions can facilitate effective, personalised and immediate
communication among students, akin to face-to-face interactions (Offir, Lev & Bezalel 2008). Bangert
(2008) concurs when tutors use constructivist-compatible practices in the design and delivery of
learning, the diagnostic nature of feedback can be used to improve the quality of educational
experiences for their students in an online environment.
In summary, effective feedback provision requires constructing the design of the student learning
experience that drives active engagement and motivation amongst them. The educational affordances
of asynchronous and synchronous technologies offer opportunities for students to become selfdirected
learners in a supportive environment. The specific aim is to encourage students to clarify,
articulate and achieve learning iteratively and develop a support network, whilst tutors actively
encourage students to create their own learning environment. Furthermore, in developing autonomy
in learning, exemplars can be utilised to effectively guide students to take on a teaching presence.
3. Methodology
The primary research utilises 2 questionnaires administered face-to-face, where all 10 semester one
tutors and 44 of their students voluntarily expressed their level of agreement to a number of
statements in a five-point Likert scale. This was immediately followed up with interviews where
qualitative data was gained by using open-ended questions to further clarify responses. This
triangulation approach was used in order to capture a more holistic and contextual portrayal of the
795

Mekala Soosay
elements under study, demanded by the nature of the research questions. More complex questions
were posed to participants and clarified by the interviewer, which otherwise may have been neglected
by single methods, leading to greater depth, vividness and clarity of meaning that enhanced the
quality of data.
4. Research findings and discussion
4.1 Student responses
The questions were framed either in simple Yes/No format or used the Likert scale where 1=strongly
disagree, 5=strongly agree on a scale of 1-5. Question 1 was aimed at students only, whereas the
rest of the 5 questions were directed to both tutors and students.
Table 1: Question 1
Please rate the following sentences on how strongly you agree or disagree with them. (1-5)
Answer Options Percentage Agreeing or Strongly
Agreeing
I believe that tutor/peer feedback is an essential part of my
learning. 95%
I read the feedback received from my tutor(s). 95%
I understand the language used in the feedback. 81%
The feedback I received was in time to apply to related
assessment. 75%
My tutor(s) prepared me in using the feedback effectively. 66%
The feedback clarified things I did not understand. 66%
The feedback helped me improve my assessment (formative
feedback). 74%
Overall, the feedback helped improve my ways of learning. 69%
Students agreed with most of the statements; however 25% of them remained neutral to the
statement ‘My tutor(s) prepared me in using the feedback effectively’, and 20% of students to the
statement ‘The feedback clarified things I did not understand’. These suggest that though a majority of
students understand the language used in the feedback and that their tutor(s) prepared them in using
the feedback effectively, they might find feedback received difficult to apply to task improvement and
progression. 23% of students were indifferent to the statement ‘Overall, the feedback helped improve
my ways of learning’, adding that they were not aware that discussions around assessment tasks
during practical sessions constituted feedback too, implying that they need to extend their ability in
recognising what feedback actually means.
There is some evidence to suggest that students could not see the feed-forward nature of feedback
as a number of them have indicated that they would prefer clearer guidance on how summative
feedback can influence subsequent modules studied. This also implies that generic feedback requires
clarification by tutors, but perhaps only to students who would demand for detailed feedback.
5. Combined tutor and student responses
The following 5 questions were directed at both tutors and students, and the findings were structured
to facilitate comparison between both sets of responses.
The responses to Question 2 (Table 2) mostly reflect consistency between the types of feedback
offered by tutors and what students generally agree as useful feedback in terms of ‘location of where
the problem lies’, ‘summarisation of issues’, and ‘problem explicitly identified’. On the other hand,
although students find feedback in terms of ‘solution explicitly offered or identified’ and ‘solution
sufficiently explained’ useful, tutors do not tend to favour them highly. They wish to see students
actively acting upon the feedback and coming up with diverse solutions, as there is no one right
answer. One tutor aptly put it, ‘the problem explained can be misconstrued’, suggesting that a bidirectional
or dialogical process is necessary for clarification of feedback to occur (Nicol 2010).
In general, students appreciated annotated work returned to them, as there is a clear indication of
where and how they were incorrect in their assessed tasks. As one student put it, ‘I prefer finding the
solutions from the feedback received because it feels good when I can finally figure it out’. This
indicates that when students feel a sense of achievement, it would boost their confidence in selfassessing
their own work. It is worth noting that the relatively low use by tutors of offering, identifying
796

Mekala Soosay
or even explaining the solution to students also indicates that they prefer feedback to be feed-forward
or formative in nature. In other words, feedback offered sooner (75% of students agreed that
feedback received was timely), encourages it to be applied for improvement of subsequent work.
Table 2: Responses to question 2
Feedback Types Percentage of students agreeing or
strongly agreeing with the statement: I find
the following types of feedback useful on
my module.
Percentage of tutors agreeing or strongly
agreeing with the statement: I offer the
following types of feedback on my
module.
(1-5).
Summarisation of
issues. 71% 90%
Location of where
the problem lies. 81% 100%
Problem explicitly
identified. 70% 70%
Solution explicitly
offered or
identified. 70% 10%
Problem
sufficiently
explained. 75% 60%
Solution sufficiently
explained. 75% 40%
In general, both tutors and students favour individual verbal and written feedback based on the
marking scheme, combined with other methods depending on the work and assessment set by tutors
(Question 3, Table 3). A number of students commented positively on the nature by which written
feedback was communicated to them, explaining that tutors put more care and effort into crafting and
conveying useful feedback. This suggests the influence that both teaching presence and a high
degree of interactivity have on student satisfaction of feedback, indicating that technology that affords
dialogue can be used for more effective feedback provision. Many students also find the provision of
model answers useful, only when combined with individual feedback.
Table 3: Responses to Question 3
(*Tutors were asked to leave blank the forms of feedback they did not use)
Feedback Methods Percentage of students
agreeing or strongly agreeing
with the statement: I find the
following methods of feedback
useful.
Percentage of tutors agreeing or strongly
agreeing with the statement: I find the
following forms of feedback support students
in improving their understanding of the
subject. (1-5).
Individual written
feedback. 85% 57%
Individual verbal
feedback. 81% 100%
Generic feedback in
taught sessions. 54% 50%
Generic feedback posted
to X-stream. 45% 40%
Peer feedback (i.e.
discussion posts) for
groups/individual on Xstream.
34% 67%
Audio/Video feedback
posted to X-stream. 34% 100%
Feedback based on
marking scheme – i.e.
related to assessment
criteria. 81% 100%
Model answers provided
to assessment questions. 65% 50%
Marks only. 20% *0%
Two out of three tutors using asynchronous discussions said that the technology afforded the
development of students’ ability in constructing feedback, therefore reinforcing the ability in students
797

Mekala Soosay
to self and peer-assess. Nicol and Macfarlane-Dick (2006) argue that this type of feedback may be
particularly helpful to lower-achieving students because it shifts the focus onto assisting them to
improve as a result of effort, rather than drawing attention to the perceived lack of ability. Typical
student comments on the tutor and peer feedback as discussion postings were:
‘It helped me define the way of learning and what was expected in terms of structure to
the assignment’.
‘…. when I did ask the answer was given directly just to put me on the right track’.
The potential benefits afforded by peer feedback through discussion boards are not immediately
understood by students, as only 34% of students found this method to be useful. A way of overcoming
these issues might be to devise strategies that guide and engage students in critical construction of
knowledge and make them explicit to students as Palloff and Pratt (2007) assert.
Three tutors who have used audio feedback, mainly to provide generic feedback rated the medium
highly, commenting on how the tone of voice can be used to motivate students and that the feedback
is accessible. Trials by Rotheram (2008) giving audio feedback have found that motivation is achieved
more easily than written feedback alone. However students commented on issues in streaming the
audio and were not appreciative of generic audio feedback. Nicol (2010) argues that there is immense
potential to exploit this form of feedback; however both tutors and students need to get accustomed to
providing and receiving feedback in audio or video format. X-stream allows the embedding of various
feedback e-tools, listed in Table 4, and Question 4 tries to capture the desired use of feedback e-tools
by students vs. tutors’ rating of the effectiveness of e-tools that they use. The most popular e-tool is
the provision of individual feedback as comments in the assignment drop box or feedback attached as
a file of written comments. Students have remarked how these are easily accessed, read and
assimilated by them. Tutors rate these e-tools highly, as they facilitate listing of comments against
criteria and are accessible. Sadler (2010) and Nicol (2010) find that explicitly mapping feedback with
the student’s work makes feedback less ambiguous or abstract to them. Typical comments from
students relating to preferred X-stream feedback opportunities were:
Table 4: Responses to question 4
(*Tutors were asked to leave blank the e-tools they did not use)
Feedback
Opportunities
Percentage of students agreeing or
strongly agreeing with the statement:
I would like to have the following
feedback opportunities included on Xstream.
Percentage of tutors agreeing or strongly
agreeing with the statement: Please rate the
following X-stream e-tools that you use, in terms
of effectiveness of meeting your requirements of
feedback provision. (1-5).
Multiple Choice
Questions
(MCQs) /
Quizzes. 46% 67%
Assignments
Drop Box. 75% 83%
Assignments
Drop Box with
Plagiarism
Detection. 64% 67%
Discussion
Boards. 60% 66%
Journals or
Blogs. 44% *0%
Audio Podcast
Creation Tools. 28% *0%
Video Podcast
Creation Tools. 31% *0%
Rubrics Marking
and feedback. 51% 57%
Written
Comments
feedback. 85% 80%
Peer Review. 43% 33%
Synchronous
Chat. 33% *0%
798

Mekala Soosay
‘It makes more sense to me to have a mark breakdown’.
‘Comments against marking criteria or report would be useful’.
A fairly high percentage of students and tutors who have used asynchronous discussions found them
to be useful in providing supplementary feedback to extend students’ understanding. Tutors who
provide audio/video feedback prefer using external e-tools to X-stream e-tools.
Responses to Question 5 (Table 5) suggest that students appreciate provision of feedback using
audio/video depending on the nature of assessment, although Table 3 suggests that they also prefer
individual instead of generic feedback, which includes audio/video media.
Table 5: Responses to question 5
Answer Options Percentage of students agreeing with
the statement: I would like to see
technology being used to provide
feedback (e.g. audio/video feedback)
for the following reason(s):
Percentage of tutors agreeing with
the statement: I would like to use
technology to provide feedback (e.g.
audio/video feedback) for the
following reason(s):
Feedback can be received
within a reasonable time. 90% 70%
Feedback can be useful in
helping me (students) to
understand where I (they)
have made mistakes in my
(their) task(s). 88% 70%
Feedback can be useful in
helping me (students) to
correct any mistakes in my
(their) task(s). 90% 70%
A majority of tutors emphasised the value of engaging students in a one-to-one dialogue when
discussing formative feedback, where students’ attention can be drawn to the exact problem area,
offering remedial actions. Most importantly, a relationship can be developed while trying to ascertain
the individual learning requirements, as several students remarked:
‘A personal conversation about the strengths and weaknesses explained works as I
would be clear on how to correct my work’.
‘The feedback received during a meeting with my tutor is much easier for interpretation’.
Given that staff and students have expressed the need for ongoing conversation, the educational
affordances offered by both synchronous and asynchronous technologies could be exploited to meet
this requirement.
Tutors are discouraged from using audio or video for feedback as it takes time to craft and record
feedback, and send it individually to students. As one tutor remarked ‘the effort put into producing
audio or video feedback does not justify its’ usefulness over and above face-to-face feedback’. A
small percentage of students who are direct entrants to the final-year mostly disagreed with the idea
of technology being used to provide feedback. Further probing suggests that if they had been
exposed to a variety of technology-enhanced feedback methods in their early undergraduate years,
they would be better placed to accept more innovative feedback methods using audio or video. This is
contradictory to the claim by Prensky (2001), who suggests that the current generation of students
whom he refers to as digital natives, are more exposed to technology than ever and therefore more
receptive to its’ use. However, there is a distinction between familiarity with the technology and an
innate ability to do something purposeful with it, and here, innovative feedback means may not be
fully appreciated by students as explicit and purposeful.
Question 6 (Table 6) tries to ascertain tutors’ and students’ perceptions of whether technology could
be used to either enhance or replace face-to-face feedback. Furthermore, it also attempts to establish
if the format of the feedback makes a difference in the appropriateness and usefulness of content that
is conveyed. Most tutors echoed the sentiment that it is time-consuming to provide clear written
feedback based on marking criteria, which a fair number of students did not turn up to collect. They
added that feedback that is useful in helping students understand where they have made mistakes in
their tasks depends more on content and less on technology. The findings indicate the need for
799

Mekala Soosay
flexible web-based approaches that are capable of responding to changes in social and institutional
priorities which engage and connect students.
Table 6: Responses to question 6
Answer Options Percentage of students agreeing
or strongly agreeing with the
statements.
Percentage of tutors agreeing or
strongly agreeing with the
statements. (1-5).
I believe that technology could be
used to enhance face-to-face
feedback. 72% 80%
I believe that technology could be
used to replace face-to-face
feedback. 25% 20%
The format of the feedback does not
matter as long as the content is
appropriate and useful. 48% 40%
Although a majority of tutors and students believe that technology could be used to enhance face-toface
feedback rather than replace it, some of them have specifically noted that synchronous web
conferencing can afford feedback opportunities similar to individual discussion between tutor and
student as the channels of communication in face-to-face interactions are preserved, such as facial
cues and tone of voice.
6. Conclusions and recommendations
The research suggests that effective feedback depends upon a range of factors, and is difficult to
measure accurately. However the research also suggests that the findings can potentially help inform
actions that can be implemented to address key concerns that it highlighted surrounding feedback,
which includes inconsistencies, quality, and the methods of communicating feedback. In bridging the
gap between current and good performance amongst students, they need to be sufficiently coached
on how to convert tutor comments into improvements using the standards that tutors routinely apply
when composing feedback (Sadler 2010).
Students and tutors expressed heterogeneous views about feedback within the course, having a
range of preferred methods for giving and receiving feedback. Although students prefer specific
methods of feedback, it does not necessarily mean that the feedback would work for them. When
students do collect feedback, they appreciate an ongoing dialogue of clarification with the tutor with
whom they feel familiar, as they perceive that tutors are interested in the work that they have
produced, and that care has been taken to provide feedback. Tutors who supported face-to-face
teaching using TEL have reaped considerable amount of benefits from flexibility in feedback
provision. The requirements ascertained from this evaluation study indicate that technology can help
increase both the accessibility of feedback and extend recognition of what feedback means, allowing
for visible and continuous feedback.
Most tutors expressed keenness on using technology-led mechanisms that would allow for the
provision of effective feedback, as long as they can be produced within the time-constraints, while
juggling teaching, management and research responsibilities. A key argument put forward by Nicol
and Macfarlane-Dick (2006) is that students are already assessing their own work and generating
their own feedback and that HE should build on this ability. Although technology can shape learning
activity and outcomes, it cannot ascertain the nature or success of that learning, and such
implementation requires careful evaluation. Setting up VLE-based or web-based asynchronous
discussions could encourage peer marking and feedback by scrutinising a range of exemplary work,
in response to the findings. Currently, the Elluminate web conferencing software is being trialled as a
more flexible medium for synchronous peer/tutor feedback on assessment tasks, offering students a
virtual group study space. Reushle and Loch (2008) advise that staff training is vital in the technical
aspects of the synchronous tools, as well as pedagogical approaches to using them. Though peer
assessment and feedback methods are pedagogically important, pivotal in this development is the
influence of the facilitating tutor (Salmon 2004) in promoting on-line engagement and independence.
As it is a novel process for most students who have been reliant on working alone so far, they may
lack confidence accepting their peers’ opinions. Further, there may be fears around plagiarism,
privacy and data protection (Franklin and van Harmelen 2007). Scaffolded delivery of TEL introduced
from the first undergraduate year would support a model for collaboration. If implemented properly,
800

Mekala Soosay
this vicarious form of feedback also encourages students to put what is learned to action and to
develop capability in complex appraisal of assessment, similar to what tutors possess (Sadler 2010).
Although this research is a small-scale evaluation and the findings are not necessarily generalisable,
the potential benefits of realigning feedback mechanisms with technology that encourages selfregulated
learning can be applied to other similar undergraduate and postgraduate learning settings.
Granted that composing individual feedback is a time-consuming process, formative peer feedback
implemented using technology can allow tutors to benefit from empowering students as feedbackconstructors,
potentially saving time. Tutors can also identify which students require more feedback
and target additional guidance towards them. This implies that detailed feedback for summative work
could be provided on demand, for diagnostic or feed-forward purposes. While it is acknowledged that
a single feedback technique would not work equally well for every student, a common theme
emerging from the findings is that heightened awareness by tutors of how students implement
feedback could positively influence how tutors prepare students to apply feedback for accelerated
learning and progression. Moreover, Sadler (2010) and Boud (2000) concur if the skills of selfregulation
are developed progressively rather than mechanistically over the course of an
undergraduate degree, this would support a model of HE where students are prepared for lifelong
learning.
It is anticipated that the findings will help develop and improve student experience in that area, guided
by good practice that tutors have found to work well in their teaching.
References
Bangert-Drowns, R.L., Kulick, J.A., and Morgan, M.T. (1991) “The instructional effect of feedback in test-like
events”, Review of Educational Research, Vol 61, No. 2, pp 213 - 238.
Bangert, A. W. (2008) “The Development and Validation of the Student Evaluation of Online Teaching
Effectiveness”, Computers in the Schools, Vol 25, No. 1, pp 25 – 47.
Bolton, P. (2011) “Changes to higher education funding and student support from 2012/13”, [online],
www.parliament.uk/briefingpapers/commons/lib/research/.../snsg-05753.pdf.
Boud, D. (1995) Enhancing learning through self assessment, Kogan Page, London.
Boud, D. (2000) “Sustainable assessment: rethinking assessment for the learning society”, Studies in the
Continuing Education, Vol 22, No. 2, pp 151- 167.
Franklin, T., and van Harmelen, M. (2007) Web 2.0 for Content for Learning and Teaching in Higher Education,
Joint Information Systems Committee, London.
Fritz, C. O., Morris, P. E., Bjork, R. A., Gelman, R. and Wickens, T. D. (2000) “When further learning fails:
stability and change following repeated presentation of text”, British Journal of Psychology, Vol 91, pp 493 -
511.
Garrison, D. R. and Anderson, T. (2003) E-learning in the 21st century, RouteledgeFalmer, London.
Knight, P. and Yorke, M. (2003) Assessment, learning and employability, SRHE/Open University Press,
Maidenhead.
Martin, A. and Madigan, D. (eds.) (2006) Digital literacies for learning, Facet Publishing, London.
Nicol, D. and Macfarlane-Dick, D. (2006) “Formative assessment and self-regulated learning: A model and seven
principles of good feedback practice”, Studies in Higher Education, Vol 34, No. 1, pp 199 - 218.
Nicol, D. (2010) “From monologue to dialogue: improving written feedback processes in mass higher education”,
Assessment & Evaluation in Higher Education, Vol 35, No. 5, pp 501 - 517.
Offir, B., Lev, Y. and Bezalel, R. (2008) “Surface and deep learning processes in distance education:
Synchronous versus asynchronous systems”, Computers and Education, Vol 51, No. 3, pp 1172 - 1183.
Palloff, R. M., Pratt, K. (2007) Building Online Learning Communities: Effective strategies for the virtual
classroom, Jossey-Bass, San Francisco.
Prensky, M. (2001) “Digital Natives, Digital Immigrants, On the Horizon”, [online], http://tinyurl.com/ypgvf.
Reushle, S. and Loch, B. (2008) “Conducting a trial of web conferencing software: Why, how, and perceptions
from the Coalface”, Turkish Online Journal of Distance Education, Vol 9, No. 3, pp 19 - 28.
Rotheram, B. (2008) Towards quicker, better assessment using audio feedback, [online], CAA Conference 2008,
Loughborough University, www.caaconference.com/pastConferences/.../Rotherham_B_formatted_b2.pdf.
Sadler, D.R. (1981) “Emphatically, not to judge?”, Journal of Education for Teaching, Vol 7, No. 2, pp 200 - 202.
Sadler, D.R. (2010) “Beyond feedback: Developing student capability in complex appraisal”, Assessment &
Evaluation in Higher Education, Vol 35, No. 5, pp 535 - 550.
Salmon, G. (2004) E-Moderating: The Key to Learning and Teaching Online. 2 nd edition, RouteledgeFalmer,
London.
Yorke, M. (2003) “Formative assessment in higher education: Moves towards theory and the enhancement of
pedagogic practice”, Higher Education, Vol 45, No. 4, pp 477 - 501.
801

Implementation and Analysis of an Online, Student Centred
Learning Environment to Support Personalised Study
Iain Stewart, William McKee and Kevin Porteous
Glasgow Caledonian University, Glasgow, UK
i.f.stewart@gcu.ac.uk
w.a.mckee@gcu.ac.uk
Kevin.Porteous@gcu.ac.uk
Abstract: The majority of University teaching is still heavily predicated towards the use of the lectures (Lammers
& Murphy, 2002). While there are many issues with the lecture as an educational tool, lectures are still an
outcome-effective and a cost-effective way of presenting information, when it is delivered by subject experts in a
structured form which supports the linear development of concepts and promotes the depth of understanding
characteristic of this level of education (Heward 2003). However, the current generation of students display a
lifestyle in which they are used to accessing information in a variety of modes across a range of platforms, often
concurrently. (Kennedy et al, 2008). In particular they tend to prefer to work with small chunks of information and
to assimilate them by relating them to other pieces of information in their own time. The challenge faced by
educators is how to take advantage of the benefits of the traditional lecture approach while also gaining from the
advantages of Web 2.0 technologies where user-driven discussion, user generated content , and the capability to
take non linear routes through information and content can all contribute to the learning experience (Ravenscroft,
2009). While traditional Virtual Learning Environments such as Blackboard are moving towards this with the
incorporation of technologies such as Wikis and blogs, there is still a rigid structure to such systems which makes
it difficult to truly integrate these elements into the learning experience. The authors have worked for a number of
years in the area of lecture capture, both to support students with disabilities and also to provide rich learning
resources. Recent work has led to the development of a software prototype which has been applied to material
designed for and gathered in a classroom environment to produce web-based, self-guided learning products.
This prototype integrates a range of resources (captured lecture video, audio and presentation slides) but in
addition to simply presenting this content as a passive viewing experience, many other resources including
tutorials, FAQs and student feedback are used to drive the navigation through the content and to allow the
students to discover and develop linkages within the content. This paper describes the evolution of the
specification of the prototype, the development of the software and the feedback received from the students who
utilised it. A critical analysis of this data is presented and from this the specification of the next evolution of the
software is presented.
Keywords: lecture capture, personalised learning, student centred, Web 2.0
1. Background
The lecture is recognised to have a number of limitations when used in education; these include the
one off nature of the event, the often impersonal nature of the presentation, the passive nature of the
event and the way that the lecture environment can discourage questions (Williams and Fardon,
2007), (Black, 2005) There are also advantages to the use of lectures. When done well a lecture can
allow a subject expert to present content in a structured and logical manner which allows the
audience to follow the evolution of concepts and ideas. (Heward, 2003), (Dolnicar, 2005).
Other advantages of the use of lectures which these and other authors (Edwards et al. 2001), (Moore,
et al., 2008) identify include:
Ensuring that all students are presented with the same baseline content.
Allowing the lecturer to get instant feedback while presenting concepts and thus recognise areas
of difficulty and address them immediately.
Any questions raised are answered to the benefit of all of the attendees.
It is a cost effective way of providing access to subject experts
The structure of the lecture can allow the content to be developed in a logical and coherent
manner so as to form a structure for the student’s learning
The authors have been working in the university sector for a number of years and initially became
interested in the use of lecture capture as a means of improving the accessibility of the taught content
for deaf and hearing impaired students. This work resulted in a product called “Talkshow” (Figure 1)
which used voice recognition software to provide live transcription of the lecture content for the
students. (Stewart & McKee, 2004) While there were a number of issues with the product (in
802

Iain Stewart et al.
particular the need to have trained the voice model before use in class) it did provide a useful
resource and the feedback from the students who used the tool was very positive.
Figure 1: Talkshow showing slide and textual material derived from presenter
An interesting output of the Talkshow project was that the hearing students also wanted the transcript
of the lecture made available to them. They found this to be a useful additional resource to add to the
normally supplied lecture slides and notes as it allowed them to refer back to the lecture content and
to follow the flow of the lecture. Similarly, students who had missed the lecture reported that they
found the transcript helpful when read with the slides. To develop these findings further, this work was
extended in 2007 to produce a first version of an integrated lecture resource.
Commercial tools do exits to provide lecture capture. A number of these are what can be described as
enterprise level systems which require a significant infrastructure commitment. (ECHO 360 etc). They
are easy to use but are not normally optimised for portable use but for purpose equipped lecture
rooms. Other products which run on local systems (eg Camtasia) capture all screen interactions that
the presenter makes. This is a powerful resource which allows for multiple forms of presentation to be
captured in a single format and makes it easy to download. As it is focussed on the screen, it can
miss out on some of the interactions of the lecturer such as gestures and other forms of body
language which are often used to reinforce meaning and content.
This product took captured video and presentations together with the live transcribed content and put
them together into a single presentation package which was navigable by the video timeline.(Figure 2)
This product was designed to support the revision of content by students in their own time and also to
support distance learning as the module where it was piloted had a number of students who were not
able to attend the weekly lectures. A class of 50 students were used for the pilot project
Feedback from the students showed that there were significant benefits to the students. The
information was extracted by a questionnaire after a series of 18 lectures, with 25 completed
responses. These were followed up by a series of random follow-up interviews. The key outcomes
were:
The deaf students were highly appreciative of the subtitles, both in the presentation and in the
recording.
Although the subtitles were intended for the deaf, several international students said they helped
them understand the local accent.
803

Iain Stewart et al.
No-one was concerned that the “production values” of the recording were not of “broadcast
quality” - the content was clear.
More than half the class filled in a questionnaire, but even those who had not filled it in said they
had viewed and used the recording.
Virtually every student wanted the mechanism employed by other presenters.
Figure 2: Integration of recorded lecture with slides and textual material
In parallel with this work, another project was working in the area of lecture capture. In this case the
underpinning concept was to support the different learning styles of the students. Irrespective of one’s
view of the importance of “learning styles”, it is evident that providing a range of resources which can
be used by students in a manner which best suits them while still within a structured framework
means that the students have greater flexibility in how they can approach learning.
Content was made available within the Blackboard Virtual Learning Environment :
Lecture Video
Timed Slides with audio
Podcasts (providing extra depth or of relevant topics)
Tutorials linked to appropriate points in the presentations
Past papers
Downloadable exercises
This content allowed the students to follow the flow of the content in the traditional way by watching
the video or running the slideshow and listening to the audio, or they could take a more enquiry driven
approach and work from questions back into the content.
The pilot was tested in a final year honours programme where 31 students took the module. At the
end of the module a questionnaire was issued to the class by an external facilitator and responses
were gathered. 17 students provided detailed answers. These responses were then used by the
facilitator to drive a discussion to elicit further points. The key points from the review were that :
804

Iain Stewart et al.
Although 50% of the respondents noticed some impact on the presentation due to the capture
process, all respondents wanted the activity continued.
There was no clear preference for the video or the audio synchronised slides. The preference
tended to vary with usage as those who expressed a preference for the video tended to use it to
catch up on a missed lecture or to see the gestures, visual cues and other interactions which
were not necessarily available from the audio content. The students who preferred the audio
synchronised slides preferred it as a resource to review a particular element of the lecture.
The learning style question showed that many students did not recognise that they had a
particular learning style but after further discussion the majority described a process where they
would take a section of content and review and summarise the content until they were
comfortable with the concept. This suggested an alignment with the reflective category as defined
by Honey and Mumford (1992). Discussion showed that their approach was also strongly biased
towards a logical sequential development of ideas and in addition in certain circumstances they
would study in groups or apply other active learning activities. Overall it appeared that rather than
having a particular learning style, many of the students were adapting their learning strategy to
the content that was available and the nature of the subject being studied. This is consistent with
the detailed discussion of the fragmented nature of much learning styles research in the work of
Coffield et al (2004).
One of the key points that came back from the review was that the students really appreciated the
availability of the resources but felt that it was still very much a one way “broadcast” learning
experience. The Blackboard VLE was also considered to be a limiting factor as it is generally
structured to act a repository for content and so allows the media and other resources to be mounted
on the VLE but does not really support integration between them.
2. Students in the Web 2.0 multiplatform environment
In the last few years the software technologies generally described as Web 2.0 have moved to the
fore in student’s interaction with web based content. From an educational perspective there are
several elements of the Web 2.0 environment which can be considered to be useful (Ullrich et al.
2008). The one key feature is ability for users to comment on content and to link to other content
allows for the establishment of a community around certain content and encourages discussion of and
interaction with the content. The students also have a desire to work with web 2.0 technologies
(Andone, 2007) which are an increasing element of their online environment.
It is also recognised that the net generation of “Digital Natives” (Prensky 2001) often have a different
way of interacting with content. In particular Stone coined the term “continuous partial attention”
(Stone 1998) to describe the way in which many users of the content will be running several activities
in parallel and focussing on a particular one when it requires attention. The prototype described
earlier allowed some element of this to happen as students described how they would run a lecture in
a window while accessing other content and then switch their focus to the lecture when a particular
point arose.
Students are also no longer tied to the desktop/laptop environment and often access educational
content when they are able to fit it into their increasingly busy lifestyles (Evans,2008) so it is important
that the content can be formatted and optimised to run on a range of mobile platforms. This was done
by several of the students on the module who downloaded the content and converted it to different
formats to run on their mobile devices. It is important to note that the target class was technically
literate and was studying in an area where this sort of skill would be common. It is not appropriate to
assume that this level of ability is present in the general student body.
3. Development of the specification of an integrated resource.
Based on the analysis from the previous activities the following specification was drawn up for the
prototype integrated resource.
The system should be easy for staff to use.
The system should not impact significantly on the ability of the lecturer to present the material in a
manner that suits their lecture style.
The presented content should be available in a variety of formats suitable for a range of delivery
platforms.
805

Iain Stewart et al.
The content should be personalisable by the student
The content should be able to be added to either by staff or students.
Ultimately the goal was to make better use of lectures and to embed them into an integrated
framework of learning resources where the lecture content was just part of the overall package that
would better support distance and asynchronous learning. From this specification a prototype was
designed and implemented. The module that it was tested with was the same one as used for the
previous prototype to allow for comparability of results from the previous version. Two topics were
selected from the range of topics covered in the module.
In each case, an integrated resource was provided which allowed the students to access the entire
content via the blackboard VLE. The content was accessible using an iFrame within Blackboard to
connect to an external server where the content was hosted. The reason for using this mechanism
was to ensure that only students registered on the module were able to access the content and also
to ensure that any copyright issues were dealt with according to the JISC guidelines for lecture
capture (JISC Legal 2010) The prototype was built on a backbone of an XML structure. This allowed
a schema to be quickly defined that identified all related materials, how they should be displayed to
the student and at what point in time these assets are required. Synchronisation data between the
captured video and presentation slides was also stored in this document, with each slide having
start/end timestamps. The benefit of using XML was that it provided a framework that would allow the
prototype to be hosted on a low cost infrastructure. This was to reduce operating costs which could be
an issue with larger databases with complex relationships.
The majority of the data processing and user interaction handling occurs on the client-side via
JavaScript. This allows the XML documents stored on the server to be converted into JSON
(JavaScript Object Notation) strings and then parsed efficiently on the client’s machine. JavaScript is
also used to synchronise the video and presentation slides. This is done via timestamp information
stored in XML documents which ensures that the content is synchronised. The prototype is shown in
Figure 3.
Figure 3: Blackboard screenshot showing the prototype running in the VLE – Note menus under
presentations
806

Iain Stewart et al.
(Note: To ensure that the students were not disadvantaged by the work, all of the previously
described resources were made available as in previous years).
The integrated framework was structured to allow access to the content on a theme basis rather than
splitting the content up into individual lectures. A typical unit would comprise:
Captured videos and slides (one set per lecture) merged into a single “presentation”. This content
can be navigated linearly.
Tutorial questions which allow the user to jump to the relevant content directly
Exam questions with links to relevant past paper questions and solutions as well as allowing the
application to navigate directly to the appropriate presentation elements.
Student FAQs which allow questions to be raised and solutions posted.
Additional Materials is where other materials that extend the interaction (eg online experiments or
demonstrations).
Further reading is linked to a range of relevant electronic articles and websites and downloadable
files.
The majority of these sections allow the students to add comments and feedback on the content and
to discuss the particular topic (Figure 4)
Figure 4: Screenshot of the prototype showing the linking of questions to content and the addition of
comments
4. Results from the prototype
Information was gathered from the student body after the assessment period to allow for the full
utilisation of the content. The students were asked to fill in an online questionnaire which had many
of the same questions as in the previous year but with an additional section on the integrated
resource. The survey was fully anonymous and used a token based system to ensure that only one
807

Iain Stewart et al.
entry per student was recorded. The option of making more detailed comments was also made
available
The survey is still open at time of writing but at present there are 18 full responses from a pool of 44
The key points from the feedback were that the students liked the access to the resources in the
integrated framework but that there were areas where it needed improvement, particularly those of
the presentation framework and the “conversation tools”. It was also recognised by the staff that while
the XML based approach for the content management was acceptable for the first prototype, it lacked
flexibility and required a certain level of skill to use and so a more sophisticated back end system
would need to be developed.
The prototype was tested across a range of platforms and feedback from the students confirmed that
it worked well across a range of PCs and Apple laptops and desktops running a wide range of
browsers. The choice of Flash as a video format and with the other content based on XML, meant that
the content was accessible from any standards compliant browser on these platforms. There were no
issues with these devices presenting the content.
While it was not the primary aim of the prototype to target handheld devices, the interface was
designed to support this form of access. The prototype has run successfully on many high end
smartphones. Figure 5 shows the application running on a Samsung Nexus S Android reference
smartphone. The decision to use Flash as a video playback standard prevents iOS devices from
being able to access the full content. To address this, the video source on the next version will
perform device detection and feed the appropriate format to the device. In addition, the future
development will include dedicated “Apps” for mobile platforms which will support offline access to
prevent high data charges.
Figure 5: The application running on an Android mobile platform
5. Design of the next stage
While the students liked the prototype and found it useful, their feedback raised several points about
the usability of the prototype. The ability to navigate to the relevant part of the content directly from
questions and from other content elements was appreciated but the need to move up and down the
screen to see the different elements needed to be improved. This will be addressed by converting the
layout of the student view to be menu driven for the text based content (eg tutorial questions) which
will ensure that the active element of the content is always directly under the presentation area.
There was an assumption that students would use the video as the prime navigation tool when
viewing the lecture. This was not always the case and so the ability to navigate by slide as well as by
video will be added.
808

Iain Stewart et al.
At present there is some confusion over the use of the communications features so the structure of
these needs to be revised.
There were relatively few issues with the original decision to use flash for the video presentation
format but to ensure full iOS compatibility, the playback will be standardised on MP4.
From an architectural perspective the design needs to be more flexible to allow for a wider range of
content to be presented and synchronised.
With this in mind a revised underpinning architecture is proposed. This will be based on an object
oriented model and will have at its core a synchronisation object. This object will contain timestamps
for key points in the presentation. In the standard model these timestamps will be stripped
automatically from the XML of the recorded PowerPoint presentation which is generated automatically
by PowerPoint. These timestamps can then be used to synchronise the other content. For example a
video capture of the same presentation may have an offset compared to the capture slides due to the
start times of the applications but once that offset property is set in the video object then it will be
synchronised to the slides via the synchronisation object. Similarly any other content (e.g. tutorial
questions) can then reference the appropriate synchronisation point and all of the content linked to
that will move to the same cue.
The architecture also allows for multiple objects of the same type to be included (eg a main lecture
presentation and tutorial discussion) as they can be uniquely identified. It also means that there is no
requirement for the content to be constrained to the format specified by the prototype. If there is no
video but for example an augmented podcast is available then that can be controlled and navigated in
a similar manner.
The front end menu and navigation will be generated dynamically and so will only present the content
that is available. For example, if there are no tutorial exercises then the menus system would not
display an empty menu element and similarly if a lecturer wishes to add different content areas then
so long as it is described in the appropriate field in the object then it will be made available through
the menu.
6. Conclusion
This is a continuing project; however it is already clear that the objective of providing an extended,
supplemented, student-centred yet structured teaching resource has been achieved. In addition it can
be implemented on a low cost basis.
At present the work has produced a tool which even at this prototype stage allows the students to
interact effectively with the content. Blackboard utilisation statistics show that it has been heavily used
and even with the limitations identified in this paper it has been popular with the students and
appreciated as a resource which they found helped with their learning activity. The work has gained
significant positive feedback to date and was nominated by the students for a teaching award at the
university. The next stage design is to take the existing strengths of the product and make it more
flexible and easy to use for both staff and students. The value of this work has been recognised by
the learning research centre at the authors’ university and the work is being supported by funding of
the development and dissemination activity.
Acknowledgements
The authors would like to thank the past and present students and staff who have supported the
project and provided the feedback that has been essential to the development of the project.
References
Andone, D., Dron, J., Pemberton, L. and Boyne, C. (2007) The desires of digital students. Proceedings of the
14th Association for Learning Technology International Conference, 4-6 September 2007, Nottingham, UK.
pp 189 -200
Black, L. (2005). Dialogue in the lecture hall: Teacher-student communication and students' perceptions of their
learning. Qualitative Research Reports in Communication, vol 6 no. 1, pp31-40
Coffield F, Moseley D, Hall E and Ecclestone K, (2004) Should we be using learning styles? What research has
to say to practice. Learning and Skills Research Centre 2004
809

Iain Stewart et al.
Dolnicar, S, Should we still lecture or just post examination questions on the web? The nature of the shift towards
pragmatism in undergraduate lecture attendance, Quality in Higher Education, 2005, vol 11 no. 2, pp103-
115
Edwards, H., Smith, B. & Webb, G. (2001). Introduction. In H. Edwards, B. Smith & G. Webb (Eds), Lecturing:
Case studies, experience and practice pp. 1-10. London and Philadelphia: Kogan Page.
Evans, C. (2008). The effectiveness of m-learning in the form of podcast revision lectures in higher education.
Computers & Education, vol 50, pp491-498
Honey, P. and A. Mumford. 1982. Manual of Learning Styles. London: P Honey .
Heward, W. L. (2003) ‘Guided Notes – Improving the Effectiveness of Your Lectures’. Factsheet published by the
US Department of Education, posted on ‘Tomorrow’s Professor Listserv’ email distribution list, 25 June
2003.
JISC Legal http://www.jisclegal.ac.uk/ManageContent/ViewDetail/tabid/243/ID/1608/Recording-Lectures-Legal-
Considerations-28072010.aspx Accessed 1/3/2011
Kennedy G, Judd T., Churchward A, Gray K, Krause K (2008) First year students' experiences with technology:
Are they really digital natives? Australasian Journal of Educational Technology 2008, Vol 24 no. 1, pp108-
122.
Lammers W.J. & Murphy J.J. (2002) A profile of teaching techniques used in the university classroom The
Journal of Active Learning in Higher Education March 2002 vol. 3 no. 1 pp54-67
Moore, S., Armstrong, C. & Pearson, J. (2008). Lecture absenteeism among students in higher education: A
valuable route to understanding student motivation. Journal of Higher Education Policy and Management,
vol. 30 no. 1, pp15-24
Prensky, M. (2001) Digital Natives, Digital Immigrants. Available from http:// www.marcprensky.com/
writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf [accessed 12
October 2010].
Ravenscroft, A. (2009), Social software, Web 2.0 and learning: status and implications of an evolving paradigm.
Journal of Computer Assisted Learning, 25: 1–5.
Stewart I, McKee W. (2004) “The application of voice recognition technology to the development and
presentation of complex engineering terminology to hearing impaired students” The British Journal of
Engineering Education Vol 4 Issue 1 pp25-32
Stone, L. (1998) Thoughts on attention and specifically, continuous partial attention. Available from
http://www.lindastone.net/ [accessed 9 October 2010].
Ullrich C, Borau K, Luo H, Tan X, Shen L, Shen R (2008) Why web 2.0 is good for learning and for research:
principles and prototypes, ACM WWW '08 Proceedings of the 17th international conference on World Wide
Web pp 705-714
Williams, J. & Fardon, M. (2007). Perpetual connectivity: Lecture recordings and portable media players. In ICT:
Providing choices for learners and learning. Proceedings ascilite Singapore 2007
810

The Danger of the Downward Spiral: Teachers and Digital
Literacy
Caroline Stockman and Fred Truyen
Catholic University of Leuven, Belgium
Caroline.stockman@arts.kuleuven.be
Fred.truyen@arts.kuleuven.be
Abstract: Results from two surveys taken in secondary schools throughout Belgium and the UK, and many
individual findings in existing literature, point towards the fact that there are not only a number of factors
influencing the successful implementation of educational technology, but also that the power of these factors can
have defining effects on the behaviour of teachers towards the technology and its educational use. Rather than
describing the actual research done, this paper intends to propose a theoretical framework to apply to the
dynamics of the adoption process of new technology in learning by teachers, as formed by the results of the
qualitative and quantitative data of the research and its comparison to existing literature. The adoption process
takes the form of a spiral, and teachers are defined by the movement on that spiral path according to the
influence of three inner and four environmental factors. The primary purpose of this paper is to provide anyone
active in the field of education with a useful tool to assess risks during the integration of new technology in an
educational setting, but especially to raise awareness of the danger of a downward spiral, which not only
undermines our investment of time and money into these promising new technologies, but which also puts the
learners at a great disadvantage when providing useful tools for their benefit which are, in the end, not fully or
even wrongly used.
Keywords: adoption process, digital literacy, secondary education, technology integration
1. Introduction
New technological means arise daily, and their educational possibilities are rapidly expanding.
Although the description and exploration of the didactic potential of these different hardware and
software tools is necessary and interesting, this paper will focus on the adoption of them by today’s
teachers. Despite the great didactic potential of computer technology, it seems its actual, effective use
is limited or at the very least, hindered by a number of elements (Maddux & Johnson, 2006; Incekara,
2011; Barge, 2009). It is important to be aware that even the most advanced and well-built technology
will not be the agent of change on purchase. Technology, once installed and ready to be used, is
static; it is simply the medium directed towards the pupils, which the teacher must put to use
effectively. The teacher, from that perspective, is the driving force of modernised education (Fisher
2006). There is a strong necessity for teachers to be innovative, digitally literate and effective as
educators, and the high investments of time, money and energy in tools with great learning potential
require us to consider the ways in which teachers deal with the novelty of the situation. Researching
which factors influence a teacher’s use of technology in the classroom, has been a task many
endeavoured to describe until now; and more importantly, we now have a sufficient base to claim that
the adoption process can be distinguished as a general pattern, which we will describe in terms of a
spiral. Three definite inner and four variable environmental factors will be determined, which influence
the dynamics of the process: either towards integration of technology in the classroom, which is
considered top of the spiral, or further away from success, which is considered a downwards
movement. Depending on the specificity of the situation, these factors will have a greater or minor
influence, but nonetheless, they will play a role. This proposed theoretical framework will help
educators and investors to understand the issues at stake when technology needs to be implemented
in an educational setting, and by being aware of the dangers which send the teachers' approach
spiralling downwards, we can administer solutions and interventions to prevent a downwards
movement, or even reverse it, so as to make our investments worthwhile, and ensure our learners
strong and effective education.
The term 'digital literacy' will remain a central notion in this paper, and it is most closely related to the
factor of 'competence'. However, digital literacy is not limited to knowing the facts about certain
technology, or knowing how to use it effectively in a certain context. It is also the ability to build and
expand the existing skillset with confidence, to retrieve information and guidelines efficiently from the
community, and to evaluate and analyse given technology and its use in a correct and objective
manner.
811

2. The research
Caroline Stockman and Fred Truyen
Two surveys were held for three hundred teachers in two different national settings, active in the field
of secondary education and colleges. Further data was gathered through semi-structured interviews
and observation of teachers during the adoption process.
The first research was held in Belgium, 2010 and aimed to determine knowledge and attitude of
secondary school teachers towards educational video gaming. The survey was presented to 102
Belgian secondary schools; no discrimination was made in gender, age, or subject of expertise when
inviting teachers to participate. Comments given during this survey and personal interviews were
gathered to allow narrative data to complement the quantitative results.
Though the description of the results is worth an article in itself, we will continue to place them in the
light of the continued research, as the actual dynamics of the process are at stake here. To study this
further, another form of educational technology was taken into consideration: language labs. Though
fairly new, many schools have already chosen to invest in the installation of this computer-based
classroom environment, and numbers are growing daily (though no precise figures are known).
All participants for this second part of the study, held in 2011, were active in secondary schools and
colleges throughout the UK. The set of teachers involved in new lab installations were closely
monitored and observed during their adoption process over a few months' time. This study confirmed
the existence of the influencing factors gathered from the first survey, but also provided more insight
into the actual development of that influence on the teacher. Though we did not set criteria prior to the
survey, the group of participants naturally formed itself as the actual user group of language labs:
MFL teachers, whose majority turned out to be female.
3. The spiral
In what follows, the dynamics of the adoption process will be discussed in terms of a spiral (see
below), whereby the factors influencing the movement in this spiral will be dealt with on the basis of
their respective occurrence in the process. Some cautious suggestions for possible interventions will
be made, though the effectiveness of these need to be researched further.
The first three elements which will be discussed are inner characteristics, which will inevitably present
themselves during the first stages of the development.
Figure 1: The adoption process of educational technology by teachers
812

3.1 The inner factors
Caroline Stockman and Fred Truyen
3.1.1 Beliefs
Though research has argued that the belief system of the teacher is an adequate factor for prediction
of behaviour towards technology (Judson, 2006; Roehrig et al., 2007; Haney et al.,2002), this paper
states it is only the beginning, as there are a lot more elements at stake and the teaching methods,
notions and prejudices which the teacher adheres as defined by past experience, will only determine
if the teacher will start at a high or low place in the spiral. Though a connection to the belief system
and the adoption process has been proven to correlate (a more traditional approach is said to have a
negative impact on the integration) (Liu, 2011; Tondeur, Hermans & Van Braak, 2008), it has also
been established that the integration of technology by teachers with a claim to student-centered,
constructivist methods and beliefs, does not result automatically in powerful, innovative or effective
teaching (Cuban, Kirkpatrick & Peck, 2001; Hermans, Tondeur, van Braak & Valcke, 2008).
During the survey on educational gaming in Belgium, teachers were asked to express their opinion on
five different statements about games.
Firstly, the statement : “Games stimulate aggressive behaviour", which is a common prejudice, readily
reinforced by public media and inherent to a 'moral panic' perspective. As our current focus is not on
the object, we will not discuss whether there is truth in that statement, only what the teachers believe
to be true.
The results show that teachers are cautious in their judgment. Roughly a third of them ‘neither agrees
nor disagrees’, while another 30% disagrees to some level.
Half of the participants also ‘disagrees’ with the statement that “You can’t learn anything from a video
game”, as show in graph A. If you add the number of teachers who ‘strongly disagreed’ with that
statement, you come to 68% of the participants who actually do believe in the educational potential of
games.
Figure 2: Opinion results to didactic potential of video games
Overall, teachers' attitude towards video games was fairly open, yet despite this largely positive
attitude, only three teachers of all participants said to have used video games in their lessons. This
shows there are more factors at play. When it came to language labs, very few teachers expressed
prejudice towards the technology, the main concern being isolation of the learners, amongst
themselves and towards the teacher. It is important to understand that these prejudices can lead to
misunderstandings about technology and its educational use. The inner disagreement becomes (yet
again) very clear when one Belgian RS teacher thoughtfully said her children "think they learn English
while playing. One of them is even pursuing higher education in English because gaming encouraged
him to it.", though during the same interview, she added dismissively that "kids already play enough
games in their spare time as it is."
813

Caroline Stockman and Fred Truyen
So even though this teacher exemplifies that games can have both a cognitive as well as a
motivational influence, she still feels reluctant to incorporate them into her own teaching method.
Again, it becomes apparent that more factors are influencing her attitude.
After that first, distanced encounter with a certain technology, the teacher is required to engage. Two
more factors come into play here: the actual knowledge and skills with which the teacher approaches
the new object, and the confidence in that personal skill set. In the schematic representation of the
theory, they are listed as '2A' en 2B' since their precise sequence seems to entwine.
3.1.2 Competence
The survey on educational gaming in Belgium showed that the most popular answer option across all
knowledge questions was the possibility to say “I don’t know”. The first part of the questionnaire used
in this study, tested the knowledge and skills of teachers concerning computer games with specific
questions on taxonomy, terminology and concrete examples of well-known video games.
If we have a closer look at Quake, for example, a gap of knowledge clearly presents itself. Quake is
undeniably a shooter. Yet only 37 teachers, which is 21% of the participants, assigned it to its proper
category. Six teachers, three men and three women, even thought it was a ‘simulator’, the least likely
possibility. It should be mentioned at this point, that gender does not play a defining role in the
answers of the participants. ESA (2008) concluded that 40% of all gamers are female, and the correct
answer to this particular question was given by 22 men and 15 women: the same equation (despite
the fact that Quake, or shooters, are considered to be a more masculine type of video game). Other
variables such as age, professional characteristics or other traits, did not have a noticeable influence
either (in the entire questionnaire).
If we look at the overall results of this survey, eight possible thresholds were identified.
Figure 3: Personally defined obstacles to integrate video games in an educational setting
Graph B lists these eight obstacles, and the level in which teachers felt that it applied to their teaching
practice. The main threshold turned out to be a lack of familiarity with games. This was very much
confirmed during the research. Teachers know very little about video games, and have even less
hands-on experience with them. This relates to the first aspect of knowledge, or relevant digital
814

Caroline Stockman and Fred Truyen
literacy levels: the actual technical and conceptual facts and necessary skillset (Lawless & Pellegrino,
2007). Even if their belief system allows them to have confidence in the fact that video games can
indeed enrich the learning process, they simply don't know how, which will cause a downward
movement on the spiral despite the fairly high starting point. Teachers are not knowledgeably aware
of the world of gaming, and so they easily miss the many opportunities of using games meaningfully
in a learning environment. This concern represents the second aspect of competence, and also the
second threshold in graph B and is described by the framework of Shulman (1986) as a separate
aspect of ‘knowledge’: a lack of practical, concrete ideas to incorporate video games in the classroom.
It goes without saying, of course, that these gaps of knowledge will lead to wrong or ineffective use –
which will in its turn influence the perception of the technology involved if the desired learning
outcomes remain to be desired. (So the movement on the spiral will continue downwards.)
Again, this is not a factor which has gone unnoticed. In 1995, for example, Borko&Putnam stated that
teacher knowledge has significant impact on teachers’ decisions, thus (p.37):”…to help teachers
change their practice, we must help them expand and elaborate their knowledge systems.” Or, as one
Belgian Technology teacher (born in 1977, male) puts it: “I think the primary obstacle to overcome
before games can be used in education, is the ignorance of the teacher.” Yet an IT competent teacher
can still shy away from new technology, as was often noticed during observations in the use of the
language labs, expressing their reluctance with exclamations such as : "I will never be able to do this!"
3.1.3 Confidence
If the levels of actual digital literacy are insufficient to deal effectively with the presented technology, a
teacher can still achieve the potential literacy levels. This requires a self-confidence to build up
computer skills and explains why, even though the participants considered themselves capable
teachers, and 9 out of 10 were enthusiastic about the newly installed system, they still approached it
with caution.
Especially novice teachers (Yan & Piper, 2003) experience this strong influence of self-efficacy. Next
to that, the greater part of the participants in the UK were female, and previous research has shown
that women seem to have a lower estimation of their own ICT skills than men (Poelmans & Truyen,
2009; Schumacher, 2001) – regardless of the actual performance.
Language labs, as well as video games, and many other types of educational technology, are very
new and the knowledge about the system, how to use it, and how to teach with it, is consequently low.
Training is necessary for the teaching staff, but those who are not confident that they will learn how to
work with it, quickly feel demotivated. Boosting confidence will happen through positive experiences
when handling the technology (Mueller et al., 2008), but the reverse is also true. The availability of
scaffolding structures (such as support and a community feel - further discussed below) can help a
teacher feel secure, decrease pressure levels, increase confidence, and make an upwards movement
on the spiral possible again. Low self-confidence creates a fearfulness to approach the technology
hands-on, and will establish a downwards movement. In a survey of over 700 teachers, Wozney,
Venkatesh & Abrami (2006) defined this IT confidence as one of the greatest predictors of teachers’
technology use. Though it does play a major role early in the development of the spiral, there are
more stakes in this process which can speed up the downward movement, slow it down, or actually
reverse it.
3.2 The outer factors
The previously discussed factors will occur in any adoption process, and in that order, and are often
considered as the major influences. However, four more aspects come into play, though their
occurrence in the process is not fixed because they stem from the teacher's environment. Therefore,
they are categorised as outer, or variable factors in the framework, their influence depending on the
prominence in the particular setting.
3.2.1 Time
A lack of knowledge, whether object-specific or in relation to the educational deployment, is
exponentially related to the need for time. Time to experiment with the technology, to perform tasks
with it, to think about effective methods to use it, or as one Economics teacher in Antwerp said:
"There's already so much to do that you don't always have time for a quick experiment, which doesn't
815

Caroline Stockman and Fred Truyen
necessarily lead to anything good." In the survey, time was felt to be the third main threshold for the
use of games during lessons. Innovation, and especially enforcing innovative ways effectively, needs
time to develop, however. There are ways to help teachers overcome this obstacle with time-effective
solutions: step-by-step lesson plans for the technology involved will save them a lot of preparation
time. Resource material, preferably in the form of open educational courses or isolated material
tailored to the technology involved, will be another valuable time-saver (creating content of course
requires a time investment, but looking for content can be equally consuming and sometimes
frustrating). Sharing examples of good practice in the local, national and international setting, lastly,
should be a commitment which educators are willing to take towards each other (Ertmer, 2005). In all
of these cases, the solutions should be as much focused to the teachers' immediate needs as
possible to save time, but to ensure long-term insights, further in-depth training is needed.
3.2.2 Technology
It matters, of course, which specific type of technology is involved. This not only determines the
relation of the teachers towards it with regards to their belief system, occurring as first influence. As
said before, video games are a much more controversial type of educational technology, so teachers
start much lower on the spiral of the adoption process than they do with regards to language labs.
That type of technology will start in a better position on the spiral, but can follow the downhill
movement easily if it doesn't function as expected. One of the greatest sources of frustration for the
teacher is the failure of technology - which, unfortunately, it sometimes does for various reasons:
because little software bugs creep into the system, because the school servers aren't powerful
enough, because infrastructures age, because the program isn't robust enough to endure wrong
use,...
If the object central to the teacher's mission acquires an image of being untrustworthy - and because
of the technical failures, a teacher loses time which he or she already lacked - they give up. Also
because, as one English MFL teacher said: "When it goes wrong every time, you just think it is you
and you lose confidence." In a situation like this, several factors are reinforcing each other and
causing a rapid downward movement.
With regards to games, a teacher of modern languages in Belgium said : "If you always have to turn
to plan B, you stop trying after a while", as was indeed the case for some of the language labs
installed in UK schools. On the other hand, games, which have a lower starting point on the spiral
because of low knowledge and cautious belief systems, can move upwards easily if they function
properly, perhaps even exceed expectations (provided teachers can gather enough confidence to try).
3.2.3 Community
In any cultural society, membership of a group is important, especially in relation to other groups
within the same community setting. Teachers form a clearly defined group in the school context, as
opposed to the group of students, school board and school administration (Ponticell, 2003; Roehrig et
al., 2007; Somekh, 2008).
Even if a teachers starts high on the spiral with a very strong belief in the technology presented, their
adoption process will waver when he or she has to deal with a lot of negativity and prejudice in the
teacher's room. Especially when other factors come into play, for example technical issues, the
teacher finds it increasingly difficult the defend the value of it and will be dragged downwards on the
spiral.
Innovation is more likely to be adopted if a group of peers stand as a united front, sharing the same
belief (Zhao & Frank, 2003). This factor does not necessarily influence the spiral of the process in a
negative way: more IT insecure teachers (factor two and three in the process) can be morally and
practically supported by the IT confident teachers in the school, thus pushing the integration upwards
(Ertmer, Ottenbreit-Leftwich, & York, 2006).
It is important to note that this community influence does not only take place in the local teacher's
room, but in the wider (online) social network. Through websites and forums, teachers can connect
and help each other with examples of good practice (factor two in the process) and questions to IT
related issues (see below: support), through participation in conferences and professional network
events, common goals can be reinforced. (Ozdemir, 2011; Green, Donovan & Bass, 2010) The
816

Caroline Stockman and Fred Truyen
teacher, being digitally literate about teaching with technology, of course knows where to look for
information or guidelines.
Not only the own cultural group is important in the adoption process of both language labs and
games, the interrelated groups matter too: school administration, members of board staff, and the
students. (Somekh, 2008) has stated the importance of the school board in the adoption process:
support and motivation 'from above' plays a crucial role to unify teachers in their community effort
towards the common goal : making the technology effective for the students. Next to that, actual
administration and organisation can be tiresome if it doesn't run smooth. One RS teacher in Belgium
declared she didn't feel motivated to work with video games at all (despite believing in the didactic
potential), because : "It is already such a hassle if I want to show the students a fragment of a movie -
scheduling the right room, getting the DVD from the library, getting equipment, hoping it actually
functions..."
When it comes to the group of students : their attitude when being taught with the new medium,
matters greatly to the pressure levels of the teacher. Students have the tendency to be more IT
literate than teachers, being Digital Natives of this era, and this can work both ways: either they will
lower the confidence of the teacher in handling the medium effectively, or they will be able to help in
moments of confusion and thus lower the stress level of teachers with regards to time and
performance. Certainly when it came to language labs, size does matter. Stress levels lowered and
teacher confidence grew when class groups did not exceed seven pupils. Once over twenty pupils,
which is an average class size, there is a lot more to manage and monitor, and teachers' behaviour
immediately became more agitated. The agitation leads to great uncertainty when something
unexpected occurs in the use of the technology. If a student can help at that time, the downwards
spiral will have been prevented. If they take advantage of the opportunity to misbehave, the teacher's
stress level will go up and it is unlikely that the right outcome will occur. Depending on the
technology's robustness, it might produce an error at this point. Already moving down on the spiral
with own IT competence wavering, which affects the IT confidence negatively, while they are losing
precious time, and while affecting their positive belief system towards the technology, the teacher now
relies on the following factor to determine the movement of the spiral: support.
3.2.4 Support
When a technical error occurs, something or someone needs to intervene to determine where the
spiral will go. Students can of course offer a helping hand, but IT staff is specifically there to help out
when technical matters are concerned. The first element at stake is the capability of the IT staff (Zhao
et al., 2002). When someone can come in quickly, but is unable to resolve the problem, the teacher's
frustration will only grow. Next to the capability of IT staff, their availability matters as well. More often
than not, IT staff are limited in schools, with a long list of 'things to do'. If they cannot come as soon as
possible when they are called, the teacher's frustration will again grow whilst waiting. The actual
occurrence of an error does not play a major role, if there is enough available and reliable IT support
in the environment of the teacher, so the problem can be solved swiftly and efficiently – not making
the teacher lose too much precious lesson time, and keeping the pressure levels of the situation
down.
Technical support matters not only in these moments of crisis. Teachers can have certain questions
about the technology involved, which IT staff needs to be able to respond to, but there are other
solutions: manuals and tutorials should be clear, concise and available, along with just-in-time training
support for teachers on the technical side of it (Brill & Park, 2011). Online forums, helplines, how-to
videos, coaching, mentoring, ongoing training, workshops etc. can all play an important role as well to
increase the teacher's feeling of being supported in their endeavours - and their levels of digital
literacy.
4. Conclusion
To conclude, the findings originating from the data of these two surveys, and their comparison with
each other as well as with many individual conclusions in existing literature, allows to form a clear
visual of the adoption process of technology in today's teaching.
Confidence and competence are the two most dominant factors in the spiral, where the attitude or
belief system of the teacher determines the starting point of the process. There is a string of
817

Caroline Stockman and Fred Truyen
subsequential factors which have their effect once they come into play. Each of these factors does not
play an absolute role in itself; but they all influence the behaviour of the teachers in an upwards or
downwards movement, if we consider the adoption process as a spiralling shape towards or away
from the integration of technology in the classroom.
The theoretical framework presented in this paper will hopefully provide educators and investors
worldwide with a useful tool for improvement. Even though the choice of educational technology was
different in the two surveys, the influencing factors are the same. Moreover, teachers in secondary
education react similarly under the influence of these factors, even though they are teaching in
different national settings. It is therefore possible to assume that we can internationally determine a
common approach to monitoring the adoption process and more importantly, influencing it into an
upwards movement. To do this, we must not only single out every factor, but be aware of the
dynamics of the spiral, noticing a downwards movement in time, and intervening with appropriate
solutions to reverse the development. Though some initial suggestions for interventions has been
made in this paper, further research into the effectiveness of these solutions is absolutely necessary
to make our investments worthwhile, and ensure learning outcomes for current and future
generations, with teachers as competent and confident agents of technological and educational
innovation.
References
Barge, M. (2009). Teaching Techniques for Multimedia Language Labs: Final Report. Report for the Centre for
Excellence in Multimedia Language Learning, on behalf of The Language Centre, Queen Mary, University
of London.
Borko, H., & Putnam, R. T. (1995). Expanding a teacher’s knowledge base: A cognitive psychological perspective
on professional development. In T. R. Guskey & M. Huberman (Eds.), Professional development in
education: New paradigms & practices . New York: Teachers College Press. pp 35-66.
Brill, J. & Park, Y. (2011). Evaluating Online Tutorials for University Faculty, Staff, and Students: The Contribution
of Just-in-Time Online Resources to Learning and Performance, International Journal on E-learning, Vol.
10(1), pp 5-26.
Bunz, U., Curry, C. & W. Voon (2007). Perceived versus actual computer-email-web fluency, Computers in
Human Behavior, Vol. 23, pp 2321-2344.
Cuban, L., Kirkpatrick, H., & Peck, C. (2001). High access and low use of technologies in high school
classrooms: Explaining an apparent paradox, American Educational Research Journal, Vol. 38, pp 813–
834.
Durndell, A., & Z. Haag (2002). Computer self efficacy, computer anxiety, attitudes towards the internet and
reported experience with the internet, by gender, in an East European sample, Computers in Human
Behavior, Vol. 18, pp 521–535.
Ertmer, P. (2005). Teacher pedagogical beliefs: The final frontier in our quest for technology integration?
Educational Technology Research and Development, Vol. 53(4), pp 25–39.
Ertmer, P., Ottenbreit-Leftwich, A., & York, C. (2006). Exemplary technology-using teachers: Perceptions of
factors influencing success, Journal of Computing in Teacher Education, Vol. 23(2), pp 55–61.
Fisher, T. (2006). Educational transformation: Is it like “beauty” in the eye of the beholder, or will we know it when
we see it? Education and Information Technologies, Vol. 11, pp 293–303.
Gee, J.P. (2003) What video games have to teach us about learning and literacy. Palgrave Macmillan, New York.
Green, T., Donovan, L. Bass, K. (2010) Taking Laptops Schoolwide: A Professional Learning Community
Approach, Learning & Leading with Technology, Vol. 38 (1), pp 12-15.
Hagoort, Thijs & Hautvast (2009). Belgian National Gamers Survey 2009. [online]
http://www.gamesindustry.com/. (accessed on 09/06/10).
Haney, J., Lumpe, A., Czerniak, C. & Egan, V. (2002). From beliefs to actions: The beliefs and actions of
teachers implementing change, Journal of Science Teacher Education, Vol. 13, pp 171–187.
Hennessy, S., Ruthven, K. & Brindley, S. (2005). Teacher perspectives on integrating ICT into subject teaching:
commitment, constrains, caution, and change, Journal of Curriculum Studies, Vol. 37, pp 155–192.
Incekara, S. (2011) The Turkish Geography Teaching Program (2005) and Technology Use in Geography
Courses: An Overview of High School Teachers' Approach, Educational Research and Reviews, Vol. 6(2),
pp 235-242.
Judson, E. (2006). How teachers integrate technology and their beliefs about learning: Is there a connection?
Journal of Technology and Teacher Education, Vol. 14, pp 581–597.
Lawless, K. A., & Pellegrino, J. W. (2007). Professional development in integrating technology into teaching and
learning: Knowns, unknowns, and ways to pursue better questions and answers, Review of Educational
Research, Vol. 77, pp 575–614.
Maddux, C. D. & Johnson, D. L. (2006). Type II applications of information technology in education: The next
revolution. Computers in the Schools, Vol. 23(1), pp 1–5.
McGee, G. W. (1987). Social context variables affecting the implementation of microcomputers, Journal of
Educational Computing Research, Vol. 3(2), pp 189–206.
818

Caroline Stockman and Fred Truyen
Ministry of Education (2007). Attainment targets. [online] http://www.ond.vlaanderen.be/dvo/secundair/index.htm.
(accessed on 20/07/10).
Mueller, J., Wood, E., Willoughby, T., Ross, C., & Specht, J. (2008). Identifying discriminating variables between
teachers who fully integrate computers and teachers with limited integration, Computers and Education, Vol.
52, pp 1523–1537.
Ozdemir, S. (2010) "To Err Is Human, but to Persist Is Diabolical": Loss of Organizational Memory and E-
Learning Projects, Computers & Education, Vol. 55(1), pp 101-108.
Poelmans, S., Truyen, F., Deslé, R. (2009). Perceived computer lireracy among different types of
(under)graduate students: findings of a survey. Proceedings of ICERI 2009 conference. Madrid, pp 4910-
4921.
Prensky, M. (2001). Digital natives, digital immigrants: do they really think differently? On the Horizon, Vol. 9 (6),
pp 1–6.
Roehrig, G. H., Kruse, R. A., & Kern, A. (2007). Teacher and school characteristics and their influence on
curriculum implementation, Journal of Research in Science Teaching, Vol. 44, pp .
Schumacher, P., & Morahan-Martin, J. (2001). Gender, internet and computer attitudes and Experiences,
Computers in Human Behavior, Vol. 17, pp 95–110.
Shulman, L. (1986). Those who understand: Knowledge growth in teaching, Educational Researcher, Vol. 15(2),
pp 4–14.
Stockman, C. (2010) Games in het Vlaams secundair onderwijs. Master's paper, Catholic University of Leuven,
Belgium.
Somekh, B. (2008) Factors Affecting Teachers’ Pedagogical Adoption of ICT, In: International Handbook of
Information Technology in Primary and Secondary Education, Springer, US, Vol. (20), pp 449-460.
Tondeur, J., Hermans, R., van Braak, J. (2008) Exploring the link between teachers' educational belief profiles
and different types of computer use in the classroom, Computers in Human Behavior, Vol. 24 (6), pp
Tondeur, J. van Braak, J., Valcke, M. (2006). Curricula and the use of ICT in education: Two worlds apart? British
Journal of Educational Technology Vol. 38 (6), pp 962-976.
Vanderlinde, Ruben & van Braak, Johan & Hermans, Ruben (2009). Educational technology on a turning point:
curriculum implementation in Flanders and challenges for schools, Education Tech Research Dev, Vol. 57,
pp 573–584.
Wozney, L., Venkatesh, V., & Abrami, P. C. (2006). Implementing computer technologies:Teachers’ perceptions
and practices, Journal of Technology and Teacher Education, Vol. 14, pp 173–207.
Yan, W. & Piper, D. (2003). The Relationship between Leadership, Self-efficacy, Computer experience, Attitudes,
and Teachers’ Implementation of Computers in the Classroom, in C. Crawford et al, Proceedings of Society
for Information Technology & Teacher Education International Conference, VA, Chesapeake. pp 1057-
1060.
Zhao, Y., Pugh, K., Sheldon, S., Byers, J. (2002) Conditions for Classroom Technology Innovations, Teachers
College Record, Vol. 104(3), pp 482-515.
Zhao, Y., & Frank, K. A. (2003). Factors affecting technology uses in schools: An ecological Perspective,
American Educational Research Journal, Vol. 40, pp 807–840.
819

PeerWise - The Marmite of Veterinary Student Learning
Amanda Sykes 1 , Paul Denny 2 and Lesley Nicolson 3
1
Student Learning Service, The University of Glasgow, Glasgow, UK
2
Department of Computer Science, The University of Auckland, Auckland, New
Zealand
3
School of Veterinary Medicine, The University of Glasgow, Glasgow, UK
amanda.sykes@glasgow.ac.uk
paul@cs.auckland.ac.nz
lesley.nicolson@glasgow.ac.uk
Abstract: PeerWise is a free online student-centred collaborative learning tool with which students anonymously
author, answer, and evaluate multiple choice questions (MCQs). Features such as commenting on questions,
rating questions and comments, and appearing on leaderboards, can encourage healthy competition, engage
students in reflection and debate, and enhance their communication skills. PeerWise has been used in diverse
subject areas but never previously in Veterinary Medicine. The Veterinary undergraduates at the University of
Glasgow are a distinct cohort; academically gifted and often highly strategic in their learning due to time
pressures and volume of course material. In 2010-11 we introduced PeerWise into 1st year Veterinary
Biomolecular Sciences in the Glasgow Bachelor of Veterinary Medicine and Surgery programme. To scaffold
PeerWise use, a short interactive session introduced students to the tool and to the basic principles of good MCQ
authorship. Students were asked to author four and answer forty MCQs throughout the academic year.
Participation was encouraged by an allocation of up to 5% of the final year mark and inclusion of studentauthored
questions in the first summative examination. Our analysis focuses on engagement of the class with the
tool and their perceptions of its use. All 141 students in the class engaged with PeerWise and the majority
contributed beyond that which was stipulated. Student engagement with PeerWise prior to a summative exam
was positively correlated to exam score, yielding a relationship that was highly significant (p

Amanda Sykes et al.
An overview of the PeerWise tool has previously been given (Denny, Luxton-Reilly and Hamer,
2008a), so we include just a short summary here. PeerWise use requires minimal staff input: the
course instructor typically begins the process by setting up a repository on PeerWise and granting
their students access to the resource, and from this point the workload is student-centred. The main
menu of PeerWise, shown for a typical student in the course that is the subject of this study, is shown
in Figure 1. This menu is divided into three sections; questions that are authored, answered and
remain unanswered by the student.
Figure 1: A student's view of the main menu of PeerWise
Students can create questions at any time, and are guided through this process by the PeerWise
interface. They are also encouraged to help organise the bank of questions by associating relevant
topics, or tags, with each question they contribute. Students can answer, evaluate and discuss any of
the questions created by their classmates at any time. When using PeerWise to practise answering
questions, students can filter questions by difficulty and by topics of interest and can use the ratings
assigned to questions by their peers to find good quality questions to answer. PeerWise employs
several basic game mechanics to encourage participation, for example, students are rewarded with
points when the contributions they make are endorsed by other students.
Although this feature was not enabled for the course in this study, students can also earn badges for
completing certain tasks within PeerWise. The idea of using badges, or achievements, for engaging
and motivating users is becoming standard practice in online gaming and social systems. In 2002,
Microsoft pioneered the first large-scale implementation of an achievement system with their XBox
Live platform. Other successful examples of these types of systems include Wikipedia's barnstars
(Kriplean, Beschastnikh and McDonald, 2008) where contributors are rewarded by other users for
hard work and diligence, the badges rewarded for constructive participation to StackOverflow's
question and answer site (StackOverflow, 2011), and the use of badges to encourage "check-ins" on
popular location-based services such as Foursquare (Foursquare, 2011). Antin and Churchill (2011)
821

Amanda Sykes et al.
recently outlined five social psychological functions for badges as used in social systems. The use of
badges in PeerWise aligns with these functions, which include "Goal Setting", known to be an
effective motivator, as well as “Instruction” and “Reputation”.
Previous studies of the use of PeerWise at the tertiary level have focused on student perceptions,
repository quality and learning gains. This research has shown that students perceive PeerWise to be
a useful and enjoyable activity (Denny, Luxton-Reilly and Hamer, 2008b), that they are capable of
creating high quality, relevant repositories (Purchase, Hamer, Denny and Luxton-Reilly, 2010), and
that there are measurable benefits to student learning (Denny, Hanks and Simon, 2010). Although
PeerWise has been used in a range of disciplines, this is the first report of the use of PeerWise with
students of Veterinary Medicine.
The University of Glasgow Veterinary undergraduate cohort is diverse, with a high number of
graduate students and overseas students, 31% and 40% respectively in 2010-11, with a range of
educational backgrounds. The high qualification tariff for entry, along with expectations of
achievement in areas other than academia including the acquisition of many hours of animal
experience, results in selection of academically gifted and highly motivated individuals. In addition, the
veterinary undergraduate curriculum is extremely intensive in terms of scheduling of classes and
volume of course material. These combined factors result in a cohort of highly strategic learners. We
elected to trial PeerWise with all 141 1st year veterinary students in Veterinary Biomolecular Sciences
as their enthusiasm levels are generally high, the workload is lighter than that in later years, and a
familiarity with and appreciation of the benefits of new technologies could be harnessed in later years
of their undergraduate degree.
2. Methodology
The Veterinary Biomolecular Sciences course is taught over the first and second years of the
Glasgow Bachelor of Veterinary Medicine and Surgery programme. Teaching is conducted primarily
by lectures supplemented with small group tutorials, laboratory and computer-based classes and a
student-directed learning assignment. The first year course comprises 11 topics delivered in two
terms from September until March. Assessments comprise 1 class exam at the end of term 1 which
contributes 10% to the year mark, and a professional exam in May, contributing 85%, both of which
include MCQs and short answer questions. As an incentive to participate in the PeerWise task, the
final 5% of the professional mark was assigned to this task and in addition students were informed
that some of their questions might be used in the class exam. In order to obtain these marks students
were required to meet 8 deadlines as indicated in Table 1. The total requirement per student was
therefore to author 4 questions and to answer 40. These were minimum participation requirements,
and students were free to contribute to a greater extent if they wished.
Table 1: PeerWise deadlines in Veterinary Biomolecular Sciences course
Deadline Task Date
1 Submit 1 MCQ on topics: cell biology, proteins and enzymes,
molecular biology
Mon 25th October 1700hr
2 Answer 10 MCQ Mon 8th Nov 1700hr
3 Submit 1 MCQ on topics: metabolism, biostatisics, genetics Mon 22nd Nov 1700hr
4 Answer 10 MCQ Mon 6th Dec 1700hr
5 Submit 1 MCQ on topics lipids and nitrogen Mon 24th Jan 1700hr
6 Answer 10 Term 2 MCQ Mon 7th Feb 1700hr
7 Submit 1 MCQ on topics nutrition, blood and signalling Mon 28th Feb 1700hr
8 Answer 10 Term 2 MCQ Mon 7th Mar 1700hr
Peerwise use was scaffolded by an introductory interactive session lasting an hour. Following a brief
overview of the session, students participated in an 8 question quiz (Nora Mogey, personal
communication allowing use of questions, originally sourced from Race and Lewis), highlighting the
limitations of poorly written MCQs. Context-based MCQs were then presented illustrating poorly and
well formed biomolecular sciences MCQs, the latter including simple and more complex examples.
Good and bad aspects of MCQs were reiterated before moving on to an introduction to the PeerWise
interface with screen-shots of each of the main screens. We emphasised the benefits to students of
engaging with the task, showing exam performance improvement data from published studies (Denny,
Hamer, Luxton-Reilly and Purchase, 2008) and informing them of the 5% they would gain for their
822

Amanda Sykes et al.
final year mark if they met all deadlines and submission criteria. The session was closed with practical
information on how to log in to PeerWise and the specifics of the assessed task.
Data was collected via the administration menu in PeerWise in the form of student activity profiles and
database content, for the period 20 th September through to the final deadline, 7 th March. Student
opinions on the PeerWise tool were gathered through standard course quality assessment
questionnaires and through a specific survey, conducted on SurveyMonkey, including Likert-based
and open-response questions (Table 2 and Table 3). Comments made in response to the openresponse
questions were coded, as driven by comment content. Some individuals made comments
relevant to more than one of the assigned codes, in which case all relevant codes were attributed.
The coded data was then allocated into categories from which overarching themes were identified
(Denscombe, 2010, p286-295). Finally, student performance in a summative exam (the class exam
originally scheduled for December but postponed until January due to bad weather) was assessed
relative to student engagement, on an individual basis, with PeerWise. For the purposes of this paper,
results presented focus on student engagement with PeerWise, student perception of the tool and
correlation between engagement and class exam grade.
Table 2: Likert questions from the SurveyMonkey survey
Q1 Developing an original question on a particular topic reinforced what I knew about that topic
and improved my understanding of the material
Q2 Answering questions written by other students helped reinforce what I knew about the subject
and improved my understanding of the material
Q3 Reading other students' comments about my questions helped me to learn
Q4 I liked to see how other students rated my questions
Q5 I thought that PeerWise was innovative
Q6 This year I found the process of developing original questions and answering other students'
questions to be useful and I would like to use PeerWise again in the future
Table 3: Open-response questions from the SurveyMonkey survey
Q7 What do you believe is the biggest benefit of using PeerWise?
Q8 What aspects of using PeerWise did you find most useful, interesting or enjoyable?
Q9 What do you believe is the biggest problem with PeerWise?
Q10 Can you recommend something that would make PeerWise more valuable or effective for
learning in Veterinary Medicine?
3. Results
3.1 PeerWise data
PeerWise was available to students throughout the Biomolecular Sciences course from the 20th
September, 2010 until the end of the academic year. Of the 141 students enrolled in the course, all
participated in PeerWise and all but 15 met the minimum requirements. From 20th September until
the final deadline, 7th March, a total of 795 questions were authored. Of the 17859 answers submitted
to these questions, 11725 (66%) matched the correct answer indicated by the question author.
With respect to question authoring, individual contributions ranged from 3-16 and the mean number of
questions authored per student was 5.6 (median = 5). The mean number of questions answered was
126.4 (median = 78) with a range of 32-785. So, students on this course answered many more
questions than required, although the number of questions they authored was similar to requirements.
Figure 2 shows the total number of questions authored and Figure 3 the total number of answers
submitted each day from the start of term until the final PeerWise deadline. The four spikes in activity
with respect to questions authored correspond to the four question authoring deadlines. Students’
823

Amanda Sykes et al.
activity became increasingly focused with each deadline, with practically no new questions authored
by students in the weeks between deadlines two and three, and three and four. Activity spikes are
also evident around deadlines three and four for answer submission, but are less obvious for the first
two. Of interest is the large spike of activity on the 13th January, the day before the class exam. As no
course credit was on offer during this time, this activity corresponds to voluntary use of the tool as a
resource for revision.
Figure 2: Number of questions authored per day
Figure 3: Number of answers submitted per day
3.2 Exam grade data
A scatter plot of the total number of answers submitted by each student in the course when using
PeerWise, against their class exam grade revealed heavy skewing of data due to a small number of
students answering a very large number of questions. The data was log-transformed to correct for this
and the resulting scatter plot is shown in Figure 4. The relationship between class exam grade and
the log of the number of questions answered is not very strong, which is to be expected given the
many factors involved in determining exam performance. However, a medium-strength (de Vaus,
824

Amanda Sykes et al.
2001) positive correlation exists (r = 0.34), and this relationship is highly statistically significant (p

Amanda Sykes et al.
students either “agreeing” or “strongly agreeing”. Q3 and Q4 drew the least favourable responses;
some students did not find the comments or ratings provided by their peers about their own questions
very valuable to their learning. As shown by the response to Q5 and Q6, the majority of the class
found PeerWise innovative and would wish to use it in the future.
Figure 5: Exam grade plotted against log (number of days of activity)
Figure 6: Responses to Likert questions from the SurveyMonkey survey
The feedback we received to the open-response questions was interesting. The students described
PeerWise as beneficial for revision, learning and receiving feedback (Q7: What do you believe is the
biggest benefit of using PeerWise? Figure 7) and these aspects were also reflected in their response
826

Amanda Sykes et al.
to Q8; What aspects of using PeerWise did you find most useful, interesting or enjoyable? (Figure 8).
Consistent with the result for Q4 (Figure 6), a number of students felt that the comments written by
their peers were problematic (Q9: What do you believe is the biggest problem with PeerWise? Figure
9). Students also commented on poorly written and unrepresentative questions, and many suggested
that moderation by staff would solve these problems. The issue of moderation was raised again in
response to Q10: Can you recommend something that would make PeerWise more valuable for
learning in Veterinary Medicine? (Figure 10). Students also expressed a desire for PeerWise use to
be extended to other courses in the Veterinary curriculum.
Category
Q7 :What do you believe is the biggest benefit of using
PeerWise?
Revision
Learning
Factual knowledge
Feedback
Appreciation of question style
Easy 5%
No benefit
Anytime, anywhere
Sense of personal responsibility
0 5 10 15 20 25 30 35 40
Frequency
Figure 7: Responses to question 7 from the SurveyMonkey survey
Category
Q8 : What aspects of using PeerWise did you find most useful,
interesting or enjoyable?
Revision
Feedback
Enhanced understanding
Writing questions
None
M otivational aspects
Diversity of style
Anonymity
Use of questions in exam
Quality
Grouping by topic
Competition
0 5 10 15 20 25 30
Frequency
Figure 8: Responses to question 8 from the SurveyMonkey survey
827

Category
Amanda Sykes et al.
Q9 : What do you believe is the biggest problem with
PeerWise?
Comments
Incorrect questions
No question moderation
Non representative questions
Lack of peer effort
Repetion of questions
Ratings
Nothing wrong
Grading process
Timing of deadlines
Too many questions
Hard to write a good question
Low efficiency
Doesn’t cover entire curriculum
Not able to repeat answering questions
0 2 4 6 8 10 12 14 16 18 20
Frequency
Figure 9: Responses to question 9 from the SurveyMonkey survey
Category
Q10 : Can you recommend something that would make
PeerWise more valuable or effective for learning in veterinary
medicine?
M oderation
Use in more courses
Alter assessment strategy
Specimen questions by staff
Remove bad questions
Update interface
Ability to re-answer questions
Quality grading
De-anonymise
Poll after each question
Ensure even topic distribution
Stop rating questions
No
0 2 4 6 8 10 12 14 16
Frequency
Figure 10: Responses to question 10 from the SurveyMonkey survey
4. Discussion
This first report of PeerWise use with students of Veterinary Medicine manifested many positive
outcomes:
Student compliance with the deadlines required to be met for the continual assessment mark was
high with 100% of students meeting a minimum of five deadlines and 90% meeting all eight,
despite only 5% being allocated to their final professional exam mark.
Student engagement with PeerWise was high: over half the class authored more questions than
the minimum although most did not author significantly more than required (consistent with
828

Amanda Sykes et al.
studies in other disciplines). Nearly two thirds of the class answered more questions than
required.
Elevated engagement by 30% of students in the two week periods prior to the original and
rescheduled class exam dates, indicated a recognition of the worth of the tool for exam revision.
Student feedback was predominantly affirmative, with the majority of positive comments focusing
on the deeper learning aspects of the tool and its value for feedback and exam revision purposes.
Positive correlation was evidenced between PeerWise engagement and the class exam grade:
the data shows a positive relationship between number of questions answered and final exam
grade, and between the days of activity with PeerWise and final exam grade. In both cases,
although the relationships are only moderately strong, they are highly statistically significant (p

Amanda Sykes et al.
Chang, S-B., Huang, H-M., Tung, K-J. & Chan, T-W., 2005. AGQ: a model of student question generation
supported by one-on-one educational computing. In: Conference on Computer Support for Collaborative
Leraning 2005. Taipei, Taiwan. May 30 – June 4 2005. International Society of the Learning Sciences.
Denny, P., Hamer, J., Luxton-Reilly, A. & Purchase, H., 2008. PeerWise: Students Sharing Their Multiple Choice
Questions. International Computing Education Reserach Workshop 2008. Sydney, Australia 6-7 September
2008. ACM, New York, NY, USA. Available at:
Denny, P., Hanks, B., & Simon, B., 2010. Peerwise: replication study of a student-collaborative self-testing web
service in a U.S. setting. SIGCSE 2010. ACM, USA, pp.421-425.
Denny, P., Luxton-Reilly, A. & Hamer, J., 2008a. The PeerWise system of student contributed assessment
questions. In: Simon Hamilton and Margaret Hamilton (Eds.), Proceedings of the tenth conference on
Australasian Computing Education - Volume 78 (ACE '08), Wollongong, Australia. 22-25 January 2008.
Australian Computer Society, Inc.
Denny, P., Luxton-Reilly, A. & Hamer, J., 2008b. Student use of the PeerWise system. SIGCSE Bull. June, 40
(3), pp.73-77. Available at:
Denny, P., Luxton-Reilly, A., Tempero, E. & Hendrickx, J., 2011. CodeWrite: supporting student-driven practice of
java. SIGCSE 2011. ACM, USA, pp.471-476.
Denscombe, M. 2010. The good research guide for small-scale research projects, 3 rd edition, pp286-295. Open
University Press, Berkshire, UK.
Draaijer, S. & Boter, J., 2005. Questionbank: Computer Supported Self-Questioning. In: M. Danson (Ed.), 9th
International Computer Assisted Assessment Conference. Loughborough, UK. 5-6 July 2005
Foursquare, 2011. Check In, Find Your Friends, Unlock Your City. [online] Available at http://foursquare.com/
[Accessed 21 June 2011]
Kriplean, T., Beschastnikh, I. & McDonald, D.W., 2008. Articulations of wikiwork: uncovering valued work in
wikipedia through barnstars. In: Proceedings of the 2008 ACM Conference on Computer Supported
Cooperative Work. San Diego, USA. 8-12 November 2008.
Laidlaw, A., Guild, S. and Struthers, J., 2009. Graduate attributes in the disciplines of Medicine, Dentistry and
Veterinary Medicine: a survey of expert opinions. BMC Medical Education, 9(28). Available at:
Luxton-Reilly, A., Plimmer, B. & Sheehan, R., 2010. StudySieve: a tool that supports constructive evaluation for
free-response questions. In: Computer-Human Interaction New Zealand 2010. Auckland, Australia. 8-9 July
2010. ACM, USA.
Nicol, D., 2007. E-assessment by design: using multiple-choice tests to good effect. Journal of Further and
Higher Education, 31 (1) pp.53-64.
Purchase, H., Hamer, J., Denny, P. & Luxton-Reilly, A., 2010. The quality of a PeerWise MCQ repository. In:
Tony Clear and John Hamer (Eds.) Proceedings of the Twelfth Australasian Conference on Computing
Education (Australasian Computing Education Conference 2010). Brisbane, Australia, 18-22 January 2010..
Australian Computer Society, Inc., Darlinghurst, Australia.
Rafaeli, S., Barak, M., Dan-Gur, Y., & Toch, E., 2004. QSIA: a web-based environment for learning, assessing
and knowledge sharing in communities. Computers & Education. November 43 (3), pp.273-289.
Rosenshine, B., Meister, C. & Chapman, S., 1996. Teaching students to generate questions: A review of the
intervention studies. Review of Education Research, 66 (2), pp.181-221.
RCVS (Royal College of Veterinary Surgeons), 2010. Day One and Year One Competences. [online] Available at
http://www.rcvs.org.uk/document-library/day-and-year-one-competences/ [Accessed 21 June 2011]
StackOverflow, 2011. Badges. [online] Available at http://stackoverflow.com/badges [Accessed 21 June 2011].
Tierney, A.M. & Sykes, A. K., 2011 (in press). Using Peerwise in a Large First Year Biology Class. Self and Peer
Assessment: Guidance on Practice in the Biosciences, 2nd Edition, Steve Maw, (Ed), Higher Education
Academy UK Centre for Bioscience, Leeds, UK.
de Vaus, D., 2001. Research Design in Social Research. 1 st edition. London: Sage Publications Ltd.
Wessa, P., 2011. Free Statistics Software, Office for Research Development and Education, version 1.1.23-r6,
Available at:
Yu, F. Y., Liu, Y. H., & Chan, T., 2005. A web-based learning system for question posing and peer assessment.
Innovations in Education and Teaching International, 42 (4), pp.337-348.
Yu, F. Y. & Hung, C. C., 2006. An empirical analysis of online multiple-choice question-generation learning
activity for the enhancement of students’ cognitive strategy development while learning Science. Lecture
series on computer and computational sciences: Recent progress in computational sciences and
engineering, pp.585-588. Chania, Crete, Greece.
830

iSELF: An Internet-Tool for Self-Evaluation and Learner
Feedback
Nicolet Theunissen and Hester Stubbé
Dep. Training Innovations, TNO, Soesterberg, The Netherlands
nicolet@ntheunissen.nl
hester.stubbe@tno.nl
Abstract: This paper describes the development of the iSELF: an Internet-tool for Self-Evaluation and Learner
Feedback to stimulate self-directed learning in ubiquitous learning environments. In ubiquitous learning, learners
follow their own trails of interest, scaffolded by coaches, peers and tools for thinking and learning. Ubiquitous
learning solutions include on- and off-line, formal and informal learning. To benefit from its possibilities, learners
need to develop competencies for self-directed learning. To do so, a self-evaluation tool can help the learner to
get insight in his/her own development, to manage and monitor his/her own learning process, to collaborate in
learning, to relate the learning to 'real life' needs, and to take control over educational decisions. The iSELF is
developed in an iterative process in three phases, complying to the following high level requirements: (1)
Enabling learning anytime, anywhere; (2) Supporting self-directed learning; (3) Evaluating learner, learning
solutions and job-needs; (4) Assessing learner competencies; (5) Using card-sort method for questionnaires; (6)
Facilitating questionnaires 'under construction'; and (7) User-friendly design. The resulting online tool contained a
card-sort module, looking somewhat like a 'solitaire' game, a profile module to evaluate core competencies, and a
feedback module to suggest learning possibilities in a ubiquitous learning environment. The iSELF was
developed to be not only a learner tool but a scientific tool as well. Using the tool, two self-evaluation
questionnaires were developed and psychometrically tested: competencies for multidisciplinary cooperation in a
Network Centric Organization and self-directed learning competencies. They are used in various populations: e.g.
students, employees from small and medium enterprises, crisis management organizations, and the military.
Usefulness and usability of the self-evaluation tool were valued positively. It contributes to an adaptive ubiquitous
learning environment in which the learner can make the educational decisions according to self-directed learning
principles. The iSELF will stimulate self-directed learning in a ubiquitous learning environment and will help to
create learners for life.
Keywords: self-evaluation, self-assessment, internet-tool, ubiquitous learning, self-directed learning, feedback.
1. Introduction
1.1 Ubiquitous learning needs self-directed learners
Technology is very much part of everyday life and work. Information and knowledge is handled and
shared by using ubiquitous technology; ICT that makes it possible to access information ‘anytime,
anywhere’ (Adkins et al. 2002). A learning environment that makes use of this technology is often
referred to as ‘ubiquitous learning’. This is a way of learning in which learners follow their own trails of
interest, scaffolded by coaches, peers and tools for thinking and learning (Dieterle & Dede 2007).
Ubiquitous learning solutions include on- and off-line, formal and informal learning.
The availability of ubiquitous learning possibilities might assume that learners are able to learn and
will develop themselves anytime, anywhere (Chen, Chang, & Wang 2008). However, this assumption
might be too ambitious. Since the ability to manage one’s own learning is becoming increasingly
important, one of the goals of education should be to create learners for life (Du Bois & Staley 1997).
Learners for life can be described as learners who have a flexible and pro-active attitude toward
learning and developing themselves. In this context the concept of self-directed learning is often
mentioned and intensively discussed (Collins 2004). In a review study of Stubbé &Theunissen (2008),
five crucial elements of self-directed learning were identified:
Learner control: Control over educational decisions and learning process.
Self-regulating learning strategies: Skills that support the learner to manage and monitor his/her
own learning process (e.g. setting goals, planning, problem solving, and strategy use).
Reflection: The combination of self-assessment and self-evaluation on both the performance and
the learning process that gives the learner insight in his/her own development.
831

Nicolet Theunissen and Hester Stubbé
Interaction with the social environment: The interaction with others, learners and
teachers/coaches, in order to determine what goal should be set, discuss in what way this goal
can be achieved, cooperate and collaborate during the learning process and ask for help.
Interaction with the physical environment: The learning experience should be set in the ‘real
world’ and should relate to ‘real-life’ (work) situations.
Reality shows that some people develop a self-directed attitude toward learning, especially in relation
to work or a hobby, others do not (Collins 2004). Explicitly teaching self-regulating learning strategies
or reflection and stimulating (perceived) learner control, helps learners to become more self-directed
(Stubbé & Theunissen 2008). Therefore, a ubiquitous learning system must not only provide the
learner with learning resources anytime and anyplace. It must also actively provide the learner with
the appropriate learning assistance for self-directed learning (Hiemstra 2006;Wang & Wu 2011).
To become a self-directed learner, one should get insight in one’s own development. A self-evaluation
instrument might be helpful. Self-evaluation (also named 'self-monitoring' or 'self-assessment') refers
to an individual's systematically observing his/her own behaviour and performance. The learner
makes a comparison between the noted behaviour and some designated standard. Although selfevaluation
presumes higher order cognitive skills, it is proven to be possible even in populations of
people with mild cognitive retardation (Hughes, Korinek, & Gorman 1991). Self-reports have their
limitations, but there are indications that self-reported abilities and competencies have some
concurrent validity (Kelso, Holland, & Gottfredson 1977). Above that, ‘People may not be right about
themselves, but their self-evaluations are the ones that most powerfully affect their future behaviour’
(page 45, (Byrnes 1984). As a result, self-evaluations are relevant for learner behaviour. However,
peer or expert observation can help learners to evaluate their own opinion about themselves.
Thus, self-evaluation will help the learner to get insight in his/her own development, to manage and
monitor his/her own learning process, to collaborate in learning, to relate the learning to 'real life'
(work)needs, and to take control over educational decisions. In this way, all five elements of selfdirected
learning, mentioned before, will be stimulated.
1.2 Assessment of competencies
For the question: ‘What to evaluate?’ we specifically look at the fifth element of self-directed learning:
Interaction with the physical environment. Learning needs to be related to ‘real-life’ (work)situations.
In our rapidly changing society, initial training alone cannot meet the need for the development of
working individuals. Training results become obsolete the moment they are obtained. A flexible and
innovative economy requires permanent adaptations of knowledge, skills and attitudes, also called
'competencies'. Competencies are indivisible clusters of skills, knowledge, conduct, attributes and
notions. They are context dependent, connected to activities and tasks, but also flexible in time (van
Merriënboer, van der Klink, & Hendriks 2002). In their essence, competencies can be used in more
situations than the current task. This means that when one's job changes, the acquired specific skills
become obsolete, but the acquired competencies can still be useful. Nevertheless, in our fast
changing society it is possible that competencies themselves become obsolete or less important.
Another characteristic of competencies is that they can be acquired by learning and development.
Competencies can be valuable to match individual performance and career planning with
organizational job needs (Whan Marko & Savickas 1998). In that context the concept 'core
competencies' is used, competencies which are essential for certain tasks or positions (Case 2003)
and as such will provide the content for the relationship with 'real-life’ work situations.
Reporting on one's behaviour poses a difficult cognitive task and participants' reports are influenced
by question wording, format, and content (Schwarz & Oyserman 2001). A self-evaluation instrument
in a ubiquitous learning environment needs an easy to use, reliable method to gather information on
selected competencies. A card-sort method is such a method, with good psychometric characteristics
(Lievens & Sanchez 2007). Card-sorts involve the placement of cards onto piles, based on how each
participant feels the concepts or statements on them are related. When using this method for
evaluation of competencies, competency statements can be placed on the cards. Former research
with this technique showed that people are able to sort a large number of separate cards in a
relatively short time (Caldwell & O'Reilly 1990), which will increase learners’ motivation to use it for
evaluation. The technique is particularly useful for identifying the common ground between a larger
832

Nicolet Theunissen and Hester Stubbé
and diverse collection of competencies with a large and diverse group of participants (Caldwell et al.
1990).
There are two approaches in card-sort: the free and the restricted procedure (Harper, Jentsch, Berry,
Lau, Bowers, & Salas 2003). In the free sort, a participant is allowed to make as many piles of related
cards as necessary, and label them. In the restricted sort, a participant uses piles that are already
defined (e.g. questionnaire Likert scales like 'not applicable at all' to 'totally applicable'). This enables
the use of statistical techniques to cluster related competency statements into core competencies.
The restricted card-sort relies less on the categorization skills of the participants. This makes it useful
for a sample with various levels of education and experience, as is the case in a ubiquitous learning
environment. Moreover, a standardized categorization makes it possible to make learner profiles that
can be related to peers and that can show development over time. The same categorization can also
be used to evaluate job-needs or the available learning solutions. An automatic match of the learner
profiles with the learning solution profiles will show if they are beneficial to the learner. The results
should be presented as suggestions so that the learner can make the educational decisions according
to self-directed learning principles.
The aim of this paper is to describe the development of the iSELF: an Internet-tool for Self-Evaluation
and Learner Feedback to stimulate self-directed learning in a ubiquitous learning environment. The
characteristics of the iSELF and the first experiences, based on the developments so far, will be
discussed.
2. The iSELF itself
2.1 High level requirements
With the introduction text in mind, a set of requirements was developed for the iSELF:
Enabling learning anytime, anywhere: available for every learner through internet, with the
possibility to embed the tool into Learning Management Systems.
Supporting self-directed learning: helping the learner to get insight in his/her own development, to
manage and monitor his/her own learning process, to collaborate in learning, to relate the learning
to 'real life' (work)needs, and to take control over educational decisions. To support control and to
provide a 'safe' learning environment, it is important that the learner is the only one who can see
personal evaluation results, until he/she decides otherwise.
Evaluating learner, learning solutions and job-needs: possibility of using the same content: (a) to
score the learner competencies of all kinds of learners (e.g. low and high educated), (b) to assess
the learner competencies by the learners themselves or by peers, colleagues, coaches or subject
matter experts, invited by the learner, (c) to score which competencies are trained by certain
learning solutions or are relevant for a job or position. Using the same content in all situations
enables a comparison between them.
Assessing learner competencies: assessing competencies that are specific for a group of learners
in their (work)situations, and at the same time generic enough to remain relevant in our rapidly
changing society.
Using card-sort method for questionnaires: this technique is less time-consuming and more
objective than other methods. This will increase motivation to use it in a large and diverse group
of participants.
Facilitating questionnaires 'under construction': with new developments in the workplace, new
competencies will become important. Therefore, new questionnaires will be developed all the
time.
User-friendly design: most people do not like questionnaires. A playful appearance, user-friendly
operation and clear, relevant content will increase motivation.
2.2 Design
The overall structure of the iSELF is presented in Figure 1 and explained in the next paragraphs.
833

Figure 1: Overall structure
Nicolet Theunissen and Hester Stubbé
2.2.1 Users
The overall structure includes the following iSELF users:
Learner: the learner in a ubiquitous learning environment.
Invitees: peers, colleagues, coaches or subject matter experts, invited by the learner to assess
the learner.
Educational expert: the evaluator of available learning solutions.
Subject matter expert: the evaluator of competency requirements for a job or position.
Administrator: administrates the tool content: competency statements, information about core
competence clusters and reference group- or norm figures.
2.2.2 Card-sort
The appearance of the card-sort tool, a front-end input module, is somewhat like a 'solitaire' game
(see Figure 2). Instead of sorting playing cards, learners sort competency statement cards on their
importance. The tool offers the possibility to get an overview or skip a statement temporarily.
The competency statements are formulated in such a way that they are easy to comprehend for
people with different backgrounds. A second person singular verb at the beginning of every statement
emphasises the fact that competencies express themselves in behaviour (E.g. ‘Cooperates with
people from other organizations’ or ‘Uses ICT systems to collect information and knowledge quickly’.)
These statements can be used with different overall opening questions. For the learner the opening
question could be: 'In the last two weeks, was this applicable to you?’ For the invitee it would be: 'In
the last two weeks, was this applicable to [name learner]?’ For the use with the learning solutions of
job-requirements: ‘Is this applicable to [name learning solutions or job]?’ The card-sort module was
built using Adobe Flex, an open source framework for building rich Internet applications that are
delivered via the Flash Player 6.0.
834

Figure 2: Card-sort module in input mode
Nicolet Theunissen and Hester Stubbé
2.2.3 Profiles
The profile module shows the results of the card-sort input to the learner. After analyses, the
competency statements are clustered into core competencies. The results are presented to the
learner in one or more graphs. It can show learner results in combination with reference scores or
scores from former sessions. The profile module was built using ColdFusion 8.0, a rapid application
development platform that includes advanced features for enterprise integration and enables the
development of rich Internet applications.
2.2.4 Feedback: Suggestions for further learning
The feedback module can be used to compare perspectives by interpreting the results of both the
learner and the invitee. The feedback can also be used to provide suggestion for learning solutions
beneficial to the learner or for job-training and selection. The feedback is based on an automatic
match of the learner scores with the learning solution scores. The feedback module was built using
ColdFusion 8.0.
2.2.5 Administration
The administration module, a back-end module, combines several input functions: overall opening
question, statements, number and content of answering categories, assignment of statements to
clusters, personal accounts, data export, reference or norm score input. The module was built using
ColdFusion 8.0 running Railo Server, the main version of Railo (a compiler) which can be integrated
into standard web servers. It is suitable for production use.
2.2.6 Content: psychometric Sound questionnaire
The content of the iSELF card sort module is created according to social sciences standards in
questionnaire construction (Schwarz et al. 01). At any moment in the development of the
questionnaire it is possible to export the data for statistical analyses for validation and reliability tests.
Using a.o. principal component analysis and Cronbach's alphas, clusters can be identified and
transformed into scale scores. The results from these analyses can be fed back through the
administration module. The data export option can also be used to perform additional statistical
835

Nicolet Theunissen and Hester Stubbé
analyses for group comparisons and determination of the influence of background variables like age
or experience. These possibilities make the iSELF not only a learner tool but a scientific tool as well.
Figure 3: A learner profile
Figure 4: Administration of competencies
836

3. The iSELF: Development in phases
Nicolet Theunissen and Hester Stubbé
The development of the iSELF was an iterative process in which the modules (card-sort, profile and
feedback) were prototyped, built and tried successively. Experiences were gathered using many
different studies. More information about these studies can be obtained from the authors.
3.1 Phase 0: Prototyping with a paper card-sort tool
3.1.1 The paper-based iSELF
In this paper-based version, the iSELF only contained the card-sort module. The selected
competence statements were placed on a set of individual paper cards. Together with this set of
cards came five envelopes with the labels ‘not important’, ‘somewhat important’, ‘important’, ‘very
important’ and ‘essential’. A participant could express the importance of a competency statement for a
position by assigning cards to envelopes. In this way, a 5-point Likert-scale was created for each
statement.
3.1.2 Context
The paper-based prototype was used for career planning using core competencies in a study group
consisting of four different training-related functions in the Army. Semi-structured interviews and a
document study resulted in 65 cards with competency statements. Subject matter experts sorted the
cards twice for their own function: on expert and on novice level. In a group discussion both the
method and the content of the cards was discussed. Using principal component analysis, five core
competencies could be identified and a career path differentiation was recommended.
3.1.3 Experiences
The participant sorted the large set of statements cards quickly: each round of 65 cards took about 5
to 15 minutes. Most participants were positive about the procedure and preferred it to a standard
questionnaire. The card-sort module was considered useful.
3.2 Phase 1: Learner-profiling
3.2.1 The iSELF alpha version
This version contained an internet based card-sort module and a half-automatic prototype of the
profile module. The card-sort module was transformed in an online version like a 'solitaire' web-game.
The prototype profile module used MS Excel to produce a graph and MS Word for handmade
individual reports.
3.2.2 Context
The alpha version was tested in two steps: in ten studies the card-sort module was tried-out and in
two studies the profile module was added and presented to the learners.
First, two different self-evaluation questionnaires were developed and psychometrically tested: (1)
Competencies for multidisciplinary cooperation in a Network Centric Organization and (2) Selfdirected
learning competencies. These questionnaires were tested in several studies with the
following populations: e.g. students, employees from various small and medium enterprises, crisis
management organizations (fire department, police force, medics), and the military. The iSELF was
presented to the participant from within a learning management system (ILIAS, MOODLE) or with a
hyperlink in an email. The data-output of the card-sort module was used to identify core competencies
using statistical package SPSS.
Next, two of these groups received a personal profile about their Self-directed learning competencies.
A group of Navy personnel received a profile after a pre-post-test intervention study. The profile
showed the individual results on three points of measurement and the overall average scores of the
whole group. Afterwards they were interviewed. The other group, studying Public Management,
received a profile that was discussed with them to influence their learning style.
837

Nicolet Theunissen and Hester Stubbé
3.2.3 Experiences
The card-sort module of the iSELF alpha version was used successfully: the technique worked,
questionnaires could be validated and the participants valued the module positively. The profile
prototype was evaluated as useful in the communication with the participant.
3.3 Phase 2: Prototyping the match of learner profiles with learning solutions profiles
3.3.1 The iSELF beta version
This version contained the complete iSELF. Although complete, it is a beta version because not all
functions were fully tested.
3.3.2 Context
The beta version was tested in two steps: First, the profile and feedback modules were used as an
addition to a 'classic' online-questionnaire embedded in MOODLE. It was developed for selfassessment
and invitee-assessment of leadership competencies in employees from the Air Force.
The suggestions that the participants received from the feedback module included formal (training, elearning)
and informal learning possibilities (e.g. documents, discussion, movies). The employees
could choose to use the iSELF or not and could invite others to assess them if they wished. The next
step was a usability pilot with the complete iSELF that was carried out with employees from crisis
management organizations. Afterwards participants were interviewed about their evaluations.
3.3.3 Experiences
In the Air Force case, using the iSELF was voluntary. Many employees did use it and their numbers
are still increasing. In the usability pilot, participants were enthusiastic about the iSELF. One of the
conclusions was that matching the learner profile with learning possibilities saves time and will
improve adaptation of the learning environment to the learner. These experiences indicate that the
tool is usable and useful.
4. Discussion and conclusions
This paper describes the iterative development, testing and evaluation of the iSELF: an Internet-tool
for Self-Evaluation and Learner Feedback. The tool is designed to stimulate self-directed learning in a
ubiquitous learning environment and our experiences so far confirm its usefulness.
The experiences with the card-sort module in a large and diverse group of participants proved that the
technique was highly appreciated. When they had the possibility to use it anytime and anywhere,
participants used it voluntarily. The playful appearance, the user-friendly operation and the clear,
relevant content increased motivation to use it.
The profile module helps the learner to get insight in his/her own development in relation to the
competencies important for his/her work. These competencies always need to be identified before
they can be used in the iSELF. We emphasize the importance to assess competencies that are
specific for a group of learners, and at the same time generic enough to remain relevant in our rapidly
changing society. This helps the learner to reflect on 'real life' (work)needs.
The possibility to compare their results with previous results or with those of invited peers, colleagues,
coaches or subject matter experts, seemed to improve reflection as intended. In future, it is possible
to include e-coach possibilities that will stimulate reflective competencies more explicitly.
The feedback module presents suggestions for learning, which helps the learner to manage and
monitor his/her own learning process and to take control over educational decisions. It is technically
possible to use the iSELF for selection purposes by combining the job-requirements with the
individual competencies. However, to support learner control and to provide a 'safe' learning
environment, it is important that the learner is the only one who can see personal evaluation results,
until he/she decides otherwise. Therefore, we use this tool exclusively for learning or career
suggestions and not for selection. Nevertheless, it is possible to combine the individual results and
present them on a group-level, for organizational purposes.
838

Nicolet Theunissen and Hester Stubbé
One of the requirements for the iSELF is that it should support learning anytime, anywhere. Some
organizations use Learning Management Systems (LMS) to provide learning solutions like e-learning,
others use the LMS as a portal and offer links to learning solutions outside the LMS. Therefore, the
iSELF needed the possibility to be embedded into Learning Management Systems. The beta version
was tested both inside and outside a LMS and could thus be used anytime and anywhere. LMS also
offer the possibility to monitor the learner’s progression. When the iSELF is used within an LMS, it is
possible to make the learner profiles available for monitoring. Again, profiles should be used to show
development over time and not for assessment purposes. As described in the introduction, ubiquitous
learning is a way of learning in which learners follow their own trails of interest, scaffolded by
coaches, peers and tools for thinking and learning (Dieterle & Dede 2007). It includes on- and off-line,
formal and informal learning. To support all that, the iSELF should be available through internet
independent of the learning solution chosen. iSELF was not tested for off-line learning solutions and
therefore we do not know if learners who prefer off-line solutions will use the on-line iSELF. Future
research should look into this limitation. However, the other way around does not present any
problems: the on-line iSELF can refer to off-line learning solutions. The profile of any learning solution
that is evaluated by educational experts can be included, As such the iSELF can be used for both
formal and informal learning.
In a self-directed ubiquitous learning environment there is no pre-defined learning content and
learners can select content from on- and off-line, formal and informal learning (Gütl, Lankmayr,
Weinhofer, & Höfler 2011). Especially informal learning can help learners to collaborate with others
when learning, a requirement for self-directed learning. Of course, it is almost impossible to provide
profiles of all possible learning solutions. To start with, the most useful or important learning
possibilities for a certain group of learners can be profiled. In addition, it might be possible to ask
learners for suggestions for useful learning solutions, which in itself increases collaborative learning.
There are interesting developments like the Automatic test item creation (Gütl et al. 2011) or the
personalisation of web-based learning solutions to knowledge level, goals and other characteristics of
individual learners (Papanikolaou, Grigoriadou, Magoulas, & Kornilakis 2002). These kinds of
adaptations prevent the learner from getting lost in the course materials by providing personalized
learning guidance, filtering out unsuitable course materials to reduce cognitive loading (Barker 2011).
Perhaps in the future, they can be helpful. However, one has to bear in mind that for self-directed
learners, technology must not take away control from the learner, but provide stimuli to increase
competencies for self-directed learning.
A limitation of the presented work is that the development in phases was performed using many
different studies and convenience samples. Some of the development iterations would have been
different if a dedicated science program with sharply defined samples could have been used. On the
other hand, the large amount of different studies provided many challenges that helped us to develop
a better instrument. As a result the iSELF was developed to be not only a learner tool but a scientific
tool as well. Extensive analyses could be made using the output of the card-sort. Plus-point of the
card-sort was the absence of missing data and a good response. It facilitated questionnaires 'under
construction' so one can keep up with new developments and new competencies needed in the
workplace.
The European labour market is faced with challenges such as an ageing labour force, increased
competition from emerging countries and on-going changes in technology and employers’ demands.
A flexible and innovative economy requires permanent adaptations of knowledge, skills and attitudes.
Formal, initial training alone cannot meet the need for the development of working individuals to face
these challenges. There is, therefore, a growing need for self-directed, flexible and innovative
employees who can and will keep on learning throughout their entire lifespan. Research had shown
that fostering students to become self-regulated learners is complicated and should be seen as a
long-term process (van den Boom, Paas, & van Merriënboer 2007). The iSELF will stimulate selfdirected
learning in a ubiquitous learning environment and will help to create learners for life.
Acknowledgements
We like to express our deepest appreciation to R. van Rijk, J.P. Sassen-van Meer, C. Six, M. van
Schaik, J. Roes and P.G.J. Storm van Leeuwen for sharing their knowledge with us. We are indebted
to many people for their participation in our projects.
839

References
Nicolet Theunissen and Hester Stubbé
Adkins, M., Kruse, J., & Younger, R. 2002, "Ubiquitous Computing: Omnipresent Technology in Support of
Network Centric Warfare", 35th Annual Hawaii International Conference on System Sciences (HICSS'02),
vol. 1, p. 40.
Barker, T. 2011, "An Automated Individual Feedback and Marking System: An Empirical Study", The Electronic
Journal of e-Learning, vol. 9, no. 1, pp. 1-14.
Byrnes, J. F. 1984, The Psychology of Religion Free Press, Virginia, USA.
Caldwell, D. F. & O'Reilly, C. A. 1990, "Measuring Person-Job Fit With a Profile-Comparison Process.", Journal
of Applied Psychology, vol. 75, no. 6, pp. 648-657.
Case, D. A. 2003, Antecedents and Outcomes of End User Computing Competence, Air Force Inst of Tech
Wright-Patterson Afb OH School of Engineering and Management, ADA415162.
Chen, G. D., Chang, C. K., & Wang, C. Y. 2008, "Ubiquitous Learning Website: Scaffold Learners by Mobile
Devices With Information-Aware Techniques", Computers and Education, vol. 50, no. 1, pp. 77-90.
Collins, J. 2004, "Education Techniques for Lifelong Learning: Principles of Adult Learning", RadioGraphics, vol.
24, no. 5, pp. 1483-1489.
Dieterle, E. & Dede, C. 2007, "Building University Faculty and Student Capacity to Use Wireless Handheld
Devices for Learning," in Ubiquitous Computing in Education: Invisible Technology, Visible Impact, M. E.
van 't Hooft & K. Swan, eds., Lawrence Erlbaum Associates Publishers; US, Mahwah, NJ, pp. 303-328.
Du Bois, N. & Staley, R. 1997, "A Self-Regulated Learning Approach to Teaching Educational Psychology",
Educational Psychology Review, vol. 9, no. 2, pp. 171-197.
Gütl, C., Lankmayr, K., Weinhofer, J., & Höfler, M. 2011, "Enhanced Automatic Question Creator EAQC:
Concept, Development and Evaluation of an Automatic Test Item Creation Tool to Foster Modern E-
Education", The Electronic Journal of e-Learning, vol. 9, no. 1, pp. 23-38.
Harper, M. E., Jentsch, F. G., Berry, D., Lau, H. C., Bowers, C., & Salas, E. 2003, "TPL-KATS-Card Sort: A Tool
for Assessing Structural Knowledge", Behavior Research Methods, Instruments, & Computers, vol. 35, no.
4, pp. 577-584.
Hiemstra, R. 2006, "Is the Internet Changing Self-Directed Learning? Rural Users Provide Some Answers",
International Journal of Self-Directed Learning, vol. 3, no. 2, pp. 45-60.
Hughes, C. A., Korinek, L., & Gorman, J. 1991, "Self-Management for Students With Mental Retardation in Public
School Settings: A Research Review", Education & Training in Mental Retardation, vol. 26, no. 3, pp. 271-
291.
Kelso, G. I., Holland, J. L., & Gottfredson, G. D. 1977, "The Relation of Self-Reported Competencies to Aptitude
Test Scores", Journal of Vocational Behavior, vol. 10, no. 1, pp. 99-103.
Lievens, F. & Sanchez, J. 2007, "Can Training Improve the Quality of Inferences Made by Raters in Competency
Modeling? A Quasi-Experiment.", Journal of Applied Psychology, vol. 92, no. 3, pp. 812-819.
Papanikolaou, K. A., Grigoriadou, M., Magoulas, G. D., & Kornilakis, H. 2002, "Towards New Forms of
Knowledge Communication: the Adaptive Dimension of a Web-Based Learning Environment", Computers &
Education, vol. 39, no. 4, pp. 333-360.
Schwarz, N. & Oyserman, D. 2001, "Asking Questions About Behavior: Cognition, Communication, and
Questionnaire Construction", American Journal of Evaluation, vol. 22, no. 2, pp. 127-160.
Stubbé, H. E. & Theunissen, N. C. M. 2008, "Self-Directed Adult Learning in a Ubiquitous Learning Environment:
a Meta-Review", Proceedings - 4th EduMedia Conference 2008 Special track -Technology Support for Self-
Organised Learners (TSSOL08), Salzburg, Austria, 2.& 3.of June 2008.
van den Boom, G., Paas, F., & van Merriënboer, J. J. G. 2007, "Effects of Elicited Reflections Combined With
Tutor or Peer Feedback on Self-Regulated Learning and Learning Outcomes", Learning and Instruction, vol.
17, no. 5, pp. 532-548.
van Merriënboer, J. J. G., van der Klink, M. R., & Hendriks, M. 2002, Competenties, Van Complicaties Tot
Compromis: Over Schuifjes En Begrenzers [Competencies, From Complications to Compromises; About
Tuning Knobs and Limiters] Onderwijsraad, Den Haag, The Netherlands.
Wang, S. L. & Wu, C. Y. 2011, "Application of Context-Aware and Personalized Recommendation to Implement
an Adaptive Ubiquitous Learning System", Expert Systems with Applications, vol. 38, no. 9, pp. 10831-
10838.
Whan Marko, K. & Savickas, M. L. 1998, "Effectiveness of a Career Time Perspective Intervention", Journal of
Vocational Behavior, vol. 52, no. 1, pp. 106-119.
840

Using a Learning Management System for Executing Role
Play Simulations
Tone Vold
Hedmark University College, Faculty of Business Administration, Institute of
Social Sciences, Norway
tone.vold@hihm.no
Abstract: Learning Management Systems (LMS’s) are generally used for storing and deploying learning
material. In this experiment, however, it is used to keep track of, and store, a role play simulation. The script and
roles for the role play simulation were invented and developed by the participants, building on what the
participants found relevant to test in a simulation. The LMS was used to capture and store the conversation to
see if this could be used for evaluation purposes. However, unexpected problems required changes to the
running of the experiment. Instead of using a chat-function (which was the original plan), the participants had to
use a discussion forum. The participants found the approach interesting and reported on a minor learning
outcome from the development workshop. They reported a lack of learning outcome from the role play due to the
fragmented dialogue and technical problems. They also indicated a preference towards shorter scenarios and
repeated plays, incorporating the information gathered from the debriefing following each play session.
Keywords: learning management systems, role play simulation, experiential learning
1. Introduction
At Hedmark University College(HUC), the Learning Management System (LMS) in use is the
Norwegian developed LMS Fronter (Fronter 2011), and was introduced in 2000.An LMS is, according
to Ryann Ellis (2009), a system that should:
Centralize and automate administration
Use self-service and self-guided services
Assemble and deliver learning content rapidly
Consolidate training initiatives on a scalable web-based platform
Support portability and standards
Personalize content and enable knowledge reuse (Ellis 2009)
The usage at HUC has generally been to deploy learning material such as links to other web pages,
articles and files that are a part of the curriculum. The students can also upload material they find
relevant, as well as handing in their assignments. For students that have enrolled for distance
education courses, the chat is also used for synchronous communication between lecturers and
students.
Among the variety of features that are available in Fronter are the Fronter chat-function and the
Fronter discussion forum. When embarking on a research project using role play simulation that
needed dialogues between the roles saved for evaluation purposes, the choice fell on using Fronter’s
chat-function. This meant that there was no need to construct a new platform for the project.
The experiment involving the role play simulation was a part of the investigations that is included in
the author’s PhD work. The research question for this paper is: “What is the evidence of enhanced
experiential learning utilizing user involvement, with special focus on simulation and gaming”. A part
of the experiment took place at Tretorget. Tretorget is a “regional innovation actor in the forestry and
wood industry in the Glåmdalen area” (Aasen 2005). As part of their continuing education program,
they organize courses for the industry in the Glåmdalen area. It was at one of these courses that the
experiment took place. The objective of the experiment was to include the users in the development
of the scenario and roles, then have them to play the scenario and record their feedback on the
different experiences. The dialogues from the role play were also a subject of the evaluation. The
dialogue would indicate if they were able to handle the situation depicted in the scenario. The ideas
for the roles and the scenario were brought forward by the participants. The course the participants
were attending was on Health, Security and Environment (HSE) issues.
841

Tone Vold
This idea phase was undertaken as a mini-workshop. A workshop is defined as “an educational
seminar or series of meetings emphasizing interaction and exchange of information among a usually
small number of participants” (Farlex 2011). The role playing was conducted using Fronter, and the
debriefing was done as an “After Action Review” (AAR). AAR is a method which gives the researcher
the ability to collate the responses from the participants. Often used in the army, this debrief is for
evaluating performances in an action, identify weaknesses and strengths, and for future action, find
ways to improve on the action (von der Oelsnitz and Busch 2006). The Learning Cycle of an AAR is
depicted in Figure 1.
Reflection on the
experiences from the
last action
Using the plan of
action in the next
action
Documentation and
reworking of a plan of
action with directions
of use and suggestions
for improvement
Figure 1: The learning cycle of an AAR (von der Oelsnitz &Busch, 2006)
The objective for this paper is to investigate the usefulness of Fronter during a role play simulation for
learning purposes. The paper presents the benefits and constraints found during the experiment.
2. Method/data collection
The study is a constructivist inquiry. The data are mainly qualitative and the collection methods
consist of questionnaires (with mainly qualitative questions), interviews, group interviews and field
notes. The interviews and group interviews are recorded for future reference due to the difficulties that
were likely to arise in contacting participants to confirm their responses. The results are discussed
with peers and fellow researchers in order to secure dependability (Guba and Lincoln 1989).
3. Theoretical foundation
Keeping the dialogue from the role play simulation is important for evaluation purposes. The written
dialogue helps in the assessment of how well they have executed the task; how well they have been
able to stick to the script and roles, and how constructively they have solved the problem.
A written input is different from an oral input. While the written word is static, the spoken word is
dynamic. There is more immediacy and less retention in the spoken word, and thereby requiring an
immediate feedback (Ferraro and Palmer 2011). It is easier to be more deliberate and thoughtful
when writing the dialogue (Hoel 2001).
Participant input was required during the creation of the scenario and roles to test whether early
involvement would encourage more participant involvement in experiential learning where there is
difficult or impossible to learn from experience in a work situation. Experiential learning is influenced
by the works of Kurt Lewin, Jean Piaget and John Dewey (Kolb 1984). An important feature that
842

Tone Vold
Dewey promotes in particular is building new experiences with the previous experience as a back
drop (Dewey 1938, Dewey 2005, Dewey 2008). In order to draw upon previous experiences, active
participation is required. Involving the participants promotes their engagement, and in other
disciplines, like systems engineering, this will induce a learning process (Sommerville 2007).
4. Structure of the paper
Following this introduction is a description of the experiment, how it was executed and what the
outcome was. The findings from the experiment are discussed before concluding with
recommendations for similar use in the future.
5. The experiment
The LMS Fronter uses a school house as metaphor, offering a “room” for each course. The role play
simulation has also a separate “room” with participants. The participants were listed from one to five.
Figure 2: Screenshot from Fronter - the Tretorget room
The participants were required to create a scenario and roles of their choice. They chose to relate the
simulation to their latest theme; drinking and drug problems at work. The complete scenario was
uploaded to the Archive in Fronter. Triggering the action was an input from the facilitator. The
facilitator, who is also the author, distributed the roles and the groups in different rooms and the
simulation could start. However, the computers were missing the program that ran the chat-function,
resulting in the decision to continue the experiment using a Forum instead. The participants were
hesitant, but gained confidence as the simulation progressed. Using the discussion forum, however,
created some confusion. Instead of using one discussion thread for all participants, it was opened up
for group discussions. This made the communication very difficult to keep track of as there were
several discussion threads running through the same forum. This was corrected by moving all the
participants to the same discussion. An unstable internet connection at this point made it difficult to
continue, and we were unable to establish a back up for this, e.g. a machine to machine network, as
Fronter required an internet connection to function. An overview indicating the time consumption for
the experiment is shown in Figure 3:
Also the idea was to let the role characters chat one-on-one as well as in a common forum. When the
chat was not available this was done with creating folders for different one-on-one-conversations.
6. Findings
The participants reported that role play simulation was engaging. They were engaged in developing
their scenario and roles and found this to be a “a new way of seeing the curriculum” which is coherent
with Dewey’s suggestion of involving the participants (Dewey 1938, Dewey 2005, Dewey 2008). This
is also suggested by Chesler and Fox (1966) to strengthen the value of the role play.
However, when it came to the simulation, they were somewhat confused about the different
instructions given due to the faulty installation on the computers. As a result, the participants felt that
their learning had been hindered. The different discussion threads and the unstable internet
connection breaking the flow of the simulation were prohibiting the learning process. Maintaining a
state of flow is important for the learning process according to Mihalyi Csikszentmihalyi (1990).
Kember et al, states that if this is unattended, it will prohibit learning (1999).
In debrief the participants reported on learning from the workshop where they developed the script.
They found the interruptions and different discussion threads irritating, which disturbed their learning
843

Tone Vold
process. A synchronous discussion is preferred as the discussion forum in Fronter was somewhat
confusing due to the number of discussion threads created there. A stricter control over the threads or
sticking to one thread would have been sufficient. This would have reduced the confusion and kept
the discussion going. While not as synchronous as the chat-function, a single Forum thread would
have been synchronous enough. Internet outages were also reported to be a major problem.
Figure 3: Overview of time consumption for experiment
The participants also reported that it was easier to start the simulation when they knew the theme for
the script. However, they also suggested that the scenario contain fewer aspects, in order to keep it
simple and easy to follow. Playing several role play simulations or even playing the same simulation
several times were among the suggestions. This is also suggested by Lauber in Silberman (Lauber
2007).
The dialogue from the role play simulation partially reveals understanding of the script, but is too
fragmented to be of any real assessment value.
7. Conclusion
There are some important lessons from this experiment. Engaging the participants in the workshop
increased the value of the script (Chesler and Fox 1966). Using an LMS requires a written dialogue.
The writing gave the participants an opportunity to express themselves more clearly. It also allows for
recording the dialogue. This dialogue was then used for evaluation purposes, including the debrief
(AAR).
However, there were some constraints regarding the use of an internet based LMS. For example, the
internet connection must be working and scaled for the usage. When using an LMS it is important to
check that the hardware has all of the software required to run the desired features. Switching from
chat-function to discussion forum became confusing, as the participants used multiple discussion
threads.
7.1 Future recommendations
For future research using an internet based LMS it is important to secure the internet connection and
also to make sure that the computers have all the necessary software installed on them.
844

Tone Vold
It is also important to keep the scenario short and avoid several parallel conversations. This applies to
either chat-function or discussion forum.
The short scenario can be played several times, using debriefing to establish the learning outcome
and possible changes to the scenario for a replay. The final debriefing can also be done in the LMS
by having the participants posting a reflection note about what they have experienced.
References
Aasen, B. (2005) Historien om Tretorget, Oslo.
Chesler, M. and Fox, R. (1966) Role-playing methods in classroom, Chicago: Science Research Associates.
Csikszentmihalyi, M. (1990) Flow: The Pshychology of Optimal Experience, New York Harper & Row.
Dewey, J. (1938) Experience & Education, New York: Touchstone.
Dewey, J. (2005) Democracy and Education, Barnes & Noble Books.
Dewey, J. (2008) Democracy and education: an introduction to the philosophy of education, [Champaign, Ill.]:
[Book Jungle].
Ellis, R. K. (2009) 'Field guide to Learning Managements Systems', 7.
Farlex, I. (2011) 'The Free Dictionary ', The Free Dictionary,
Ferraro, V. and Palmer, K. C. (2011) 'Differences Between Oral and Written Communication', [online], available:
http://www.mtholyoke.edu/acad/intrel/speech/differences.htm [accessed 09.05.2011]
Fronter (2011) 'Fronter', [online], available: http://com.fronter.info/mnu5.shtml [accessed 02.05.2011]
Guba, E. G. and Lincoln, Y. S. (1989) Fourth generation evaluation, Newbury Park, Calif.: Sage.
Hoel, T. L. (2001) 'Samtalar" på e-post og kommunikative vilkår for læring', Norsk Pedagogisk Tidsskrift, 2(3),
172-183.
Kember, D., Jones, A., Loke, A., McKay, J., Sinclair, K., Tse, H., Webb, C., Wong, F., Wong, M. and Yeung, E.
(1999) 'Determining the level of reflective thinking from students’ written journals using a coding scheme
based on the work of Mezirow', International Journal of Lifelong Education, 18(1), 18-30.
Kolb, D. A. (1984) Experiential learning: experience as the source of learning and development, Englewood
Cliffs, N.J.: Prentice-Hall.
Lauber, L. (2007) 'Role Play: Principles to Increase Effectiveness' in Silberman, M., ed. The Handbook of
Experiential Learning, San Francisco: Pfeifer, 185 - 201.
Sommerville, I. (2007) Software Engineering, 8th ed., Essex: Pearson Education Limited.
von der Oelsnitz, D. and Busch, M. W. (2006) 'Teamlernen durch After Action Review', Personalführung, (2), 54-
62.
845

The Effects of Self-Directed Learning Readiness on
Learning Motivation in Web 2.0 Environments
Chien-hwa Wang 1 and Cheng-ping Chen 2
1
Graphic Arts and Communications, National Taiwan Normal University, Taipei,
Taiwan
2
Information and Learning Technology, National University of Tainan, Tainan,
Taiwan
pw5896@ms39.hinet.net
chenjp0820@yahoo.com.tw
Abstract: Recent studies argue that the emergence of Web 2.0 has significantly advanced learner-centred
eLearning. However, critics have also suggested that learners could easily become overwhelmed in a completely
learner-centred Web 2.0 environment. This indicates that learners are not easily motivated to concentrate on
Web 2.0 learning activities. The researchers of this study speculated that learners’ self-directed learning
readiness might affect learner motivation in Web 2.0 environments. Two research hypotheses were made for the
study: 1. learners’ Internet experience affects their self-directed learning readiness; 2. self-directed learning
readiness is related to Web 2.0 learning motivation. A questionnaire survey was conducted and pilot study
showed that overall reliabilities of the questionnaire were satisfied. The subjects of the study were sampled from
the college students who were frequent users of local Yahoo! Answers. Data from 334 valid questionnaire
responses were collected and statistically analysed. The results indicated that learners with 10 years or more
web experience exhibited a significantly higher average score for self-directed learning readiness than learners
with less web experience, suggesting web experience significantly influenced the self-directed learning
readiness. The results also indicated that the correlation coefficients between each subcategory of self-directed
learning readiness and each subset of Web 2.0 learning motivation were highly significant, suggesting that the
higher the self-directed learning readiness, the higher the Web 2.0 learning motivation. These findings are
especially useful as a reference for educational practitioners to adapt learners’ self-directed learning readiness to
Web 2.0 environments. Moreover, the findings can also be used to reduce the concerns of people who are
sceptical of web 2.0 learning.
Keywords: Web 2.0, self-directed learning readiness, learning motivation, learner control
1. Introduction
Due to the recent rapid increase of blogs, wikis, and social networks, research on Web 2.0 learning
has become popular in Educational Technology. Huang, Yang, and Tsai (2009) asserted that one of
the essential goals of applying Web 2.0 to interactive eLearning is to enhance learner-centric
communication in web-based learning. However, learner-centric or learner-controlled instruction
(hereafter referred to as “learner control”) has been wildly discussed over the past 20 years. Many
eLearning studies regarding learner control were conducted before the introduction of Web 2.0, and
many focused on developing learner control mechanisms and examining the effectiveness of welldesigned
instructional systems. The use of navigational aids (Burke, Etnier and Sullivan, 1998) and
intensive practice (Shute, Gawlick and Gluck, 1998) are two of the many learner control eLearning
strategies developed in the 1990s. In current research and development, a series of Web 2.0 studies
similarly focused on designing learner control strategies and testing their effects. For example,
Mendenhall and Johnson (2010) designed a social annotation model learning system embedded with
multiple learner control strategies including examples, practice, reflection, and collaboration; the
findings indicated that users’ experiences were positive. Westera, de Bakker and Wagemans (2009)
used peer tutoring as the learner control strategy and a computational model selected the most
appropriate peer. However, the results were not self-evident as students frequently used alternative
methods to request assistance. Regardless of the positive or negative findings of these Web 2.0
studies, the controversy over the use of learner control strategies cannot be overlooked. Learnercontrolled
eLearning courses have also been criticized for learners “entering with insufficient reasons
(motivation) to remain and the instructional methods they encounter fail to hold them,” therefore
learner-controlled “eLearning courses often suffer a high dropout rate” (Molenda and Bichelmeye,
2005).
Another complaint against learner control strategies relates to the unique attributes of Web 2.0.
Instead of passively absorbing static information, a Web 2.0 environment provides a knowledgesharing
platform that enables collective intelligence (O’Reilly, 2005), anyone has the right to improve
and revise the information. Learners are able to more actively search, select, acquire, and create
846

Chien-hwa Wang and Cheng-ping Chen
information suitable for their own learning purposes. Therefore, learner control strategies should not
be self-defeating, becoming an invisible constraint that limits learners’ freedom while navigating within
Web 2.0 environments. To solve the aforementioned problems, further study is necessary to identify
more effective learner characteristics and to design the guiding algorithms that can adequately adapt
individual differences to Web 2.0 environments.
2. Unique media attributes of Web 2.0
The effectiveness of educational media was debated for more than 15 years, from approximately
1980 to the mid-1990s. The original debate was probably initiated by Salomon (1979) who argued
that it was not the medium that influenced learning, instead the attributes of a medium that can be
modelled by learners are what shapes the development of unique cognitive processes. Clark (1994)
concluded that the media not only does not influence learning, but it also does not motivate learning.
His argument is that only adequate instructional methods can influence learning, and the instructional
methods should be developed according to the unique attributes of a particular medium. These early
debates over media were primarily hardware oriented. With the convergence of digital media since the
late 1990s, the term media has become increasingly software oriented. Studies emphasizing digital
media and eLearning have shifted the evaluation of hardware effectiveness to the development of
eLearning environments. Subsequently, research on computer-based instruction (CAI), Internet
application, and Web 2.0 have been conducted, and the effectiveness of instructional strategies
embedded in eLearning systems is frequently evaluated.
However, in accordance with media attributes theory, a careful re-examination of the unique attributes
of Web 2.0 is necessary. Web 2.0 provides a platform that allows learners to not only build their own
virtual learning society, but also enables the freedom to navigate, communicate, and construct
information within cyberspace. As stated by Reinartz (2009), the new instructional design paradigm
moves from providing learners with systematic knowledge-acquisition models for learning to providing
learners with learning-environment accommodation. He also asserted that the learning process
requires learners to be actively involved decision makers, knowledge builders, and designers, rather
than simply receivers of information, to develop higher cognitive skills. This study suggests that
additional research should be conducted from the learner control perspectives and more adaptive
strategies should be developed. A similar suggestion can be found in the research by Scheiter and
Gerjets (2007). They asserted that the results of studies on the effectiveness of eLearning were
ambiguous because self-controlled learning is difficult to demonstrate due to usability issues and
moderating learner characteristics. More recently, Uzunboylu, Bicen, and Cavus (2011) also
recommended that students’ expectations and individual differences should be analysed before
integrating an eLearning environment into educational applications. Before developing additional Web
2.0 instructional systems and strategies, it seems greater attention should be given to the learner
perspective, and more learner characteristics that affect motivation toward Web 2.0 learning activities
should be identified.
3. Learner characteristics
Learner characteristics have been widely discussed in research over the past 10 years. Scheiter and
Gerjets (2007) categorized the main learner characteristics for hypermedia environments as prior
knowledge, self-regulatory skills, cognitive style, and attitude toward learning. Scheiter et al. (2009)
then found that learners with greater favourable characteristics tended to display increased adaptive
example utilization behaviour, reported less cognitive load, and solved more problems correctly than
learners with less favourable characteristics. They defined the favourable characteristics as higher
prior knowledge, more complex epistemological beliefs, greater positive attitude, and better cognitive
and metacognitive strategy use. This line of research has focused on hypermedia instruction with
specific built-in instructional content and the drawback is that the research was limited to a Web 1.0
perspective. Luckin et al. (2009), however, provided a more Web 2.0-perspective; they determined
that Web 2.0 learners could be categorized into four main groups: researchers, collaborators,
producers, and publishers. They further argued that few learners were familiar with the complete
spectrum of Web 2.0 activities and only a small number were producing and publishing self-created
content for wider consumption; therefore, learners’ self-management or metacognitive reflection
cannot be found. From a Web 2.0 perspective, the recognition of hypermedia instruction should be
extended from “information utilization” to “knowledge construction,” and the higher order thinking skills
of student must be encouraged. However, higher order thinking skills, such as cognitive style, selfregulatory
skills, and attitude towards learning, cannot be activated or inspired unless motivational
factors are initiated. These factors may affect learners’ motivation and willingness to participate in the
847

Chien-hwa Wang and Cheng-ping Chen
shared construction of knowledge, which is an essential attribute of Web 2.0. Unfortunately, there are
few studies of Web 2.0 learning that focus on learner characteristics. Vandewaetere, Desmet and
Clarebout (2011) also highlighted that current studies on adaptive learning failed to bridge theory with
practice. Therefore, a framework that integrates current and past research results is urgent.
4. Self-directed learning readiness (SDLR)
In a promising development, Hung et al. (2010) verified a five dimensional model of learner
characteristics for online learning environments. The five dimensions were categorized as: selfdirected
learning (SDL), motivation for learning, computer/Internet self-efficacy, and learner control.
These learner characteristics are distinctive and useful for building the research framework of this
study. Considering the limited scope of this study, the first two dimensions were selected as the main
research variables and then examined.
As Long and Agyekum (1983) highlighted, the attributes of self-direction in learning are becoming
increasingly important. Educators have been challenged to assist in the development of SDL skills
and to encourage learners to use self-direction more freely in their learning activities. Long and
Agyekum are not the first researchers to introduce the idea of SDL. Tough (1966) first determined that
SDL involves a person with the ability to learn how to plan and maintain motivation. Knowles (1975)
suggested SDL involves a person with the ability to diagnose their own learning demands, draft
learning objectives, choose appropriate learning strategies, and evaluate the learning effect.
In a major application of SDL, Guglielmino (1977) introduced the term SDLR and developed a selfreport
questionnaire with Likert-type items. Eight factor aspects were included in the SDLR
questionnaire. These eight factor aspects underlying the SDLR ascertain learner readiness for SDL
(McCune and Guglielmino, 1992). The SDLR questionnaire is widely known as the self-directed
learning readiness scale (SDLRS). Though there have been some criticisms of SDLRS, (Brockett,
1987; Field, 1989; Straka and Hinz, 1996), the vast majority of studies have supported the
questionnaire’s reliability and validity (Delahaye and Smith, 1995; Long and Agyekum, 1983; McCune
and Guglielmino, 1991; Russell, 1988). The SDLRS and its self-scoring form, the Learning Preference
Assessment, are the most frequently used methods for assessing SDL readiness (Merriam, Caffarella,
and Baumgartner, 2007).
According to Chang (2006), Guglielmino’s SDLR questionnaire has been translated into Chinese and
carefully revalidated by Teng (1995); the number of questions was reduced from 58 to 55 and the
original eight aspects were reduced to four aspects in consideration of cultural differences. The four
aspects are (1) effective learning (EL), (2) active learning (AL), (3) independent learning (IL), and (4)
creative learning (CL).
With the current information technology advancements, the SDLR can be adapted to a learnercontrolled
eLearning environment. The link between SDLR and eLearning was determined by Warner,
Christie, and Choy (1998). They defined the readiness for online learning as follows: (1) students’
preferences for the form of delivery compared to face-to-face classroom instruction; (2) student
confidence when using electronic communication for learning and, in particular, competence and
confidence in the use of Internet and computer-mediated communication; and (3) ability to engage in
autonomous learning.
5. Learning motivation
Motivation significantly influences learners’ intentions of remaining in a self-regulated, learnercontrolled
eLearning environment. According to Keller (1983), motivation affects individuals’ efforts
and attitudes toward learning. In other words, motivation influences the selection or avoidance of
specific experiences or goals as well as influences the degree of effort learners apply to the
experience or goal. Keller (1983) suggested that learning motivation is affected by four perceptual
components: attention, relevance, confidence, and satisfaction. Each component plays a critical role
in motivating students throughout the learning process. This is the origin of the ARCS model, which
has been wildly employed as the theoretical foundation of various instructional strategy developments.
Keller (1987, 2006) further recognized several categories of motivational factors that may affect the
learning outcome. These factors were used to develop his motivational design models. The models
are categorized into four groups: person-centred models, environment-centred models, interactioncentred
models, and omnibus models. The first two models are in agreement with recent theoretical
848

Chien-hwa Wang and Cheng-ping Chen
arguments. For example, (Ryan and Deci, 2000) defined the intrinsic and extrinsic factors of
motivation. Intrinsic motivation refers to individual interests that one grows in knowledge and skills,
while extrinsic motivation originates from an environment external to learning. The factors influencing
intrinsic motivation in the Web 2.0 environment may include attitude in knowledge acquisition (AKA),
expectation of professional performance (EPP), and solving practical problems (SPP). The factors
influencing extrinsic motivation may include attraction of virtual society (AVS), convenience of Internet
usability (CIU), and self-efficacy of learning (SEL). Ryan and Deci (2000) asserted that motivation can
be determined intrinsically by individuals and externally by sources due to situational variables and
environmental factors. These two orientation types significantly affect student learning performance.
The intrinsic and extrinsic orientation of learning motivation has been applied in recent research on
motivation in eLearning circumstances. For example, Law, Lee and Yu (2010) employed this
dichotomously oriented scheme as the framework for designing computer programming courses.
Hung et al (2010) also employed the same orientation as the theoretical background when defining
the five dimensional model of learning readiness for online learning.
6. Research framework
This study argues that Web 2.0 eLearning research should focus on the learner aspect rather than the
instructional design or instruction aspects. Furthermore, for Web 2.0 eLearning studies, the unique
media attributes of Web 2.0 (freedom of searching, selecting, acquiring, and creating information for
learners’ own purposes) should be considered and the effects of specific learner characteristics within
a Web 2.0 environment should be examined. Recent Web 2.0 eLearning studies tended to focus on
developing learner control strategies and testing their effectiveness. However, these learner control
studies did not consider the effects of Web 2.0 learner characteristics. Fortunately, some dimensions
of the learner characteristics of online learning have been identified. Among these characteristics, this
study considers SDLR to be the important factor influencing Web 2.0 learning. In summation of the
literature reviewed above, this study speculates that SDLR is positively related to learners’ intrinsic
and extrinsic motivations of Web 2.0 learning. Furthermore, we also speculate that learners’ Internet
experience affects self-directed learning readiness, since research evidence has shown that Internet
experience influences on the learners’ intention to utilize the Internet, and the level of web
experiences play a significant role on the intention to use a particular Internet environment (Alenezi,
Abdul Karim, and Veloo, 2010; Fusilier and Durlabhji, 2005). The main research hypotheses of this
study are as follows:
H1: Internet experience affects SDLR
H1.1: Internet experience affects overall SDLR
H1.2: Internet experience affects each subcategory of SDLR
H2: There is a relationship between SDLR and learning motivation
H2.1: There is a relationship between overall SDLR and overall learning motivation
H2.2: There are relationships between each subcategory of SDLR and each factor of learning
motivation
7. Methodology
A questionnaire survey was conducted to collect data from college students in Taipei City, Taiwan. All
subjects of the study are frequent users of a Web 2.0 knowledge-sharing site widely used by college
students. The data collected were analysed statistically and the research hypotheses were tested.
Figure 1 shows the research framework of the study.
7.1 The instrument
The SDLR and learning motivation questionnaire was designed according to the research framework
and comprised three parts: the demographic information, the SDLRS, and the learning motivation
inventory. The demographic data included gender, college grade, learning domain, Internet
experience, and prior knowledge. The SDLRS modified from Teng (1995) had four factors: effective
learning, learning motivation, independent learning, and creative learning. Finally, the learning
849

Chien-hwa Wang and Cheng-ping Chen
motivation inventory had two aspects: intrinsic and extrinsic motivations, both of them had three
levels, as described in the previous section. Some sample questions are listed in Table 1.
Figure 1: Research framework
Table 1: Sample questions of the questionnaire
SDLRS
Learning
Motivation
Inventory
Factors Questions
IL I understand my learning achievement
EL It is important for me to understand learning methods
AL I will try my best to learn even I’m very busy
CL I like to solve problems by employing innovative methods
I like to use Yahoo! Answers because
AKA I am interested in the information embedded
EPP it helps me solve academic problems
SPP it is more efficient in finding useful solutions
AVS I am able to interact with more virtual friends
The questionnaire was designed with reliability and validity confirmation. Firstly, an expert validation
process was conducted and several questions were modified as per experts’ suggestion, and some
questions were eliminated from the original of 86 questions. The final questionnaire comprised 69
questions in total, 10 for demographic information, 29 for the SDLRS, and 30 for the learning
motivation inventory. Secondly, a Cronbach α reliability test (Crocker and Algina, 1986) was
conducted during a pilot study with 35 samples drawn from the same population as the formal survey
to confirm the internal validity. An overall reliability value of .964 was reported for the SDLRS, and
.977 for the learning motivation inventory.
7.2 Sampling and data processing
The study subjects were university students who are frequent users of a local Web 2.0 knowledgesharing
site Yahoo! Answers of Taiwan. From a total of 21 universities, 6 universities in Taipei City
were randomly selected. A systematic sampling process was then taken to randomly select frequent
Yahoo! Answer users from each of the 6 universities. Approximately 350 students were selected to
answer the questionnaire.
The Web 2.0 site Yahoo! Answers is a community-driven question-and-answer site that allows users
to both submit and answer questions asked by other users. Yahoo! Answers is currently available in
up to 19 countries and areas in different languages. Yahoo! Answers Taiwan was launched in 2004.
More than a million items were posted onto the site in the first six months following its debut.
Therefore, this site was selected because it is one of the most widely used knowledge-sharing tools of
students in Taiwan.
The data from 344 valid questionnaires were collected and analysed descriptively and statistically. A
one-way Analysis of Variance (ANOVA) was performed to examine the effect of learners’ web
experience on self-directed learning readiness, and a Pearson product moment correlation process
was conducted to measure the coefficient of self-directed learning readiness and web 2.0 learning
motivation.
850

8. Results and discussions
8.1 General findings
Chien-hwa Wang and Cheng-ping Chen
By analysing the collected data, the general findings of this study are described as follows: (1) Gender
distributions are roughly even, there were 4.8 % more females than males, reflecting that the use of
Yahoo! Answers Taiwan is equal between gender. An additional t-test was conducted to examine the
gender effects on learning motivation, and as expected, no significant difference (p=.324) was
observed. (2) Junior and senior students comprised the majority of the study samples (more than 70
%) possibly indicating that higher-grade university students tended to utilize Web 2.0 environment
more. (3) 53 % of the samples had 6 to 10 years Internet experience, and no one had an Internet
experience of less than one year. Statistical analysis of the influence of Internet experience on SDLR
is shown in the following section to test the research hypothesis 1. Table 2 shows the distributions
and percentages of demographic information.
Table 2: Demographic information
Variable Level No. Per cent
Gender
male
female
159
175
47.6
52.4
freshman 30 9.0
Grade
sophomore
junior
64
115
19.2
34.4
senior 125 37.4
Internet
Experience
< 1 year
1-5 Years
6-10 Years
0
67
178
0%
20.1%
53.3%
> 10 Years 89 26.6%
8.2 Hypotheses testing
8.2.1 The effect of Internet experience on SDLR
The effects of Internet experience on each subcategory (IL, EL, AL, and CL) of SDLR were
statistically examined by independent ANOVAs’, with the significant level set to .05. The results
indicated that: (1) Internet experience had an effect on overall SDLR scores (F=4.86, p=.008); (2) it
also had an effect on IL (F=4.19, p=.016) and EL (F=5.76, p=.003), but no significant result was found
in CL. Although the difference was statistically insignificant, descriptive data indicated that the more
years of Internet experience, the greater CL scores were. Table 3 shows the means, standard
deviations (SD), F value, and p value of each SDLR categories.
Table 3: Statistical data of SDLR and Internet experience
Sub-SDLR Internet. Experience Mean SD F P
IL
EL
AL
CL
Overall
1-5 years. 3.46 .48
6-10 years. 3.52 .52
>10 years. 3.69 .56
1-5 years. 3.32 .56
6-10 years. 3.36 .59
>10 years. 3.59 .62
1-5 years. 3.36 .62
6-10 years. 3.48 .64
>10 years. 3.58 .70
1-5 years. 3.53 .67
6-10 years. 3.56 .70
>10 years. 3.76 .71
1-5 years. 3.47 .48
6-10 years. 3.55 .52
>10 years. 3.72 .55
851
4.19 .016
5.76 .003
4.52 .012
2.78 .064
4.86 .008

Chien-hwa Wang and Cheng-ping Chen
8.2.2 The correlation between SDLR and learning motivation
Multiple correlation processes were conducted to examine the cross-relationships between each
subcategory of SDLR and each dimension of learning motivation. The overall result indicated that
SDLR is significantly correlated with learning motivation (r=.396, p

Chien-hwa Wang and Cheng-ping Chen
Clark, R. (1994) “Media will never influence learning”, Educational Technology Research and Development, Vol
42, No2, 21-29.
Crocker, L., and Algina, J. (1986). Introduction to Classical and Modern Test Theory. Harcourt Brace Jovanovich
College Publishers, Orlando, U.S.A.
Delahaye, B. L. and Smith, H. E. (1995) “The validity of the Learning Preference Assessment”, Adult Education
Quarterly, vol 45, pp 159-173.
Field, L. (1989) “An Investigation into ihe Structure, Validity, and Reliability of Guglielmino's Self-Directed
Learning Scale”, Adult Education Quarterly, vol 39, pp 125-139.
Fusilier, M. and Durlabhji, S. (2005) “An Exploration of Student Internet Use in India', Campus-Wide Information
Systems, Vol. 22, No. 4, pp 233-246.
Huang, Y. M., Yang, S. J. H., & Tsai, C. C. (2009) “Web 2.0 for Interactive ELearning”, Interactive Learning
Environments, Vol 17,No. 4, 257-259.
Guglielmino, L. M. (1978) “Development of the Self-Directed Learning Reading Scale”, Dissertation Abstracts
International, vol 38, pp 64-67.
Hung, M. L., Chou, C., Chen, C. H. and Own, Z. Y. (2010) “Learner Readiness for Online Learning: Scale
Development and Student Perceptions”, Computers and Education, Vol 55, No. 3, pp 1080-1090.
Knowles, M. S.(1975). Self-directed Learning, Association Press, New York.
Law, K. M. Y., Lee, V. C. S. and Yu, Y. T. (2010) “Learning Motivation in ELearning Facilitated Computer
Programming Courses”, Computers & Education, vol 55, No. 1, pp 218-228.
Long H. B. and Agyekum S. K. (1983) “Guglielmino's Self-Directed Learning Readiness Scale: A Validation
Study” Higher Education, vol 12, No. 1, pp 77-87.
McCune, S. K. and Guglielmino, L. M. (1992) “Factor Analytic Study of The Self-Directed Learning Readiness
Scale”, Paper presented at the Sixth International Symposium on Adult Self-Directed Learning, West Palm
Beach, FL, U.S.A., January.
McCune, S. K. and Guglielmino L. M. (1991) “Validity Generalization of the Self-Directed Learning Readiness
Scale”, In Long H. B. and Associates (ed.) Self-directed Learning: Consensus and Conflict, pp 147-154
Oklahoma Research Center for Continuing Professional and Higher Education, Norman, OK., U.S.A.
Mendenhall, A. and Johnson, T. E. (2010) “Fostering the Development of Critical Thinking Skills, and Reading
Comprehension of Undergraduates Using A Web 2.0 Tool Coupled with A Learning System”, Interactive
Learning Environments, Vol 18, No. 3, pp 263-276.
Merriam, S. B., Caffarella, R. S. and Baumgartner, L. M. (2007). Learning in Adulthood 3rd edition, John Wiley &
Sons, San Francisco, U.S.A.
Molenda, M. and Bichelmeye, B. (2005) “Issues and Trends in Instructional Technology: Slow Growth as
Economy Recovers”, in Orey, M., McClendon, J. and Branch, R. (ed.) Educational Media and Technology
Yearbook: 2005, pp 3-28. Westport, CT: Libraries Unlimited.
O’Reilly, T. (2005) “What Is Web 2.0 Design Patterns and Business Models for the Next Generation of
Software”, O'Reilly Media, Inc, U.S.A.
Ryan, R. M. and Deci, E. L. (2000) “Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social
Development, and Well-Being, American Psychologist, vol 55, No. 1, pp 68-78.
Russell, J. W. (1988) “Learner Preference for Structure, Self-Directed Learning Readiness and Instructional
Methods”, Dissertation Abstracts International, vol 49, p 1689.
Scheiter, K. and Gerjets, P. (2007) “Learner Control in Hypermedia Environments”, Educational Psychology
Review, vol 19, No. 3, pp 285-307.
Scheiter, K., Gerjets, P., Vollmann, B. and Catrambone, R. (2009) “The Impact of Learner Characteristics on
Information Utilization Strategies, Cognitive Load Experienced, and Performance in Hypermedia Learning”,
Learning and Instruction, vol 19, No. 5, pp 387-401.
Shute, V. J., Gawlick, L. A. and Gluck, K. A. (1998) “Effects of Practice and Learner Control on Short- and Long-
Term Gain and Efficiency”, Human Factors, vol 40, No. 2, pp 296-310.
Straka, G.A. and Hinz (1996) The original SDLS (Self-Directed Learning Readiness Scale) reconsidered,
Waxmann, Münster, Germany.
Teng, Y. (1995) Adult Instruction and Self-Directed Learning. Wu-Nan, Taipei, Taiwan.
Tough, A. M. (1966) “The Assistance Obtained by Adult Self-Teachers”, Adult Education, vol 17, pp 30-37.
Uzunboylu, H., Bicen, H. and Cavus, N. (2011) “The Efficient Virtual Learning Environment: A Case Study of Web
2.0 Tools and Windows Live Spaces”, Computers & Education, vol 56, No. 3, pp 720-726.
Vandewaetere, M., Desmet, P. and Clarebout, G. (2011) “The Contribution of Learner Characteristics in the
Development of Computer-Based Adaptive Learning Environments”, Computers in Human Behavior, vol 27,
No. 1, pp 118-130.
Westera, W., de Bakker, G. and Wagemans, L. (2009) “Self-Arrangement of Fleeting Student Pairs: A Web 2.0
Approach for Peer Tutoring”, Interactive Learning Environments, vol 17, No. 4, pp 341 - 349.
853

Usage Cases: A Useful way to Improve Effectiveness of
eLearning web Based Platforms
Cristina Wanzeller 1 and Orlando Belo 2
1
Departamento de Informática, Escola Superior de Tecnologia e Gestão e
CI&DETS, Instituto Politécnico de Viseu, Viseu, Portugal
2
Departamento de Informática, Escola de Engenharia, Universidade do Minho,
Braga, Portugal
cwanzeller@estv.ipv.pt
obelo@di.uminho.pt
Abstract: In this paper we propose a more effective use of the eLearning platforms resources, in order to
decrease the effort and time required to derive useful knowledge and bringing up together multiple valuable
contributions to overcome resource maintenance difficulties. The primary target of our work is the role of a Web
based platforms’ administrator (and promoter) that hasn’t enough experience on site optimization and resource
maintenance. Our idea to deal with this challenge is to assist such tasks, providing a system that acquires and
reuses the knowledge gained from the experience in solving problems concerning administration activities. The
system explores the case-based reasoning approach to accomplish its duty of suggesting a suited solution for
particular usage data analysis problem, on an eLearning Web based platform, as well, supporting the collecting
and structuring of the knowledge contained on successful solved usage problems. We describe the system and
propose its application to the specific domain of eLearning Web based platforms optimization. We are specifically
engaged in facilitating the collection and cataloguing of knowledge in this area, so it can be shared, evaluated
and finally reused in new situations.
Keywords: eLearning web based platforms, web usage mining, clickstream analysis, case-based reasoning,
web usage analysis, eLearning platforms optimization
1. Introduction
Web based eLearning platforms are very common and important. Currently the goal is to improve and
maximize their potential. Like any other web site, such platforms must realise their mission.
Nevertheless, implementing and administrating Web sites are tasks very difficult to accomplish in an
effective manner. The experience tells us that eLearning Web sites’ usage differs very often from
expectations, demanding deep decision support to define and apply some improvements and
refinements. Therefore knowing and understanding users’ behaviour is strategic to achieve site goals,
perceiving the impact of its structure, and to maximize the potentialities of the platforms and
guaranteeing their success. Web Usage Mining (WUM) technology can help us in a significant way to
reach such goals. With it, we can develop and apply Knowledge Discovery (KD) or Data Mining (DM)
processes to data related with the interaction activity between users and eLearning platforms, using
information about the usage of all the resources available to them. Yet, WUM doesn’t provide itself a
solution with the ability to answer to the most critical issues about the obsolescence of resources. It
neither assures KD process effectiveness.
Past experience in WUM is strategic and vital to the success of the site evaluation process. To
acquire such experience and use it latter in site restructuring processes, we decided to catalogue and
store WUM past experiences, in a specific oriented knowledge base that could be applied over usage
data analysis. The Case Based Reasoning (CBR) paradigm provides us the necessary means to
create and maintain our knowledge base. Its methods favour a flexible similarity based comparison,
even if the involved features are not objective and precisely defined. CBR can cope with incomplete
and subjective information and makes possible to consider only the relevant features and to use
specific importance levels, increasing the potential of answering the real user’s needs. The intuitive
nature of usage cases as knowledge representation and the sustained ability to learn incrementally
can reduce significantly the knowledge acquisition and the maintenance efforts related to eLearning
platforms organization and resources.
WUM potential (Rafaeli and Ravid, 1997) and application (Zaiane 2001; Pahl 2004; Lu 2004) to
eLearning environments was proposed for long time. Now educational DM is a prominent direction of
active research (Romero and Ventura 2007; Herhskovitz and Nachmias 2009). Still, the idea of mixing
WUM and CBR, in order to assist Web based eLearning systems enhancement is unusual. CBR is
854

Cristina Wanzeller and Orlando Belo
often used in combination with eLearning, but for different purposes, such as content categorization
and retrieval (Rodrigues et al. 2007) and for personalizing courses (Chorfi and Jemni 2004).
In this paper we present the Mining Plans Selector (MPS) system, focusing the practical use of the
two major assistance tasks that the system intents to fulfil: 1) aiding users in WUM processes
development, by recommending the most suited mining plans; and 2) semi-automated knowledge
acquisition support, from past successful WUM processes, into a centralized case base. The system
has the ability to suggest the most suited WUM plans to solve a specific usage data analysis problem.
The MPS also provides support to collect and organize the knowledge acquired from the experience
on solving WUM problems (usage cases, reflecting each one different usage past profiles), bringing
such knowledge up to date and promoting the system’s sustained incremental learning. New WUM
processes are stored on a collective case base, centralizing a key resource to the system’s capacity
to solve problems in a subject usage oriented knowledge base. When adopting and using a WUM
plan suggested by the system we’ll be able to explore in a more effective way the usage information
gathered inside the target eLearning platform and, with the results, defining better site organizations
and detect obsolete resources and know what kind of use users do to current eLearning platform
resources. Consequently, we have the basis to optimize site functionalities, reducing operational and
maintenance costs. Section 2 discusses main characteristics and challenges of eLearning usage
analysis. Sections 3 and 4 describe some issues and the utility of the MPS system. Finally, section 5
presents some conclusions and future work.
2. Usage analysis in eLearning systems
Several organizations explore eLearning platforms and need to make the process of creating, using
and maintaining the contents more efficient and effective. In order to obtain feedback and to
guarantee platforms effectiveness, we should analyse its usage. Web based eLearning platforms are
the most promising and interesting kind of platforms, due to their success, given the advantage of
easy access (Pahl 2004). They are a specific type of a Web site, and so, we may apply similar
principles, techniques and approaches to deal with analogous problems. This strategy is corroborated
by the increasing research involved on applying WUM in this area.
Within the context of vulgar Web sites, clickstream data is logged using non-intrusive forms, being a
valuable source of users’ behaviour information. This data may be gathered on a periodic basis to be
analysed consistently. Usage data may also be combined with other types of information about users,
pushing profiling to completely new levels (Srivastava et al. 2002). Moreover, we can anticipate
discontent and even new user’ needs and, so, react faster. Exploring WUM to extract knowledge from
this and related data has hereby potential enormous benefits to organizations. Some important and
actionable areas of WUM exploration consist of Web personalization, business intelligence, system
performance improvement and site content and structure enhancement (Srivastava et al. 2000)
On the context of eLearning Web sites, usage data provides insights to understand students learning
behaviour. There are specific requirements, mainly the need to take into account pedagogical aspects
of the learner and of the system (Romero and Ventura 2007). The ultimate goal differs from ecommerce
sites, i.e. “turning learners into effective better learners” (Zaiane 2001). Although the ways
to achieve such goal are similar, there are several important differences between the two areas. Main
differences may be structured in terms of (Romero and Ventura 2007): domain; data; objective and
techniques. The domain differs due to the purpose of guiding learning. The data in eLearning
comprises more information about student’s interaction and the user model is different (e.g. users are
not anonymous). Besides, learning sessions are longer (may span many access sessions and spread
over various days). The objective of improving learning is considered more subjective and more subtle
to qualify/quantify (Zaiane 2001) and the process of learning more complex than shopping (Pahl
2004). At the last, the techniques, DM and WUM were not tailored to eLearning, but the methods are
general enough and may bring benefit to these platforms. However, some can be adapted and other
cannot. Our view is that the extracted knowledge will be very helpful and that common features
number is higher, particularly the ones regarding KD process characteristics.
Web based eLearning systems provide usage data to analyse and turn discoveries into actions.
Though, this approach has some other real limitations. Data mining today is at best a semi-automated
process. A fundamental challenge is to develop KD systems easier to use, even by casual users. KD
and specially WUM are very useful but even more complex processes. ELearning sites administrators
and promoters, usually, are not skilled WUM users. Additionally, we point the unavailability of
855

Cristina Wanzeller and Orlando Belo
sufficient processes’ descriptions, properly structured and applicable, according to the organization’s
own needs. If we have such description, we will ask for an effective and flexible mechanism to help
the matching of the current problem with the most promising mining processes. Knowledge
management systems focus on knowledge and experience sharing. They enhance organization
capabilities by externalizing knowledge of its employees and combine them in novel forms. It breaks
up rigid processes and enables much richer interaction possibilities and creativity.
3. Representing usage analysis experience
Case representation is a key issue in CBR. The case based representation model must support a
comprehensive description of WUM experiences, regarding the nature and requirements of such
process development. A case is defined as a WUM process, corresponding to a successful usage
analysis example. The case description is structured according to the domain problem and the
applied solution (Table 1). We also defined a set of fundamental dimensions (D, T, P, A and K) to
organize items of case’s description. The P dimension contains general attributes of WUM process
description, involving problem (Pp) and solution aspects (Ps).
Table 1: WUM process main item descriptors
Case representation
Problem Dataset and variables metadata (D)
Problem type (T): activity, analysis goals and application areas
Process date and requisites (Pp)
Solution Mining Activities (A): DM tool and modelling and transformation stages
Knowledge (K): Discoveries and general and specific facts
Process general information (Ps)
The case problem description comprises: metadata characterizing the data (D), at dataset and
individual variables level; categorizations of the WUM problem type (T), mainly in terms of
abstractions such as Web site main activity, analysis goals and application areas; and process data
and evaluation criteria (Pp). The site activity may be “higher education” or something more detailed.
An analysis goal reflects a kind of WUM problem, for example some general like “determine access
order of pages and items”. The application areas are meant as the possible uses of the discoveries
(e.g. “impact analysis”). To allow several levels of subdivision in more sub-areas, the application
areas are organized in a dynamic hierarchy. This hierarchy provides a flexible way to define more
subtle and refined application areas concerning the eLearning domain. The process requisites
respects to criteria desired for the process results (e.g. level of precision).
The applied solution is essentially defined by: a sequence of activities (A), including transformation
and modelling stages, the involved data items and the model parameter settings; prior and derived
knowledge (K), concerning to facts that affected the analysis, the extracted knowledge and the
relations to such facts; and general information about the WUM process (Ps). The most important
aspects of the solution description are the details which may be used to reproduce with fidelity the
WUM process. Such details are usually the main ingredients of the DM activity success.
We also have a context description item to organize cases in terms of a site or particular sections
(e.g. curse). This item is a logic container for cases description features. The context may be
associated with some aspects, namely dataset, activity, specific fact and general fact. Besides CBR
principles (Kolodner 1993; Aamodt and Plaza 1994; Richter 1995; Mantaras et al. 2005), the
Predictive Model Markup Language (PMML) standard was an important orientation. PMML is a XML
based standard to define and share statistical and DM models across compliant applications The
DM/KD/WUM domains are complex, the respective vocabulary is diverse, even at basic aspects,
hindering the knowledge modelling. So, PMML provided accepted vocabulary to adopt and insights on
how to represent DM processes.
4. Acquiring and reusing experience
A straight reuse of one solution is quite possible in the WUM domain, since recurrent problems are
common. Still, becomes necessary to enable to help users on identifying the most plausible strategies
to address the problem at hands. We also need simple ways for acquiring items of problem and
solution description, in order to submit easily new problems and gather efficiently the relevant know
how of successful processes. The CBR paradigm brings a key opportunity to our knowledge base,
providing inherently a proper way for attending these demands. Coupling CBR intrinsic functionalities
with components to attend additional needs, we devised an integrated system to fulfil our intents.
856

4.1 The Mining Plans Selector system
Cristina Wanzeller and Orlando Belo
Figure 1 shows the MPS system fundamental functional components. We adapted the acknowledge
CBR cycle (Aamod and Plaza 1994) to the tasks at hands, devising (six) MPS constituent steps and
forming a problem solving and learning from experience strategy. The R’s steps, (Retrieve, Reuse,
Revise and Retain) are the original ones. Other steps are specific of the MPS system.
Figure 1: MPS functional components
The problem solving part of the system takes as inputs the clickstream data and the requirements
and, based on the cases kept on the knowledge base, delivers appropriate WUM plans. One MPS’s
specific task is to “characterize” the target usage data, producing metadata. We need a data
characterization consistent and independent from KD tool, in order to compare different datasets
systematically. Further, the extraction of some metadata must be automatized, as much as possible,
since clickstream data is typically huge. Another particular task is to “construct” a new WUM problem,
guiding, gathering and organizing the user’s explicit analysis constraints specification, through the
provided abstractions (e.g. analysis goals and application areas).
The “retrieve” step is a typical one and plays a vital role on problem solving. This step selects the
most plausible cases to found the construction of mining plans to recommend, according to the target
problem. The retrieve is based on the similarity threshold value, to choose the cases potentially more
effective. Next, the “reuse” task generates WUM plans, mostly based on the mining methods and the
levels of similarity of the retrieved cases, and considering also the evaluation criteria most important
to the analyst. This step does not performs extensive adaptation of the solution to the current
problem, namely in the wide sense intended by the original step. Yet, it focus the main parts of the
candidate cases that may be transferred to the target problems, by recommending mining methods
instead of mere cases. So, it prepares the reuse of the methods that constructed the solution. The
“revise” step is accomplished by the analyst, outside of the system, using a KD tool.
From the learning point of view, the system operates accepting heterogeneous descriptions of new
WUM processes and acquiring knowledge, through a semi-automated approach. The goal is to
simplify arduous activity, due to the great number of details. The accepted incomings are: documents
describing mining activities, generated by the KD tool in PMML; the process complementary
description, which would be exhaustive, if the used tool does not supports the PMML standard. The
learning starts with a “conciliate” task, to transform and combine the heterogeneous descriptions
items, supplied by user interaction and documents in PMML. Then the traditional “retain” task
essentially adds a new case to the knowledge base. This is realised by integrating and structuring the
incoming elements, considering the internal schema of the cases’ representation. After that, the case
becomes available for edition, to obtain additional information, such as items omitted in the
documents, decisions’ and settings’ explanations/justifications, discoveries and even new stages.
Indeed, the conciliate step may be accomplished in several iterations to enrich progressively the
description and benefit from the availability of the items that are extracted automatically.
4.2 Exemplifying problem solving
To exemplify problem solving we use a simple dataset and specify requisites comprising all the D, T
and P dimensions of problem description. We consider the improving site structure WUM typical
problem. We want to discover meaning relationships among learning materials. The idea is to find out
contents relevance and relations, taking into account the student’ point of view.
857

Cristina Wanzeller and Orlando Belo
Table 2 shows the main extracted and specified values for the example problem description. The
used clickstream data was a server log file, describing data at page view or access level. Concerning
the explicit requisites: (i) for goals we used all the ones regarding to relationships among pages; (ii)
we selected two sub-areas of the “quality of service” application area, since both are relevant and the
closest ones to the intended actions; (iii) we gave the maximum relative importance to Interpretability
evaluation criteria (followed by the other criteria items).
Table 2: Main values for the features of problem description
Problem description: category, features (and values)
(D)
Metadata at
dataset
level:
(D)
Metadata at
variable
level:
(T)
Problem
type
(P)
Evaluation
criteria
Number of lines (7128) and variables (8); granularity (PageView)
% of numeric (0.375), categorical (0.625), temporal (0.125) and binary (0) columns
Type of visitor’s identification (not available) and information recording (not available)
Access order (true) and access repetition availability (true)
Access data and hour availability (true)
Data type: 3 Numeric (1 DateTime; 2 Integer) 5 categorical (String)
Number of distinct values (…) and number of null values (0 for all)
Semantic category (…)
Goals (Discover relationships among pages and items; Determine access order of pages and
items)
Application areas (Impact analysis; Content and structure optimization)
(value; relative importance)
Precision (5; 4); Time of reply (5; 2); Interpretability (5; 5); Resources requirements (5; 1);
Implementation simplicity (5; 3)
The solution proposed by the MPS system is presented in Figure 2. Using a similarity threshold of 0.5,
the retrieve step finds eight cases, grouped into three model categories. Explaining the figure (from
right to left), the proposed solution includes mining plans of the (three) DM functions and model
categories (columns IV and V). The evaluation criteria average values are presented on column III.
Each model category is instantiated with the most similar case, providing: on (II) its similitude to the
target; at the left side of (I), the (most similar) case’s number hyperlink, to access its detailed
information; at the right side of (I), the combo boxes to access further information about other (less
similar) cases of the model category; each combo box may be expanded to see the case’s number
and its similitude to the target, as well to access further information of the case.
I II
III IV V
Figure 2: A problem solving solution description excerpt
The proposed plans are all appropriate. Association rules is frequently used to discover relations
among learning activities and sequential analysis to extract interesting patterns in the sequences of
on-line activities, and clustering to group similar access behaviours (Zaiane 2001). Hierarchical
clustering model (case 6) was a similarity value substantially inferior and is a different king of
clustering. By default and as intended, the system gives emphasis to the similarity between datasets.
This is considered a good result, from the experimental work, since dataset characteristics are the
most important features of mining methods choice. The analyses from cases 8 and 9 were performed
using datasets most similar to the target. The selection of the goal “Determine access order of pages
and items” also favours the sequential model. The association rules model represents a good option,
given the better compromise between precision and coverage. Yet, the inclusion of the hierarchical
clustering plan may be useful, although the model is lesser suited to the problem than the association
rules, which is more informative and accurate (i.e. provides rules with support and confidence).
4.3 Exemplifying learning
WUM processes description encompasses data from several dispersed and heterogeneous sources.
Sources include databases or files, containing “unlimited” datasets, pre-processing and KD tools,
858

Cristina Wanzeller and Orlando Belo
used to accomplish transformation and analysis activities, as well humans to provide complementary
descriptions. The data collection begins when a new problem arises, within a problem-solving use of
the system. Nevertheless, a new case may be integrated into de system independently from problem
solving. The analyst describes some datasets characteristics (e.g. variables semantic category) but
most metadata is extracted automatically (e.g. variables data type, number of distinct and null values),
as exemplified on Table 2. The data collection proceeds, with more intensity, during the Conciliate
and Retain steps. Again, user interaction with the analysts is required to gather information. However,
concerning mining activities, PMML documents are a very important source of knowledge to the MPS
system semi-automated learning approach.
Figure 3 shows a PMML document excerpt. We focus the mining activities gathering from PMML
documents, presenting the main item types which may be extracted (on Table 3). The used dataset
describes the page visits from msnbc.com, on September 28, 1999, whose dataset is available from
the UCI KDD Archive. We have few cases on MPS repository, based on this well know dataset.
…

Cristina Wanzeller and Orlando Belo
several values of parameter settings. The learning approach was used, with success for all the WUM
processes from which was possible to obtain PMML documents, despite the need to complement the
description through explicit user interaction. We may say that the approach is effective.
Table 3: Description of the main extracted PMML items
PMML elements/attributes
(Header) Application version name KD Tool version and name
DataDictionary name, optype dataType field name, operation type (e.g., ordinal, continuous) and data
type
SequenceModel: functionName,
DM Function (e.g. sequences ) and Model (e.g. sequence)
algorithmName
SequenceModel: remain attributes parameter names and settings
MiningSchema fields used in the model and the played roles
5. Conclusions and future work
Web based eLearning platforms promoters and administrators, like other Web site sponsors, have
pressures to provide better services, using less resources and more efficiently. WUM may be a tool to
bridge the gap between the unknown within massive clickstream data and actionable knowledge, in
order to devise opportune enhancements. At least, site sponsors have to know how eLearning
platform is being used in general terms. However, clickstream data analysis and particularly WUM
learning curves are serious obstacles to inexperienced users, being pertinent to have a strategy
showing the way to proceed.
The proposed and developed work aims at promoting a more efficient, effective and synergetic
exploration of WUM potential. To achieve this aim we designed, developed and implemented a
prototype of a CBR system, specifically devoted to assist users on WUM processes, mainly on
selecting proper mining methods and approaches to address analysis problems. The system also
provides support to users on documenting and organizing the knowledge gained from the experience
on solving new WUM problems, through a semi-automatic learning approach.
We believe that the MPS system is a good tool for knowledge creation, sharing and reuse. The
general and specific tests conduct so far confirm the systems effectiveness. Currently, we have cases
with complexity substantially higher than the ones showed. However, one concern is to support cases
from the eLearning domain. We may say that the system is capable of dealing with experiences from
this area. The case representation approach is flexible and wide and, moreover, MPS is extensible.
For the future we plan to further evaluate the current implementation, using cases from the eLearning
domain. Furthermore, we intend to study the need to capture additional context information from the
eLearning sites. We want to study this specific area of application of WUM, in order to construct a
specialized case base. This specific WUM area, the problem types, the kinds of mining activities, the
related practical applications and the key data items are less studied and structured. Given so, our
first goal is to assure that the knowledge base can cope with specific domain, maintaining its
generality, but, at the same type, accommodating the rich details of these particular experiences.
References
Aamodt, A. and Plaza, E. (1994) “Case-Based Reasoning: Foundational Issues, Methodological Variations and
Systems Approaches”, Artificial Intelligence Communications, IOS Press, Vol7, pp.39-59.
Chorfi, H. and Jemni, M. (2004) “PERSO: Towards an Adaptive eLearning System”, Journal of Interactive
Learning Research, 15(4), pp. 433-447.
Herhskovitz, A. and Nachmias, R. (2009) “Learning about online learning processes and students’ motivation
through Web usage mining”, Interdisciplinary Journal of ELearning and Learning Objects, Vol. 5, pp.197-
214.
Kolodner, J. (1993) Case Based Reasoning, Morgan Kaufmann, San Francisco.
Lu, Jie (2004) “A Personalized eLearning Material Recommender System”, Proceedings of the 2nd International
Conference on Information Technology for Application. pp.374-379.
Mantaras, R., McSherry, D., Bridge, D., Leake, D., Smyth, B., Craw, S., Faltings, B., Maher, M.L., Cox, M.,
Forbus, K., Keane, M., Aamodt, A. and Watson, I. (2005) “Retrieval, Reuse, Revision, and Retention in
Case-Based Reasoning”, The Knowledge Engineering Review, Cambridge University Press DOI.
Pahl, C. (2004) “Data mining technology for the evaluation of learning content interaction”, International Journal
of ELearning, 3(4), 47-55.
Rafaeli, S., and Ravid, G. (1997) “Online, web based learning environment for an information systems course:
Access logs, linearity and performance”, Proc. of the Information Systems Education conference, USA.
860

Cristina Wanzeller and Orlando Belo
Richter, M (1995) “The Knowledge Contained in Similarity Measures”, (Invited Talk) at the First International
Conference on Case-Based Reasoning, Lecture Notes in Artificial Intelligence, Springer Verlag.
Rodrigues, L., Antunes, B., Gomes P., Santos, A. and Carvalho, A. (2007) “Using Textual CBR for eLearning
Content Categorization and Retrieval”, Textual Case-Based Reasoning Workshop at ICCBR-07, Belfast.
Romero, C. and Ventura, S. (2007) “Educational data mining: A survey from 1995 to 2005”, Expert Systems with
Applications, 33(1), 135-146.
Srivastava, J., Cooley, R., Deshpande, M. and Tan P.-N. (2000) “Web Usage Mining: Discovery and Applications
of Usage Patterns from Web Data”, SIGKDD Explorations, Vol. 1, No 2, pp. 1–12.
Srivastava, J., Desikan, P. and Kumar, V. (2002) “Web Mining - Accomplishments and Future Directions”. Invited
paper in National Science Foundation Workshop on Next Generation Data Mining, Baltimore.
Zaiane, O. R. (2001) “Web usage mining for a better Web-based learning environment”, Proc. of the International
Conference on Advanced Technology for Education, Canada.
861

The Virtual Path to Academic Transition: Enabling
International Students to Begin Their Transition to
University Study Before They Arrive
Julie Watson
Modern Languages, Faculty of Humanities, University of Southampton, UK
J.Watson@soton.ac.uk
Abstract: Institutions receiving international students for postgraduate study are now committing time and energy
to the development of online transition resources to enable students to prepare for the demands of a different
academic culture before they arrive. Important questions underlying such initiatives are identifying what kind of
digital resources will both engage international students and be of most use to them in preparing for this
transition, and how to effectively reach students. Current institutional initiatives are taking several forms. A
popular model is to offer browsable advice/tips or FAQs about life and study at a particular institution together
with, for example, video clips of other international students describing their experiences there. These may be
open and web-hosted or accessible through a password protected area on an institutional website or VLE. Less
commonly found are video and other media embedded in learning resources developed in the form of ‘learning
objects’ which have been designed to offer key information through structured interactive learning activities
supported with answers and feedback. Importantly, these also offer opportunities for language improvement at
the same time since they are supported by help, feedback and transcripts. This case study focuses on a project
to develop and deliver a pre-arrival online course of interactive learning resources for all incoming international
students to one UK institution. Building on five years of experience in delivering pre-arrival, tutored online courses
to pre-sessional course international students, the project team developed institution-specific learning objects and
incorporated open resources from the website, ‘Prepare for Success’, developed by the same institution. The
project seeks to deliver a self-access online course with three strands to it to address students’ concerns and
needs. These are to prepare international students for the location in which they will be living and studying (the
city of Southampton - its key features and amenities); to introduce them to practical aspects of British life and
culture (e.g. setting up a bank account, shopping in a UK supermarket) and to familiarise them with key study
skills and other aspects of UK academic culture which may present challenges for them (e.g. academic writing
conventions; dealing with course reading lists). This paper will be of value to institutions embarking on similar
ventures. It will describe the rationale for the online course; refer to the pedagogic approach taken; showcase
course content, and report on the first phase of its delivery which begins in late spring 2011.
Keywords: international students; transition; pre-arrival elearning; online course design; learning objects
1. Introduction
A key finding of a recent benchmarking survey into the state of international student services in UK
colleges and universities conducted by the UK Council for International Student Affairs (UKCISA) is
that the delivery of pre-departure information to students has changed from hard copy to electronic in
the majority of cases (UKCISA 2011). UK institutions have also realised the benefit of preparing their
incoming international students for life and study at their specific destination before they arrive by
using the affordances of the Internet. As any search of UK institutional websites will show, institutions
are increasingly turning to the development of online transition resources in order to do this.
Moreover, using online multi-media resources to provide an insight into everyday student life at a
particular institution and highlight aspects of academic life can also serve the secondary purpose of
promoting that institution and possibly converting undecided applicants into real students in the future.
In regard to the primary aim of better preparing international students for the challenges that lie
ahead, an important question for institutions is to identify the type of digital resources that can both
engage the interest of students and help them start on the transition process, particularly in terms of
recognising the demands of studying within a different academic culture. This paper presents a case
study of a project at one UK university to develop and offer an online transition course for international
students during the period before their departure for the UK. It will explain the rationale for the use of
an online course, outlining the pedagogic approach taken and the development process, and referring
to the choice of course content. It will also report briefly on the first phase of delivery.
2. Approaches to facilitating international student transition
Together with the use of institutional websites as the currently preferred means of providing
international students with pre-departure information in UK Further and Higher Education Institutions
(92%) (UKCISA 2011), there also appears to be an increasing interest among providers in moving
862

Julie Watson
beyond text-based linear information and exploiting the affordances of different digital media,
particularly audio-visual resources and scope for social networking. A survey of institutional websites
and their content directed at international students reveals increasing use of self-access audio-visual
resources, which take the form of video clips or podcasts and images of current international students
describing their experiences at an institution, and short FAQs or observations about life and study.
These offer scope in terms of both browsability and choice for the student.
Social networking is also gaining in popularity as a means of providing pre-departure information and
of putting prospective students in touch with current students or alumni. Recently, it has replaced
email and online forums as the preferred method of enabling contact between new international
students and their peers (UKCISA 2011). It is evident that new ways of informing and preparing
incoming international students are being sought by receiving institutions, utilising digital innovations
and changes in the ways we are choosing to interact thought the Internet.
3. Building on a tutored, course-specific transition course
Between 2005 and 2010, the University of Southampton offered a short online pre-arrival course
every summer for international students who, a few weeks later, arrived at the University to undertake
a five or ten week Pre-sessional Course in English for Academic Purposes before starting their main
course of study in the autumn term. Although the design of this course-specific transition tool evolved
over time and included experimentation with additional emerging technologies, a number of core
features persisted. These were that the course:
Was limited to 5 weeks in length
Focussed on practical aspects of living in the UK
Offered acculturation and preparation for students coming to study on the Pre-sessional Course
at the University of Southampton
Used a customised VLE for delivery (Moodle)
Included activities designed in the form of Learning Objects, podcasts and vidcasts as core
content
Was e-tutored through Discussion Forum and Chat Room
During the final summer of its delivery (2010) over 200 students responded to their invitation to join
the course, representing c. 40% of those contacted by email. While feedback from those students
who had taken the course was very positive, a number of issues became apparent:
The 5 week window for doing the course did not suit all students ( e.g. those still struggling to get
visas arranged; those working to pay for their forthcoming studies)
E-tutoring mostly entailed responding to a number of recurrent questions through the forum or
chat room; interestingly, students were less interested in discussion
A significant number of their questions related to study expectations
Many students, particularly those who had come through agents, reported on arrival that they had
not known about the online course
At the same time, certain aspects of the pre-arrival course were regularly receiving praise from the
students in their post-course evaluations:
This course made me familiar with the (pre-sessional) course, the people and the School
which I have chosen. I appreciate this course.
The main benefits are to know someone else before coming to UK and it’s very good for
making friends. It was really beneficial to me. By means of (the course) I learned many
things about British culture, cost of living, studying in the UK, and many, many useful
other things.
When I arrived in Southampton everything was familiar to me and I didn’t feel alone. I
could find much information about the University and Southampton through this and it
was so fun.
Significantly, most of the informative content of the course was delivered in the form of activity-based
Learning Objects, and data tracking through the VLE showed that these self-contained learning
resources, together with weekly podcasts and vidcasts, were the most consistently used parts of the
863

Julie Watson
course overall, with students continuing to access them after the course had officially finished (when
the tutor had signed off), during the following week leading up to their arrival in the UK.
4. A national transition tool for international students entering UK institutions
Concurrent with this, in 2008, a UKCISA-commissioned project funded through the Prime Minister’s
Initiative (PMI) for International Education was awarded to the same development team at
Southampton. The brief was to create a tool to help prepare all UK-bound international students for
the challenges of study within any UK Further or Higher Education institution, and at the same time, to
deliver a tool that would allow individual UK institutions to make use of it with their own students. The
outcome was the development of the web-based transition tool for international students and their UK
receiving institutions, Prepare for Success (PfS) (www.prepareforsuccess.org.uk). The key features of
the PfS website include:
A main menu of 23 multimedia learning resources designed as Learning Objects, introducing
aspects of academic life, study skills, and teaching and learning styles in the UK
Study Pathways: a choice of routes through the learning resources by skill or by topic
FAQs: an interactive wordle for browsing a bank of simple questions and answers about
academic study in the UK
FAQs for Further Education (FE): an interactive spinning globe for browsing a bank of specific
questions and answers about FE
Webpage for teachers and institutions: downloadable resources and guidance about modes of
use
Integrated social networking sites: Facebook and Twitter
Lessons learned from the experience of delivering the tutored pre-arrival course informed the
development of PfS, particularly in terms of the need for learning resources that students could use
independently and at their own pace to inform themselves, and the desirability of including podcasts
and vidcasts especially of their peers/other international students already embarked on study in the
UK. The focus of PfS is on introducing generic aspects of UK academic life: the different teaching and
learning styles that students would encounter and the study skills needed, rather than practical
aspects of living in the UK or the specific cultural context of study at a single institution.
Since its launch in July 2008, the PfS website has received over 350,000 visits from student users in
211 countries. In addition, c. 90 UK institutions (HEIs 80%; FECs 20%) have so far made use of it
with their own international students in a variety of ways, as well as a number of UK secondary
schools and sixth form colleges, and institutions overseas. Pre-arrival links to the homepage are the
most popular method of incorporating the resource into an institution’s own provision but increasingly,
integrations with pre- or post-arrival student induction programmes are occurring (Watson 2011).
Inclusion in the pre- or post-arrival induction programme of a specific institution is possible by linking
to selected learning resources in the PfS main menu from an institutional VLE- or web-hosted course
or resource such as a wiki or blog. The learning objects are designed as separable html pages and
can function as course ‘building blocks’ to link to alongside an institution's own transition resources for
its international students. This feature has been of particular benefit to the current project.
Ultimately, a further aim of both PfS and the earlier course-specific transition tool which gave rise to it,
has been to achieve an institutional transition tool for international students which combines
preparation for generic challenges (academic culture and study skills in the UK) by re-using the
educational resources available through PfS, with preparation for the more specific context of study
(that of the destination institution and its location) using an institution’s own resources. As our
experience with the course-specific transition tool showed, students appreciated a focus on future
study skills as well as content relating to their specific destination institution and more practical
matters.
The findings of an earlier feasibility study (Watson 2007) undertaken before the development of
Prepare for Success, also confirmed this to be the view of other institutions. Moreover, the responding
institutions in that study expressed the desire for a transition tool which, ‘covered generic issues
common to UK HE, but also (allowed for) an individual induction tool hosted by our institution’. The
separable learning objects in PfS are designed to meet this need, allowing institutions to incorporate
them into their own institutionally-hosted induction tool by selective linking. This approach was
864

Julie Watson
adopted at the University of Southampton in order to create the institution-specific transition tool for
this project, providing content on aspects of general and practical cultural issues and concerning the
institution itself, and incorporating PfS resources to broaden the study skills and academic culture
dimension. The project has resulted in the development of ‘Get Ready for Southampton’, an online
transition course offered to all international students coming to the University of Southampton.
5. Developing an institution-specific transition tool
The rationale for migrating from a tutored online pre-arrival course to one which is untutored and
allows automated self-access is partly explained above in terms of optimal timing and use of tutoring.
Other factors also played a role. Due to the blending of three strands: synchronous (chat sessions),
semi-synchronous (discussion activities working within accepted time frames) and asynchronous (the
self-access learning content) on the tutored course, the management of a large number of students
by one tutor was possible (e.g. c.115 students per tutor in 2011). However, a new aim was to extend
the provision of the transition course beyond the Pre-sessional course students to encompass all
potential direct entry international students as well. Some of these students would hold confirmed
offers and some would still be in the process of applying through their faculty admissions office. Since
this would involve potentially a very large cohort of students, the preferred option was to deliver wholly
self-access content. The features required of the course design for ‘Get Ready for Southampton’
(GRfS), then, were that it:
Alllows study pace and length of use to be decided by the individual student;
Provides a broader focus: practical aspects of living in Britain + preparation for study at the
University + UK academic culture and study skills;
Delivers automated course invitation, sign up process and account creation through a single entry
point (see Figure 1);
Offers self-access content (activity-based learning objects – see section 5.1);
Offers an optional community-building dimension through the use of a social wall.
The outcome is an institution-specific transition tool that delivers a compromise between a fully
tutored and time-restricted online course and a collection of open and readily accessible multi-media
resources, structured loosely or unstructured. At the current time this latter option appears to be the
approach being favoured by many receiving institutions. However, our aim has been to package
informational content in such a way as to maximise its potential for learning by the student. The
pedagogic approach to doing this is explained in the next section.
5.1 Pedagogic design
Both the generic resources in Prepare for Success ( which were selectively linked to) and the
institution-specific content in GRfS employ video and other media embedded in learning resources
developed in the form of ‘learning objects’, and are designed according to a shared pedagogy (see
Watson 2010). The aim here is to offer informational content about the destination institution, the city,
aspects of UK academic culture and life as a student through structured interactive learning activities
supported with answers and feedback. These offer flexibility in terms of how the individual student
chooses to use them but importantly, content is scaffolded so as to provide opportunities for language
improvement at the same time. Design features which facilitate this include help and feedback
sections, and additional resources such as transcripts. Figure 2 below shows some highlighted design
features (1. Instruction; 2. Activity; 3. Feedback). Figure 3 shows additional design features (1. Help;
2. Transcript; 3. Video resource for the Flash activity which follows). With structured learning activities
at their centre and these extra layers of support, the learning objects offer more scope for interaction
and reflection, and consequently, greater potential for learning than less rich, pure media resources.
A further benefit of this approach is that learning objects from different sources can be pulled together
to form a new course offering. Older in-house resources can be given a facelift and repurposed to suit
new contexts of use. Any new content that is required can be developed. Where content can be
disaggregated and re-aggregated in this way, online course creation is achieved in a much shorter
time span. In Figure 4 the juxtaposition of selected and linked learning object resources from PfS
(particularly under topic 2) with learning objects initially created for the tutored, course–specific
transition tool and then repurposed as necessary for GRfS (several learning objects under topic 1)
can be seen. This use of separable reusable ‘building blocks’ in online course design proves both
865

Julie Watson
practical and economical. As an important aspect of instructional design, learning object reusability
has also long been recognised as desirable although the debate continues about how best to achieve
it (Wiley 2000, Harvey 2005, Sicilia and Garcia 2003).
Figure 1: Entry point to the self-access transition tool ‘Get Ready for Southampton’
Figure 2: Screenshot of Learning Object showing key elements of an activity
A number of studies report on the use of peer or alumni contact at different institutions either before or
after arrival (e.g. Lai et al. 2008; Quintrell and Westwood 1994). As UKCISA report, putting students
in direct contact with their peers or alumni using social networking tools, email or even simply through
online forums is a method increasingly being favoured by UK institutions (UKCISA 2011). Due to its
large scale, our project precluded direct contact with students already studying at the University. Our
own previous experience had suggested that students enjoyed introducing themselves and making
initial contact with one another through a discussion forum but that most of their postings had
concerned specific questions arising from visa delays; uncertainty about what kind of accommodation
to find etc. In other words, at this stage they had not sought extended contact with each other. For this
reason, it was decided to take an experimental approach during the first year and replace the
866

Julie Watson
discussion forum with a ‘Southampton social wall’ using freely available social software, wallwisher
(www.wallwisher.com), which was linked to from within GRfS.
Figure 3: Screenshot of other design features of our learning objects
Figure 4: Two topic menus within GRfS showing a blend of learning object resources
Wallwisher is an internet application which does not require user registration. A GRfS social wall has
been created and short student posts comprising text (max.160 characters), image and/or video can
be easily added from within the course simply by clicking on the wall. The use of tweet-size messages
reflects the typical size of the posts made by students on our course-specific transition course and,
unlike a discussion forum, the wall allows the content of all messages to be immediately seen. In
GRfS students are invited to post a short message at the end of the GRfS course and look among the
other messages for future friends at the University. It is hoped that this will also help provide a sense
of community. The use of this software will be reviewed at the start of the coming academic year and
user opinion sought at the student evaluation stage.
867

6. First phase
Julie Watson
Following a brief pilot phase, the GRfS site was launched in May 2011. The use of a customised
Moodle site and automated generation of unique usernames and passwords allows tracking and
useful data gathering to take place. Although still in the very early stages in this respect, some initial
data has been collected from the first seven days since the launch (see Table 1).
Table 1: Data gathered 7 days after launch of GRfS
Aspect of GRfS use investigated Summary of data
No. of students registering and accessing GRfS in the first 7 days 142
No. of those students who will join the summer Pre-sessional Course 92
No. of those students who will be direct entry in Sept. 2011 50
No. of countries students accessed GRfS from 35
No. of student posts to social wall 5
No. of viewings of individual items of course content Min. 8; Max. 134
Total no. of item viewings and average no. of views per item 1015; av. 30
The number of international students who responded to their email invitation and registered for the
online course within the first seven days of its launch represented 14% of those contacted in the first
emailing, but it should be noted that this action will continue throughout the summer and only a small
proportion have as yet been sent the email so the eventual number of GRfS users is expected to be
much higher. The first emailing also contained a disproportionate number of international students
who will be studying at the University on a 5 or 10 week Pre-sessional Course between July and
September. Further mailings are likely to focus on direct entry students to other faculties and those
coming to study at the University under the Erasmus scheme. Nevertheless, these early figures are
felt to be very encouraging.
7. Conclusion
The aim of this project has been to put into place a practical method of facilitating international
student transition on a much wider scale than has been hitherto possible. It is too early to tell yet how
effective this can be, or even what the eventual level of take-up will be, although the very first data
suggest that a promising start has already been made. Student feedback being gathered on the
course, will be reviewed in the autumn together with the data from tracking, It is our intention to keep
this transition gateway open year round for all incoming international students, including those who
are in the initial stages of course enquiry. Through the various mechanisms in place, we hope to
harvest some rich data which will help us to continue to improve our pre-arrival support and
preparation of international students.
References
Harvey, B (2005) Learning Objects and Instructional Design. International Review of Research in Open and
Distance Learning, Vol 6 No. 2. (online) http://www.irrodl.org/index.php/irrodl/article/view/227/310
Lai, K-W. Berg,D. and F.McDonald, (2008) Establishing an online peer-support and mentoring community for
international students: Some cultural and design considerations. In Joint Open and Working IFIP
Conference ICT and Learning for the Net Generation. (online)
http://cs.anu.edu.au/iojs/index.php/ifip/article/view/13532
Quintrell N, and M. Westwood (1994)The Influence of a Peer-Pairing Program on International Students' First
Year Experience and Use of Student Services
Higher Education Research & Development. Vol 13, No 1, pp.49 - 58
Sicilia, M. and Garcia, E. (2003). On the concepts of usability and reusability of learning objects. International
Review of Research in Open and Distance Learning, Vol 4 No. 2. (online)
http://www.irrodl.org/index.php/irrodl/issue/view/16
UKCISA (2011) International student services in colleges and universities 2010-11: a benchmarking survey.
UKCISA:London.
Watson, J. (2007) A Feasibility Study for the Development of a Pre-arrival Induction Tool for International
Students. Unpublished report for UKCISA (UK Council for International Student Affairs).
Watson, J. (2010) A case study: developing learning objects with an explicit learning design. Electronic Journal of
e-Learning, Vol. 8, No.1, pp. 41-50. (online) www.ejel.org/issue/download.html?idArticle=159
868

Julie Watson
Watson, J. (2011) Designing a Self-Access Website of Pre-arrival Learning Resources to Support International
Student Mobility. Ed Media Conference Proceedings. AACE (online) http://www.editlib.org/ (Forthcoming)
Wiley, D. (2000) Connecting learning objects to instructional design theory: A definition, a metaphor, and a
taxonomy. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects (online).
http://reusability.org/read/chapters/wiley.doc
869

Identifying and Locating Frames of Reference to Inform the
Design of Virtual Worlds in Higher Education
Katherine Wimpenny, Maggi Savin-Baden, Matt Mawer, Nicole Steils and
Gemma Tombs
Coventry University, Coventry, UK
m.savinbaden@coventry.ac.uk
Abstract: In the changing context of higher education a series of pedagogical shifts have occurred and with
them a number of interactive learning approaches have emerged. Learning in immersive worlds (simulations
and virtual worlds such as Second Life) has received significant attention, but to date the impact of virtual world
learning on higher education remains relatively under-researched. This paper will draw on 3 distinct but
interrelated funded studies that have explored the socio political impact of virtual world learning on higher
education, with a specific focus on Second Life (SL). It will argue that there are multiple frames of reference
which inform the design of and response to virtual worlds as learning technologies. Such frames of reference
were evident in the practices of those involved in using virtual worlds, but have largely been over-looked in the
literature in terms of their impact.
Keywords: virtual worlds, Second Life, qualitative study, participatory action synthesis
1. Introduction
Although the higher education landscape continues to change and evolve there is still relatively little
data that indicates how tutors make pedagogical design decisions and what impact this may have on
students. This paper will draw on 3 distinct but interrelated funded studies that have explored the
socio political impact of virtual world learning on higher education, with a specific focus on Second
Life (SL). It will argue that there are multiple frames of reference which inform the design of and
response to virtual worlds as learning technologies. Three particular frames of reference have been
found to be evident across the studies, and they provide the focus for this paper, namely:
Understandings of games and gaming media
Disciplinary values
Institutional space and ownership
We have drawn upon O’Donoghue’s interpretivist position on ‘perspective’ in our use of the term
‘frame of reference’ (O’Donoghue, 2007 p. 26). However, the stance we have taken is to use the
notion of 'frames of reference' as a lens through which it is possible to see the impact of different
stances, approaches and beliefs on the use of virtual world technologies in higher education. Further,
such frames of references may, or may not change in the light of experience. Taken together, we
assert these frames of reference inform understandings of the variation in approaches taken by
students and tutors when using immersive virtual worlds, which, in turn, inform decisions made about
learning designs and pedagogic response.
2. Literature
Over the last decade there has been much criticism about interactive media environments that fail to
create effective settings for learning (Noble, 2001; Reeves, 2002). One of the reasons for this has
been because the focus in interactive media environments has been on technological rather than
pedagogical design. Although there is a range of literature that reflects diverse disciplinary use of
immersive worlds (as exemplified in Savin-Baden, 2010) there are few expositions of the complexities
of the use of SL or indeed transdisciplinary research studies. Yet other studies might be overlaid to
help our understanding of the use of SL in the disciplines. For example, Jenkins & Zetter (2003) argue
that disciplines shape the nature of pedagogy and such pedagogies reflect the practices and culture
of the discipline. In addition, Trowler & Trowler’s (2010) recent review of the literature brought
together three reported dimensions of student engagement in relation to learning, identity and
structure and process. However, there does appear to be a decontextualisation of teaching methods
and technical developments from both the learners and the disciplines resulting in a worrying trend
towards ignoring the particularities of teaching in a given discipline (Becher & Trowler, 2001), along
with the assumption that teaching and learning are necessarily the same thing. By contrast, engaging
in learning and play has been recognised in both schooling and higher education as being useful for
encouraging effective learning (Dewey, 1938; Bruner, 1991; Gee, 2004). Games such as the Quest
870

Katherine Wimpenny et al.
Atlantis Project (Barab, Dodge, Tuzun, et al., 2007) a 3D game for children, and the River City MUVE
(Galas & Ketelhut, 2006) have embraced play as a central component of learning. The paper
examines the ways in which particular frames of reference relating to immersive virtual worlds might
affect the way we approach learning design in higher education. Specifically we investigated the
following questions;
How do frames of reference articulated around gaming and virtual worlds influence expectations
and engagement with SL?
What disciplinary influences are prominent in the use of SL?
How do tutors’ perceptions of ownership of space inform approaches to pedagogy?
3. Methodology
The three studies have each adopted the use of different methodologies which remain soundly
qualitative. The range includes case study (Simons, 2009), narrative inquiry (Clandinin & Connelly,
2000), and modified grounded theory (Charmaz, 2006). Whilst separate in their study design,
examples of data from the three studies have been brought together for the purposes of this paper to
form a ‘synthesis' through an interpretivist lens. Thus both the experiences and structures reported
upon by research participants have been examined in context. This synthesis of such accounts has
demanded naturalistic approaches to the translation of field data and emerging concepts from the
individual studies into one another, thereby evolving overarching concepts. We have termed this
process participatory action synthesis (Wimpenny and Savin-Baden, forthcoming), which is presented
in more detail below.
3.1 Data collection
Data were gathered by three PhD students, through individual and group interviews with students,
tutors and known experts over an 18 month period, from diverse research sites across a variety of
disciplines subject areas, including the arts and humanities, computing, professional education and
employability.
Data analysis, synthesis and interpretation
The participatory action synthesis process involved simultaneous phases of data collection and
inductive approaches to analysis, building on the process of reciprocal translational analysis (RTA) as
outlined by Noblit & Hare (1988) and further adapted by Sandelowski & Barroso (2007). Figure 1
illustrates the process used to locate knowledge gaps by making connections between findings and
themes, moving beyond breaking down, reassembling and describing the findings to offering new
forms of representation, contextualised by the literature as suggested by Major & Savin-Baden,
(2011).
Figure 1: Process of participatory action synthesis
871

Ethical approval
Katherine Wimpenny et al.
Approval was granted through research ethics at Coventry University and renegotiated with all other
university research sites. Research integrity has been assured through researcher reflexivity and
trustworthiness throughout the study.
4. Findings
The findings presented here reflect the frames of reference that emerged through the participatory
action synthesis. It should be noted at the outset that these ‘frames’ were not imposed on these data,
but rather emerged as issues of tension and troublesomeness across the studies. Whilst there are
other cross study themes it is these frames of reference that appeared to be most poignant and are
therefore presented here.
4.1 Understandings of games and gaming media
Within our data at least seven unique reference points related to digital games emerged, including:
Console first-person shooters (FPS),
Online FPS,
Beat-‘em-up/fighting games,
Online casual games (for example. Facebook games),
Role-playing games,
Simulation and world builders
Virtual world platforms (both gaming and metaverse).
The disparities between the norms, standards, and expectations drawn from these frames of
reference can be stark, and what is translated from digital games to virtual worlds by individual
participants can be diverse. Frames of reference may, for example, influence expectations of
behavioural norms, as evidenced by one participant:
I've only ever played games when you beat people up. So someone would be in front of
me and I'd be like ‘oh, how do I hit them, how do I hit them?!
Here the individual’s previous gaming history is rooted in a specific type of game (a beat-‘em-up),
where behavioural norms (such as hitting) are quite specific. Translating this behaviour into the virtual
world of SL was therefore problematic. Not only were there disparities in systems of action (i.e. how to
hit someone), but such actions carried differing significance within much of SL compared with a
fighting-oriented digital game.
Frames of reference were also evident in participants’ estimations of personal competence:
I used to play Sims, but I was never good at it…and so, when they were, like, ‘oh, you’re
going to be able to build a set and you’re going to’, I was, like, ‘oh, (explitive) it’s like
Sims!’. And it was, it was just daunting to think that, like, I was going in this place.
Here a digital game frame of reference (The Sims), influenced both the expectations of action in
world, and the participant’s perceived personal competence at successful completion of those actions.
Previous experiences with The Sims appeared to have lowered this student’s confidence in the
possibility of a positive engagement with SL. The sense of anxiety evident in this quotation highlights
how a gaming history can affect motivation and self-belief.
In addition to the diversity in the types and the ways in which digital games affect
engagement with the virtual world, we found virtual worlds were positioned by
participants in a variety of ways. For example, as digital games, non-games, a replication
of the physical (‘real’) world, an augmentation of the physical world, or a distinct and
separate fantasy world. The link to digital gaming was sometimes made explicitly: I was
probably one of those people that, prior to the course, that was guilty of thinking that
Virtual Worlds were just games effectively.
In this case, the ‘game’ as a frame of reference is applied to understand the virtual world. The use of
‘guilty’ and ‘prior’ were also of significant interest here, in that it indicated a shifting of perspectives
872

Katherine Wimpenny et al.
between frames of reference i.e. away from games towards other possibilities. This shift was
observed amongst numerous participants who began from this position:
Well, I keep saying RPG [role-playing game] because I do see Second Life as an RPG to
some degree. Um, it’s not a game, I know that, I’m very aware of that, but it is in that
same category.
Yet here the continuity between digital games and virtual worlds is less clear. From a cognitive
perspective what we are seeing here is the individual locating the virtual world in relation to their
cognitive frame of reference, and trying to make sense of it based on prior learning and experience
(Ausubel, 1975).
It became clear that frames of reference were particularly murky at the interstices between gaming
and non-gaming media, and fantasy and reality. The virtual world can be part chimera (like a clone:
Friese, 2010) and part shapeshifter (potentially like its users: Savin-Baden, 2010); a positional
conundrum. Such frames of reference at an individual level were also affected by the particular
disciplines in which virtual worlds were being used.
4.2 Disciplinary values
By use of the term disciplinary values we mean not only the impact that learning disciplinary
knowledge, skills, and behaviours has on learning and teaching, but also the way in which disciplinary
traditions and beliefs affect what it means to learn within a given discipline.
The findings indicate that initially students did not understand what and how they were expected to
learn when using SL. For example, individual frames of reference appeared to draw upon pre-existing
images, knowledge, and experience of the particular discipline that may or may not be relevant. What
is of note here is whether tutors are able to help students make connections between complex subject
matters and constructive ways of learning in SL, especially if tutors themselves are uncertain of the
learning technology, which may require a radical transformation of their practice (Kalogiannakis,
2004). In our data we found students and tutors framed their experience alike, in that they were not
able to make sense of SL use intuitively. Some showed signs of reservation or resistance, as
commented upon here by a designer for e-learning:
Prior to this I’d tried it out at home on the PS3 for kind of five minutes; I tried Second Life
for minutes and kind of run away screaming ‘this is just rubbish!’
Whilst this varied depending on the individual and level of the course, there was a tendency for an
enhanced and applied understanding to emerge as students became more familiar with the
application – as demonstrated in this quote from a rather sceptical environmental management
student:
But it was definitely better than I thought, easier than I thought. Although yeah, in the
beginning I thought ... this is rubbish, I’m not going to learn anything from this and that
changed. […] They’re not going to make us do anything that’s going... that’s going to
have benefit or use. So yeah, definitely changed my mind on that.
Here we see a clear link between the use of SL and the values implicit in the discipline: team work
and presentations, for example. Yet in contrast the framing of experience shared by a performing arts
student suggests more of a struggle to make disciplinary links:
I was a bit like 'that's, that's not theatre!’, but then, I was left working with it for a while
and that was it, it is, it's, in its own little way, it can be used as a performance tool, as well
as a lot of other things.
What became apparent by such frames of reference was that the use of SL did not provide immediate
disciplinary ‘fit’. By this we mean that when students came into these learning spaces, they did not
immediately recognise the disciplinary shape of them or were able to marry them with previous
experiences of disciplinary values and discipline-based pedagogy. However, there were examples
demonstrating a clear fit between disciplinary values and the use of virtual worlds. One example was
the use of SL to simulate a disaster scenario for environmental managers. Here the potential benefits
of SL were clear; a SL simulation provided a safe, but complex space for trainees to practice their
future professional roles, and the transferability of the training ‘into the real world’ was obvious. As two
students acknowledged:
873

Katherine Wimpenny et al.
SL was a good starting point. Great to try things out first before doing it in reality. It’s safe
preparation for placement, I’ll be able to bring some skills in and try them out again.
(It) was a bit like a role play, because we don't get to do any kind of practical things
really, or that many, so as it was, it was a good kind of tool to use, where we could
actually put skills into practice without actually physically having to go and do it and we've
got not anything like that, so that was, that was really good.
SL was framed as a space that provided opportunity for the development of disciplinary values and
capabilities (such as ‘soft’ skills practice, building, designing) within a specific discipline, yet as Savin-
Baden (2008) argues, does it also provide scope for the level of critique necessary for life and work?
Tutors and students tended to build and visit spaces within SL that reflected their discipline, and such
spaces were designed within disciplinary assumptions. Yet our data also suggested that the intricacy
of how disciplinary values may be conveyed requires more thoughtful consideration. Immersive
worlds offer possibilities for, and the desire to do things differently, whilst also confirming and imbuing
a sense of disciplinary values. Yet at the same time understandings of games and disciplinary values
are also affected by the institutional spaces into which they are placed, and it is this we next explore.
4.3 Institutional space and ownership
These findings were drawn from interviews with tutors involved in teaching in SL, and they represent
the complex understandings of ownership that frame the rationales and approaches to the use of SL.
As Temple (2008: 239) notes, the university’s use of space is intimately connected to the student
learning experience and thus the implications of spatial practice should be closely considered. We
suggest that as new spaces emerge in higher education (such as virtual spaces), they must be shown
the same regard. Here we draw upon Lefebvre’s (1991) notion of (social) space; specifically space as
a means of control, through which some understanding of ownership is developed. Lefebvre’s
constitution of spaces, along with territorial, disciplinary and institutional spaces impact on learning
spaces by preventing or enhancing the development of creative spaces, yet an understanding of the
diversity and complexity of learning spaces can also inform the ways that they are (re) created,
managed and owned.
At least five different reference points emerged from our data regarding the perceived ‘ownership’ of
SL:
Student-owned social space
Student-owned learning space
Practitioner-owned replicated classroom
Institutionally owned extension of the campus
Institutionally owned marketing space
Tutors often viewed SL ‘islands’ as encompassing multiple frames of reference, thus precluding an
easily discernible notion of ownership; drawing again on Lefebvre, understandings of distinct
ownership seemed to relate to the ‘everyday life’ and spatial practice. For example, spaces between
people and places are important learning spaces. Lefebvre (1991) suggested social space might be
seen as comprising a conceptual triad of spatial practice, representations of space and
representational spaces. Spatial practice indicates the way in which space is produced and
reproduced in particular locations and social formations. Yet in the context of SL it would seem that
such a formulation of space has created different and diverse spatial zones along with imaginary
geographies. For the purpose of this paper, two distinct but interrelated frames of reference are
delineated: SL as a student owned space, and as an institutionally owned space.
Assumptions of institutional ownership in SL have often been related to the representation of space
and the re-creation of physical university buildings (for example, Savin-Baden, 2010). However, as
representations of space have altered throughout SL’s lifespan, assertions of institutional ownership
have altered. For participants in this project, the framing of SL as an institutionally owned space often
related to the level of control exerted by the institution. The structuring and formalising influence of the
institution through these processes was also recognised, as exemplified by one tutor:
874

Katherine Wimpenny et al.
It’s somewhere where you get that nice crossover between the informal and the formal…
You know, it’s one of those places where you can see the bringing together of those two
spaces.
Here, SL was framed as a space in between the formal (the institution) and the informal (social
media); as a way to bring the two spaces together. However, the control and authority implicit in this
quote are initiated by the educator. Intricate understandings of ownership are thus indicated, implying
an ease of unity which we suggest does not exist in practice, (where routine and ritual prevail).
Exploring perceptions of student ownership represented a range of complexities, two examples of
which are now considered.
The use of SL as a way to engage with students in a ‘shared’, informal space was often promoted by
staff. However, for one tutor, his students’ social use of SL was perceived to be a key factor in
safeguarding their ownership as opposed to the institution’s ownership and control:
But when [students] go down to the student union bar then, well that's fair enough, they
can do what they want to do. They might be chatting about the learning stuff, discussing
assignments - that's their environment, their space, and that's what I think of Facebook
as being like. And in a way that's what I think of Second Life as being like. It’s that it's not
a space that we can intervene in too forcefully, or interfere with. You know, it’s fine for
[tutors] to pop in every now and again, but - a sense of ownership I guess it is, over
different environments.
The comparison of SL to the student union bar and to Facebook (seen as both learning and social
spaces, but definitively student-owned spaces), for this participant, was seen as demanding a ‘light
touch’ from the institution. Students did not actively exert control or authority in the claiming of SL as
‘their’ space; rather, this was seen by this participant as being the institution’s responsibility to ensure
it did not encroach on that ownership.
An alternate view of ownership emerged through the use of islands other than the educator/institutionowned
space for learning. These were often referred to as "field trips", thus automatically situating
these SL spaces as owned by another. However, it also represents a further claim to institutionally
owned space. The discursive construction of a field trip automatically presupposes that there is a
space to leave that is not a field trip. In terming the visit to another island's space as a field trip, their
own institutional island becomes the SL 'home' from which they leave and to which they will return.
Yet for one tutor working in a science-based discipline, the use of other SL islands was perceived as
a challenge to student ownership of space and of learning:
I do feel that the eye candy aspect of SL can lead to a degree of "tourism". I want... to get
students to modify the environment. To achieve their own ends... So it is important for me
that they generate physical artefacts.
Here, student ownership of the SL space is supposed through the creation of the objects and the
modification of the environment. Framing ownership in this way meant a move away from the
ownership of physical space as discussed thus far (ownership of their SL island, ownership of the SL
technology as a whole) and establishes ownership as the enactment of spatial practices in Lefebvre's
terms. When ownership is perceived in this manner, students can own any SL space in which they
can build, for example: public sandboxes, their institutional island, and the sandboxes of
other institutional islands. Whereas ownership has largely been understood in terms of the formal
(institutionally owned) and informal (social media/student ownership) values attributed to the space,
for this participant, were a means of ensuring student ownership through creative learning processes:
'achiev[ing] their own ends'.
5. Discussion
In the following section we move on to discuss how participants multiple frames of reference served to
influence and inform virtual world design. Firstly we contend that a continuity of frames of reference
between digital games as a media and virtual worlds as a media cannot be assumed. For example,
some participants viewed virtual worlds as a type of game, whilst others held alternate positions and
meanings, and understandings were not necessarily translated in straightforward ways. Therefore it is
not easy to predict what influence on action emerges from the positioning of virtual worlds as akin to
(or actually as) games. The framing of virtual worlds that influence actions are neither determined
wholly in isolation of pedagogy and engagement in higher education, nor solely by that engagement.
875

Katherine Wimpenny et al.
This in turn raises questions about the relationship between disciplinary values and virtual worlds,
which we now discuss.
There are few expositions of the complexities of the use of SL in disciplinary research studies. These
findings indicate that students struggle to see the disciplinary relevance of their learning in SL, and
staff do not always realise the impact of discipline-based pedagogy on their use of SL. The result is
that individually held disciplinary assumptions result in students holding different positions regarding
professional understandings, which are not translated in straightforward ways. Although the play
elements and visual stimulus of SL is explored in the literature, what remains relatively hidden is the
potential for SL to project disciplinary understandings, enabling students to see its relevance for their
subject field. We contend that tutor confidence, knowledge and skill in using SL within the disciplines
are key. Whilst not directly related to SL, studies such as Kalogiannakis (2004) demonstrate the value
added when social support networks are used to support tutors in their role to effect the diffusion of
technology within a profession. Further, what is important to consider when designing virtual world
learning is not only knowing ‘how to do it’ in SL, but also how to do it in new ways in SL, and under
which circumstances, and how this can affect the way that students learn particular subject matter.
Shulman’s (2005) work on signature pedagogies can be applied here in terms of considering ‘surface
level interaction’, or the working principles, employed by the tutor to enhance learning, to give
students good reason to be part of the learning community, to ensure learning expectations are
explicit and responsive and foster social connections. If disciplines shape the nature of pedagogy and
such pedagogies reflect the practices and culture of the discipline, how can use of an immersive
learning environment influence teaching practices and the methods by which future practitioners will
be educated for their profession?
Finally suggest there is a tension between designing learning for the disciplines within SL, which
makes best use of its creative space, capitalising on the one hand a sense of novelty and surprise,
(Jankowska and Atlay 2008), whilst being aware of expectations and reference points for the learner.
As Savin-Baden (2008) identifies, the opportunity to do things differently when designing learning for
the disciplines within SL, where there is less order, forces a reconsideration of how learning spaces
are to be constituted. Further, understandings of ownership play a role in how practitioners perceive
virtual worlds as learning technologies and how this can influence the design of pedagogy within it.
Yet the balancing of SL as a social space and SL as a learning space include issues of ownership.
Within the study ownership (and associated themes of implied control and power exertion) emerged
as a complex frame of reference which differed from individual to individual, and characterised
different aspects of SL. Such a range of perspectives in turn raise questions about what is allowed
and disallowed including how tutors may seek to control and contain space. For example, SL was
viewed as an institutionally owned extension of the campus, as a replicated classroom and, or as a
useful marketing feature. Such perspectives were due in part to the design of the space, but arguably
were more a reflection of views of ownership framed by self-positioning. Our data revealed how one
tutor framed SL as offering a useful crossover space in which to bring together social media and the
institution; in other examples we see SL as being framed as a type of game, or, seen as both a
learning and social space, but definitively a student-owned space. Our findings demonstrate a tension
expressed by tutors who seek to push boundaries of structure and appearance, and encourage
students to make use of SL spaces, yet also want students to get something from their learning, in
ways they perceive to know best.
6. Conclusion
This paper has examined frames of reference relevant to the design and experience of virtual worlds
in higher education. This participatory action synthesis suggests that previous explorations of learning
in virtual worlds in the research literature have neglected to look more closely at frames of reference
and how these serve to inform expectations. Whilst not always deleterious, there is potential that
frames of reference may interfere with, and or collide with each other, with interesting consequences
during the student / tutor encounter. Of further note is how frames of reference are reconceptualised
temporally, thus in the (re) configuring of perspectives how will learning designed within virtual worlds
respond.
References
Ausubel, D.P., Novak, J.S. &Hanesian, H. (1978). Educational Psychology: a Cognitive View. New York: Holt,
Rinehart and Winston.
876

Katherine Wimpenny et al.
Barab, S., Dodge, T., Tuzun, H., Job-Sluder, K., Jackson, C., Arici, A., Job-Sluder, L., Carteaux, R., Jr.,
Gilbertson, J., and Heiselt, C. (2007). The Quest Atlantis Project: A socially-responsive play space for
learning. In B. E. Shelton, & D. Wiley (Eds.), The Educational Design and Use of Simulation Computer
Games; Rotterdam, The Netherlands: Sense Publishers.
Becher, T. & Trowler, P.R. (2001). Academic Tribes and Territories, Second Edition. Buckingham: Open
University Press/SRHE.
Bruner, J. (1991). Acts of Meaning. Cambridge MA: Harvard University Press.
Charmaz, K. (2006). Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis. London:
Sage.
Clandinin, D. J. & Connelly F. M. (2000). Narrative Inquiry: Experience and story in qualitative research, San
Francisco: Jossey-Bass.
Dewey, J. (1938). Experience and Education. New York: Collier and Kappa Delta Pi.
Friese, C. (2010). Classification Conundrums: Categorizing Chimeras and Enacting Species Preservation.
Theory and society, 39, (2) 145-172.
Galas, C., & Ketelhut, D. J. (2006). River City, The MUVE. Leading and Learning with Technology, 33(7), 31- 32.
Gee, J.P. (2004). What Video Games Have to Teach Us about Learning and Literacy. Basingstoke: Palgrave
Macmillan.
Jankowska, M. & Atlay, M. (2008). Use of creative space in enhancing students’ engagement. Innovations in
Education and Teaching International. 45 (3), 271 – 279.
Jenkins A. & Zetter R (2003). Linking Teaching and Research in Departments. York: Higher Education Academy.
Retrieved from http://www.heacademy.ac.uk/resources.asp?process=full_record&section=generic&id=257.
Kalogiannakis, M. (2004). A virtual learning environment for the French physics teachers, Education and
information technologies, 9 (4), 345 – 353.
Lefebvre, H, (1991). The production of space. 15 th Edition. Oxford: Blackwell.
Major, C. & Savin-Baden, M. (2011). Integration of qualitative evidence: Towards construction of academic
knowledge, Qualitative Research, 11 (6) 1-19.
Noble, D. F. (2001). Digital diploma mills: The automation of higher education. New York: Monthly Review Press.
Noblit, G. W. & Hare, R. D. (1988). Meta Ethnography: Synthesising Qualitative Studies. Newbury Park, CA:
Sage.
O’Donoghue, T. (2007). Planning your qualitative research project: an introduction to interpretivist research in
education, Routledge, UK.
Reeves, T. (2002). Storm clouds on the digital education horizon, in: A. Williamson, C, Gunn, A. Young, & T.
Clear, (Eds.), Winds of change in the sea of learning Proceedings of the 19th Annual Conference of the
Australasian Society for Computers in Learning in Tertiary Education (ASCILITE), Auckland. Auckland, New
Zealand: UNITEC Institute of Technology 17-26.
Sandelowski, M. & Barroso, J. (2007). Handbook for synthesising qualitative research, Springer: New York.
Savin-Baden, M. (2008). Learning Spaces: Creating opportunities for knowledge creation in academic life,
McGraw-Hill, Open University Press.
Savin-Baden, M. (2010). 'Changelings and Shape Shifters? Identity Play and Pedagogical Positioning of Staff in
Immersive Virtual Worlds.' London Review of Education, 8, (1) 28-38.
Shulman, L. (2005). Signature pedagogies in the professions, Daedalus, 134, (3) 52-59.
Simons, H.(2009). Case Study Research in Practice, London SAGE.
Temple, P. (2008). Learning spaces in higher education: an under-researched topic. London Review of
Education, 6, (3) 229-24.
Trowler, V. & Trowler, P. (2010). Student engagement evidence summary. The Higher Education Academy.
Wimpenny, K. & Savin-Baden, M. (submitted) Exploring and Implementing Participatory Action Synthesis
Qualitative Research.
877

Reaction Lecture: Text Messaging to Increase Student
Engagement in Large-Scale Lectures
Koos Winnips, Joost Heutink, and Hans Beldhuis
University of Groningen, Netherlands
koos.winnips@rug.nl
j.h.c.heutink@rug.nl
h.j.a.beldhuis@rug.nl
Abstract: By helping to be active during large scale lectures, students remember better what is presented, and
can better connect new knowledge to entry knowledge. In large scale lectures with more than 150 students, it is
nearly impossible to manage personal, face-to-face discussions and interaction. Based upon positive
experiences with enabling interaction via mobile phones a new form of interaction was tried in a second year
Bachelor’s course (722 students). Students could send in open comments and questions via SMS, Twitter or via
mobile Internet. The lecturer built in blocks of time to respond to these questions and comments. Unanswered but
relevant questions are forwarded to the forum of the electronic learning environment (Blackboard), enabling an
online response. An experimental design was set up, dividing the lectures of the same lecturer between
“reactionlectures” and traditional lectures. Three primary questions for this study were defined: Would students
learn better in a reactionlecture than in a traditional lecture? What are the opinions of students on giving direct
reactions during a lecture? What forms of providing open comments during lectures can be used? A difference in
learning results (as perceived by the students) was not found. Although students indicated that by giving open
comments, the lecturer could better adapt to their entry knowledge. Further, questionnaires showed that students
were happy both with the options for interaction, and with the connection between the lecture and the discussion
forum in Blackboard. Students indicated they would like to have more blocks of interaction during the lecture, so
that responding via their mobile phone doesn’t distract from listening to the lecturer. Contrary to expectations
about the “Facebook generation”, students did not respond or comment often. To make the reactionlecture more
effective, blocks of interaction on key lecture topics should be scheduled.
Keywords: backchannel, audience response systems, student interaction, large-scale lectures
1. Introduction
This paper describes a way to give students a new role during lectures that supports active learning.
Traditionally, large-scale lectures (more than 150 students) provided a passive experience. A lecturer
provides information to students, and students are listening. But, the attention span of students is
limited. Students themselves report that the longest time they can endure uninterrupted lecturing is 20
to 30 minutes (MacManaway, 1970). To “reset this attention clock” several measures have been tried
out, such as “structured interactive sessions” (Kumar, 2003), audience response systems (Caldwell,
2007), attention breaks in lectures (Johnstone & Percival, 1976) to using clothespins for audience
paced feedback (described in Poulis, Massen, Robens & Gilbert, 1998). By clicking an electrically
wired clothespin, the audience could give reactions to the lecturer. Students could indicate whether
the content is understood, give answers to multiple choice questions, or indicate whether they are
ready to continue. All these measures are designed to increase interaction and engagement with the
audience during a lecture.
Further, we have seen that contact with fellow students, and especially contact with a lecturer
promotes positive study behaviour, and thus increases the chances of students successfully finishing
a course (Torenbeek, 2011). Consequently, measures that promote contact between students and
lecturer, even in very large lecture halls, would increase the efficiency of courses.
Via an earlier study (Winnips & Thlaoele, 2010) we found several didactic models to promote
interaction in large courses:
Concept tests: as proposed by Mazur (2009), concept tests are challenging multiple choice
questions that are presented after 15 to 20 minutes of lecturing. After answering the questions,
students discuss with their neighbour. At the end of the lecture, students are asked to give their
answers again, after which the “correct” answer is given and discussed.
Group answers: a variation on the “Mazur” model is proposed by Thloaele (2011). Groups of
students are asked to discuss their answers to questions first, before answering and getting
feedback on the answer.
878

Koos Winnips et al.
Formative testing: reactions via SMS can be used to carry out testing during lectures for large
groups. For example, Hancock (2010) used personal response systems to write a summative test.
Interactive tutorials: answering questions via SMS or audience response systems can be used in
interactive tutorials (D’Inverno et. al., 2003): a small problem was explained at the start of the
tutorial, after which groups of students would walk through that, answering questions at
intermediate stages. Based on the answers of the questions the contents of the tutorials would be
changed. In this way the answers to questions determine which path through the materials is
followed, and thus which contents within lectures are covered.
Entry testing: getting to know the entry knowledge of students helps better suit course contents to
the level of the students. For students, entry testing can make it clear if they have a gap in their
entry knowledge.
Backchannel: Backchannel tools (Atkinson, 2010) can improve engagement by allowing students
to react during a lecture. A backchannel enables students to input their questions and thoughts
online, while a verbal presentation is going on simultaneously. Currently, on many conferences a
twitter backchannel is opened, giving participants the option to react and interact before, during,
and after the conference.
For this study the backchannel model was chosen. This was based on assumptions that better
connections between the lecturer and students improve learning outcomes (Torenbeek, 2011), and
that use of open comments could improve these connections. Sending in open comments, questions
and thoughts can be used to:
Assist higher order thinking (for example, by stimulating discussion during a lecture).
Adapt to entry-knowledge (by adapting the level or pace of the lecture).
Increase student time-on-task (for example, by posting questions and thoughts on a forum after
the lecture to stimulate discussion there). A discussion started during a lecture can thus be
continued online.
This paper describes a study to try this concept, now called the “reactionlecture” in the setting of a
large classroom at the University of Groningen. The course was a second year Bachelor’s Psychology
course “Brain and Behaviour” (722 students). The course consisted of a series of lectures,
accompanied by a book. At the end of the course a written examination determined the course mark.
Lectures were given in a cinema room (as shown in Figure 1), that could host 600 students. In a
second cinema room the remainder of the students (often the students that arrived late) could attend
the lecture live via a video screen. In cooperation with one of the two lecturers the reactionlectures
were set up.
Figure 1: Lecture in cinema in Groningen, students send in their responses to the lecturer via Mobile
Internet
To implement the backchannel a few options were considered: using audience response systems (or
“clickers” as they are sometimes called), SMS or Mobile Internet (via the students smartphones). In
879

Koos Winnips et al.
order to make a choice between these tools, it was necessary to know what students have available.
Earlier studies (Winnips & Thlaoele, 2010, Brakel, 2011) showed that about 50% of students at
University currently have a smartphone available. Further, with Gartner’s hype cycle (Gartner, 2010)
predicting a rise in popularity of low-range to mid-range handsets (i.e. handsets with increasing
opportunities for mobile Internet access) this will certainly be an area of development over the next
five years. Comparing this to handing out 722 clickers (and getting them back…), using what students
already have in their pockets (i.e. their phones) seemed like a good idea.
For this study, a tool called Shakespeak was used (http://www.shakespeak.com/). Students can send
in reactions via SMS, mobile Internet or Twitter. Students send their reactions to a moderator screen
on the lecturers computer. The lecturer decides to show relevant reactions in Powerpoint. In a pilot
study (Winnips & Thlaoele, 2010) we used this tool to regain attention using multiple choice
questions.
Based on an overview of Atkinson (2010) Table 1 gives a number of expected student reactions.
Table 1: Expected backchannel reactions of students and lecturer
Student reaction Example Reaction of lecturer
Minute paper Writing a “minute paper” at the end of a lecture,
summarizing the most important point of the
lecture.
Sharing sources A student pointing out that information can be
found in an earlier course, or from a different
source (television, Youtube)
Commenting Discussion amongst students (agree, disagree,
arguments)
Feedback to the lecturer
on what was learned
Lecturer can check
correctness of source, or
compare viewpoint
Checking arguments
Making linkages Linking concepts to concepts of an earlier course Placing link in context of
current course
Reinforcement/moderation This is interesting! / We’ve covered that before Continuation with the topic,
or moving on to next
Selftesting So, this concept is comparable to …? State correct/incorrect
Asking for clarification Could you give an example of this concept? Give the explanation, or
point out source
Helping each other Student explaining concept to other student None
Opinion/emotion Asking the lecturer for opinion on a topic Provide well grounded
opinion, connect to course
Suggestion to the lecturer Asking to provide the importance of the topic (Is
this covered in the exam?), asking for an
overview, technical problems (lighting in room,
talking to fast, sheets not readable, etc.)
“Community building” Students joking about course content, sharing
personal experiences
Opening up the classroom Enabling ill students to get a touch of the “live”
lecture, asking outside experts to react
content
Address in next lecture,
answer, solve directly
None
Lecturer can initiate
contact
With students giving reactions as given above, the questions we had were:
Would students learn better in a reactionlecture than in a traditional lecture?
What are the opinions of students on giving direct reactions during a lecture?
What forms of providing open comments during lectures can be used?
To answer these questions, a study was set up in the context of the second year Bachelor’s course
Brain & Behaviour (722 students enrolled).
2. Design
The reactionlectures were done in the second, fourth, and sixth lecture of the course, with regular
lectures in between, in order to make a comparison. These lectures were provided by the same
lecturer. Student responses during the reactionlecture were voluntary and anonymous.
The reactionlecture differed from the regular lecture as students could give reactions to the lecturer,
using the Shakespeak software. In the beginning of each lecture, explanation was given about the
880

Koos Winnips et al.
use of the software. Students sent in their reaction via SMS, Mobile Internet or Twitter. Figure 2
shows how reactions can be given via Mobile Internet.
Figure 2: Sending in a reaction to the lecturer via a (mobile) web browser
During the break the lecturer views the reactions on his own computer, and responds after the break.
Student reactions that were not covered during the actual lecture were posted in the forum of the
(Blackboard) learning environment, allowing students and lecturers to reply and discuss.
After the final exam of the course, students were asked to fill out (anonymously) a web questionnaire,
consisting of 14 Likert-scale type of questions and two open ended questions. Main questions for the
questionnaire were:
Are students better motivated in lectures because of being able to respond?
Do students understand course content better by being able to respond?
Do students see the use of posting on questions in the forum of Blackboard?
Do students think they have better contact with their lecturer?
In what ways would students like to respond during lectures?
Technical: did the software work? Was it user-friendly?
3. Results
Student reactions
In total, 38 responses were collected, with reactions varying widely (from sending in the message
“test”, to technical issues (“could you please number your slides?”) to content questions (“ What would
happen in the experiment if subject X with brain damage to the right half would have the paper in his
left hand side?”). The lecturer responded directly to a number of questions in class, the rest of the
questions were posted on, and answered via the forum in Blackboard.
Table 2 gives an overview of the type of reactions given:
The bulk of reactions concerned course content (with 13 out of 38 reactions). These were direct
questions regarding what was presented, sometimes directly referring to a slide of the lecturers’
presentation. The reactions stating an opinion/emotion were mainly responding to this study, with all
four reactions responding positively to the study (“kudos for the technical advances”). Suggestions to
the lectures concerned issues such as numbering slides, or speaking a bit louder. The two reactions
concerning community building were personal questions to the lecturer (“How was your trip to Kuala
Lumpur?”). Technical questions concerned placing content on the electronic learning environment,
course issues (“Will there be guidelines for reading the book?”).
881

Table 2: Overview of student reactions
Koos Winnips et al.
Student reaction Number of
reactions
Note taking 0
Sharing sources 1
Commenting 0
Making links 0
Reinforcement/moderation 0
Selftesting 0
Asking for clarification 13
Helping each other 0
Opinion/emotion 4
Suggestion to the lecturer 3
“Community building” 2
Opening up the classroom 0
Technical 9
Irrelevant 7
Total 38
Reactions that were not answered during the lecture were posted in the discussion forum on
Blackboard. These comprised 21 out of in total 123 discussion items (threads) on Blackboard. Of
these 21 threads, 16 were further responded, either by the lecturer, or by fellow students. Figure 2
shows an example of how questions were forwarded to Blackboard.
Figure 2: A reaction in the lecture, posted on to the discussion forum in Blackboard, with a response
from a peer student, and a final response from the lecturer
Student Evaluations
In total 113 students responded to the questionnaire. Below, their answers are summarized, along the
main evaluation questions.
Are students better motivated in lectures by being able to respond?
47.8 % of the students liked being able to respond via their mobile phone during a lecture.
They did not attend more lectures because of the possibility to react (50.5 % very much disagreed to
the statement to attend more lectures).
Do students understand course content better by being able to respond?
Students did not find they were better able to understand course content (34.8 %) However, they did
find the questions that were sent in and discussed during the lecture useful (47.8 %).
882

Koos Winnips et al.
Do students see the use of posting on questions in the forum of Blackboard?
Students did find it useful (40.2 %) to view back posted question in the Blackboard forum.
Do students think they have better contact with their lecturer?
Students were lightly positive on the statement that via the reactions by mobile phone the lecturer
could better respond to what students’ did or did not understand (29.7 % agreed, 40.7 % neutral).
They did agree (29.7 %) to have a better contact with the lecturer by being able to respond to the
lecturer via their mobile phone.
In what ways would students like to respond during lectures?
Students wanted to send in reactions to the lecturer (76.1%) with very little interest in sending
reactions to peer-students inside or outside the lecture room. The open questions showed that nearly
all reactions would be a question to the lecturer (which was also shown the overview of actually sent
in questions). Some students mentioned giving positive feedback (as a “boost” to the lecturer), or
critiquing the course content. Some students found it distracting to send in questions, and would
prefer to do so in dedicated moments. Others found it more useful to ask the questions “on the spot”
as they could otherwise forget the question.
Technical: did the software work? Was it user-friendly?
40.7 percent of the students has a phone available with the possibility to view web pages.
The cost of reacting (students paid for responding via SMS via their own accounts) varied, with 31.5
percent finding the cost of SMS problematic, 51.4 % of students had no problems with cost (probably
as they have an unlimited SMS bundle).
Other comments mentioned that opening up the room for questions showed the lecturer has an
interest in the students’, which was taken as a positive. A number of students mentioned they had to
get used to the system of asking questions via their mobile phone, and they would probably use it
more as they got more used to it.
Common in the reactions of students was that they would like more fixed blocks to ask and respond to
questions. More questions could be asked for practice with the course content, with direct discussion
of the answers.
Technically, it would be better to always show the number where SMS messages could be sent to
(down below in a fixed place on the sheets), instead of the few slides where it was now shown during
the lecture (at the beginning of the lecture).
4. Conclusions and discussion
Going back to the main questions of this study, this section gives and discusses the conclusions.
Would students learn better in a reactionlecture than in a traditional lecture?
Students did not think that asking questions and reacting, in itself, helped their learning, nor did they
attend more lectures because they were now able to respond. Direct effects on learning (such as
differences in grades) were not found. Indirectly, students did find the questions and discussions
around the questions useful, and they spent time after the lectures discussing the answers via the
Blackboard learning environment of the course.
Even if there would be a direct effect on learning via the reactionlecture, the effect would be small,
given the relatively low number of answers. Possibly, by building in more dedicated moments to give
reactions, reactionlectures can be used more effectively (as discussed in the third study question).
What are the opinions of students on giving direct reactions during a lecture?
883

Koos Winnips et al.
Students liked to respond and answer questions during the lecture, and found the discussed
questions during and after the lecture (via Blackboard) useful. These effects on motivation can be
explained in three ways:
Contact between lecturer and students is improved. Students indicate that the lecturer can better
adapt to what they know. Students did not mention that they got a better view of what is expected of
them. There was no mention that students got better personal contact with the lecturer, or that they
would like this. Except for one student’s question (about the lecturers conference visit) the contact
between lecturer and student was purely focused on course content.
Students indicate they are taken seriously, as the lecturer makes an effort to hear their opinions
and comments. The lecturer indicated that, because of opening up the lecture for reactions,
students were more likely to visit him for face-to-face questions.
Confidence of students can increase. Being provided with direct feedback on their answers can
give students confidence that they will be able to master the course content.
As a side effect, lecturers indicated their own motivation for lecturing increased, as they were now
able to “get something back” from the audience. Uninterrupted lecturing can be unpleasant for a
lecturer: “Do they understand?”, “Was this covered in another course?”, “Is the lecture getting
boring?”. Getting audience reactions thus improves the lecturers’ motivation.
What students did not appreciate was that they themselves had to pay for the SMS messages. When
using mobile Internet to respond, this was not a problem. Most of the lecture rooms at the University
of Groningen have wifi connection.
What forms of providing open comments during lectures can be used?
From students open comments in the questionnaire, a general issue was that students would have
liked more moments for interaction in the lecture. The lectures now had only one fixed block of
reactions (after the coffee break). By specifically building in more moments for interaction, students
indicated they would be less distracted. For future use of the reactionlecture, it would be a good idea
to build in these specific blocks of interaction, so students do not get distracted by sending in their
comments, but use it as a way to help the lecturer adapt to the entry knowledge of the students.
Some students indicated that they wanted to ask a question, but lost the phone number. Showing the
phone number to send questions to, on every slide of the lecturers presentation might have helped to
get more reactions.
It took more time than expected for students to think up and send in comments. The time for reacting
needs to be taken into account. Reacting via SMS messages takes more time then reacting via
Personal Response Systems (sometimes called Clickers). Using the mobile Internet, reacting would
take about the same time as reacting via Clickers. Reactions and discussions take time, which needs
to be planned into the lecture. This stresses the point that the reactions and discussions need to focus
on the main topics of the lecture. In this way, instead of wasting valuable lecture time, the time for
reactions is used to help students focus on what is important, and will help students to process this
information.
During conferences nowadays, a backchannel seems to be a “must-have”. The audience sends in
reactions on presentations via Twitter, resulting in further reactions, lively discussions, and making
contact between the participants (getting “followers”). For this study, this model of interaction could
not be copied to the lecture room, as the audience and setting are different. This group of second
year Bachelor’s students is more reticent in asking questions and making comments. They prefer to
have a structure planned, with dedicated response time. Furthermore, students prefer to have direct
contact with their lecturer based on course content, not with their peers (in or out of the lecture room).
Planning in more moments of interaction can make the reactionlecture more effective, with more
student reactions and dedicated key topics for discussion.
With students using tools such as Twitter and Facebook, and phones with mobile Internet being used
more and more in lecture halls, a more free form of the reactionlecture can work. But for now, it
seems this “Facebook generation” does not behave like the earlier mentioned conference audience.
884

Koos Winnips et al.
Rather, they like to have structured activities around main lecture topics, taking the lecturer as their
source of knowledge.
References
Atkinson, C. (2010) The Backchannel: How Audiences are Using Twitter and Social Media and Changing
Presentations Forever, New Riders Press, Berkeley.
Brakel, R. (2011) Smartphones onder studenten [Students’ use of smartphones], [online],
http://onderwijsingrafieken.blogspot.com/2011/05/onderwijsgrafiek-62-smartphones-onder.html
Caldwell, J. E. (2007) “Clickers in the Large Classroom: Current Research and Best-Practice Tips”, Life sciences
education, Vol 6, No. 1, pp 9-20.
D’Inverno, R., Davis, H. and White, S. (2003) “Using a Personal Response System for Promoting Student
Interaction”, Teaching Mathematics and its applications, Vol. 22, No. 4, pp 163-169.
Hancock, T. M. (2010) “Use of audience response systems for summative assessment in large classes”,
Australasian Journal of Educational Technology, Vol 26, No. 2, pp 226-237.
Johnstone, A. H. and Percival, F. (1976) “Attention Breaks in Lectures”, Education in Chemistry, Vol 13, No. 2, pp
49-50.
Poulis, J., Massen, C., Robens, E. and Gilbert, M. (1998) “Physics lecturing with audience paced feedback”,
American Journal of Physics, Vol 66, No. 5, pp 439-441. Online: http://tinyurl.com/mrmvsb
Kumar, S. (2003) “An Innovative Method to Enhance Interaction during Lecture Sessions”, Advances in
Physiology Education, Vol 27, No. 1, pp 20-25.
Mazur, E. (1997) Peer instruction: a user’s manual, Prentice Hall, Englewood Cliffs NJ.
Macmanaway, L. A. (1970) “Teaching methods in higher education-innovation and research”, Universities
Quarterly, Vol 24, No. 3, pp 321-329.
Tlhoaele, M., Hofman, W.H.A., Naidoo, A., and Winnips, J.C. (2011) “Using Information and Communication
Technology to support interactive engagement in higher education” Article submitted to South African
Journal of Higher Education.
Torenbeek, M. (2011) “Hop, skip and jump? The fit between secondary school and university”, [online], Doctoral
dissertation, University of Groningen, http://irs.ub.rug.nl/ppn/331690829.
Winnips, J.C., and Thlaoele, M. (2010) “Using SMS to increase interaction in large lectures: models and results”,
Paper read at Online Educa Berlin, Berlin, Germany, December.
885

A Holistic Approach to Instructional Design for Blended
Learning Environments
Li Zhong Zhang
Regional School of Business, La Trobe University, Bendigo, Australia
L.Zhang@latrobe.edu.au
Abstract: This paper aims at presenting an integrated framework, comprising three key components, namely,
instructional design, pedagogy and learning technologies, as a holistic approach to designing technology-based
blended learning environments. Each component will be reviewed commencing from the earlier developments
that have served as part of the foundation to the development of new trends and emerging ideas or concepts in
the field. The dynamic inter-relationships among the three components will be discussed. The design implications
and challenges in creating technology-based blended learning environments in support of contemporary
educational and technological trends and developments will be presented.
Keywords: Instructional design, pedagogy, learning technologies, blended learning
1. Introduction
The prevalence of online learning and information access is growing exponentially in various areas of
our society. Increasingly more teachers, educators, and instructors are beginning to see and
experience the importance and potential of using learning technologies in their courses and
programmes. However, they are often being rushed into using various learning technologies without
due considerations on pedagogical and instructional design aspects of instruction and learning
(Convery, 2009).
It is evident to educators alike that in moving from face-to-face setting to technology-mediated
environment would call for the need to change the way we teach and learn, this is more so for a
blended learning environment (Conole and Oliver, 2007).
From a macro perspective, blended learning could entail a hybrid mode of delivery, i.e, face-to-face
and online components. However, from a micro perspective, blended learning could mean
deployment of a mix of different instructional strategies and media within a setting. The discussion in
this paper encompasses both perspectives of blended learning. In addition, the integrated design
framework presented in this paper attempts to introduce another dimension of blended learning by
adopting a mix of learning approaches as the pedagogical bases for design.
2. Instructional design: Perspectives and considerations
Instructional design is increasingly gaining importance and popular among education and training
industry for creating effective instruction and learning in curriculum and training programmes. It is
useful for creating learning structure which keeps learners focused during the course of learning on
the intended learning outcomes, goals, process of learning (via various learning tasks) and the
product of learning (what is expected of the learners from the learning).
Instructional design is a systematic approach for the design, development, implementation and
evaluation of instruction to improve learning and performance in educational settings and workplace.
It is a procedural system for all phases of instructional system design (Dick and Carey, 2004, p.4).
The iterative and self-correction nature of instructional design process emerges as one of its greatest
strength (Reiser and Dempsey, 2005, p.19).
There are various instructional design models and processes cited in the literature (e.g, Dick and
Carey, 2004; Morrison, Ross and Kemp, 2007; Pachler and Daly, 2011). The following section
describes some components of instructional design process in four phases, namely, planning and
analysis, design, develop, delivery and management. For the purpose of this paper, only the first two
phases will be included in the discussion.
2.1 Planning and analysis phase
This phase comprises of analyses of learners, instructors, environment and tasks. It includes
identifying learning or performance needs and problems, learner profile, and learning setting. The
886

Li Zhong Zhang
results from the analyses form the basis for subsequent design phase. A statement on intended
learning outcomes and learning goals along with the description of intended target audience forms the
most significant deliverable at this phase.
2.2 Design phase
This phase provides the bulk of the work in instructional design, comprises of a series of design
decisions on various aspects of content (what to learn) and learning (how to learn). The deliverables
at this phase include statements on learning objectives, content outline, learning strategies, learning
tasks, learning media, assessment methods and criteria, course and programme evaluation methods.
The height of this phase is at designing learning strategies. The designer needs to be familiar with
various learning theoretical frameworks for pedagogical considerations as well as with various
learning technologies available for deploying in the learning setting. The perspectives and
considerations of the two areas will be discussed in more details in the next sections.
3. Pedagogy: Perspectives and considerations
This section of the paper attempts to review and present the various theories of learning in two broad
perspectives, the instructive and constructive paradigms.
3.1 Instructive paradigms
Examples of learning theories associated with these paradigms include behavourism and cognitivism.
Some of the underlying beliefs are that the environment constitutes as an important determinant in
learning, in which knowledge exists outside of learners, and learning occurs when this knowledge is
transmitted or input from the environment to the learners. Another fundamental belief is the focus on
instruction or teaching, in which the learning content and tasks are to be broken into smaller units and
organized by the teacher and then delivered to the learners (Chou, 2008; Pachler and Daly, 2011). A
distinct departure of cognitivism from behavourism lies in the assumption of internal processes within
learner when learning occurs. Therefore, it advocates learning strategies that direct attention for
facilitating information encoding, decoding and retrieving in a learner from short and long term
memory. This approach can be applied in designing technology-based instruction by using cognitive
mapping, associative learning, motivational graphics, animation, sound, and visual presentations to
aid in recall and learning
Instructive learning environments are generally teacher-centered or instructor-led, structured and
prescriptive, teacher is viewed as the content or information provider, learners are passive recipients
of information and content, and content is viewed as the ‘king’, a central part of the learning. Practice
and feedback are deemed as part of learning.
Formal education and training, traditional classroom teaching and majority of online and technologybased
learning, have largely reflected this instructive approach.
3.2 Constructive paradigms
There are many variations of constructivism. Examples of learning theories associated with the
paradigms included in the discussion are constructivism, social constructivism, and situated learning.
Constructive learning environments are characterized as being learner-centered, less prescriptive,
teacher is viewed as the facilitator or guide in the learning, learners are actively involved in the
construction of knowledge, with emphasis on processes of learning such as problem solving,
evaluation of information resources. Learning is deemed as contextual and social. Knowledge is
perceived as being meaningfully constructed and not via transmission or reproduction. A learner is
viewed as an active agent in the learning process. Situated learning stresses that learning is greatly
enhanced when it is situated in meaningful and authentic contexts (Lim, 2008; Kirkwood, 2009).
In this paradigm of learning, learners are responsible for seeking, interpreting and constructing
meanings and understandings based on individual experiences and evolved beliefs which are tied to
their social and cultural contexts. Also authentic group-based learning activities aid in providing
opportunities for more knowledge construction via multiple perspectives and interaction.
887

Li Zhong Zhang
There is a wealth of information in the literature which reports and describes highly viable application
of constructivism in technology-based or online learning (Hung, 2001; Chou, 2008). Zhang (2006)
reports a principled approach to language education by adopting constructivist principles. Oliver &
Herrington (2003) explore ways in which contemporary pedagogical principles rooted in constructivist
principles can guide and the design of technology-mediated learning environments.
It is also evident that new technologies have the potential to provide a constructivist learning
environment for both the instructor and learners if learning strategies are appropriately designed to
facilitate technology-based instruction (Zhang, 2006). From a constructivist perspective, simply
reproducing conventional teaching and learning concepts in a computational environment does not
utilize these new technologies. Technologies should not be used as conveyors and delivers of only
information messages, but should engage both the instructor and learners in meaningful learning,
where they are intentionally and actively processing information while pursuing authentic tasks
together in order to construct personal and socially shared meaning for the phenomena they are
exploring and manipulating.
4. Learner-centered psychological principles
Another dimension of theoretical framework for learning to be noted is that of Learner-centered
Psychological Principles, a framework for new designs in curriculum and instruction as well as in
assessment systems for evaluating accomplishment of learning goals (Jara and Mellar, 2010). There
are fourteen Principles which present a paradigm shift from traditional teacher-centered approach to
one that emphasizes the active and reflective nature of learning and learners.
4.1 Adult learning theory
With an increasing demand for technology-based distance learning or online learning among mature
learners, an understanding of how adults learn can guide in more effective design of online courses
and training programmes offered. Knowles’ theory of andragogy (1999), is a theory developed for
adult learning, Among the six principles proposed, self-directed and learner-relevancy bear direct
implications on course design.
4.2 Learning technologies : Perspectives and considerations
A distinction should be made between instructional strategies and instructional media. While the
former indicates the methods of instruction or learning strategies, the latter constitutes the media
through which instruction for learning is delivered (see examples in Table 1).
Table 1: Examples of instructional strategies and possible corresponding instructional media
Instructional strategies Instructional media
Eg. 1. Drill & practice
2. Case studies
3.Group discussion
1.Computer-assisted instruction
2. Videos, audio
3. Chat, BBS
The use of media for instructional purposes has been predominantly influenced by the development
and advancements of technologies, particularly with the emergence of the Internet / Web technologies
since early 1990s. There has been a tremendous growth in online learning and e-learning industry
with the convergence of information and communication technologies (ICT). The convergent
technologies can serve both as a delivery medium for flexibility in delivering courses online as well as
a powerful medium of instruction, collaboration and communication (Lim, 2008).
With the proliferation of ICT, we see the emergence of more open-ended systems and tools which are
capable of supporting learner-centered, knowledge construction and collaborative learning
environments.
Learning technological tools and systems can be represented under five instructional environments
(Hung, 2001):
Individual instructive tools
Individual constructive tools
888

Informative tools
Social communicative tools
Social constructive tools
Li Zhong Zhang
Table 2 presents an overview of various categories of learning technological tools for instruction and
learning.
Table 2: Overview of learning technological tools (adapted from Hung, 2001)
Learning
technological
tools
Individual
instructive tools
Individual
constructive
tools
Informative
tools
Social
communicative
tools
Social
constructive
tools
Descriptions and examples of existing tools Emerging technologies as integrated
systems
Computer –assisted instruction (CAI) for drilland-practice,
tutorials, simulation software.
Range from presentation software to
interactive multimedia tools such as
Macromedia Director and Authorware,
ToolBook, Macromedia MX series, Office
Suite.
Learning resources in the form of Internet
/Web, search engines, online encyclopedia,
dictionaries, and various information and
knowledge databases such as OPAC.
For group communication such as chat,
video/audio conferencing tools and systems,
groupware such as Exchange, Lotus,
FirstClass.
For group / collaborative work such as BBS,
file and application sharing, electornic
whiteboard.
“Push” technology based on user
profiling and audience targeting
Allow the creation of personalized sites
for constructing personalized knowledge
base and collating resources, allow
sharing capability
“Pull” technology based on user profiling
and subscribed services
Support the creation of surveys, polls,
discussions of both inline and threaded
Allow creation of team sites and shared
workspace for collaboration and online
community building
Tools listed in (Table 2), under the five instructional or learning environments consist of some of the
existing tools and those emerging technological tools. The latter are delivered via an integrated
system such as IBM WebSphere Portal solution and Microsoft SharePoint Portal Technologies and
Products.
However, there are other categories of learning technologies which are not included in the Table.
Among these are :
Learning management systems (LMS) e.g. WebCT, BlackBoard, Docent, Moodle and many
others. These systems are used to administer, deliver and manage courses and track student
learning
Learning content management systems (LCMS) such as Aspen LCMS, Evolution, Techniq,
LogicBuilder. These systems simplify the tasks of creating, managing, and reusing of learning
content. Ideally, a system of this nature works best when it is integrated with the LMS chosen
Learning Object Technology – the underlying fundamental concept of this emerging trend and
development is the creation of content in the form of database-driven objects, which can be
centrally stored, searched, retrieved and re-assembled according to learners’ needs and
requirements. The development is well-suited for supporting learner-centered instruction
(Morrison, Ross and Kemp, 2007).
Rich-media content creation tools such as Cyberlink StreamAuthor 3.0. These tools basically
integrate PowerPoint slides with video and any other documents such as Web pages, Word, PDF
files and deliver the package, instead having to open different ‘windows’ for each of the files.
5. An integrated framework for designing technology-based blended learning
environments
The previous sections of the paper present separate accounts of perspectives and considerations on
instructional design, pedagogy and learning technologies respectively. This part of the paper attempts
to describe the dynamic inter-relationships of the three key components to form an integrated design
framework for creating technology-based blended learning environments (Figure 1).
889

Figure 1: An integrated design framework
Li Zhong Zhang
Instructional
Design
Pedagogy Learning
Technologies
The dynamic inter-relationship among the three components are indicated by the two-way arrows.
5.1 Instructional design and pedagogy
“Instructional design practices have been greatly influenced by a variety of different
theories of learning and instruction……. Behavioral learing theory, cognitive information
processing theory, Gagne’s theory of instruction – have had a major influence on
instructional design. In recent years, situated learning theory and constructivism have
offered a different view of learning and instruction and have influenced the practices of
many of those involved in the design of instruction” (by Reiser and Dempsey, 2005,
p.57).
The emerging trend with the emphasis on learner-centered instruction has impacted on instructional
design to become more focus on learners rather than on systems, on learning rather than on
teaching.
In any technology-based blended learning setting, instructional design process provides the macro
component of the design in
Identifying learning needs and outcomes
Specifying learning goals based on the learning needs
Formulating learning objectives
Providing structure for instruction and learning, which include the information on nature & scope
of domain knowledge, skills & attributes to be covered, course structure, course duration, prerequisites,
learning resources etc.
Assessing the learning
These fundamental elements of instructional design will continue to hold true in the future for
specifying the necessary structure and support for learning in any learning or training settings,
whether it is for technology-based or traditional classroom learning, for teacher-centered instruction or
learner-centered learning, for individual or group learning.
Based on the instructional input and framework gathered during the instructional design process, the
course instructor or designer will decide on an appropriate pedagogy to be deployed in designing
learning strategies, tasks and activities for units of lessons or modules of learning, with due
considerations on various theoretical frameworks on pedagogy as presented in the previous section.
The designer may choose Constructivism as the entire pedagogical framework or deploy a mix
approach of using Constructivism as the predominant pedagogical framework to be complemented
with instructive approach for creating the most appropriate learning environment for that particular
setting. However, the processes for both the instructional design and deployment of the most
appropriate pedagogical framework are not static in that both are iterative in nature. After completing
the cycle of development & implementation, the course designer or the team will be able to improve or
adjust the learning design based on the evaluation data collected. Hence, the inter-relationship
between the two components, instructional design and pedagogy is a complementary and dynamic
890

Li Zhong Zhang
one, in that the former provides the macro aspects while the pedagogical considerations form the
micro aspect of a course design.
5.2 Instructional design and learning technologies
“…. the field of instructional design and technology is hugely affected by the enormous
changes that have been brought about largely through the pervasiveness of networked
computers. ” (by Reiser and Dempsey, 2005, p.279).
As mentioned in the previous section, the emergence of ICT not only changes the delivery and
reception of instruction and information, it also exerts a widespread effect as a medium of instruction
and learning for enhanced interactions and interactivity as well as in creating and sustaining learning
communities.
The development in instructional design will continue to be influenced and impacted by emerging
technologies. An evidence of a technology-induced paradigm shift that is currently underway in the
field of instructional design is the distributed cognition in response to the emerging team-based,
collaborative and distributed applications and systems. Thus, learning spaces are becoming more
complex and dynamic, which are defined by the tools, the interface and interactivity, and sharing
workspaces provided by more sophisticated and intelligent technologies and systems within an
integrated environment (e.g, Table 2, under the column of Emerging technologies). An integrated
environment provides additional benefits to learners and instructors in that they need not have to go
to different systems and tools for the learning tasks. Furthermore, being an integrated system, the
data is centrally stored in the same database for quick search and retrieval as well as for information
sharing.
5.3 Pedagogy and learning technologies
Traditionally emphasis has been placed on technologies without regard to teaching and learning
aspects and therefore technologies often fail to create the learning impact (Oliver & Herrington, 2003;
Zhang, 2006). It is important to avoid this pitfall by ensuring that the adoption of innovative teaching is
educationally sound.
Technologies alone cannot provide solutions to teaching and learning problems and needs. Neither
can technologies themselves transform teaching, learning and assessment. Transformation comes
from re-structuring or re-designing of existing teaching and learning practice with incorporation of
technologies. As the deployment of technologies in teaching and learning goes beyond acquisition
and set-up of technical infrastructure, the integration of technologies should not be technically-driven
(Jones, Ferreday and Hodgson, 2008).
Pedagogical-driven approach begins with considering teaching and learning problems, needs and
requirements. The course designer will first decide on the learning paradigms, instructive or
constructive or a mix of both with one predominant over the other. Learning technologies will be relied
on when there is a need to accomplish the desired learning outcomes. For instance, if the
pedagogical underpinning is for creating a learning environment for supporting social construction of
knowledge, then social communicative and constructive tools (see examples in Table 2) could be
deployed to meet the needs in group communication and collaboration. The designer can then
examine an array of such tools and systems, e.g, groupware, discussion board, chat, or virtual
classroom, to find the best ‘fit’.
Working with technological change, pedagogical-driven approach is a dynamic process in which the
course designer will need from time to time to adopt improved or newly developed functions of
learning technologies to enhance course content presentations and learning tasks. Therefore,
pedagogical enhancement can be achieved by taking advantage of technological change and
integrating this changing dimension into the process of teaching and learning.
6. Design implications and challenges
The design implications & challenges in creating technology-based blended learning environments in
support of contemporary educational and technological trends and developments are to be presented
under the respective sub-headings.
891

6.1 Design implications
Li Zhong Zhang
The dynamic inter-relationships of instructional design, pedagogy and learning technologies will
continue to shape the landscape of one another. Except for the fundamental design elements in
instructional design and pedagogy, learning design will continue to be dynamically impacted by
new educational and technological trends.
At the surface, instructive and constructive paradigms are discordant or seem to be mutually
exclusive. However, learning is as much a social as an individual activity. Hence the two
paradigms of learning can actually complement each other for determining design requirements in
supporting the learning.
The emerging trend of distributed learning environments and that of an integrated blended
technologies, will inevitably lead course designers to rely more on emerging learning technologies
to shift towards a learner-centered environment
Distributed learning and cognition coupled with distributed technological systems are providing
more interaction between human and technologies. Learning communities, made up learners,
instructors, and domain experts, will constantly interact not only with one another via
technologies, but also with technologies in constructing and retrieving both explicit and tacit
knowledge from the knowledge base and database stored. Under these conditions, the traditional
instructional system approach and pedagogical design that are rooted in linear sequencing needs
to give way to one that places a premium on learner control and community building (Kirkwood,
2009).
6.2 Design challenges
A design challenge lies in choosing appropriate learning technologies, i.e., to find the “best fit”, to
match the technological tools and systems with the pedagogical underpinning in a learning
setting. Hence, the course designer should examine closely the pedagogical aspects of learning
before considering technological tools and systems
While most of the learning theories and principles provide solid frameworks for new educational
approaches and paradigms, the actual implementation of the concepts still requires much effort in
translating them into feasible or practical learning strategies
In responding to new educational and technological trends and developments, alternative
teaching and learning methods and strategies are deployed to create more interactive,
constructive, process-oriented and collaborative learning environments. The existing assessment
methods and techniques for learning may not be totally suited for assessing learning occurred
under the said environments. Hence, alternative assessment methods and techniques are to be
developed and validated for use in these kinds of learning environments
7. Conclusion
Good technology-based learning is defined by design and not by technology. The author has
proposed a holistic approach to designing technology-based blended learning environments by
introducing an integrated design framework, comprising three key components, namely, instructional
design, pedagogy and learning technologies. Based on the literature review of various learning
theories as theoretical frameworks for pedagogy, though the contemporary dominant theme
underpinning the approach to technology-based blended learning is constructivist with social contexts,
the authors proposed a blended approach to deployment of instructive and constructive paradigms
as pedagogical bases in designing technology-based blended learning environment. A blended
approach to the pedagogical design would be more effective in addressing different learning
requirements by situating it in appropriate learning contexts based on the intended learning outcomes,
goals and objectives.
References
Chou, C.C. (2008) “Formative Evaluation of Synchronous CMC Systems for a Learner-Centered Online Course”.
Journal of Interactive Learning Research, Vol.16, No. 2, pp.173-192.
Conole, G., and Oliver, M. (2007) Contemporary Perspectives in E-Learning Research. Routledge, New York.
Convery, A. (2009) “The Pedagogy of the Impressed: How Teachers Become Victims of Technological Vision”,
Teachers and Teaching: Theory and Practice, Vol. 15, No. 1, pp.25–41.
Dick, W., Carey, L. and Carey, J.O. (2004) The Systematic Design of Instruction (6th ed.). Addison-Wesley
Educational Publishers, New York.
892

Li Zhong Zhang
Hung, D. (2001) “Theories of Learning and Computer-Mediated Instructional Technologies”. Education Media
International, Vol. 38, No. 4, pp.281-287.
Jara, M., and Mellar, H. (2010) “Quality Enhancement for e-Learning Courses: The Role of Student Feedback.”
Computers & Education, Vol. 54, No.3, pp.709–914.
Jones, C., Ferreday, D. and Hodgson, V. (2008) “Networked Learning: A Relational Approach”, Journal of
Computer Assisted Learning, Vol. 24, No. 2, pp.90–102.
Kirkwood, A. (2009) “E-Learning: You Don’t Always Get What You Hope for”, Technology, Pedagogy and
Education, Vol. 18, No. 2, pp.107–121.
Knowles, M. (1999) The Adult Learner: A Neglected Species (5th Ed). Gulf Publishing, Houston, TX.
Lim, G. (2008) “ICT-Supported Learning Strategies and Learner-Centred Instruction”, CDTL Brief, Vol.12, No.9,
pp.2-5.
Morrison, G., Ross, S., and Kemp, J. (2007) Designing Effective Instruction (6 th ed). Wiley, New York.
Oliver, R., and Herrington, J. (2003) “Exploring Technology-Mediated Learning from a Pedagogical Perspective”,
Interactive Learning Environments, Vol. 11, No. 2, pp.111-126.
Pachler, N. and Daly, C. (2011) Key Issues in E-Learning: Research and Practice. Continuum Publishing,
London.
Reiser, R.A. and Dempsey, J.V. (2005) Trends and Issues in Instructional Design and Technology. Pearson
Education, New Jersey.
Zhang, L.Z. (2006) “Online Language Education: A Principled Approach to Instructional Design”, Journal of
Language Education and Technology, Vol.10, No. 3, pp.33-40.
893

894

PhD
Research
Papers
895

896

Evaluating the Impact of an Arabic Version of an Adaptive
Learning System Based on the Felder-Silverman’s
Learning Style Instrument
Nahla Aljojo 1 , Carl Adams 2 , Huda Saifuddin 3 and Zainab Alsehaimi 4
1
Faculty of Computing and Information Technology, King Abdul Aziz
University, Jeddah, S.A
2
Computing department, Technology College, University of Portsmouth,
Portsmouth, UK
3
Cognitive Psychology department, Arts and Humanities faculty, King Abdul
Aziz University, Jeddah, S.A
4
Public administration department, Economics and administration faculty,
King Abdul Aziz University, Jeddah, S.A
Nahla_Aljojo@yahoo.com
carl.adams@port.ac.uk
hbourhan@gmail.com
Zalsehaimi@msn.com
Abstract: This paper presents an approach to integrate learning styles into adaptive e-learning hypermedia
system and an approach to evaluate the impact of such a learning system. The main objective was to develop an
adaptive e-learning system based on individual student’s learning style, then to try and assess the effectiveness
of the system on the students’ learning. From a technical perspective, the system development involved the
combination of SQL server 2005, SQL database and Active Server Pages were used to implement the system
based on learning styles to present the appropriate subject matter, including the content, Teaching strategies and
Electronic Media. The system was organized into 3 models; domain model, learner model and adaptation model.
The 3 models interact together to perform adaptively. From an experiment design perspective, experiments
involved applying using the system on two cohorts of students and evaluating the impact on learning
achievement. Inferential statistics were applied to make inferences from the sample data to more general
conditions. Descriptive statistics were applied simply to describe what's going on in the sample data. Results
showed that students taught using learning style adaptive system performed significantly better in academic
achievement (p

Nahla Aljojo et al.
Adaptive hypermedia research has received more attention during the last two decades, however it
still is unclear which aspects of learning styles are worth modeling, and what can be done differently
for users with different learning styles. An adaptive e-learning hypermedia is an approach whose
target is to personalize the learning experience for the learner (De Bra et al., 2004; Henze and Nejdl,
2004). A number of adaptive educational systems have been developed based on learning styles as
a source for adaptation, including: AEC-CS (Trantafillou et al., 2002), INSPIRE (Grigoriadou et al.,
2001), iWeaver (Wolf, 2003), MASPLANG (Peña et al, 2002; Peña, 2004), LSAS (Bajraktarevic et al,
2003), EDUCE (Kelly, 2005) and ILASH (Bajraktarevic et al., 2003a). One of the key challenges in
such adaptive learning systems is the development of robust experimental evaluation mechanisms to
assess their impact on student's achievement. For instance, Brown et al. (2009) investigated adaptive
e-learning hypermedia that specially utilizes learning style as their adaptation mechanism; they found
that out of 10 systems, 6 systems did not seem to have published any quantitative evaluations in their
recent research. Typical would be AES-CS (Triantafillou et al., 2003) and INSPIRE (Papanikolaou et
al., 2003) which uses some empirical data in the form of descriptive statistics and but no inferential
statistics testing, and the number of users were relatively small (n = 10 and n = 23, respectively).
A common evaluation approach involves comparing performance on an adaptive learning system with
non-adaptive versions for different cohorts of users. However, there are many challenges with
comparing non-adaptive with adaptive version of learning systems (De Bra, 2000). Any difference
between the groups’ performances might be attributed to users’ features (e.g. initial knowledge, goals
etc) or wider environment.
This paper presents an approach to integrate learning styles into adaptive e-learning hypermedia
system and an approach to evaluate the impact of such a learning system. This paper hopes to make
contribution by presenting a further case study of a dedicated adaptive educational system and
providing guidance and discussion on both development issues and how to evaluate the effectiveness
of an adaptive learning system. Existing adaptive learning systems are predominantly English based.
This paper hopes to make further contribution by bring adaptive learning capability to on-English
speaking communities.
1.2 Felder-Silverman Learning Style Model
In Felder-Silverman learning style model (FSLSM) (Felder and Silverman, 1988), learners are
characterized by values on four dimensions. Table 1: summarizes learning environment preferences
of typical learners from each of the four dimensions of the Felder Silverman model. The FSLSM
provides a good base to develop adaptive learning systems (Aljojo and Adams 2009; Aljojo and
Adams 2010) and was used as the base in the Teacher Assisting and Subject Adaptive Material
(TASAM) system described in this paper.
Table 1: Felder’s learning dimensions
Active Explains to others Tries things out, works within a group, discusses and
Reflective Thinks before doing something, works alone
Sensing Learns from and memorizes facts, solves problems by well-established
methods, patient with details, works slower
Intuitive Discovers possibilities and relationships, innovative, easily grasp new concepts,
abstractions and mathematical formulation, works faster
Visual Learns from pictures, diagrams, flow charts, time lines,
films, multimedia content and demonstrations
Verbal Learns form written and spoken explanations
Sequential Learns and thinks in linear/sequential steps
Global Learns in large leaps, absorbing material almost randomly
The reasons to choose this learning style theory in this research are:
Its Index of Learning Style (ILS) questionnaire (Felder and Soloman, 2003) provides a convenient
and practical approach to establish the dominant learning style of each student.
The results of ILS can be linked easily to adaptive environments (Paredes And Rodriguez,
2002).
It is most appropriate and feasible to be implemented for hypermedia courseware (Carver, et al.,
1999; Kinshuk and Lin, 2003).
898

2. TASAM design and production
2.1 System architecture
Nahla Aljojo et al.
The main characteristic of TASAM is that it can be adapted to the learning style. The system was
organized in the form of three basic components: The domain model, the learner model and the
adaptation model. These three components interacted to adapt different aspects of the instructional
process. Figure 1 illustrates the system architecture.
Learner Model
SQL Database
Adaption Model
Visual basic.net
SQL server 2005
Web Browser
Domain Model
Active Server Pages 3.5
Figure 1: Illustrates the system architecture
When learners enter TASAM for the first time, they signed up to the system by using a registration
form. Once a learner registers, a learner profile will be created to store all his/her information and will
be saved in the database, a unique identification (ID) is generated for the learner. After that he/she
will submit the answered questionnaire to get the results that will show his/her learning style. TASAM
uses an Arabic version of the Felder and Soloman’s (1997) Index of Learning Styles (ILS) to generate
the learning profile which consists of a personal preference for each of the four dimensions of FSLSM
expressed with values between +11 to -11 per dimension (see figures 2&3).
TASAM adapts the content sequencing of the course material to match the learning style profile for
the student. The lesson contents appear in the navigation area as tree-like structure of hyperlinks,
whilst in the content area the learning content is presented by the media matched for the learner
preference. TASAM offered many signs to prevent learner from getting lost. First, the learning tree
shows already visited pages in a different color (purple instead of blue). Secondly, the learner typically
progresses through TASAM in a hierarchical manner. As the learning tree grows, new pages will be
added below the last branch. The new branch expands and the first content page is displayed when
the learner enters a new lesson. Finally, link annotations have been added to learning contents to
show the currently viewed content pages.
Learning tree: The learning content was accessible in a hierarchical, tree-like fashion with the aid of
a collapsible Active Server Pages tree menu .The tree grew with the progression of the learner.
Intra lesson navigation: A small navigation bar offered “previous” and “next” arrows for the content
pages of the current lesson.
Learning content: The central screen area was reserved for the learning content, presented in the
different teaching strategies and electronic Media.
899

Nahla Aljojo et al.
Email: The email icon to send email for any person.
Chatting: The Chat icon for chatting with other people
Print: The print icon for printing the lessons of the course.
Help: for browsing the tutorial of the website.
Figure 2: Screenshot of Index Learning styles questionnaires
Figure 3: Screenshot of the result of learning style
2.1.1 Domain model
We organized each chapter of statistics material by using instructional design theories (Elaboration
theory and Component Display Theory (CDT)). Each chapter is generated for a learning goal and is
organized around specific outcome concept and each outcome concept is associated with specific
learning outcomes as well as with prerequisites and related concepts by using Elaboration Theory
(Reigeluth and Stein, 1983).
900

Nahla Aljojo et al.
On a micro level the learning content was structured according to Merrill’s component display theory
(CDT) (1994). CDT was one of the first instructional design theories that separated content from
instructional strategy and it was therefore an important contribution to the field of educational
technology (Kovalchick & Dawson, 2002; wolf 2003). The theory comprises four primary presentation
forms: rules (general form), instances (concrete examples), practice, and recall. A secondary layer of
components includes prerequisites, objectives, helps, mnemonics and feedback. According to CDT,
instruction is most effective if all primary and secondary components are present in the instructional
materials. In line with CDT, learners should be able to select and jump between components that best
suit their needs and preferences. See table 2.
Table 2: Components of an exemplary TASAM learning sequence
Component TASAM equivalent content
Objective Content page :Objective of each related concept
Example Content page :Example of each related concept
Elaboration Content page: summary of each related concept
Elaboration Content page: outline of each related concept
Practice Content page :Practice or of each related concept
Recall Content page :Test end of each related concept
feedback Correct answers of test
The concept for providing adaptivity is based on representing specific course elements, or topics,
grouped into chapters for a course. The courses chosen to apply the TASAM adaptive system were
short introductory statistic courses aimed at first level undergraduates used across two faculties at the
King Abdul-Aziz University in Saudi Arabia: The ‘Arts and Humanities’ faculty. The Statistics topic was
chosen for several reasons. Firstly, expert-refined and validated learning materials were available,
which were kindly provided by evaluation of teacher. Secondly, it was a relatively straightforward task
to re-design the materials of a Statistics -related topic for a computer-based environment. Thirdly,
Statistics was considered a timely and desirable learning objective for potential participants. Lastly,
Statistics course is an abstract topic, which opened opportunities to develop different representations
for the same concept by employing different electronically media. The statistics TASAM system ran
between 2010 and 2011. Content improvement suggestions and general feedback was collected from
participating tutors and students.
2.1.2 Learner model
A distinct feature of an adaptive e-learning system is the learner model it employs, that is, a
representation of information about an individual learner. Learner modeling and adaptation are
strongly correlated, in the sense that the amount and nature of the information represented in the
learner model depend largely on the kind of adaptation effect that the system has to deliver.
The learner model in TASAM represents the knowledge of the system about the learner. It reflects
several characteristics of the learners and supports the communication between learner and system.
In our approach, the learner model includes general information about the learner, his/her dominant
learning style, username, password, unique ID, age, and e-mail. The learning style state stores values
for objects concepts to match learner's learning style that is, media type. It associates a number of
learner preferences with each object concept of the domain sub-model resources structures.
2.1.3 Adaptation model
The adaptation model in TASAM specified the way in which learning style modify the presentation of
the content. It was implemented as a set of the classical structure: If condition, then action type rules.
These rules form the connection between the domain model and learner model to update the learner
model and provide appropriate learning materials. Following Kinshuk and Lin (2003) moderate and
strong preference were grouped together to enable 16 types of combination of leaning style
dimensions from which representation templates were generated (see table 3). This provided the
basis for enabling learners with different learning styles to view different presentations of the same
educational material ( AlJojo and Adams 2009).
901

Nahla Aljojo et al.
Table 3: 16 types of combination of leaning style dimensions
3. Participants and study design
3.1 Test the TASAM system
combination of leaning style dimensions
active/sensing/visual/sequential
active/sensing/visual/global
active/sensing/verbal/sequential
active/sensing/verbal/global
active/intuitive/visual/sequential
active/intuitive/visual/global
active/intuitive/verbal/sequential
active/intuitive/verbal/global
reflective/sensing/visual/sequential
reflective/sensing/visual/global
reflective/sensing/verbal/sequential
reflective/sensing/verbal/global
reflective/intuitive/visual/sequential
reflective/intuitive/visual/global
reflective/intuitive/verbal/sequential
reflective/intuitive/verbal/global
Testing of the TASAM system took place in the main laboratory of the college of Economics and
Administration in King Abdul-Aziz University, after the mid exams of the first semester (academic year
2010-2011). All the computers used in the experiment were connected to the Internet and participants
accessed the TASAM website through a common web browser. Participants consisted of 53 first
levels of statistics students from Arts and Humanities collage and consisted of two different group
tests. Each of the two groups in the system testing was engaged in using two chapters within the
statistics course see the following table 4:--
Table 4: procedures of test system (TASAM)
Group A
Test
Group B
Test
Stats chapters
covered
1) Measures of
Variability
2) Correlation
Professor teaching
only
Central tendency 28
1) Measures of
Variability
2) Central tendency
TASAM only Professor
teaching and
TASAM
28 28
25 25
Correlation 25
Testing comparisons consisted of the following:-
Compared the result of group (A) using the TASAM system without professor explanation of the
chapters correlation and Measures of Variability with students of group (A) not using the TASAM
system only using the professor explanation of the chapter (Measures of Central tendency).
Compared the exam result of group (A) using the TASAM system without professor explanation of
the chapters correlation and Measures of Variability with students of group (A) using the TASAM
system with professor explanation of the chapters correlation and Measures of Variability).
Compared the exam result of group (A) students not using the TASAM system with professor
explanation of the chapters’ correlation and Measures of Variability with second group (B)
students using the TASAM system without professor explanation of the chapters Measures of
Variability and Measures of Central tendency.
902

Nahla Aljojo et al.
Compared the exam result of students of group(B) using the TASAM system with no professor
explanation of the chapter (Measures of Central tendency and Measures of Variability) with the
exam result of group (B) not using the TASAM system with professor explanation of the chapter
Correlation
In addition feedback was gained from a survey of the students using the TASAM system. The
feedback of student’s Survey overall students seemed to have enjoyed using the TASAM system and
there seemed to have been a positive impact on learning performance (see table 11). The results of
these comparisons and the survey will be discussed in the next section.
4. Results and discussion
4.1 Test system (TASAM)
4.1.1 Result of A comparison of first case of group (A) with second case of group (A)
H0: group (A) using the TASAM system no professor explanation of the chapters correlation and
Measures of Variability (after adaptive) will learn significantly better than students of group (A) not
using the TASAM system only using the professor explanation of the chapter (Measures of Central
tendency) (before adaptive). To determine if the students of first case of group (A) (after adaptive)
significantly better than second case of group (A) (before adaptive), The main results of the one way
repeated measures analysis of variance are presented in Table 6
Table 5 shows the results of the ANOVA for within subject variable. This table can be read much the
same as for one way independent ANOVA. There is a sum of squares for the within subject effects of
system test, which tells us how much of the total variability is explained by experimental effect (i.e
differences in before adaptive and after adaptive. there are an error term, which is the amount of
unexplained variation across the conditions of the repeated measures variable these sums of squares
are converted into mean squares by dividing by the degrees of freedom. The F-ratio is obtained by
dividing the mean squares for experimental effect (12410.012) by error mean squares (31.067). As
with between –group ANOVA, this test statistics represents the ratio systematic variance to
unsystematic variance. The value of the F-ratio (12410.012/31.067 = 399.46) is then compared
against a critical value for 1 and 27 degrees of freedom. The significance of F is 0 which is
significance because it is less than the criterion value of .05 we can, therefore, conclude that there
was significance difference in scores of students before adaptive and after adaptive.
The main values for the scores students of first case of group (A) (after adaptive) and students of
second case of group (A) (before adaptive) and standard deviation are listed in Table 5 and it appears
that the mean scores for after adaptive are much higher than the main scores for before adaptive
(12.46> 11.75). Analysis of student performance indicated first case of group (A) after adaptive
significantly better than students of second case of group (A) before adaptive are also listed in the
Table 6. From the before adaptive and after adaptive scores, it has been found that there was a very
significant difference of scores; P=.045

Table 7: The results of pairwise comparisons
Measure: MEASURE_1
(I) test
1
2
3
(J) test
2
3
1
3
1
2
Nahla Aljojo et al.
Pairwise Comparisons
Mean
Difference
(I-J) Std. Error Sig.
-.714* .340 .045 -1.411 -.017
a
95% Confidence Interval for
Difference
Lower Bound Upper Bound
a
-.500 .670 .462 -1.875 .875
.714* .340 .045 .017 1.411
.214 .496 .669 -.803 1.231
.500 .670 .462 -.875 1.875
-.214 .496 .669 -1.231 .803
Based on estimated marginal means
*. The mean difference is significant at the .05 level.
a. Adjustment for multiple comparisons: Least Significant Difference (equivalent to no
adjustments).
4.1.2 Result of A comparison of first case of group (A) with third case of group (A)
H1: group (A) using the TASAM system with no professor explanation of the chapters correlation and
Measures of Variability) will learn significantly better than group (A) using the TASAM system with
professor explanation of the chapter (correlation and Measures of Variability).
To determine if the students of group (A) using the TASAM system with no professor explanation
significantly better than the students of group (A) using the TASAM system with professor
explanation, The main results of the one way repeated measures analysis of variance are presented
in Table 5.
The main values for the scores students of group (A) using the TASAM system with no professor
explanation and the scores students of group (A) using the TASAM system with professor explanation
and standard deviation are listed in Table 5 and it appears that the main scores for the scores
students of group (A) using the TASAM system with professor no explanation are much higher than
for the scores students of group (A) using the TASAM system with professor explanation (12.63>
12.37). The scores students of group (A) using the TASAM system with no professor explanation and
the scores students of group (A) using the TASAM system with professor explanation, it has been
found that there wasn`t significant difference of scores P=.453 >0.05, see table 7.
4.1.3 Comparing the two different groups
H2: students of group(B) using the TASAM system with no professor explanation of the chapter
(Measures of Central tendency and Measures of Variability) will learn significantly better than group
(A) not using the TASAM system with professor explanation of the chapters Measures of Central
tendency.
An Independent – samples t-test was conducted to evaluate the hypothesis that students of group(B)
using the TASAM system with no professor explanation will learn significantly better than group (A)
not using the TASAM system with professor explanation. The test was significant, t (51) =2.244, p =
.029

Table: 8: Descriptive statistics
Scores
Type
before Adaptive
After Adaptive
Table: 9: Independent samples test
Scores
Equal variances
assumed
Equal variances
not assumed
Levene's Test for
Equality of Variances
F Sig.
4.2 Student’s evaluation questionnaire
Nahla Aljojo et al.
Group Statistics
N Mean Std. Deviation
Std. Error
Mean
28 11.7500 4.22405 .79827
25 13.7600 1.56205 .31241
Independent Samples Test
t df Sig. (2-tailed)
t-test for Equality of Means
Mean
Difference
95% Confidence
Interval of the
Difference
Std. Error
Difference Lower Upper
15.985 .000 -2.244 51 .029 -2.01000 .89562 -3.80804 -.21196
-2.345 34.981 .025 -2.01000 .85723 -3.75029 -.26971
Section 4.2 covers Test-Retest Reliability of student`s evaluation survey, Result of Student`s
evaluation Survey and the final evaluation and assessment of the adaptive learning system by
students.
4.2.1 Test-retest reliability of student`s evaluation survey
In estimating test-retest reliability, the same test is administered to the same or similar sample, on
more than one occasion. Time elapsing between the measurements is critical. Typically the longer the
time-gap is, the lower the correlation. In the study, the time lapse of one month was dictated by the
classroom realities, as described above. Table 10 shows a moderate to strong correlation between
the test and the retest questions of evaluation.
Table: 10 Paired samples correlations
N Correlation Sig.
Pair 1 Q1_before & Q1_after 48 -.185 .209
Pair 2 Q2_before & Q2_after 48 -.021 .888
Pair 3 Q3_before & Q3_after 42 -.013 .934
Pair 4 Q4_before & Q4_after 47 -.189 .204
Pair 5 Q5_before & Q5_after 49 .082 .576
Pair 6 Q6_before & Q6_after 48 .048 .745
Pair 7 Q7_before & Q7_after 47 -.121 .416
Pair 8 Q8_before & Q8_after 48 .011 .942
Pair 9 Q9_before & Q9_after 49 -.010 .948
Pair 10 Q10_before & Q10_after 49 .070 .633
Pair 11 Q11_before & Q11_after 49 -.096 .511
Pair 12 Q12_before & Q12_after 47 -.117 .435
Pair 13 Q13_before & Q13_after 49 -.064 .662
However, the difference between the means of answers fifth, ninth, tenth and thirteen questions were
borderline significant (P = 0.002, P=.000, P=.017, P=.027 respectively ) and the correlation between
the before and after of answers fifth, ninth, tenth and thirteen questions were the second lowest, at 0.
.082,.010, .070 , 064 respectively (Table 11). Yet, the use of such standard statistical tools may be in
905

Nahla Aljojo et al.
fact misleading as a stability predictor of questions evaluation, in such answers for fifth, ninth, tenth
and thirteen questions. Homogeneity or heterogeneity of scores affect score reliability since a small
change in raw scores leads to large changes in rankings and thus low correlation of the evaluation
questions.
Table 11: Paired samples test
Mean
Std.
Deviation
Paired Differences t df
Std.
Error
Mean
95% Confidence
Interval of the
Difference
Sig. (2tailed)
Lower Upper
Pair 1 Q1_before - Q1_after .02083 .56454 .08148 -.14309 .18476 .256 47 .799
Pair 2 Q2_before - Q2_after -
.10417
.59213 .08547 -.27610 .06777 -1.219 47 .229
Pair 3 Q3_before - Q3_after .07143 .60052 .09266 -.11571 .25856 .771 41 .445
Pair 4 Q4_before - Q4_after .14894 .65868 .09608 -.04446 .34233 1.550 46 .128
Pair 5 Q5_before - Q5_after .28571 .61237 .08748 .10982 .46161 3.266 48 .002
Pair 6 Q6_before - Q6_after .10417 .47219 .06815 -.03294 .24128 1.528 47 .133
Pair 7 Q7_before - Q7_after .14894 .65868 .09608 -.04446 .34233 1.550 46 .128
Pair 8 Q8_before - Q8_after .06250 .69669 .10056 -.13980 .26480 .622 47 .537
Pair 9 Q9_before - Q9_after -
.36735
.66752 .09536 -.55908 -.17561 -3.852 48 .000
Pair Q10_before -
.20408 .57661 .08237 .03846 .36970 2.478 48 .017
10
Pair
11
Pair
12
Pair
13
Q10_after
Q11_before -
Q11_after
Q12_before -
Q12_after
Q13_before -
Q13_after
-
.08163
.64021 .09146 -.26552 .10226 -.893 48 .377
.04255 .46426 .06772 -.09376 .17887 .628 46 .533
.18367 .56544 .08078 .02126 .34609 2.274 48 .027
4.2.2 Result of student`s first evaluation questionnaire
Questionnaire of student evaluation was answered by 112 students who learned the material from the
site related to learning styles(www.adaptivelearningstyle.com) table 12 shows that most students
used the learning style belongs to me. In general, table 12 shows most students thought that the
material in this manner is easy and clears (82 percent). Table 12 shows students hope the rest of
professors use a similar method of teaching so they can learn in a way that they prefer. As of table 11
shows students showing information are clear and easy. It’s also easier to teach myself that way. See
table 12 for more details.
Table 12: Questionnaire of evaluation student
Questions Yes NO
1. I learned the material from the site related to learning
styles(www.adaptivelearningstyle.com)
77 23
2. I used the learning style belongs to me 62 30
3. Used my friends learning styles 24 76
4. The material in this manner is easy and clear 82 18
5. I hope the rest of professors use a similar method of teaching so we can learn in a way
that we prefer
73 27
6. Information are clear and easy 90 10
7. It’s easier to teach myself that way 70 29
8. There are a number of points I didn’t understand 57 44
9. I prefer that the professor explain material related to their study 36 64
906

Nahla Aljojo et al.
Questions Yes NO
10. I learned a great deal 79 21
11. I didn’t learn much 36 64
12. I did not understand the contents of the subject related to their study because, it was
difficult
10 90
13. I think it’s a great experience 86 14
4.2.3 Feedback of students about system (TASAM)
Most learners appreciated the integration of the adaptation to learning styles adopted in TASAM and
the support offered by the system. All of them found that the system is user-friendly, The material in
this manner is easy and clear, hope the rest of professors use a similar method of teaching so we can
learn in a way that we prefer, and It’s easier to teach myself that way . High rates were given to the
media format and adaptation techniques implemented in the system. The participant’s opinion to use
the system in the future was very high. The feedback provided valuable positive indications of
participants belonging to different learning style categories towards the system.
5. Conclusion
This paper described the initial development and testing of the first Arabic adaptive learning system
the TASAM (The Teacher Assisting and Subject Adaptive Material). The system dynamically tailors
the learning environment, using Felder-Silverman learning style Theory, to match the individual
learning preferences of individuals.
In this paper , with its emphasis on students of the ‘Arts and Humanities’ faculty at the King Abdul-
Aziz University in Saudi Arabia , the main hypotheses postulated, regarding the main scores
difference, were found to be particularly pertinent and well founded. The findings suggest that
students benefit from the learning materials being adapted to suit their learning preferences. The
results revealed that students have obvious different preferences for lesson presentation type. The
results suggest that the learning outcomes can be improved if designers of hypermedia statistics
course provide a different sequence and presentation of materials to accommodate individual learning
style differences. Hence possibilities for promoting more effective learning are these solid results
indicate that learning styles provide a good basis with which to adapt hypermedia to individual needs.
Hypermedia design features, based on student’s learning styles, such as Adaptive taxonomy:
Learning style ( LS) dimensions and Electronic media (EM) relationships for course material of
statistics and linking mechanisms, have significant bearing for the future development of adaptive
hypermedia systems. Feedback adaptation in the TASAM context has been studied very selectively in
the e-Learning research communities. Therefore, the results of our summarizing analysis of
recommendations are highly speculative and await further validation in extensive experimental
studies. These experimental studies are necessary to discover the positive patterns of relations
between individual LSs and the adaptable feedback parameters increasing the efficiency of
interaction and learning processes. The results of experiments should obtain useful and actionable
knowledge that could be used by an adaptation system of a TASAM.
Findings showed that students taught using the system with adaptation to learning style performed
significantly better in academic achievement than students taught the same material without
adaptation to learning style (p

Nahla Aljojo et al.
Aljojo, N. & Adams, C (2010) Applying learning styles to Arabic speaking groups, International Association of
Technology, Education and Development, Spain, Madrid
Bajraktarevic N, Hall W, Fullick P (2003a) ILASH: Incorporating learning strategies in hypermedia. Paper
presented at the Fourteenth Conference on Hypertext and Hypermedia, August 26-30, Nottingham, UK.
Bajraktarevic, N., W. Hall and Fullick.( 2003) Incorporating learning styles in hypermedia environment: Empirical
evaluation. The Workshop on Adaptive Hypermedia and Adaptive Web-Based Systems Conference 12.
Briggs Myers, I. (1962). Manual: The Myers-Briggs Type Indicator. Consulting Psychologists Press, Palo Alto,
CA.
Brown E, Brailsford T, Fisher T, Moore .A. (2009) Evaluating learning style personalization in adaptive systems:
Quantitative methods and approaches. IEEE Trans. Learn. Technol., 2(1): 10-22.
Carver, C. A., R. A. Howard. and W. D. Lane. (1999) Enhancing Student Learning Through Hypermedia
Courseware and Incorporation of Student Learning Styles. IEEE Transactions on Education. 42(1). pp. 33-
38.
De Bra (2000) Pros and cons of adaptive hypermedia in webbased education. Cyber Psy. Behav., 3(1): 71-77.
Felder, R. M. and B. A. Soloman (2003) “Index of Learning Styles Questionnaire”, [online],
http://www.ncsu.edu/felder-public/ILSdir/ilsweb.html.
Felder, R. M., and Silverman, L. K. (1988) Learning and Teaching Styles in Engineering Education. Engineering
Education, 78 (7), 674–681.
Grigoriadou M, Papanikolaou K, Kornilakis H, Magoulas G (2001) INSPIRE: An intelligent system for
personalized instruction in a remote environment, Proceedings of the Eighth International Conference on
user modelling. Sonthofen, Germany.
Honey, P., and Mumford, A. (1982) The Manual of Learning Styles. Peter Honey, Maidenhead
Kelly, D. 2005. On the dynamic multiple intelligence informed personalization of the learning environment.
University of Dublin, Trinity College, Dublin.
Kinshuk. And T. Lin. (2003) Application of Learning Styles Adaptivity in Mobile Learning Environments. ASEE
Annual Conference and Exposition. Nashville, Tennessee
Kolb, D. A. (1984) Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall,
Englewood Cliffs, New Jersey
Kovalchick, A. and K. Dawson (2002) Component display theory. Retrieved 15 May 2006, from
http://opencontent.org/docs/cdt.pdf.
Manochehr N (2006) The influence of learning styles on learners in elearning environments: An empirical study,
Computers in Higher Education Economics Revew, 18(1): 10-14.
Merrill, M. D. (1994) Instructional design theory. Englewood Cliffs, N.J.: Educational Technology Publications.
Pask, G. (1976b) Styles and Strategies of Learning. British Journal of Educational Psychology, 46, 128-148.
Peña, C.,I. (2004) Intelligent Agents to Improve Adaptivity in a Web-Based Learning Environment. University of
Girona.
Peña,C.I, J.L. Marzo. and J.L. de la Rosa (2002) “Intelligent agents in a teaching and learning environment on
the web”, IEEE International Conference on Advanced Learning Technologies (ICALT 2002). Kazan,
Russia.
Paredes, P. and P. Rodriguez. (2002) Considering Learning Styles in Adaptive Web-based Education.
Proceedings of the 6th World Multi conference on Systemics, Cybernetics and Informatics en Orlando,
Florida, pp. 481-485.
Reigeluth, C. M. and F. S. Stein. (1983) ‘The elaboration theory of instruction’. In: .M. Reigeluth (ed.):
Instructional Design Theories and Models: An Overview of Their Current Status. Lawrence Erlbaum
Associates. Hillsdale, New Jersey.
Trantafillou E, Pomportsis A, Georgiadou E (2002) AES-CS: adaptive educational system based on cognitive
styles. In P. Brusilovsky, N. Henze and E. Millan (eds), proceedings of the workshop on adaptive systems
for web-based education. Held in conjunction with AH 2002, Malaga, Spain.
Wolf C (2003) iWeaver: Towards learning style-based e-learning. In Greening T, Lister R (eds) Conferences in
Research and Practice in Information Technology. Proc. Fifth Australasian Computing Education
Conference (ACE2003), Adelaide, Australia., pp. 273-279.
908

Negotiating Doctoral Practices and Academic Identities
Through the Adoption and Use of Social and Participative
Media
Andy Coverdale
University of Nottingham, UK
ttxac20@nottingham.ac.uk
Abstract: This paper describes current doctoral research into how PhD students are using social and
participative media (web 2.0) in their academic studies. It examines the role these media can play in identityformation
and induction into academic scholarship and professional development. The highly contextualised and
situated approach of this study challenges some of the dominant discourses and idealised concepts within the
educational technology field to address the significant gap between the potential of web 2.0 and the reality of low
rates of adoption and use. The study reconciles social media adoption and use with the self-efficacy and
heterogeneity of doctoral practice. By taking an ecological approach, it recognises that doing a PhD requires the
negotiation of multiple and interrelated academic and occasionally non-academic contexts. Such an approach
legitimises doctoral practice beyond those related purely to thesis-development, and challenges models of
doctoral education defined by a trajectory of increased participation and enculturation within a single, localised
institutional research community. In addition, rather than focusing on one particular tool or platform, the study
adopts a holistic perspective to social media that recognises the multiplicity, interrelatedness and transiency of
web 2.0. The empirical research uses a small sample of social sciences, humanities and interdisciplinary PhD
students as participants. Adopting a qualitative approach and mixed-method design, data were collected through
the observation of online activities across a range of social media, participant-reported accounts, and a series of
in-depth participant interviews. Activity theory is used to support a grounded and recursive approach to analysing
participant-produced digital artefacts, field notes and interview transcripts through open coding and thick
description. From these data, an analytical framework of interrelated object-oriented activity systems was
generated with which to identify and describe shifting patterns in social media practice through key phases in the
participants’ doctoral experiences, and across a range of academic contexts. Emerging findings indicate the role
of social media in contributing to, and revealing, the tensions inherent in negotiating multiple and interrelated
practice contexts through boundary crossing and interdisciplinary activities. The study reveals how participation in
emergent online research networks and communities is enabling new forms of engagement with the research
field, often beyond the immediate scope of thesis-related work, and examines how this contributes to the act of
mapping the research field by providing additional insights into the socio-cultural infrastructure that underpins
academic discourse. It also highlights how the development of doctoral social media practices and identity
agendas are influenced by localised research cultures and often compromised by ambiguous or perceived
audiences.
Keywords: doctoral practices, social media, web 2.0
1. Introduction
This paper describes current doctoral research into how PhD students are using social and
participative media (web 2.0) as part of their academic studies. It examines how these media can
facilitate identity-formation and induction into research scholarship and professional development.
Using an activity theory-based research design, the study takes a qualitative and situated approach to
examine how PhD students adopt and use social media whilst engaging in multiple and interrelated
doctoral practices.
Academic practices are becoming increasingly orchestrated through the web, and social and
participative media (also referred to as web 2.0) are seen as enabling increased opportunities for
developing an online presence, engaging in discussion, and sharing content (Procter et al., 2010).
Social media incorporates a range of web-based tools - such as blogs, wikis and social network and
bookmarking sites – that support communicative, participatory and collaborative practices; core
values that are seen as complementary to a number of social learning models and pedagogies
(Conole, 2010). Social media and related web 2.0 practices have potentially profound implications on
research scholarship, providing new forms of academic discourse, research dissemination, peer
review and collaboration. Yet despite increasing interest and evidence in how these technologies can
be used effectively within an educational context, there is a clear and significant disparity between
their potential and the reality of adoption rates and widespread use (Selwyn, 2010).
Doctoral students may be seen as representing a new generation of researchers inclined to adopt
new technologies that challenge established practices. But currently, the majority do not use social
909

Andy Coverdale
media and related online research environments in their studies, and what use there is tends to be
experimental, localised and inconsistent (British Library and JISC, 2009). PhD students represent an
extremely heterogeneous group; with a wide range of disciplines, ages, experiences and skills and a
diverse set of research foci. This heterogeneity generally precludes generational or cultural
distinctions in how they adopt and use the social web for their academic practice (British Library and
JISC, 2009). Generational factors characterised by ‘digital natives and immigrants’ (Prensky, 2001)
have become increasingly challenged in recent years (for example, Bennett et al., 2008). Rather, key
differences in how researchers carry out and communicate their work within different disciplines or
institutional settings may be more influential on how they adopt – or don’t adopt – social media
(Procter et al., 2010).
Learning institutions inherit the role as custodians of a PhD student’s learning process by formally
inducting and integrating students within a supportive research environment through supervision,
research training, and institutional support services (Chiang, 2003). Feuer et al. (2002; 8) suggest ‘a
[research] culture typically grows naturally within a fairly homogeneous group with shared values,
goals, and customs,’ and Deem and Brehony (2000) argue PhD students are far more likely to be
influenced by academic cultures specific to their discipline rather than any so called ‘research
cultures’ that may be cultivated by themselves or through institutional training programmes. A
community of practice is frequently adopted as a metaphor for students’ socialisation into an
institutional and disciplinary community of scholars (for example, Janson and Howard, 2004). Such
communities provide a social regulation of the learning process. Wenger (1998) describes practice as
a dialectic relationship between the increasing participation in the community and the interpretation of
signs and familiarity with discourse that reify that participation.
However, increasingly holistic perspectives of doctoral education recognise the multiple activities that
PhD students are engaged in. Cumming (2010) describes a doctoral enterprise of mutually inclusive,
interrelated practices describing curricular, pedagogical, research and work-based activities.
Cumming’s model indicates doctoral practices are in a state of flux, embedded within a diverse range
of relationships, networks, resources and artefacts between participants, the academy, and the wider
research community. McAlpine et al. (2009) make the distinction between ‘primary’ and ‘secondary’
research foci in which a range of formal and informal activities are seen as ‘doctoral-specific’ or
‘academic-general’. Recent neo-liberal and knowledge economy agendas have been seen as
instrumental in transforming doctoral education from a scholarship model to a training model, in which
the acquisition of a set of research skills and methods is given precedence over traditional values
defining the PhD as intellectual, theoretical and critical enquiry (Thomson and Walker, 2010).
Scholarly discourse is culturally embedded within traditional forms of research dissemination
dominated by peer-reviewed journal articles, conference papers, posters and presentations (British
Library and JISC, 2009). Their pre-eminence means they remain the core currency with which
academic status is recognised. Yet postgraduate study traditionally combines formally structured and
informal community-based learning (Brooks and Fyffe, 2004). Increased independence and
responsibility of postgraduate study requires students negotiating shifts in academic authority. Chang
et al. (2008) identify the concerns over academic integrity in Web 2.0 activities, highlighting the
tensions that can arise between students’ desire to engage in ‘student-based pedagogies’ and their
dependability on authoritative sources.
In response to these key issues, the following research questions were identified:
How do PhD students use social media in their doctoral studies?
How does the use of social media enable and define academic identities?
2. Methodology
Research Design
Activity theory provides a systematic approach to examining and understanding complex, real-world
learning (Nardi, 1996). It provides a manageable framework with which to organise and examine
comprehensive data sets representing authentic participant experiences by formulating and
describing how activities and their settings evolve over time (Yamagata-Lynch, 2010).
910

Andy Coverdale
The unit of analysis is the activity system. This is defined by its object, which Engeström (1995)
describes as representing a horizon of possible actions. Leont’ev’s activity system (Fig.1) extends
Vygotsky’s notion of tool mediation, to include social and cultural components. Engeström’s (1987)
‘third generation’ of activity theory develops this further by recognising multiple and interrelated
activity systems, in which emergent objects can be partly shared, fragmented and contested.
Figure 1: Based on Engeström (1987)
For Engeström (1983), activity theory does not offer ready-made procedures for research, but rather
provides a conceptual framework for developing an ecological perspective on human activity,
providing the means of studying human actions and interactions with artefacts within a historical,
cultural and environmental context. In contrast to the largely interventionist and collaborative
approaches associated with Engeström’s work, this study adopts a largely descriptive approach,
developing activity systems as a heuristic to facilitate understanding of the activities enacted by a
cohort of participants.
Activity theory provides a procedural and systematic method for revealing and examining interrelated
incentives, disincentives, opportunities and barriers through identifying systemic contradictions and
tensions rooted in the socio-cultural components of specific activity systems (Yamagata-Lynch, 2010).
Contradictions are systemic sources of influence that exist outside of, and across, multiple activities
within the subject’s context. These influence the tensions that subjects encounter while participating in
an activity, stimulating or disrupting their abilities to attain the object.
Using activity theory requires a commitment to understanding things from the learner’s viewpoint.
Activity includes both observable experiences and cognitive actions, and is best explored through a
qualitative research design using mixed methods associated with naturalistic inquiry; observations,
interviews and collecting of artefacts. Nardi (1996) advocates a prolonged engagement with
participants to determine broad longitudinal patterns of activity rather than narrow episodic actions
with which to reveal the individual trajectories. The twelve-month observational period, alongside
historical accounts of participants’ prior experience and intended future use of social media, ensured
a sufficient timeframe to identify key changes within multiple interrelated object-oriented activities.
Six participants were selected from sampling various traditional school-based and departmental
research communities, and doctoral training centres at a number of universities. The following profiles
present a summary of the participants. In accordance with ethical procedures, participant anonymity is
assured by using aliases.
911

Andy Coverdale
A former interdisciplinary postgraduate student, Emily is in her 2nd year at a doctoral training
centre exploring spiritual technology. A self-confessed social media sceptic, she has cautiously
engaged in blogging and using Twitter.
James is a part-time student studying for a Humanities PhD. He set up several blogs and Twitter
to initially connect with specialist online communities for his research into collecting cultures, but
has since used these to establish connections with other researchers in his field.
Hannah is a 3rd year PhD English student and an early adopter of social networking sites. More
recently, she has become a prolific user of Twitter and she set up a PhD blog to develop a
professional online profile. Hannah is also engaged in social media activities with a number of
activist and special interest student groups.
Chris is a 2nd year PhD at an interdisciplinary doctoral training centre examining approaches to
using technology as advocacy for people with learning disabilities. With an arts and teaching
background, he has retained a web-based network of previous work contexts.
Rebecca is a 2nd year doctoral training centre student with a design and entrepreneurial
background. She has used a range of social media for first time to establish connections between
academic and professional networks. She was also instrumental in establishing the Google group
and external facing group blog for the doctoral training centre.
Paula, a 4th year humanities student at a Russell Group university, set up a Facebook group that
was to become instrumental in creating an international network of doctoral students in her field.
She has also used social media to engage key communities as part of her fieldwork, and to
support a number of interdisciplinary student initiatives and an external internship project.
Data Collection and Analysis
The study adopted a holistic perspective of social media activity based on a number of key
assumptions:
Academics who engage in using social media often use multiple tools and platforms
Underlying technologies (for example, hypertext and RSS) and related processes (such as
aggregation, subscription and tagging) provide persistent forms of interconnectivity between
different social media
The adoption and use of social media is transient in nature, subject to changes in technological,
social and cultural trends
Membership of academic communities and networks formed through, or supported by, social
media tend to be interrelated and overlapping in nature
Participants’ social media activities consisted primarily of using personal and group blogs, social
networking sites, microblogging and content sharing sites. Specific social media were selected as
sites for observation in agreement with each participant in accordance with ethical procedures.
Observable data were defined as that resulting in new or modified digital artefacts, such as a blog
post or a tweet, and semi-permanent features such as profiles. Participants also submitted links to
any additional activities on sites not observed by the researcher (e.g. commenting on other blogs).
Field notes were taken to examine contextual information regarding individual, community, and
institutional factors, and participants’ relational values within their various online communities and
networks (such as those identifying location, academic disciplines and hierarchies). Three individual
90-minute interviews were conducted with each participant at roughly four-month intervals, using a
semi-structured format of open-ended questions. All interviews were recorded and fully transcribed.
Multiple object-oriented activity systems were drafted and refined throughout the study in an iterative
process between key stages of observation and participant interviews. All digital artefacts were
logged and codified, and key exemplars were selected as sources of reference and prompts during
interviews. Rather than using activity systems to pre-define the coding process, a constant
comparative method (Glaser and Strauss, 1967) of open coding was adopted to provide an inductive
and systematic process of examining the data. At each key stage, initial codes were formed, refined
and merged until a point of saturation was reached, after which a process of axial coding was used to
develop the larger categories or themes on which the activity systems were based. Thick description
from field notes and interview data helped develop participant narratives from which patterns of social
media adoption and use could be determined.
912

3. Findings
Andy Coverdale
With the final set of participant interviews forthcoming, data analysis is ongoing at the time of writing.
The following findings are therefore to be considered as emergent and not fully formed. Key
discussions around emerging themes and implications for the remainder of the study and are
presented in the next section.
Localised and Distributed Research Cultures
Academic cultures, evident in participants’ pre-doctoral and current institutional environments, have
shown to have significant influence on the role of social media in facilitating established and emergent
peer networks. Either through choice or through programme requirements, participants have also
engaged in additional and varying practice contexts external to thesis-specific commitments, including
internships and work placements, and participation in student initiatives and special-interest groups.
This has required them to negotiate a wide range of practice contexts that have included
ethnographic, entrepreneurial and activist agendas.
Paula’s direct transition from Masters to PhD within the same university department ensured
continuity with an established peer group, replicated in her Facebook group, which channelled the
occasional, inconsistent and fragmented academic-related discussion within the mainstream chat into
a bounded, recognisable and purposeful community. For Paula, subsequent network development
through this and other social media (especially Twitter) corresponded with her increasing participation
in the conferencing circuit, as the group quickly extended beyond the geographical limits of the
university campus to attract considerable numbers of postgraduate students worldwide.
In comparison, Hannah’s doctoral experience was far less smooth. Obliged to follow her supervisor to
a different university, to be then transferred across several departments, she was denied the
opportunity to develop a sustainable relationship with a supportive research community. Crucially, this
reinforced her reliance on previous peer groups from her former university, particularly strengthening
ties through her established blogging community. Similarly Chris, with strong reservations about the
‘business-orientated’ focus of his doctoral training centre, actively sought to participate in alternative
teaching and learning spaces aligned with fees and cuts protests, establishing links through with an
emerging and interdisciplinary network of radical academics. He saw the explicit connections revealed
through his blogging and social networking activities as instrumental in establishing a left-field profile
and dissenting voice.
This contrasts with Rebecca’s close affinity with the research culture and agenda of her doctoral
training centre, embracing its entrepreneurial spirit and the opportunities for establishing key
connections with external design-based companies early in her PhD. She was active in promoting
social media initiatives within the centre, participating regularly in the student Google group and
helping establish an externally facing group blog. James’s attempts at establishing a sustainable
online platform with his departmental colleagues proved less fruitful. Returning to academia after a
long break, his part-time mode and requirement to travel long-distance to his university left him
isolated from his departmental research community. Increasingly drawn to seeking other doctoral
students online, James found emergent academic communities oriented around Twitter hashtags and
the Tumblr blogging network.
Research Scope, Foci and Peripherality
Participants have used various indicators with which to understand the scope and nature of their
doctoral enquiry. Literature reviews and the conference circuit have constituted primary contexts for
participants to map their research field and initiate a process of locating themselves and their
research. Understanding how the scope of doctoral enquiry may be represented and conceptualised
through social media practice requires an approach which recognises it is implicit not only in the
content of participants’ digital artefacts, but also in the content they curate and share (links, retweets
etc.), and in the identifiable communities and networks they participate.
Delimitations regarding scope and peripherality are traditionally defined by disciplinary boundaries,
and to an extent all participants described themselves as interdisciplinary. In her initial experiences of
conference networking, Emily found it difficult locating people with similar interdisciplinary ‘footprints.’
913

Andy Coverdale
Whilst new initiatives in her doctoral training present a new vision of doctoral education, they do not
necessarily support the academic infrastructure appropriate for effective dissemination and
networking that traditional disciplines have had years to evolve. Taking a more optimistic view,
Rebecca values the shared attitudes of an emerging culture of interdisciplinarity, and how they have
initiated new ways of working and communicating, recognising the potential contribution of social
media. Chris believes the fluid, multicontextual nature of social media is ideally suited to the
challenges of his interdisciplinary practice, enabling individual practitioners like him to both coalesce
around and transcend shifting communities of practice and social aggregates.
In contextualising the research scope within the trajectory of the doctoral programme, the inclination
to continue exploring the research field is countered by the necessity to ‘focus in,’ as research
questions and methodologies become formalised. Participants consistently expressed concerns about
sharing work directly linked to thesis development online, for fear of disclosure or revealing academic
naivety. Consequently, Hannah has been more inclined to blog about peripheral topics in her
research field, or examine wider socio-cultural and political issues that underpin her core topic.
Relating wider or peripheral contexts with thesis-focussed work has not only enable her to engage a
wider academic audience, but has also contributed to the later stages of her PhD, as it has become
necessary to evaluate research findings, and formalise discussions and conclusions.
Identity Management and Boundary Negotiation
The data indicate commonalities in the memberships of academic networks across different social
media. Some participants have attempted to make clear distinctions between online spaces for
recreational and social networking and those for their studies, and most have developed a
‘centralised’ site (usually a blog) with which to link distributed media. Hannah specifically set up her
PhD blog at the start of her third year to develop a more coherent and ‘professional’ online research
persona.
With multiple blogs, Chris saw his Twitter account as the ‘heartbeat’ of his social media practice: ‘the
place where most of my online activities collide.’ For several of the participants, this aspect of Twitter
networks has been a particular concern, as followers and followees represent potentially conflicting
and problematic social aggregates, and content related and conversational factors – such as
swearing, use of slang and academic lingo, radical views, and trivialised content – are seen as having
the potential to ‘interfere’ or conflict with the sensibilities of overlapping audiences. Paula has been
careful not to be ‘over-opinionated’ in her tweets, generally adopting a neutral persona, whilst Chris
has to modify some of the more outspoken nature of his tweeting when colleagues from his doctoral
training centre began following him.
Despite the rhetoric around the ‘participatory’ web, the generally low levels of interaction in this study
have indicated that non-interactive aspects of social media play a crucial role. Social media, and how
they are appropriated, have provided wide-ranging levels and orientations of interaction, discussion
and feedback. Participants’ audience perceptions have been integral to these activities, and have
been based as much on an imagined social context as on an informed or experiential understanding.
Participants tend to transfer (often perceived) audiences from one social media to another. Most for
example, assumed that people from their identifiable networks such as Twitter make up most of their
blogging audience. Similar assumptions also relied on corresponding relations in the physical world.
The participants expressed real concerns over the ambiguity of social media audiences, particularly
around blogging. By choosing to use social media, they have demonstrated a commitment to engage
in communication and dissemination processes that are more public, distributed, and potentially
uncontrollable. Their needs to identify and engage with real or imagined audiences is seen as
instrumental in the decisions they made about the content, style and tone of their social media
activities and output. When they blog, tweet and create other digital artefacts across interrelated
platforms and audiences, practice and identity agendas are further compromised whenever those
audiences are unknown or ambiguous.
Mapping the Research Field
As discussed previously, the act of mapping the research field, and ‘locating’ or ‘positioning’ oneself
within the research field is implicit in doctoral practice, particularly in the processes of reviewing
914

Andy Coverdale
literature and conferencing. Engagement in social media encourages and supports an approach to
academic enquiry that links the discourse, arguments and perspectives with the contextual and social
dynamics of the research field. Web 2.0 environments have the potential to provide access to the
personal traits, professional circumstances and social relations that underpin academic discourse,
and in doing so, make the nuances, cliques and hiearchies of faculty and the wider academic field
more explicit, potentially reinforcing or challenging assumptions based on offline and formal academic
discourse.
Rebecca suspects that some of the insights she has gained from following several academics through
social media give her the ‘edge’ over other doctoral students who may not be as well connected. As
she has become increasingly participative and engaged herself, social media have also revealed her
own emerging positioning and allegiances in the discursive environment. Chris suggests this more
informed perspective has helped him ‘signpost’ key arguments and their proponents. Exploiting this
‘insider knowledge’ is dependent on the nature of visibility and interaction fellow academics choose to
engage in. Hannah greatly values academics who embrace the informality of social media, presenting
a richer, more authentic voice by disclosing activities, interests and beliefs on matters external to, or
on the periphery of, their core research interests. Such representational approaches have influenced
Hannah’s own social media practices and the ways in which she presents herself online.
Observational data from this study suggests that whilst relations within local research communities
are heavily influenced by departmental roles, non-localised academic networks formed largely
through the web are strongly stratified along hierarchical reputations (i.e. individuals indicating a
tendency to communicate with academics of equal or equivalent academic status). Participants have
expressed conscious decisions to engage in networking strategies that may lead to opportunities to
‘be on the radar’ of senior, respected and influential academics in their field, with the view that they
may possibly engage in direct discussion.
4. Discussion
Social media are enabling access to emerging online research networks and communities within,
across and outside of departmental and institutional boundaries, yet collocated research
environments have shown to be highly influential in determining how and why PhD students begin or
continue to engage in academic peer networks. The boundary crossing and interdisciplinary activities
demonstrated by the participants of this study would seem to support the need to adopt holistic
perspectives of doctoral education.
All the participants have broadly followed a trajectory that has seen them increasingly engaged in
activities online through multiple and distributed social media and typically overlapping online
communities and networks, requiring them to manage their own status reputation and identity within
and across different socio-technical and academic contexts. The identity management and boundary
negotiation inherent in their social media practices are manifest in decisions related not only to their
digital outputs, but also in their profiling strategies, the links they create across different platforms, and
the memberships of their online communities and networks.
Becher and Trowler (2001) use the metaphor of academic tribes and territories to explore the
relationship between the normative mode of disciplinary and professional contexts, and the
operational mode of academic participation and social interaction. It is into this landscape that
doctoral students must orient themselves, and seek to establish their own scholarly identities in the
contested academic territory (Thomson and Kamler, 2010). In contrast to the primarily territorial
possessions defined by their institutional departments and training centres, participants have
demonstrated tribal tendencies which operate in a state of flux through convergent and divergent
patterns of mutuality and fragmentation inherent in academic migration, interdisciplinarity and multiple
memberships. The tension between the potential for curiosity and expansive learning, and the
legitimating forces of enculturation is manifest in how they appropriate research foci and frameworks,
epistemologies, tools, methods and norms of enquiry. The participants’ use of social media has been
shown to be instrumental in making these ongoing decisions explicit to a distributed research
community, and in doing so, is inherent in the processes of socialisation and identity production of
their doctoral practices.
Even within the small number of participants, levels of critical and reflective thinking about how they
used social media varied considerably. Dissemination of the activity systems analysis has facilitated
915

Andy Coverdale
the participant interviews, and further dissemination will play an increasing role in the remaining
participant interviews. enabling a negotiated understanding of their use of web 2.0. Outside the
limitations of the researcher-participant relationship described here, opportunities for a shared, critical
and reflective dialogue should be seen as important for the development of effective social media
practices.
References
Becher, T. and Trowler, P. R. (2001) Academic tribes and territories (2nd Ed.) Buckingham: Open University
Press.
Bennett, S., Maton, K. and Kervin, L. (2008) “The ‘digital natives’ debate: A critical review of the evidence”, British
Journal of Educational Technology, Vol. 39, No. 5, pp. 775-786.
British Library and JISC (2009) “Researchers of tomorrow: A three year study tracking the research behaviour of
'Generation Y' doctoral students. Interim Research Report 1: Summary Report”. London: Education for
Change.
Brooks, C. and Fyffe, J. (2004) “Are we comfortable yet? Developing a community of practice with PhD students
at the University of Melbourne”, Ascilite Conference, Perth.
Chang, R., Kennedy, G. and Petrovic, T. (2008) “Web 2.0 and user-created content: Students negotiating shifts in
academic authority”, Ascilite Conference, Melbourne.
Chiang, K.-H. (2003) “Learning experiences of doctoral students in UK universities”, International Journal of
Sociology and Social Policy, Vol. 23, No. 1-2, pp. 4-32.
Conole, G. (2010) “Facilitating new forms of discourse for learning and teaching: harnessing the power of web
2.0 practices”, Open Learning, Vol. 25, No. 2, pp. 141-151.
Cumming, J. (2010) “Doctoral enterprise: a holistic conception of evolving practices and arrangements”, Studies
in Higher Education, Vol. 35, No. 1, pp. 25-39.
Deem, R. and Brehony, K. J. (2000) “Doctoral students’ access to research cultures – are some more unequal
than others?”, Studies in Higher Education, Vol. 25, No. 2, pp. 149-165.
Engeström, Y. (1987) Learning by expanding: An activity-theoretical approach to developmental research.
Helsinki: Orienta-Konsultit.
Engeström, Y. (1993) Developmental studies of work as a testbench of activity theory. In S. Chaiklin & J. Lave
(Eds.), Understanding practice: Perspectives on activity and context. Cambridge: Cambridge University
Press, pp. 64-103.
Engeström, R. (1995) “Voice as communicative action”, Mind, Culture, and Activity, No. 2, pp. 192-214.
Feuer, M. J., Towne, L. and Shavelson, R. J. (2002) “Scientific culture and educational research”, Educational
Researcher, Vol. 31, No. 8, pp. 4-14.
Janson, A. and Howard, L. (2004) “The odyssey of PhD students becoming a community of practice”, Business
Communication Quarterly, Vol. 67, No. 2, pp. 168-181.
McAlpine, L., Jazvac-Martek, M. and Hopwood, N. (2009) “Doctoral student experience: Activities and difficulties
influencing identity development”, International Journal for Researcher Development, Vol. 1, No. 1.
Nardi, B. A. (1996) Activity theory and human-computer interaction. In B. A. Nardi (Ed.), Context and
consciousness: activity theory and human-computer interaction, Cambridge, MA: MIT Press, pp. 69-103.
Prenksy, M. (2001) “Digital natives, digital immigrants”, On the Horizon, Vol. 9, No. 5, pp. 1-6.
Procter, R., Williams, R. and Stuart, J. (2010) “If you build it, will they come? How researchers perceive and use
web 2.0”, London: Research Information Network.
Selwyn, N. (2010) “The educational significance of social media – a critical perspective”, Ed-Media conference
2010, Toronto.
Strauss, A. and Corbin, J. (1998) Basics of qualitative research: Techniques and procedures for developing
grounded theory (2nd Ed.). Thousand Oaks, CA: Sage Publications.
Thomson, P. and Walker, M. (2010) Doctoral education in context: The changing nature of the doctorate and
doctoral students. In P. Thomson & M. Walker, The Routledge Doctoral Student’s Companion. London:
Routledge. 9-26.
Wenger, E. (1998) Communities of Practice: Learning, meaning and identity, Cambridge: Cambridge University
Press.
Yamagata-Lynch, L. C. (2010) Activity systems analysis methods: Understanding complex learning
environments. New York: Springer.
916

Enabling Disruptive Technologies for Higher Education
Learning and Teaching
Michael Flavin
The Open University, Milton Keynes, UK
The IFS School of Finance, London, UK
m.flavin@open.ac.uk
mflavin@ifslearning.ac.uk
Abstract: Higher Education Institutions (H.E.I.s) have invested significantly in technologies for learning and
teaching, with Virtual Learning Environments (V.L.E.s) being more or less universal (Britain and Liber 2004).
However, technologies provided by H.E.I.s have not been universally successful in terms of adoption and usage.
Meanwhile, both students and lecturers use a range of technologies not controlled by H.E.I.s to enhance their
learning and teaching on one hand, their social lives on the other, and to blur the boundaries between the two.
There is therefore a need to understand how non-institutional technologies influence learning and teaching, and
to understand how they can be incorporated into institutional contexts. In order to address this issue, this
research investigates how H.E.I.s can engage constructively with “disruptive technologies” (Christensen 1997).
Disruptive technologies in the context of this research are technologies that are not designed explicitly for
learning and teaching, but which transpire to have learning and teaching potential. The research uses Activity
Theory and Expansive Learning (Engestrom 1987, 2001) and the Community of Practice theory (Lave and
Wenger 1991, Wenger 1998) as the primary lenses through which to analyse the impact of disruptive
technologies. The research uses questionnaires and interviews in its pilot study phase to identify the technologies
people use, the purposes for which they use them, and the extent to which uses of technology may be regarded
as disruptive. The research is also interested in how disruptive technologies impact on activity systems
(Engestrom 1987, 2001), and interested in how disruptive technologies impact on online identity formation. The
findings suggest that a more bottom-up and less top-down approach to the implementation of technologies to
support learning and teaching in H.E.I.s is more likely to lead to the enhanced adoption of technologies.
Moreover, the findings suggest that users create their own meanings for technologies. In addition, the findings
suggest that learners’ uses of technologies blur the lines between work, study and recreation, which carries
implications for where learning takes place, the means by which it is delivered, and the power relations that
operate within a learning and teaching situation.
Keywords: disruptive technology; expansive learning; community of practice; eLearning
1. Introduction
H.E.I.s in the UK have invested significantly in technologies for learning and teaching, with Virtual
Learning Environments (V.L.E.s) being more or less universal (Britain and Liber 2004). However,
technologies provided by H.E.I.s have not been universally successful in terms of adoption and usage
(Selwyn 2007; Conole et al. 2008; Blin and Munro 2008). Meanwhile, both students and lecturers use
a range of technologies not owned or controlled by H.E.I.s to enhance their learning and teaching,
and their social lives, and to blur the boundaries between the two (Conole et al. 2008).
A number of researchers anticipated that the use of technology in learning and teaching “would
transform and disrupt teaching practices in Higher Education” (Blin and Munro 2008, p.475; Sharples
2003). However, this has tended not to happen. There is therefore a need to understand how
technologies used outside formal educational contexts shape learning, and how they can be
effectively incorporated into formal structures for supporting learning and teaching. In order to address
this issue, this paper investigates how H.E.I.s in the UK can engage constructively with “disruptive
technologies” (Christensen 1997).
2. Disruptive technology
Disruptive technologies are technologies that disrupt established practices. Conversely, “sustaining
technologies” are technologies that enhance performance along pre-established lines, as Christensen
(1997) outlines:
What all sustaining technologies have in common is that they improve the performance
of established products… Disruptive technologies bring to market a very different value
proposition than had been available previously… Products based on disruptive
technologies are typically cheaper, simpler, smaller, and, frequently, more convenient to
use. (Christensen 1997, p. xv)
917

Michael Flavin
In a subsequent work, Christensen and Raynor (2003) change the term “disruptive technology” to
“disruptive innovation”, arguing that the disruption is not an intrinsic feature of the technology, but is
determined by practices within the organisation confronted by the innovation. Disruption, therefore,
emerges from usage. This paper, however, uses the term “disruptive technology” for clarity’s sake, as
the research is fundamentally interested in technologies for learning and teaching.
Christensen (2006) argues that it is not the case that innovators have mastery over a technology,
whereas established providers don’t. Instead, the established provider finds that the new technology
does not fit within its business model. Hence, it is undesirable, not unattainable. To apply this principle
to technology-enhanced learning in H.E., H.E.I.s have no technical issues with setting up social
networking technologies. Existing, institutional V.L.E.s have chat facilities. However, students are
electing to have their discussions elsewhere. It may be that students have already established an
online presence and identity via existing social networking technologies and therefore feel no need to
have a further online presence in an institutional V.L.E.. It may also be the case that institutional
V.L.E.s are perceived as less attractive and less interesting environments than the more popular
social networking technologies. In addition, students may prefer to hold discussions in environments
where the institution has no control, and no means of monitoring usage. Moreover, students may be
opting to blur the boundaries between a range of social, work and educational activities; this was one
of the conclusions drawn by Conole et al. (2008), in their survey of 427 students.
In Disrupting Class (2011), Christensen et al. take Disruptive Innovation theory and apply it to the
school system in the U.S., arguing that schools’ implementation of technology-enhanced learning has
followed the Sustaining Innovation path, instead of prompting a fundamental rethink of learning and
teaching: “Schools have crammed the computers into the existing teaching and classroom models.
Teachers have implemented computers in the most commonsense way – to sustain their existing
practices and pedagogies rather than to displace them” (p. 84). In this sense, Christensen’s argument
repeats the research findings of Blin and Munro (2008) in their study of technology-enhanced learning
in higher education. Where Blin and Munro conclude, “although use of the V.L.E. is widespread
within the university, little disruption of teaching practices… has occurred” (2008, p. 488), Christensen
argues, “traditional instructional practices have changed little despite the introduction of computer and
other modern technologies” (2011, p. 83).
Christensen concludes by arguing that disruption works not by confronting established practice, but
by doing something new: “A major lesson from our studies of innovation is that disruptive innovation
does not take root through a direct attack on the existing system. Instead, it must go around and
underneath the system” (p. 243). Hence, applying technology-enhanced learning within established
pedagogic models is a mistake, because the technology gets contorted to suit the existing pedagogy,
and thus only a small portion of the learning and teaching potential of the technology is realised.
3. The Community of Practice
The term Community of Practice was coined originally by Lave and Wenger (1991), who observed a
range of different learning communities, including Yucatan midwives, Liberian tailors, and Alcoholics
Anonymous in the United States. Lave and Wenger found that learning had a similar structure in each
case. Individual learners start on the periphery of communities and travel towards the centre. Good
progress relies on a supportive structure within the community.
Subsequently, Wenger (1998) argues that learning is about the formation of an identity rather than the
acquisition of a product, and focuses on learning as social participation. Wenger also argues that
learning is an unavoidable aspect of existence, is ongoing, and “may use teaching as one of its many
structuring resources” (p. 267). Recognising that learning is not contained in a classroom, we begin to
recognise how technology can facilitate effective learning, irrespective of the learner’s location (a
point also made by Koszalka and Ntloedibe-Kuswani [2010]) .
Wenger further argues that learning is controlled by the learning community, not by the external
drivers that prompted the formation of the learning community and adds, “Learning cannot be
designed. Ultimately, it belongs to the realm of experience and practice” (1998, p. 225). He later
argues “Learning is a matter of imagination” (p. 227) and: “In a world that is not predictable,
improvisation and innovation are more than desirable, they are essential” (p. 233). Therefore, learning
for Wenger is a creative process, not classroom bound, and an inevitable, ongoing fact of human
experience.
918

Michael Flavin
The Community of Practice theory is relevant to this research because the research is interested in
how the disruptive use of technologies blurs boundaries between working and recreational identities.
Moreover, in arguing that learning is controlled by the learning community (“Even when a community
of practice arises in response to some outside mandate, the practice evolves into the community’s
own response to that mandate” [1998, p. 80]), Wenger agrees with Engestrom (1987), who writes,
“The instructor’s task and the learner’s perceived task are seldom the same thing.” This research is
interested in how the different components of an activity system (see below) interact dialectically to
produce new activity systems and new knowledge. Communities of learners construct knowledge,
understanding and identities which, themselves, are dynamic rather than immutable.
4. Activity theory and expansive learning
The theory of Expansive Learning (Engestrom 1987) derives from Activity Theory (Leont’ev [1981]),
which was first formulated from Vygotsky’s (1978, 1997 [1927]) theory of human development. Activity
Theory argues that human actions are not a direct transmission between subject and object, but are
mediated through the use of (broadly defined) tools.
Vygotsky (1978) represented the first generation model of human activity as a simple triangle.
Mediating Artefact (Tools)
Subject Object Outcome
Figure 1: First-generation activity system (based on Vygotsky 1978).
Vygotsky’s model illustrates his theory that human beings do not interact directly with their
environment. Instead, they use tools (including signs and codes as well as physical apparatus) as
mediators. Engestrom (1987) developed the expanded model of human activity (the activity system)
to include and highlight the collaborative nature of human activity by adding social elements to
Vygotsky’s original model of human activity, as shown in Figure 2.
The bottom row of the triangle (the layer added by Engestrom) features the rules, the community, and
the division of labour as its nodes: “The rules element represents the norms, expectations, and
conventions that constrain and influence the means by which an activity is carried out” (Greenhow
and Belbas 2007, p. 367). Community refers to the environment in a broad sense, and the division of
labour node represents who does what in an activity, thereby illustrating both the distribution of tasks,
and the hierarchy of power.
There can, however, be contradictions in the interaction of the nodes, and it is these contradictions
that Engestrom (1987, 2001) identifies as significant in Expansive Learning, as the contradiction can
enable the construction of new knowledge: “When an activity system adopts a new element from the
outside …, it often leads to an aggravated secondary contradiction where some old element (for
example, the rules or the division of labor [sic]) collides with the new one. Such contradictions
generate disturbances and conflicts, but also innovative attempts to change the activity” (2001, p.
137). For example, a lecturer (subject) works with students (objects) with the intended outcome of
high-quality learning. Technology (tools) can be used to facilitate the learning. However, if a new tool
is available, over which the students (rather than the lecturer) have mastery, then this may require
new practices within the activity system in order for the object of high-quality learning to be
accomplished. The lecturer may have to relinquish some of their authority (the “rules” and “division of
919

Michael Flavin
labour” nodes) in order to enable enhanced learning. The analysis overlaps with Christensen’s
Disruptive Technology theory (1997) in the sense that a new technology can disrupt existing practices
(thereby risking rejection), but also that the new technology can go on to change the practice itself.
Rules
Tools
Subject Object Outcome
Community Division of Labour
Figure 2: Second generation activity system (based on Engestrom 1987)
A feature of Engestrom’s approach is that tools do not need an instruction manual: “A tool always
implies more possible uses than the original operations that have given birth to it” (1987), and “the
material form and shape of the artifact [sic] have only limited power to determine its epistemic use”
(2007a, pp. 34-35). This leads Engestrom to conclude, “In Expansive Learning… reconfiguration of
given technologies by their users is essential” (2007a, p. 35). Engestrom’s argument implies that
people establish the meanings of technologies through their uses of them. Meaning is not constrained
by design. Moreover, and in common with Wenger (1998, pp. 227, 233) Engestrom suggests learning
requires imagination, improvisation and innovation.
Expansive Learning and the Communities of Practice theory agree that learning involves identity
formation. Engestrom (2007b) further agrees with Wenger (1998) by arguing that the Communities of
Practice model sees learning as “an inevitable aspect of all productive practices, not a specific
process mainly or exclusively limited to schools and other institutions of formal learning” (2007b, p.
41). However, Engestrom sees a different model of learning emerging, one in which learning is
equally inevitable. He promotes knotworking, defined as collaborative work without clear rules or a
hierarchy (2007b, p. 44).
5. Research methodology
The main research question this study is focused on is to understand how H.E.I.s can engage
constructively with disruptive technologies for learning and teaching. In order to investigate this issue,
a pilot study with a sample size of 28 was designed. The purpose of the study was to explore the
following questions: (1) what technologies are participants aware of? (2) What do participants use
technologies for? (3) To what extent do participants use individual technologies for more than one
purpose? (4) To what extent do participants construct their own meanings for technologies? (5) To
what extent are technologies used disruptively? The questionnaires included tick box questions, and
longer questions inviting participants to be more reflective. Questionnaires were used for this study in
order to obtain base line data about the uses of technology for learning and teaching, and the uses of
technology for other purposes. Questionnaires also enabled the identification of particular
technologies that were used most widely, and for more than one purpose.
The sample comprised 13 undergraduates studying an Arts subject by distance learning. There were
also 4 postgraduate students (3 studying at campus universities, and 1 by distance learning) and 1
postgraduate researcher. The sample also included 6 lecturers working at a specialist H.E.I. teaching
a specific range of subjects (one of whom also teaches at a campus university), and 4 academic-
920

Michael Flavin
related staff (3 from the same specialist H.E.I., and 1 from a campus university). The questionnaires
were issued and returned in the period November 2010-March 2011.
The second phase of the pilot study comprised two semi-structured interviews of participants from the
original sample, an approach which made it possible to identify what learners have done with
technologies, and how participants articulate their experiences with technologies, thereby helping to
generate an understanding of the meanings participants create for and with technologies.
The interview transcripts were analysed with the primary aim of identifying disruptions and
contradictions, and exploring how purposes for technologies are established through usage. Analysis
of the interviews was also interested in how participants gained competence in technologies, and in
exploring the interactions between nodes in an activity system.
6. Findings
The questionnaires show specific technologies being used for more than one purpose. For example,
Facebook is used for recreation by 16 participants, but also for work (7) and informal learning (3), with
informal learning signifying learning not undertaken in the context of a formal course. Twitter is used
for recreation by 10 participants, but also for work (7) and informal learning (5). This phenomenon
echoes research by Conole et al. (2008), in the sense that technology blurs the lines between study,
work and recreation.
The questionnaire findings for LinkedIn and Wikipedia are particularly noteworthy. LinkedIn
encourages individuals to create a profile foregrounding “professional expertise and
accomplishments” thereby “linking you to a vast number of qualified professionals and experts”
(LinkedIn 2011). Its focus and raison d’être is therefore clearly professional, yet while 4 participants
use it for work, 5 use it for recreation and 4 for informal learning. The meanings of the technology are
not dictated by design, but emerge from usage. While the evidence base for this assertion is small, it
would be worthwhile seeing if the same results were replicated in a larger sample beyond the pilot
study phase.
The findings for Wikipedia indicate its ubiquity, as it crosses boundaries between work, study and
recreation. Twenty-one participants use it for recreation, 20 for informal learning, 15 for formal
learning, and 16 for work. In addition, Wikipedia has displaced, or at least challenged, more
established encyclopedias, which are sustaining technologies in the sense that they update their
knowledge along existing lines and publication formats, but find it difficult to complete with a rival
encyclopedia which is free, and available to anyone with access to a networked device. One
participant stated, “The biggest advantage of Wikipedia is that the answers are at your finger tips, you
can ask a question and the answer appears without the need for flicking from chapter to chapter in a
book.” Like many disruptive technologies, Wikipedia is cheaper (free in this particular case) and more
convenient than its more established rivals.
The questionnaires do not reveal any significant patterns of usage among distinct groups. Students at
levels 4, 5, 6 and 7 in the UK Framework for Higher Education Qualifications (QAA 2008) were
questioned, but did not provide any data to suggest that individuals at different stages in a community
of practice have different levels of technology awareness and usage.
The two interviews were conducted with a lecturer who teaches at both a campus university and a
smaller, specialist H.E.I., and with a Learning Technologist at the same specialist H.E.I., who is also a
postgraduate student studying by distance learning. Both interviews were conducted in February
2011.
The interview with the lecturer referred to her experiences teaching on an undergraduate degree at a
campus university, where students can post materials on the V.L.E.:
Q. Instead of all the learning and teaching resources at [name of institution] coming from
you – some of them maybe posted by you, but essentially coming from the students.
How do you feel about that?
A. I think it’s absolutely brilliant because they’re engaging in that process. And they’re
not just waiting to be spoon fed. They’re actually actively seeking materials and they’re
sharing it with everyone else.
921

Michael Flavin
The lecturer also spoke about her own experiences with technologies outside work and, specifically,
her experiences on LinkedIn, which she uses for social rather than business networking. The lecturer
stated she experienced no awkwardness about using LinkedIn for social networking and described
how a LinkedIn contact had led to her finding a violin shop that assisted her children’s progress in
music lessons. The interviewee uses LinkedIn for a purpose other than its declared purpose, but
without any sense of transgression or protracted reflection. The technology serves a personal
purpose, and has thus acquired a meaning through usage.
The second interviewee, a Learning Technologist, spoke about her own experiences when she first
started working with learning technologies: “we learnt on the job pretty much… We probably learnt
from our own mistakes as we went along.” She then talked about her own experiences as a
postgraduate student: “I am not really very well organised when I study, so I tend to just dive in and
see how it works for myself really… you just have a go and hope that you don’t embarrass yourself
publicly.”
Q. So there were induction materials available, but would it be fair to say that you didn’t
use those and just used experience?
A. Yes. absolutely… For me It’s the most effective way to learn – just go in and click
things and see what happens.
The interviewee also spoke about her experiences with Twitter: “You could watch five thousand
tutorials on Twitter and I still don’t think you would get it. You have to capture your own purpose in
using it… you have to actually use it before you begin to see the value in it.” However, while the
interviewee uses Twitter primarily to support her work, her use of Facebook serves a different
purpose: “Facebook for me is purely a social tool.” Thereafter, the interviewee spoke about her online
identities.
Q. So is it fair to say that you have more than one online identity?
A. Definitely, yes. I keep them all separate deliberately.
Q. Why is that?
A. Facebook relates to what I do at the weekend, socialising with friends. My friends
take pictures and I don’t have any control over the pictures they put up. I don’t
necessarily want my boss at work seeing me mingling… And Twitter for me, I use it as a
learning tool primarily. I don’t actually tweet that much myself, I’m more kind of
consuming stuff at the moment. But I don’t put up things relating to what I had for dinner
and what I’m doing that night. Sometimes I do, but generally I keep that sort of work
related persona.
The interviewee’s responses echo the findings of Creanor et al. (2006) in the sense that learners can
choose to separate their uses of technologies, and may opt to use different technologies to support
different online identities. The interviewee maintains some demarcation between work, study and
recreation in relation to her uses of technologies. However, she also establishes her purposes for
technologies through her use of them.
Both interviewees are self-taught to a large extent in their uses of technologies to support learning,
thereby supporting one of Conole et al.’s contentions, that learners rely more on trial and error than
on formal training (2008, p. 515). Moreover, both interviewees create their own meanings for
technologies, using LinkedIn in one case for non-professional networking, and Twitter in another for
solely professional networking.
7. Conclusion
A number of the questionnaire responses suggest the validity of the Disruptive Technology theory.
For example, only one participant stated that they use Second Life (online virtual world). The
participant enjoys using Second Life but states it has “a steeper learning curve”. In addition, a further
participant stated a disadvantage of using technologies for learning is that they are “not always
transferable to other teachers (due to their lack of knowledge or desire)”. The Disruptive Technology
theory argues that ease of use is a significant factor in the take up of a disruptive technology.
Wikipedia has this capacity, with one simple search box. Mastering Second Life requires more
advanced competencies. If technologies are kept simple, people are more likely to use them.
922

Michael Flavin
From an Activity Theory perspective, Wikipedia comprises a tool that disrupts the rules of the activity
system, and the division of labour. Students are no longer reliant on their lecturers or their institution
for information. Tools (technologies in this case) also impact on the subject and the community by
blurring the lines between study and recreation. Learning is less easily contained, and less easily
demarcated away from other activities. Hence the perception of learning may be shifting away from an
activity confined to a particular time (the working day within the context of the conventional academic
calendar) and a particular place (the campus). New tools, through which human activity is mediated,
are shifting the terms of human activity.
Technologies for learning can be disruptive in the sense that access to learning is increasingly
controlled by learners. Technologies have prompted change, as the H.E.I. does not exert control over
what material is accessed, when it is accessed, or the scale and nature of learner collaboration.
Applying Activity Theory to one of the interviews, the fact that students can post learning materials on
the University’s V.L.E. comprises a change in the division of labour within the activity system, yet the
lecturer is working positively with the change, and sharing some control of the learning. Thus a new
tool (and a flexible approach within the explicitly social nodes in the bottom row of the activity system)
is enhancing learning for the whole community.
Hereafter, the research will further explore the impact of disruptive technologies, using Activity Theory
to evaluate the impact of disruptive technologies in higher education learning and teaching, especially
the extent to which the disruptive uses of technologies to support learning are blurring the lines
between study, work and recreation. The research will also use the Community of Practice theory to
examine the multiple identities formed by learners, and the extent to which different identities meld as
lines blur between different economic and social practices.
References
Blin, F. and Munro, M. (2008) ‘Why hasn’t technology disrupted academics’ teaching practices? Understanding
resistance to change through the lens of activity theory’, Computers and Education, vol. 50, pp. 475-490.
Britain, S., and Liber, O. (2004) ‘A framework for the pedagogical evaluation of eLearning Environments’, JISC,
http://www.jisc.ac.uk/uploaded_documents/VLE%20Full%20Report%2006.doc (accessed 6 June 2011).
Christensen, C. M. (1997) The innovator’s dilemma: when new technologies cause great firms to fail, Boston,
Mass., Harvard Business School Press.
Christensen, C. M. and Raynor, M. E. (2003) The Innovator’s Solution: Creating and Sustaining Successful
Growth, Cambridge MA, Harvard University Press.
Christensen, C. M. (2006) ‘The Ongoing process of Building a Theory of Disruption’, The Journal of Product
Innovation Management, vol. 23, pp. 39-55.
Christensen, C. M., Horn, M. B., and Johnson, C. W. (2011) Disrupting Class: How Disruptive Innovation Will
Change the Way the World Learns, New York, McGraw Hill.
Conole, G., Laat, Maarten de, Dillon, T. and Darby, J. (2008) ‘“Disruptive technologies”, “pedagogical innovation”:
What’s new? Findings from an in-depth study of students’ use and perception of technology’, Computers
and Education, vol. 50, pp. 511-524.
Creanor, L., Trinder, K., Gowan, D. and Howells, C. (2006) ‘Who’s learning and how? Researching the learner
experience’, Proceedings of the 23rd annual Ascilite conference: Who’s learning? Whose technology?
University of Sydney, 4-6 December 2006.
Engeström, Y. (1987) Learning by expanding: an activity-theoretical approach to developmental research,
Helsinki, Orienta-Konsultit Oy. http://lchc.ucsd.edu/MCA/Paper/Engestrom/expanding/toc.htm (accessed 6
June 2011).
Engeström, Y. (2001) ‘Expansive Learning at Work: toward an activity theoretical reconceptualization’, Journal of
Education and Work, vol. 14, no. 1, pp. 133-156.
Engeström, Y. (2007a) ‘Enriching the Theory of Expansive Learning: Lessons From Journeys Toward
Coconfiguration’, Mind, Culture and Activity, vol. 14, nos. 1-2, pp. 23-39.
Engeström, Y. (2007b) ‘From communities of practice to mycorrhizae’, in J. Hughes,
N. Jewson and L. Unwin (eds) Communities of Practice: Critical Perspectives, London, Routledge.
Greenhow, C. and Belbas, B. (2007) ‘Using activity-oriented design methods to study collaborative knowledgebuilding
in eLearning courses within Higher Education’, Computer-Supported Collaborative Learning, vol. 2,
pp. 363-391.
Koszalka, T. A. and Ntloedibe-Kuswani, G. S. (2010) ‘Literature on the safe and disruptive learning potential of
mobile technologies’, Distance Education, vol. 31, no. 2, pp. 139-157.
Lave, J. and Wenger, E. (1991) Situated Learning: Legitimate Peripheral Participation, Cambridge, Cambridge
University Press.
Leont’ev, A.N. (1981) The Development of Mind, U.S.S.R., Progress. Also available at
http://marxists.org/archive/leontev/works/development-mind.pdf (accessed 6 June 2011).
923

Michael Flavin
Quality Assurance Agency for Higher Education (2008) The framework for Higher Education qualifications in
England, Wales and Northern Ireland, Mansfield, The Quality Assurance Agency for Higher Education,
http://www.qaa.ac.uk/academicinfrastructure/FHEQ/EWNI08/FHEQ08.pdf (accessed 6 June 2011).
Selwyn, N. (2007) ‘The use of computer technology in university learning and teaching: a critical perspective’,
Journal of Computer Assisted Learning, vol. 23, no. 2, pp.83-94.
Sharples, M. (2003) ‘Disruptive Devices: Mobile technology for conversational learning’, International Journal of
Continuing Engineering Education and Lifelong Learning, vol. 12, no. 5-6, pp. 504-520.
Vygotsky, L.S, (1978) Mind in Society - The Development of Higher Psychological Processes, ed. by M. Cole, V.
John-Steiner, S Scribner, and E.
Souberman, Oxford, Harvard University Press.
Vygotsky, L. S. (1927/1997) ‘The historical meaning of the crisis in psychology: A methodological investigation’,
in R. W. Rieber, and J. Wollock (eds.) The collected works of L. S. Vygotsky, vol. 3: problems of the theory
and history of Psychology, New York, Plenum.
Wenger, E. (1998, repr. 2005) Communities of Practice: learning, meaning and identity, Cambridge, Cambridge
University Press.
924

Exploring the Potential of a Mobile Computer lab in a
Developmental Context: The Teacher’s Perspective
Fortunate Gunzo and Lorenzo Dalvit
Rhodes University, Grahamstown, South Africa
G08g6256@campus.ru.ac.za
l.dalvit@ru.ac.za
Abstract: This paper discusses the design phase of an intervention involving the integration and use of a mobile
computer laboratory in South African schools. We outline a model we have developed as part of an Advanced
Certificate in Education (ACE-ICT) course; a professional development course which targets in-service teachers.
As part of the practical component of the ACE-ICT course, teachers are required to plan and implement an
intervention involving the use of computers in their schools and or classrooms. The model we have developed is
an attempt to address some of the problems teachers face in their schools when they try to use computers for
teaching and learning. We used a qualitative research approach and collected data using a questionnaire with
only open ended questions. With the questionnaire, we gathered in-depth descriptions of some of the challenges
teachers faced and suggestions of how they would use the mobile computer lab in their schools. Data was
collected from 20 in-service teachers enrolled for the ACE-ICT course at a South African institution. Teachers
who came from schools that do not have computers suggested using the mobile computer lab to create basic
awareness of how a computer works first for their learners. These teachers acknowledged their role in preparing
their learners for the future whereby their learners will encounter the computer in one way or the other. Most
teachers proposed using the mobile computer lab for teaching and learning as they regarded these as very
important areas in which computers and ICT in general can make some contribution in education. Findings from
this design phase of the intervention were useful in highlighting the areas that the mobile computer lab can be
used in, in a developing context. Based on these findings, the interventions will now be implemented in the
various schools.
Keywords: mobile computer lab, access to ICT, marginalised schools
1. Introduction
According to the South African White paper on e-Education (2004) every school in the country is
expected to be equipped with computers by 2013. All learners and teachers are expected to become
competent users of Information Communication Technology (ICT) in the school context. The
Advanced Certificate in Education (ACE) in ICT run by our university is designed to equip teachers
with the appropriate ICT skills and understanding of ICT use in a school context. The course runs
over two years. It accommodates 20 teachers from marginalised schools in townships, farm and rural
areas. All contact sessions take place at the university, in the Education Department’s computer lab.
As part of the course, teachers are required to plan and implement an intervention involving the use of
ICT in their school. Although most schools do have ICT infrastructure, this is often haphazard, outdated,
poorly maintained (Kabede, 2006). We believe that by using the same, well supported and fully
functional mobile computer lab for all interventions we can make the experience more rewarding for
teachers and other participants and support meaningful sharing and reflection on the experience
within a community of practice.
As part of a PhD research project, we intend to involve teachers in the design and implementation
phases of an intervention using a mobile computer lab in their schools. The intervention must aim to
address specific problems in any area of their professional life through the use of ICT. Issues of
physical and epistemological access to ICT by the teachers, their learners and colleagues are of
particular importance. Narratives of all participants are collected and analysed to tease out the “story”
behind each intervention from different points of view. Teachers then present and reflect on their
experience in the ACE-ICT class. In this paper, we discuss the design phase, highlighting teachers'
ideas about the possible use of the mobile computer lab in their schools.
2. Context
2.1 Mobile computer labs and netbooks
In this paper we describe the design phase of a computer intervention at schools. We used a group of
netbooks to populate a mobile computer lab. A netbook is a 10inch lightweight laptop which has
similar functions and specification as those of a standard laptop. The size of the netbook makes it
925

Fortunate Gunzo and Lorenzo Dalvit
portable (Staplesson, 2010), a very important characteristic for the context within which the mobile
computer lab will be used in this study. We are aware that portability of the netbook increases the
security risk of theft (Daly, 2005). A netbook also caters for increased productivity because of the long
battery life (3-8hours) particularly for areas where electricity is still problematic. The netbook also has
built-in wireless network connection for easy access and connection to the internet. In the developed
countries where the mobile lab has been implemented in schools (Martin, 2010; Daly, 2005; Manbeck,
2005; The Journal, 2003) a “trolley” is used to house the machines when they are not in use and
recharge or update/install software. In that instance the mobile computer lab becomes “mobile”
because it can be moved from one class to the next by pushing the trolley. In this paper, the mobile
computer lab we describe is slightly different from this. Instead of only moving from one class to the
next, the lab will move from one school to the next. This is an important and useful feature of the lab
in the developing context in which we work, since one lab can serve a number of schools in
marginalised areas.
2.2 Access to ICTs in South Africa schools
In this paper we focus on the physical access to computers. A significant proportion of schools in the
Eastern Cape Province of South Africa in which this research is conducted still do not have physical
access to ICTs. According to the National Education Infrastructure Management System (NEIMS)
report, 90% of ordinary schools (not including private/special schools) do not have a computer lab
(NEIMS, 2009). It must be noted however that South Africa has one of the most uneven and unequal
societies, such that while this province has this backlog there are other provinces that boast of over
90% of their schools having computer labs(NEIMS, 2009). The Eastern Cape Province is one of the
poorest provinces in South Africa (Herskovitz, 2010). Many schools lack basic infrastructure such as
decent classrooms, sanitation, running water, electricity, and furniture. Coupled with this, is the
challenge of large classes that make it difficult to share the few available resources (Dalvit et al,
2006).
In terms of physical access to ICT, there are several problems commonly associated with using ICTs
in South African schools. According to Farrell and Isaacs (2007) most schools face infrastructure
problems such as a lack of decent, secure buildings, no or inconsistent electricity supply, lack of or
unaffordable connectivity, lack of access to ICT infrastructure, lack of technical support services,
physical security of the computers etc. (Brandt, 2006; Smith, 2005). In some instances the problem of
school management acting as a “gatekeeper” to technology, because of its perceived cost (Brandt,
2006) hinders access to ICT.
In terms of epistemological access, lack of qualified or confident teachers to teach learners how to
use computers and subsequently use computers in their classrooms is a huge challenge (Dalvit et al,
2006). Therefore the importance of training teachers to teach with ICT cannot be overstated. Another
tension is for teachers to know when to let children explore and for how long without losing control of
the class. We are aware of the potential exploration has for computer skills acquisition in children as
indicated in the Hole in Wall project (Dangwal, 2005). Language is also a key issue, since computers
are a domain of the English language and most of the learners at marginalised schools do not have a
good command of the English language (Mapi et al, 2007)
2.3 Rationale for using ICT in marginalised schools
There are several reasons why marginalised schools in South Africa are receiving attention when it
comes to ICT. Since the end of apartheid in 1994, the South African government has been working on
delivering an equal education system to all its citizens. Making sure that resources and skills training
for teachers and learners in marginalised schools happened was one of the ways of building a
balanced education system. Issues of digital divide also became prominent in the discussion
surrounding South Africans access to ICT. Provision of ICT in marginalised schools was and still is an
endeavour to stop the widening of the gap between those who have access to ICT and those who do
not. Marginalised schools are also still faced with most of the challenges that hinder access to
computers as discussed above.
A mobile computer lab supported by the University and that can be shared by a group of schools
could be a solution to the problem of physical access to computers. Integrating an experience in using
ICT within a marginalised school context as part of a teacher's professional development course is
intended to unpack and address issues of epistemological access. Gauging teachers' perceptions of
926

Fortunate Gunzo and Lorenzo Dalvit
the potential of mobile labs is an important initial step in this regard. The findings presented here can
serve as a basis for further research on the efficacy of mobile computer labs.
3. Methodology
3.1 Data collection and analysis
We used a qualitative research approach to gather data for this paper. Data collection started from
the beginning of the ACE-ICT course. Teachers were asked to write down and submit their
expectations of the course. Later, they were asked to fill in a questionnaire that had seven open
ended questions. It was from this initial questionnaire that we learnt of the various challenges
teachers were faced with in their schools. The challenges which are outlined in the findings section
were cited as the reasons teachers found it impossible to undertake one of their assignments-
designing and implementing a research project involving the use of computers in their schools. Then
we got the idea of using a mobile computer lab. We realised that using a mobile computer lab would
provide the same well maintained computers to all teachers and give them an equal opportunity to
fully integrate ICT in their schools/classrooms. Since the teachers we were working with had no prior
experience using the netbook and a mobile computer lab, we devised a model that would allow them
to use the netbooks during the entire design phase which we report on in this paper. We did this so
that the teachers would become familiar and comfortable using the netbook before using it for their
research projects in their schools/classes.
Teachers worked in groups to brainstorm on how the mobile computer lab could enhance teacher
experiences within their schools. After this, each teacher had to prepare and make a presentation on
what they thought would be the advantages and disadvantages of using the mobile computer lab in
their schools. All this was done as a way of preparing the teachers to write an individual proposal of
what they thought a mobile computer lab could be used for in their own school. In the proposal,
teachers had to outline the context of their school, paying attention to the resources and
infrastructure. They also had to articulate exactly what they would do with a mobile computer lab. In
this paper we present findings from these proposals. Data was analysed by grouping the responses of
the teachers into different themes/categorises. We found that the model we employed for the
research was useful and effective. We realised that teachers struggled to use the netbooks at first
mainly because of their size. After using it for few times they started to feel confident and comfortable
using the netbooks. A challenge that we faced as we undertook this project was the limited number of
netbooks-we had 12 netbooks. The 20 teachers had to share these and this sometimes meant that
some of them could not finish the work they had to do. We resolved this by identifying teachers who
were reasonably comfortable with the netbooks and so worked faster and asked them to use the
netbooks first. Also we allocated more time to the work that had to be done on the machines. This
was all important for teachers as they planned their own projects with their learners.
3.2 Profile of the schools
Teachers who participated in this study were from marginalised schools located in townships, farms
and rural schools. In the South African context, the term township usually refers to “the urban living
areas (often underdeveloped) that during the apartheid era were reserved for non-whites i.e. blacks,
coloured and working class Indians (Bond, 2008). Townships were usually built on the periphery of
towns and cities. On the other hand, farm schools were created as a way of providing cheap labour to
farm owners (Wilson, 2002). These schools depended on the farm owner heavily. Today, farm
schools still exist, and the government is now involved in the running of such schools (Herskovitz,
2010). Farm schools usually have very low student enrolment and as a result a few teachers who
usually teach across grades and subjects. Rural schools have similarities with farm schools both are
characterised by underdevelopment and poverty (Herskovitz, 2010). The number of teachers that
came from each school is represented in the table below:
Table 1: Research participants
Type of school Number of teachers
Township school 12
Rural School 5
Farm School 2
District official 1
Total participants 20
927

4. Possible scenarios
4.1 Computer literacy
Fortunate Gunzo and Lorenzo Dalvit
Thirteen (65%) of the twenty teachers said they suggested learners because they believed that they
deserved to be introduced to computers as they would need computer skills to make it in the world
despite of their backgrounds. Six (30%) teachers had learners who had never been exposed to
computers in a school context but expressed confidence that learners would be able to cope. One
teacher from a rural school said, “What children lack is not capacity, it is opportunity and resources”.
While teachers agreed that learners in any grade deserved to have access to computers, there was
special mention of learners in the final grade- 12. A teacher said “I am mentioning grade 12 because
they are going to tertiary institutions next year and most except for Computer and Applications
Technology (CAT) students, know nothing about computers. It will be very difficult for them to cope at
tertiary level if they don’t know how to operate a computer”.
Five (38%) of the thirteen teachers who suggested targeting learners for their intervention, said that
they wanted to do an introduction to computers. Though computer literacy was deemed particularly
important for learners in the lower grade (3 to 7) it must be noted that depending on the school the
teacher was from and the resources available at the school, basic computer literacy was suggested
even for learners up to grade 12. An example was a teacher from a rural school, which did not have a
computer. Learners at the school had therefore never used computers within the school context and
an introduction to computers for all grades was therefore appropriate. Another five (38%) of the
thirteen teachers suggested conducting a computer literacy course with their learners. These were all
teachers from schools that had computer facilities and whose learners were in the senior phase
(grades 8-12) and had been exposed to computers before.
4.2 Teaching across the curriculum
Three (23%) of the thirteen teachers wishing to target learners suggested integrating computers into
the subjects they were currently teaching. The key dimensions emerging from the data was the use of
ICT to avoid duplication of effort and to access existing resources. One teacher said:
“In my school, we still make use of chalkboards for teaching. For instance, I have five
grade nine Natural Sciences groups. When they come to class, I have to write on the
board over and over again. By the time group 3 comes, I am exhausted and doing
disservice to them- sometimes, leaving out some important stuff which I have already
mentioned to others. In this instance, I will have 1 well prepared presentation for all the
groups and they will have it too in their netbooks”
Teachers also suggested using the mobile computer lab to access online resources. Teachers
suggested teaching learners basic web searching/browsing techniques for learners. This suggestion
was made since teachers were aware that the netbooks that we used to populate the lab had wireless
internet connectivity and made it possible to connect to the internet. Suggestions to cut on the costs
of connecting to the internet included that the teacher could make a list of websites for learners to visit
and monitoring that they were not downloading anything else during the class time. A teacher said “I
think that online resources available on Thutong and educational software such as Encarta would be
very useful for me and also I can show my learners how to search for information on Google for the
activity that I would have given them”. Thutong Portal is an online resource that provides information
about Education in South Africa including the curriculum (http://www.thutong.doe.gov.za ). Encarta on
the other hand is a digital multimedia encyclopaedia. More information about Encarta can be found on
(http://www.microsoft.com/uk/encarta/default.mspx).
4.3 Skills development for colleagues
One participant suggested that the mobile computer lab should target teachers. This participant
works for the district Department of Education as e-learning subject advisor. Since the beginning of
the course she expressed interest in the intervention as it is in line with her work. As she is not based
in a school, we asked her to make a proposal based on a nearby school if she felt she wanted to
target learners or school teachers. She was also free to choose to work with her colleagues at her
place of work. The proposal she prepared was part of her job. She stated in her proposal that they
visited a particular rural school to assess whether it was e-ready, i.e. whether or not a school is ready
to implement ICT. This participant stated in her proposal that while they found the school met all the
928

Fortunate Gunzo and Lorenzo Dalvit
requirements for it to receive government assistance to set a computer laboratory- the staff was not
enthusiastic about it as none of them were computer literate. She suggested that teachers should be
targeted with any technology intervention arguing that “the teacher must first know how the
technology works so that he/she can be in a position to assist learners to become familiar with the
technology so that the teacher does not feel embarrassed in the class when learners show her how
the computer works....because children are like that, they learn fast”.
Six (30%) of the twenty teachers suggested using the mobile computer lab for administration. These
teachers mentioned that due to limited computer facilities, their clerk did everything they needed on
the computer. There were also suggestions of training colleagues to use South Africa School
Administration and Management System (SA-SAMS). SA-SAMS is an electronics school
management system implemented by the South African Department of Education
(www.education.gpg.gov.za
4.4 Teachers’ expectations and possible challenges
It was the hope of the teachers that the use of the mobile computer lab in their classrooms might
interest learners and with time increase learners’ motivation to learn. We made sure that all the
teachers understood that the interventions we were going to make at a later stage were going to be
very brief and that we were not going to provide a mobile computer lab for their schools. This was
important to ensure that teachers did not think of a permanent mobile computer lab in their school as
they prepared for the project. Despite this teachers were optimistic that permanent access to
computers could be realised from our initial intervention, once the school management had seen the
potential of the mobile computer lab.
Teachers had high expectations of the parents and the school management teams (SMTs)
involvement in any computer programs at schools. For teachers, their involvement and support would
increase the chances of computer programs being successful and effective. The teachers would need
permission from the schools SMTs so that they can be granted access to the school outside school
hours. This is important given the ‘gatekeeper’ role that most SMTs often play due to the costs
associated with computers.
Teachers identified lack of computers as a key challenge. Six (30%) of the teachers came from
schools that did not have computers. For the teachers that came from schools that had computers 14
(70%), they stated that these were either very few, old or not working properly. Teachers expressed
concern over their inability to practice what they would have learnt with their learners in their schools,
one said “the mobile lab will allow me the chance to go back to my class and plough what I have
gained here in the course”. Closely linked to this was the problem of limited to no access to
computers by learners. Teachers cited problems with infrastructure at their schools. This included
decent buildings and telecommunications.
Another challenge that teachers spoke about was the large number of learners which they said made
it difficult for the teachers to take their learners to the computer lab where facilities are available. One
teacher from a township school said “there is a computer lab at my school; the only challenge is that it
has 25 computers and my class has 45 learners”. In the case of this teacher, the addition made by the
mobile lab would enable almost all learners to sit at their own computer. Another teacher said “Large
class sizes at our school make it impossible for each and every learner to be admitted to CAT; also it
will be difficult for one teacher to employ interactive teaching strategies or gain insight of different
problems encountered by the learners”. Teachers went ahead to offer possible solutions, for instance
a teacher at one township school suggested using the data projector in the class so that his learners
can follow from the board “I will use a data projector since my learners are 36, I will not be able to
show each learner or group what they must do, the data projector will make it easy for me to control
what happens in terms of the activity that I would have given them”.
Teachers also noted that learners come from different backgrounds and their computer knowledge
differs based on that. Teachers noted that some learners had access to computers at home while the
majority did not. It was their assumption that learners with access to computers would be more
comfortable with computers and able to finish their tasks early, while the ones who have no access at
home would take much longer to finish their work. As such, trying to work with all learners on one
activity would be a challenge. One teacher said:
929

Fortunate Gunzo and Lorenzo Dalvit
One big challenge in using computers is the different backgrounds that the learners are
coming from, for an example there are learners that have computers at home. Those
pupils can have their enthusiasm dampened if the activities they are asked to do are not
challenging. Such pupils can also become frustrated if the school equipment is slower
than the computer at home.
Issues of classroom management were also raised by the teachers. One teacher said “I am aware
that children will probably do other things as they like to explore, I will give them the chance to do that
once they finish the activity”. Another teacher said “I know that it is children’s nature to get excited
when they see visitors and new things in the classroom. So to reinforce good behaviour I will reward
the best behaved group with a merit”. These two examples show the resourcefulness and
innovativeness of the teachers in trying to make the intervention successful.
Language was noted as one of the main challenges especially for younger learners. Teachers
indicated that they felt the computer terminology might be a bit confusing to the learners, given that
computers are the domain of the English language. One teacher expressed concern of whether or not
the learners would understand the terms used for computers and resolved that she would code switch
to try and explain and make it easy for learners to understand “I have started teaching in English now
since my learners are in grade four as is stated by the education policy, but I know that my learners
will struggle to understand computer terms so I will have to explain to them in IsiXhosa (local
language)”. Teachers were optimistic that learners would be able to improve their command of the
language through the use of the computer. One teacher said “their language (English) will improve
since they will be using the spell checker during their writing times (on the computer)”. As an
alternative, open-source software already available in indigenous South African languages could be
installed on the netbooks (see Dalvit et al 2006).
5. Conclusion
In this paper, we discussed the perspectives of a group of 20 teachers working at various
marginalised schools in South Africa on a possible intervention using a mobile computer lab in their
schools as part of their professional development course. Most teachers supported the use of
computers for learners. The most popular suggestion was to use the mobile lab to teach computer
literacy (and introduction to computers) for learners. Special emphasis was made on the final grade
12 since they would be going into tertiary institutions and would need computer skills there. There
were suggestions for the use of the lab to teach subjects to learners. In this case teachers anticipated
that they could use online resources to search for information for the subjects they taught. Another
suggestion was to upskill colleagues to use computers for administrative purposes.
Although a longer term intervention would be necessary in order to have a real impact, these
suggestions seem reasonable within the constraints of the proposed intervention. Teachers
mentioned the support of the school's management bodies as a key success factor and see an
experience using the mobile lab as a way to spear-head the implementation of ICT in their schools.
Lack of infrastructure and large classes were cited as challenges to the use of ICT in school which the
an intervention with a mobile lab could help to overcome. Teachers could foresee problems of
classroom management due to the heterogeneous levels of prior experience with computer and by
excitement about the novel technology in their classes. Particularly for younger learners, language
was also identified as a possible problem. Teachers collectively devised strategies to overcome these
possible challenges in the upcoming implementation phase.
References
Bond, P. (2008) Townships, In International Encyclopaedia of the Social Sciences, 2 nd edition: Macmillan, USA
Brandt, I. (2006). Models of Internet connectivity for secondary schools in the Grahamstown district. Unpublished,
Masters Thesis, Rhodes University
Conway-Smith, E. (2011) In South Africa, attending school in a mud hut. Independent Education. % April,
available from: http://www.ieducation.co.za/in-south-africa-attending-school-in-a-mud-hut/
Herskovitz, J. (2010). Rural Schools from apartheid cloud South Africa’s future. Reuters. Wednesday, July 21
Dalvit, L., Alfonsi, R., Isabirye, N., Murray, S., Terzoli, A. and Thinyane, M. (2006). “A case study on the teaching
of computer training in a rural area in South Africa”. Paper presented at the 22 nd Comparative Education
Society of Europe (CESE) Conference, 3 - 6 July 2006, Granada (Spain).
Dalvit, L., Murray, S., Mini, B., Terzoli, A. and Zhao, X. (2006). “Production of and Access to ICT-based
Knowledge through English and African Languages at a South African University”. South African Journal of
Higher Education (SAJHE), 19, 1486 – 1498.
Daly, U. (2005). The hidden costs of wireless computer labs. The Journal. Available at: www.highbeam.com/doc
930

Fortunate Gunzo and Lorenzo Dalvit
Dangwal, R., Jha, S. and Kapur, P. (2005). ‘Impact of Minimally Invasive Education on Children: An Indian
Perspective’. British Journal of Education Technology 37(2): 295.
Department of Education (2008). Gauteng Education Department- Recording and reporting from SA-SAMS.
Available from www.education.gpg.gov.za
Department of Education, (DoE). (2004). White paper on e-Education: Transforming learning and teaching
through Information and Communication Technologies. September, 2004
Farrell, G.; Isaacs, S. (2007). Survey of ICT and Education in Africa: A summary Report based on 53 country
surveys, Washington DC: InfoDev/World Bank. Available at: http://www.infodev.org/en/Publication.353.html
Kabede, T. (2006). SchoolNet Lesotho (ed). Newsletter p1-8
http://www.thutong.doe.gov.za/
Manbeck,M.; Harris,A.and Adeyemi,A. (2005). EarthWalk Whitepaper- Making the case: Mobile wireless
computer labs as a cost-effective alternative to fixed desktop computer labs in schools [11.08.05]
Mapi, T., Dalvit, L. and Terzoli, A. (2007). “A study on the adoption of ICTs in a marginalised area in South
Africa”. Africa Media Review, 15(2).
Martin, M. (2010). Mobile computer lab a big hit at West Salem Middle School. West Salem Coulee News Online,
09 April, Available at: www.couleenews.com
NEIMS report. (2009) South Africa Department of Education. Available from www.education.gov.za/neims
Smith, S. (2005). ICT and School Security available online at:
http://www.cumbridagridforlearning.org.uk/getfile.php?src=59/ICTindicators/ICTin EDchap2.pdf
Staplesson,M. (2010). What is the point of a mini notebook? Available at:
http://EzineArticles.com/?expert=Mika_staplesson
The Journal. (2003). Implementing a mobile lab in a faculty of Education. Available at
http://thejournal.com/articles/2003/10/01/implementing-a-mobile-lab-in-a-faculty-of-education
Wilson, S. (2002). Transforming Farm Schools. Issue Paper 3
931

Collaborative eLearning in a Developing Country: A
University Case Study in Uganda
Evelyn Kigozi Kahiigi 1 , Henrik Hansson 1 , Mats Danielson 1 , F.F Tusubira 2 and
Mikko Vesisenaho 3
1
Department of Computer and Systems Sciences, Stockholm University, Kista,
Sweden
2
Knowledge Consulting Ltd, Kampala, Uganda
3
University of Eastern Finland, Joensuu, Finland
kahiigi@dsv.su.se
henrik.hansson@dsv.su.se
mad@dsv.su.se
tusu@kcl.co.ug
mikko.vesisenaho@uef.fi
Abstract: Universities in developing countries are increasingly adopting and using eLearning in their teaching
and learning processes as one of the means for leapfrogging into the knowledge driven world. However, despite
the recognition eLearning has received, it has not fulfilled the expectations in terms of impact on the delivery and
quality of learning, pointing to the need for improved or new approaches. This paper explores the collaborative
eLearning approach as one of the strategies for effective adoption and use of eLearning. Using the first stage of
development research this paper presents an empirical study that aimed to explore and understand the current
practices of collaborative eLearning in a developing country context. The study was carried out with university
students in Uganda and placed focus on the value derived from and challenges encountered in adopting and
using collaborative eLearning. From a general perspective, it was evident that learning and teaching methods are
predominantly traditional, with limited/no integration of eLearning and there were inconsistencies in
understanding the integration of technology into teaching and learning processes. The findings obtained indicated
that students through collaborative eLearning were able to share and gain knowledge, understand course
concepts and access various views and learning material. Factors such as inadequate bandwidth, inadequate
Internet/computer access, conflict resolution, adequate ICT skills and face to face interaction challenged the
adoption and use of collaborative eLearning in this context.
Keywords: collaborative eLearning; developing countries; eLearning; ICT; higher education; Uganda
1. Introduction and aim
Developing country contexts (DCC) have been mainly exemplified by low living standards, high rates
of population growth, low income per capita, and general economic and technological dependence on
developed economies (Bakari, 2005). It is without doubt that, higher education is increasingly
becoming a vital avenue for students to leapfrog into the competitive economic market and improve
their quality of life. The drive to join the growing global economy is attributed to the rapid
transformation in various business sectors that require a skilled workforce that is more effective and
competitive. The increasing need to rely on faster and reliable access to information and knowledge
have also become crucial aspects. It comes as no surprise that recent developments in the education
sector have seen a widespread recognition for the need to position eLearning in a broader context of
the emerging knowledge economy.
However, Garrison and Anderson (2007) point out that we are yet to fully experience the
transformative effect of eLearning to support learning. For instance, the inherent assumption in DCC
is that putting in place ICT infrastructure and creating online courses equates to eLearning and thus
leads to changes in the teaching and learning process (Kahiigi et al., 2009). Contrary to this, Zurita
and Ryberg (2005) affirm that changes in the mode of education delivery does not translate into a
positive development, nor a change or development in the teaching styles or the pedagogical
practices. This can be attributed to the limited understanding of integrating eLearning into the
teaching and learning process in DCC (Vesisenaho, 2009).
Nonetheless, several higher education institutions in DCC have adopted eLearning to improve the
quality of teaching and learning; to increase access to learning materials; to reduce the total cost of
education and to enhance their academic profiles (Engelbrecht, 2003). However, attempts to enhance
and reform the education sector through eLearning in DCC have been negatively affected by factors
such as unreliable and inadequate ICT infrastructure and services, low levels of ICT literacy and
932

Evelyn Kigozi Kahiigi et al.
experience, lack of pedagogy in their curricula and lack of lecturer and management support to drive
the eLearning initiatives (Kahiigi et al., 2009). Consequently, eLearning has not yet fully penetrated
the existing education systems in most DCC (Iahad et al., 2004). While acknowledging these
challenges as having a negative impact on eLearning in DCC, we affirm that suitable approaches for
its effective adoption and use to support student learning can be instituted.
Collaborative learning (CL) is such an approach that has been attested to by several studies to
support student learning (Kamba, 2009). CL has been applied as an instructional method in traditional
and distance learning environments. The underlying phenomenon involves students working in
groups through mutual interaction to achieve common learning goals. Pedagogically, interactions
derived through CL scenarios can be related to deep learning, critical thinking, higher cognitive
development and long term knowledge retention (De Wever et al., 2006). Recent research in the CL
field has mostly placed focus on investigating how collaboration contributes to group or individual
knowledge construction and learning outcomes (Arvaja et al., 2007). However, the significance of
factors influencing students’ opportunities to learn collaboratively still remains under-researched
(Östlund, 2008). This has resulted into limited empirical evidence on the effectiveness of Computer
Supported Collaborative Learning (CSCL) at the higher education level, especially in DCC.
Understanding the collaboration process and what impacts on it to support effective, meaningful and
sustainable interactions in eLearning environments within any context is equally important. CSCL is
hence forth referred to as collaborative eLearning in this paper.
From the discussion above a question that requires examination is; what is the current state and
practice of collaborative eLearning (CEL) in a DCC? This paper therefore aims to investigate the CEL
concept and how it has been applied within a DCC, specifically in Uganda, as the first stage of
development research. More specifically, this study investigates the value derived and challenges
encountered from adopting and using CEL. The empirical results are obtained from third year
undergraduate students.
2. Theoretical perspective on collaborative learning
CL is based on the notion that knowledge construction is a social event and interaction is particularly
important for learning and attainment of higher order cognitive skills (Lehtinen et al., 1999). This view
builds on Vygotsky’s (1978) and Piaget’s (1926) frameworks that emphasizes social interaction as a
means of individual cognitive development and learning. This implies that through social interactions
individuals with different perception tend to improve their understanding of concepts. Lehtinen et al.
(1999) further assert that deep conceptual understanding is fostered through explaining a problem to
others during the process of inquiry. The resultant effect is that individual’s level of understanding of
concepts tends to become salient in a collaborative environment.
CL can be explained basing on social constructivism that relates to individuals constructing their
knowledge through the process of negotiating meanings with others within the learning community
(So and Brush, 2008). The constructivist view of learning is associated with Vygotsky’s (1978) zone of
proximal development that relates the learners level of understanding and cognitive development to
social interaction and collaboration from expert guidance and capable peers. This implies that CL can
allow learners develop an understanding and master various aspects in a course better than when
working alone.
Notwithstanding the potential of CL in supporting student learning, there still exists obscurity in finding
a common definition of CL. As exemplified by Lipponen et al.(2004), from a theoretical perspective the
variation can be reflected through three metaphors of learning, which are: acquisition, participation
and knowledge creation. Acquisition implies individual knowledge gain; participation emphasizes
interaction, while knowledge creation relates to continued advancement of shared knowledge.
Emanating from these metaphors various definitions have been presented. For example: Roschelle
and Teasley (1995) defines collaboration as “a coordinated synchronous activity that is the result of a
continued attempt to construct and maintain a shared conception of a problem”. Dillenbourg (1999)
views CL as “a situation in which particular forms of interaction among people are expected to occur,
which triggers a learning mechanism”. Liaw and Huang (2006) cite CL as “a social activity involving a
community of learners and teachers through which information is acquired and knowledge shared
through social constructivism approaches”. It is evidently difficult to provide a generalized definition of
CL as various perspectives are taken on by different disciplines and contexts in which CL is
933

Evelyn Kigozi Kahiigi et al.
implemented and studied. However, these definitions point to joint participation and social interaction
as dominate aspects that are required to facilitate CL situations.
The distinction between collaboration and cooperation is another aspect that has created numerous
debates. Some researchers have used and referred to these terms interchangeably, while others
have had counter arguments to distinguish them. Dillenbourg et al.(1996) argue that in cooperation a
task is split into independent subtasks and coordination occurs when results are collated, while in
collaboration cognitive processes maybe divided into intertwined layers that facilitate continued
attempts to share concepts in order to achieve the desired results. Roschelle and Teasley (1995)
affirm that cooperation involved division of labour among participants to accomplish a task; while
collaboration involves mutual engagement of participants in a coordinated effort to solve a problem
together. According to Panitz (1996), collaboration is a philosophy of interaction and personal lifestyle
while co-operation is a structure of interaction designed to facilitate the accomplishment of an end
product or goal through people working together in groups.
It is clear that there exists obscurity in finding a common distinction between cooperation and
collaboration. This can be attributed to the fact that individuals have different perspective for which
they define these concepts. However, Kirschner (2001) affirms that of utmost importance is that the
two concepts have similarities that are significant in supporting the learning process. In both cases,
the learning process is active, teaching and learning is a shared experience, students are engaged in
group activities, reflection on ones contribution is enhanced through discussions and articulations of
one’s ideas within a group and social and team skills are developed through consensus building. This
paper subscribes and adopts Roschelle and Teasley’s (1995) distinction of cooperation and
collaboration, based on participation and different roles played by group members during a learning
activity.
3. Collaborative eLearning as an alternative learning approach
The wide acceptance and availability of the internet means that CEL facilitates knowledge sharing
and creation, social interaction and cognitive development regardless of geographical boundaries,
time and socio-economic status. Consequently CEL has increasingly become appealing to students
and educational institutions as an alternative learning approach. CEL supports a faster learning curve,
since students can interactively customize their learning and have more control of their learning
process (Cantoni et al., 2004). The belief is that shared understanding through interaction is a natural
way for students to learn (Kreijns et al., 2003). This implies that collaboration and social interaction
coexist and are dependent on each other in any learning environment.
Technology developments supporting CL have played a critical role in facilitating and mediating the
interactive process (So and Bonk, 2010). Students are able to establish online social and academic
support networks while becoming constructively involved in their learning activities. Additionally
students are exposed to various online learning resources and conditions for quality interactions (with
peers, teachers and content) in order to achieve worthwhile learning goals and profound levels of
understanding (Garrison, 2007). Indeed studies (Kahiigi et al., 2009, Kamba, 2009) have established
that students engage in peer interactions in order to achieve their academic goals. Prosser and
Trigwell (1999) argue for a holistic approach that focuses on student learning experience, learning
context and learning outcomes as correlated variables and processes vital for CEL. Thompson and
MacDonald (2005) further assert that although it is important to establish effective communication and
develop social bonds, there is need to create a secure environment that facilitates open
communication in order to sustain the community. This implies that online courses have to be
designed in a way that facilitates acquaintance and trust among students (Östlund, 2008). Studies
have also indicated that aspects such as motivation, self discipline and self directed learning;
experience with ICT use and learning styles are facilitators of CEL (Kahiigi et al., 2009, Fahy and Ally,
2005).
Challenges for effective adoption and use of CEL have been realized in various contexts. These
challenges include unclear expectations from teachers, additional workload, slow access and
absence of synchronous communication (So and Brush, 2008). Cantoni et al. (2004) indicate that in
some instances technology and online environments can be frustrating, pointing to the lack of
technology skills among the students and teachers. Kahiigi et al. (2009) emphasize the limited
understanding of integrating technology and pedagogy as a major challenge in implementing CEL.
Koponen et al. (2011) view the lack of teacher training for modern educational technology, user
934

Evelyn Kigozi Kahiigi et al.
environment, culture and language as aspects that challenge adoption and use of CEL. Other studies
have reported scaffolding, group cohesion, lack of timely feedback, persistent technical problems,
time, resources and motivation as having a negative effect on CEL (Kirschner and Kreijns, 2004,
Lehtinen et al., 1999). However, worth noting is that appropriate design strategies can eliminate most
of the challenges, while without accurate and informed instructional design the benefits become
unachievable in any given context.
4. Case description and methodology
4.1 Case description
In 2002, Makerere University, Uganda embarked on an eLearning project with support from
development partners. This project was in line with the University policy of leveraging academic units’
effectiveness by using ICT in teaching, learning and research. The overall project goals were: a) to
improve the quality of graduates, by utilizing modern instructional materials and methods; b) to
provide greater access to university education, by developing capacity for increased enrolment
through non-conventional approaches to teaching and learning. Through this project, an eLearning
policy was formulated and approved; over 300 academic staff have been trained in online course
authoring; an end user training program to cater for computer literacy was instituted; and various
supporting ICT infrastructure and resources (which include: local area networks, computer labs,
internet kiosks and a wireless networks on campus grounds, etc) have been set up.
However, to date Makerere University is still struggling to attain the minimal educational benefits from
implementing eLearning. While the ICT environment to some extent is conducive for eLearning
development, there are currently partially developed online courses that are not used, and the
Makerere University ELearning Environment (MUELE) is mostly used as an information repository by
a few lecturers. A review of the academic sphere at Makerere University indicates that the issue is not
access but attitude towards adopting eLearning in the teaching and learning processes. For instance,
while the lecturers attended the training courses and appreciated eLearning as an alternative
approach, they were concerned about the increase in workload to support large student numbers;
increased course preparation time. In addition to this, there is the mind-set challenge, most of the
lecturers are used to the traditional teaching methods and view eLearning as a waste of time and
resources.
Conversely, the students at Makerere University frequently have face to face group meetings to
discuss their course material, work on class assignments and study for their tests and examinations
(Kahiigi et al., 2009). In addition students engage in “informal eLearning” which encompasses the use
of the internet, access to e-content and mailing lists. This has created an environment through which
significant learning has been realized. These findings indicate that transferring traditional interactions
into the online environment can present students with opportunities to enhance and support their
learning. The students interact and share knowledge through a wider online learning community. This
strategic approach can support Makerere University in achieving its eLearning implementation goals.
Adoption and use of CEL in the teaching and learning processes is therefore proposed as a research
agenda in this context.
4.2 Methodological approach
Development research was adopted for this study. The underlying phenomenon of development
research is based on solving real problems while at the same time constructing design principles that
can inform future decisions (Reeves, 2000). Development research involves interactions with
practitioners (in this case lecturers and students) to gradually clarify on both the problem within the
research context and the eventual solution (Van den Akker, 1999). According to Collins et al. (2004)
development research addresses several issues central to the study of learning, these include: a) the
need to address theoretical questions about the nature of learning in context; b) the need for
approaches to the study of learning phenomena in the real world rather than the laboratory; c) the
need to go beyond narrow measures of learning; and d) the need to derive research findings from
formative evaluation. These aspects form a basis of choice of development research as a method for
pursuing the aim of the study.
Although development research provides suitable means to facilitate the process of understanding
and developing an appropriate approach to CEL in this context; it also poses challenges which
935

Evelyn Kigozi Kahiigi et al.
include an immature methodology that has not been widely accepted in the research community;
generation of numerous unstructured data that requires a lot of time to analyse and report; complexity
arising from engaging teachers and students in the design process that may cause bias in the
research output (Wang and Hannafin, 2005). These challenges are crucial aspects to be considered
during the course of the study in order obtain justifiable results. This is so specifically since the
application of development research is limited in DCC.
The current study employed a process of inquiry between students, lecturers and the researcher,
encompassing the first stage of development research - analysis of practical problems by researchers
and practitioners (Reeves, 2000). The study was carried out in three stages as presented in Table 1.
The first stage of the study adopted a design approach that focused on introducing the research
concept, soliciting views on how the study was going to fit in the course structure. Through
involvement in the course design and consultative meetings with the lecturers, a CEL component was
developed and included within the course structures.
Table 1: Methodological approach
The second and third stage were based on an interpretive approach that involved observation of
learning activities, deriving perception about online vs. face to face interactions and application of the
CEL component. Data generated at this stage was derived from attending classroom sessions,
holding informal discussions with students and reviewing students’ activity in the MUELE. Constant
engagement with the students during the semester term was used to explore their perception about
online vis-à-vis the traditional interactions during their learning activities. In the third stage, qualitative
responses were solicited through a questionnaire that aimed to facilitating rich empirical data.
At the end of the course, an online questionnaire was disseminated to students however this resulted
into a 17 completed responses, which equated to 2% of the students in the study. The low response
rate can be attributed to lack of time as students were preparing for their examinations. It could also
be attributed to the limited access to computers facilities or to the fact the students are not
accustomed to the online questionnaire culture. These aspects raise concern, especially if online
learning methods are to be adopted and used in this context. As a result 400 questionnaires were
randomly distribution to the students of which 266 out of the 276 returned questionnaires were usable.
The questionnaire items covered the following general open ended questions: (a) What value do you
derive from collaborative eLearning? (b) What challenges do you encounter/would you encounter
when working in CEL environments? The responses were compiled and categorized into emerging
themes for analysis using Microsoft Office Excel 2007.
4.3 The case study
The data presented in this paper was generated from two cases of third year undergraduate students
pursuing the Bachelors of Information Technology (BIT) and Bachelor of Medicine and Surgery (BMS)
936

Evelyn Kigozi Kahiigi et al.
degrees at Makerere University, Uganda. The field research was undertaken between September
2010 and January 2011. The cases used in the study were selected through purposeful sampling to
derive information-rich cases for the study (Patton, 1990). The selection criterion was based on: a)
students using eLearning in their learning activities; b) and the will of the lecturer to participate and
drive students’ involvement in the study. Students perception of eLearning systems have been
attested to by several studies as one of the crucial elements in providing management with better
understanding of what is required for effective eLearning development and use (Yaghoubi et al.,
2008). Consequently there was a need to understand how students perceive and react to CEL since
their perception and attitudes are critical in motivating them as well as enhancing their learning.
The first group of students, 700 BIT students, studying Strategic Management as part of their degree
program was regularly exposed to the use of ICT and online resources incorporated in their learning
activities. The second group of 120 students undertaking BMS studying the Head and Neck Region
course were subjected to a problem based approach at the centre of their learning activities. To
achieve the course objectives, both students groups were expected to attend face to face lectures
and tutorial sessions and also to engage into discussion on various course issues largely through the
online environment.
The first activity aimed at introducing students to the online learning environment Moodle also referred
to as MUELE. In order to familiarize students with the MUELE, the students were given online quizzes
and also encouraged to make journal contributions about the course (see Figure 1). The online
quizzes and journals were used to assess the students’ level of understanding of the course
concepts.
Figure 1: Online quiz and journal in MUELE
The second activity aimed at introducing the CL approach. Students were divided into groups of 10.
Each group was supposed to complete the case study assignment using MUELE’s online discussion
forums see (see Figure 2). The groups then had to present their findings to the class using poster
presentations. At the end of the semester a questionnaire was disseminated in order to obtain
students perception and experience about the CEL approach in relation to the value they derived and
challenges faced.
5. Results and discussion
The study presented in this paper aimed to examine the current practices of adopting and using a
CEL approach to support student learning. Resulting from observations, open-ended questions and
informal discussions with students, issues relating to CEL emerged. The results presented and
discussed in this section relate to students value derived and challenges encountered in a CEL
environment.
5.1 Students background characteristics
The student background characteristics indicated a gender composition of 38% female and 62% male
distribution. 35% of the students were taking the BMS course, while students taking the BIT course
accounted for 65% of the students. In relation to knowledge of ICT and its use: 38% of the students
937

Evelyn Kigozi Kahiigi et al.
had very good knowledge, 30% had fairly good knowledge, 31% had fairly good knowledge, 23 % had
basic knowledge and 9% had limited knowledge. 74.4% of the students had access to a shared
computer and 98.1% of the students owned personal mobile phones.
Figure 2: Online discussion forum in MUELE
5.2 Value derived from collaborative eLearning
Table 2 lists students’ responses relating to derived value of adopting and using collaborative
eLearning. Students reported that they are able to improve their ICT skills, understanding course
concepts, access to various views and learning material, interact with fellow students, access course
information in a timely manner, share and gain knowledge and well as self evaluate and build
confidence in their learning process. These results affirm to the claims that CEL is an alternative
learning approach that support student learning (Zhu et al., 2009).
Access to various views and learning material recorded the highest occurrence. The learning
management system - MUELE was used as an information repository. Students had access to
various resource links and documents which supported their understanding of the course concepts in
addition to having timely access to relevant information.
In addition, students were actively involved in traditional discussion groups and thus viewed CEL as
an approach that would support and enhance their interaction. Indeed, Puckdeepun et al.(2010) affirm
that interaction is a key element in CEL through which shared understanding can promote learning.
During the group assignments, students carried out individual research and then collated information
for further discussions. This was an ideal case for the student as they would work independently at
their own pace and time. This facilitated knowledge sharing and learning as students had varying
views on a given concept.
Continued engagement with the online environment over the semester led to changes in students’
attitude towards using MUELE that resulted from improved ICT skills. This finding is supported by Doll
and Ahmed (1983) who established that user expectations change as they become more familiar with
the technology. Students also gain confidence with working in a CEL environment and were able to
self evaluate their contributions. Indeed Lehtinen et al.(1999) recognized that interaction between
participants forces them to consider their conceptions from the view point of others, thus facilitating a
growing awareness of one’s own knowledge and belief.
938

Evelyn Kigozi Kahiigi et al.
Table 2: Students responses relating to value derived from collaborative eLearning
5.3 Challenges encountered with collaborative eLearning
In relation to challenges encountered with adoption and use of CEL two categories emerged. These
were: technical and pedagogical issues (see Table 3). The technical issues included inadequate
bandwidth, inadequate Internet/computer access, lack of ICT skills, lack of support, and high cost.
Pedagogical issues included lack of timely feedback, lack of self esteem and trust, conflict resolution,
lack of face to face interaction and workload.
A typical approach adopted by students while working on their group assignments was to conduct
offline discussions in their face to face discussion sessions and then post their work in MUELE. This
resulted from the limited internet/computer assess that would have otherwise supported
asynchronous and in some cases synchronous discussions. Students also lacked adequate ICT skills
to support their engagement in the online environment, or access to, and manipulation of the
information in MUELE, thus resulting into the students’ apprehension in engaging in online learning
activities.
Although students viewed CEL as an approach to support their learning, it was difficult to see
student’s involvement in online activities during the observation period. This pointed to the fact that
CEL was a new learning approach that the students were not accustomed to and thus did not
understand. In addition students missed the face to face component, which constrained their ability to
express themselves in the online environment. The lack of immediacy of responses and interactive
cues during dialogues resulted in the online forums not to be effectively utilized. Reviewing the online
forums indicated that questions posed by the lecturer through MUELE hardly received any responses
and requests for contributions did not bring forth an increase in postings.
The study further established that different learning styles and levels of understanding of the course
concepts, triggered conflicts during student discussion. This led to low self esteem and lack of trust
among some of the students, thus affecting students’ online interactions. However, studies (Kong,
2006) have shown that positive constructive cognitive conflicts have promoted positive learning
outcomes. This then implies that if the conflicts within the students groups are geared towards
cognitive development, then students are given an opportunity to learn from their peers. In such
situations, teacher presence is crucial in supporting and sustaining the process of inquiry. Timely
feedback from lecturers is another aspect that challenged students’ involvement in CEL; this could be
explained by the large student numbers that overwhelmed the lecturers. In order to increase
effectiveness in this respect there is need to exploit the student to student interaction where
939

Evelyn Kigozi Kahiigi et al.
considerable learning can be achieved by encouraging collaborative activities and processes of
inquiry. The resultant effect can rationalize the lecturers’ workload, while the students receive timely
feedback to their queries.
Table 3: Students responses relating to challenges encountered from collaborative eLearning
Although the occurrence of cost and increased workload were low in this study, their impact on CEL in
this context is worth discussing. The cost for procuring computers is still high, making it difficult to
meet the growing demand. 74.4% of the student in the study had access to shared computers but
could not afford personal computers. Students have to rely on computer laboratory opening hours to
book for computer access time. However, campus grounds have wireless internet installed to support
students’ internet access 24/7. This then implies that there is need to institute student computer
purchase loan schemes or avail low cost computers that are students can afford. Computer/internet
access is crucial to drive CEL developments. 98.1% of the students in the study had personal mobile
phones. An implication could be to explore avenues that will support affordable mobile phone
pedagogies using basic functionalities to support CEL. In relation to perceived increased workload, it
was observed that students were used to receiving information from lecturers and are not encouraged
to getting involved in their learning process.
6. Conclusion
This paper aimed to explore and understand the current practices of CEL as an alternative learning
approach. More specifically, the paper investigates and discusses the value derived and challenges
encountered with the CEL approach in Uganda. In so doing the paper provides insight into both
research and practice on aspects that are particularly important when implementing CEL in a DCC.
This is so since delivery of innovative education that supports the development of cognitive skills, selfdirected
learning and research creates competitiveness in the global information society. The study
established that CEL is one of the approaches that can have a positive impact on student learning,
giving them opportunities to interact and share knowledge. However, it was evident from the study
that learning and teaching methods are predominately traditional, with limited/no integration of
eLearning. In addition there were inconsistencies in understanding the integration of technology into
teaching and learning processes. As a result students’ possibility or willingness to adopt and use CEL
was negatively affected. Future research therefore should be geared towards in-depth analysis and
elaboration of factors that facilitate a holistic online learning environment. In so doing the major
research question on how CEL can be effectively implemented to support students learning in a
developing country context should be pursued.
References
Arvaja, M., Salovaara, H., Häkkinen, P. & Järvelä, S. (2007) Combining individual and group-level perspectives
for studying collaborative knowledge construction in context. Learning and Instruction, 17, 448-459.
Bakari, J. K. (2005) Towards a Holistic Approach for Managing ICT Security in Developing Countries: A Case
Study of Tanzania. Stockholm University and Royal Institute of technology.
940

Evelyn Kigozi Kahiigi et al.
Cantoni, V., Cellario, M. & Porta, M. (2004) Perspectives and challenges in eLearning: towards natural interaction
paradigms. Journal of Visual Languages & Computing, 15, 333-345.
Collins, A., Joseph, D. & Bielaczyc, K. (2004) Design Research: Theoretical and Methodological Issues. The
Journal of the Learning Sciences, 13, 15-42.
De Wever, B., Schellens, T., Valcke, M. & Van Keer, H. (2006) Content analysis schemes to analyze transcripts
of online asynchronous discussion groups: A review. Computers & Education, 46 6-28.
Dillenbourg, P. (1999) What do you mean by collaborative learning? In P. Dillenbourg (Ed) CollaborativeLearning:
Cognitive and Computational Approaches., 1-19. Oxford: Elsevier.
Dillenbourg, P., Baker, M., Blaye, A. & O'Malley, C. (1996) The evolution of research on collaborative learning.
Learning in Humans and Machine: Towards an interdisciplinary learning science. Spada, E. & Reiman, P.
Doll, W. J. & Ahmed, M. U. (1983) Managing user expectations. Journal of Systems Management, 34, 6-11.
Engelbrecht, E. (2003) A look at eLearning models: Investigating their value for developing an eLearning
strategy. Progressio, 25, 38-47.
Fahy, P. J. & Ally, M. (2005) Student Learning Style and Asynchronous Computer-Mediated Conferencing (CMC)
Interaction. The American Journal of Distance Education, 19, 5-22.
Garrison, D. R. (2007) Online community of inquiry review: Social, cognitive and teaching presence issues.
Journal of Asynchronous Learning Networks, 11, 61-72.
Iahad, N., Dafoulas, G. A., Milankovic-Atkinson, M. & Murphy, A. (2004) ELearning in Developing Countries:
Suggesting a Methodology for Enabling Computer-Aided Assessment. Advanced Learning Technologies,
IEEE International Conference on. Los Alamitos, CA, USA, IEEE Computer Society.
Kahiigi, K. E., Danielson, M., Hansson, H., Ekenberg, L. & Tusubira, F. F. (2009) An Empirical Investigation of
Students’ Perceptions of ELearning in Uganda. IN CUNNINGHAM, P. C. A. M. (Ed.) IST-Africa 2009
Conference Proceedings. Kampala, Uganda.
Kamba, M. A. (2009) Information and Knowledge Transfer: the Impact of Collaborative Learning (CL) in
Enhancing Students Academic Performance in Nigerian Universities. IEEE Multidisciplinary Engineering
Education Magazine, 4.
Kirschner, P. A. (2001) Using integrated electronic environments for collaborative teaching/learning. Research
Dialogue in Learning and Instruction 2, 1-9.
Kirschner, P. A. & Kreijns, K. (2004) The sociability of computer-mediated collaborative learning environments:
Pitfalls of social interaction and how to avoid them. In R. Bromme, F. Hesse, & H. Spada (Eds.). Barriers
and biases in computer-mediated knowledge communication-And how they may be overcome. Dordrecht,
NL: Kluwer.
Kong, S. C. (2006) The Design of a Collaborative Learning Environment in a Mobile Technology Supported
Classroom: Concept of Fraction Equivalence. Proceeding of the 2006 conference on Learning by Effective
Utilization of Technologies: Facilitating Intercultural Understanding Amsterdam, The Netherlands, IOS
Press.
Koponen, T., Tedre, M. & Vesisenaho, M. (2011) An Analysis of the state and prospects of eLearning in
developing countries. IN CUNNINGHAM, P. C. A. M. (Ed.) IST Africa 2011. Gaborone, Botswana.
Kreijns, K., Kirschner, P., A & Jochems, W. (2003) Identifying the Pitfalls for Social Interaction In Computer-
Supported Collaborative Learning Environments: A Review of the Research. Computers in Human
Behavior, 19, 335 - 353.
Lehtinen, E., Hakkarainen, K., Lipponen, L., Rahikainen, M. & Muukkonen, H. (1999) Computer supported
collaborative learning: A review. CL-Net Project.
Liaw, S. S. & Huang, H. M. (2006) Investigating Learner Attitudes toward a Collaborative ELearning System.
Proceedings of the 10th International Conference on Computer Supported Cooperative Work in Design.
Nanjing
Lipponen, L., Hakkarainen, K. & Paavola, S. (2004) Practices and orientations of CSCL. In J.W. Strijbos, P.
Kirschner & R. Martens (Eds.), What we know about CSCL: And implementing it in higher education. Kluwer
Academic Publishers, 3, 31-50.
Östlund, B. (2008) Interaction and Collaborative Learning: If, Why and How? European Journal of Open,
Distance and ELearning, 1-9.
Panitz, T. (1996) Collaborative versus cooperative learning.
Piaget, J. (1926) The language and thoughts of the child, New York, Harcourt.
Prosser, M. & Trigwell, K. (1999) Understanding learning and teaching. , Buckingham, Society for Research into
Higher Education and Open University Press.
Puckdeepun, T., Jaafar, J., Hassan, M. F. & Hussin, F. A. (2010) Investigating collaborative interaction using
interactive table and IR devices. International Conference on User Science Engineering (i-USEr). Shah
Alam, Malaysia.
Reeves, T. C. (2000) Enhancing the Worth of Instructional Technology Research through "Design Experiments"
and Other Development Research Strategies. Session 41.29, Annual Meeting of the American Educational
Research Association, New Orleans, LA.
Roschelle, J. & Teasley, S. D. (1995) The Construction of shared knowledge in collaborative problem solving. In
C. O’Malley, ed. C. O’Malley (Ed.). Computer-Supported Collaborative Learning, 67-97. Berlin Springer-
Verlag.
So, H. J. & Bonk, C. (2010) Examining the Roles of Blended Learning Approaches in Computer- Supported
Collaborative Learning (CSCL) Environments: A Delphi Study. Educational Technology & Society, 13, 189-
200.
941

Evelyn Kigozi Kahiigi et al.
So, H. J. & Brush, T. A. (2008) Student perceptions of collaborative learning, social presence and satisfaction in a
blended learning environment: Relationships and critical factors. Computers & Education, 51, 318-336.
Thompson, T. L. & MacDonald, C. J. (2005) Community building, emergent design and expecting the
unexpected: Creating a quality eLearning experience. The Internet and Higher Education, 8, 233 - 249.
Van den Akker, J. (1999) Principles and methods of development research
In J. Van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, T. Plomp (Eds.) . The Netherlands: Kluwer
Academic Publishers.1-14.
Vesisenaho, M. (2009) Developing contextualized ICT Education. Case Tumaini University, Tanzania. Lap
Lambert Academic Publishing.
Vygotsky, L. (1978) Mind in Society: The development of Higher psychological process, Cambridge, Havard
University Press.
Wang, F. & Hannafin, M. J. (2005) Design-based research and technology-enhanced learning environments.
Educational Technology Research and Development, 53, 5-23.
Zhu, C., Valcke, M., Schellens, T. & Li, Y. (2009) Chinese Students' Perceptions of a Collaborative ELearning
Environment and Factors Affecting Their Performance: Implementing a Flemish ELearning Course in a
Chinese Educational Context. Asia Pacific Education Review, 10, 225-235.
Zurita, L. & Ryberg, T. (2005) Towards a Collaborative Approach of Introducing ELearning in Higher Education
Institutions: How do University Teachers Conceive and React to Transitions to ELearning. IFIP (The
International Federation for Information Processing).
942

Applying the Multimedia Learning Theory in the Primary
School: An Experimental Study About Learning Settings
Using Digital Science Contents
Fabio Serenelli 1 , Enrico Ruggeri 1 , Andrea Mangiatordi 2 and Paolo Ferri 2
1
CPM - Università degli Studi di Milano-Bicocca, Viale dell'Innovazione, Italy
2
nuMedia Bios – Università degli Studi di Milano-Bicocca, Italy
fabio.serenelli@unimib.it
enrico.ruggeri@unimib.it
andrea.mangiatordi@unimib.it
paolo.ferri@unimib.it
Abstract: The introduction of digital technologies in educational contexts led to eLearning contents proliferation,
thanks to their cost-cutting production and to the social networks that are favoring their widespread circulation.
These objects are often the collaboration results between graphic designers and subject-matter experts
disregarding the empirical evidence of Instructional Design research. Moreover they are frequently introduced to
the classes without appropriate delivering strategies. The purpose of this paper is to describe an experimental
study we ran in 2010 in Uruguay to identify effective learning object formats and adequate conditions for using
multimedia contents with kids in “real world” learning contexts. Uruguay is part of the OLPC (One Laptop Per
Child) initiative that aims to distribute low cost laptop PCs (called XOs) in developing countries schools to foster
kids' learning according to the instructional principles of constructionism, learning-by-doing and social
constructivism theories. This country is the only one that reached the "saturation" goal, covering the entire
primary school population (teachers included). For this reason the capital Montevideo was an appropriate “en
plein air” research field because most of the students haven’t evident impairments using educational technologies
and digital learning contents. In order to find out how to reduce cognitive load and increase learning
performances using infographics, animations and interactivity, we arranged an experimental study that involved
360 early adolescents from 16 classes of critical context schools in Montevideo. We identified a scientific topic,
the food chain process, and presented it in 4 different ways. We modified supports and instructional formats
according to Mayer’s ‘Multimedia Learning Principles’ and the ‘First Principles of Instruction’ theory by Merrill. The
first part of the research focused on Self-Directed Learning in real contexts and investigated the use of different
instructional strategies (e.g. topic-centered vs. task-centered; linear vs game-based) handling the learner’s User
Experience in order to increase the engagement for the proposed formats. Considering the target of early
adolescents, we introduced a likeable virtual tutor to manage explanations, feedbacks, and focus on relevant
information. We used infographic techniques to combine analytical and synthetic schemas and to enhance the
aesthetic perception. The second part of the study aimed to identify the best use of multimedia contents in
classrooms comparing 3 learning settings: Self-Directed Learning, Cooperative Learning and Teacher-Directed
Learning. In order to measure the impact of content design format and learning setting we identified 5
instructional objectives using a Content/Performance Matrix, and evaluated the outcomes by 4 kinds of tests:
Retention, Comprehension, Problem Solving Transfer, Delayed Problem Solving Transfer (after one week). We
also used a qualitative tool: a self-administered questionnaire for the User Experience satisfaction, to discover
relationships between students performances and individual preferences matched with satisfactory learning
experiences.
Keywords: multimedia learning theory, critical context, learning settings, instructional content design, user
experience, graphic organizers
1. Introduction
eLearning vision is still focused on trainers' needs and e-teaching processes rather than on the
invoked learner centered approach. In the last decade, a lot of improvements have been implemented
in eLearning environments crossbreeding Learning Management System and Virtual Learning
Environment with Social Networks; they actually offer a rich range of features based on constructivist
and socio-constructionist principles. Unfortunately, teachers and trainers often rely only on the basic
functionalities of these complex tools in order to easily replicate the offline formal education
experience. Most of these environments give the opportunity to run and manage eLearning contents
as Learning Objects. Moreover these digital materials are often the collaboration results between
graphic designers and subject-matter experts disregarding the empirical evidence of Instructional
Design research so they are frequently introduced to the classes without appropriate delivering
strategies.This paper presents the experimental study we ran in 2010 in order to test learning object
formats with kids in “real world” learning contexts. The main goal of the study is to find out effective
ways to use infographics and interactivity for digital scientific contents to reduce cognitive load and
balance the learning performance levels with the quality of the studying experience. The second goal
943

Fabio Serenelli et al.
is to identify the best use of eLearning contents in classrooms comparing 3 different learning settings:
Self-Directed Learning, Cooperative Learning and Teacher-Directed Learning.
2. Research field and participants
To compare digital and analogical learning object effectiveness, we had to find a sample of students
that hadn’t impairments in using personal computers. We chose Uruguay because of its unique public
educational system that aims to provide digital learning skills to the primary school students by the
implementation of the One Laptop Per Child (OLPC) program. The OLPC project started in 2005 at
the MIT Media-Lab in Boston (USA) by the founders Nicholas Negroponte and Seymour Papert,
whose intentions were to develop a low cost (price lower than 100$) laptop pc model called XO and to
distribute it in the schools of developing countries. These models are made to boost children's
learning according to instructional principles of constructionism, learning-by-doing and social
constructivism theories. Uruguay is an “en plein air” research laboratory with a vast scholar sample:
510.000 XO personal computers have been distributed to primary school children and teachers.
This experimental study aims to evaluate the impact of content design on the achievement of specific
instructional goals by developing and testing 3 analogical and digital Learning Objects (LOs) to
assess how information display methods and interactivity levels can affect learning in early
adolescents. The population sample is composed of 360 subjects from 16 classes of the capital
Montevideo. The selected students (11-14 years old), according to the teachers reports had not
significant previous knowledge of the instructional content we delivered.
3. Theoretical background and research design
In order to discover which LO format promotes meaningful learning processes in early adolescent
students, we opted for a theoretical framework based on the Cognitive Theory of Multimedia Learning
(ML) by R. E. Mayer and First Principles of Instruction (FPI) by D. Merrill.
16 teachers were involved as subject matter experts in a participatory design process using interviews
and focus groups to identify a scientific content that were part of the official school program but was
not previously treated in classes. We chose the food chain process as main topic including - as
prerequisites - the animal diet and the trophic levels.
We defined learning objectives in conformity with the type of knowledge and the performance level
expected by the learner. We evaluated different instructional design tools including the classic and the
revised Bloom’s taxonomy and the Merrill’s Content/Performance Matrix (CPM). We created a new
hybrid matrix (see Table 1) based on both the original and the Clark’s modified version of the CPM.
The types of information provided in the materials were defined as: Facts, bits of information that
identify specific items (ex. lion); Concepts, a category of objects or ideas that share common features
and have a common name (ex. carnivores include lions, hawks and sharks); Processes: concern how
a system works and operates and how its parts are interrelated (ex. the food chain process). The
adopted levels of cognitive performance are: Remember, the learner memorizes or recognizes the
information provided; Apply, the learner uses new information to solve a problem or perform a task;
Find, the learner creates a new schema through reorganization of the concepts.
Table 1: Hybrid content/performance matrix
REMEMBER
FACTS CONCEPTS PROCESSES
Remember specific data
and facts.
APPLY -
Remember and understand
definitions
Apply and transfer the concepts
classifying new, unknown
examples
FIND - -
944
-
Solve a problem by
relating the acquired
concepts in the right way
Reorganize existing
concepts to derive a new
abstract scheme

Fabio Serenelli et al.
4. Multimedia materials development and learning settings
4.1 Self directed learning (setting A)
The first part of the research focused on Self-Directed Learning in real contexts, and investigated the
use of different instructional strategies (e.g. Topic-Centered Vs Task-Centered; Linear Vs Game-
Based) handling the learner’s User Experience in order to increase the engagement for the proposed
formats. Considering the target of early adolescents, we introduced a likeable virtual tutor to manage
explanations, feedbacks, and focus on relevant information. We used infographic techniques to
combine analytical and synthetic schemas and to increase the aesthetic satisfaction.
The content design phase was implemented by a small team that produced three multimedia
instructional formats for Self Directed Learning based on to the performance outcomes isolated with
the teacher’s help.
LO1 – Infographic Resume: this analogical learning object was printed on A4, full-colour sheets.
It displays the information by using graphic organizers to visually explain concepts and
connections (see Figure 1).
Figure 1: Examples from LO1 – Infographic Resume
LO2 - Step by Step Multimedia: (OLPC Laptop support) this multimedia explanation consists of
a digital step by step animation synchronized with the pedagogical agent narration (see Figure 2).
Audio track of a real voice (in earphones) substitutes most of the texts. The learner manages the
linear animation progress.
Figure 2: Screenshots of LO2 – Step by Step Multimedia
LO3 – Learning-Game: (OLPC Laptop support) similar to a videogame (see Figure 3), the
learner (with earphones) plays with the information, experiencing the causality effects of his/her
actions.
945

Figure 3: Screenshots of LO3 – Learning-Game
Fabio Serenelli et al.
Each of the LOs displays the same information set and uses the same virtual pedagogical agent (the
character named Prof. Haragan, see Figure 1,2,3,) to introduce relevant concepts and processes. A
synthetic comparison is presented in Table 2.
The three learning object formats vary in:
Instructional Media: printed material Vs digital content.
Instructional Strategy: Topic Centered Vs Problem Centered learning.
Instructional Sequence: information straight from the virtual tutor Vs provided as a positive
feedback.
LO3 is the more complex learning object and consists of a sequence of 3 Learning-Games based on
Merrill’s Task Centered Principle (see Figure 4): “…performance is enhanced when learners
undertake a simple-to-complex progression of whole tasks (T) with a corresponding decreasing
amount of learner guidance and coaching…”.
Figure 4: Task-Centered Instructional Strategy and fading guidance in the interactions in LO3
The first part of each interaction is the activation phase in which the learner is invited to apply relevant
cognitive structures by making inferences on the solutions; considering that learners have limited prior
information, the instruction provides relevant challenging experience that can be used as a foundation
for the new knowledge.
As in LO2, follows the demonstration phase where the tutor synthesizes the topic chunk with
multimedia messages using synchronized audio, animated diagrams and infographic resumes. Most
of the texts are replaced by the human voice of the animated character.
Game Mechanics: the virtual tutor in LO3 invites the learner to become a young scientist passing
through activities related to the main topics. The flow of the problem-centered game is conditioned by
the right solution of previous tasks; the virtual tutor delivers objects and symbolic rewards when the
learner passes to the next level (notepad, camera, camcorder, lab alb, final certificate and medal).
Feedbacks: in LO3 before every activity, the virtual tutor introduces the topic and the rules,
challenging the student to solve the problem without previous information. During the interaction, the
tutor provides minimal negative feedback in order to guide toward the right answer, and uses
significant positive feedback when the learner completes part of the task. The real reward offered by
946

Fabio Serenelli et al.
the game is the acquisition of new information about the topic. The progression is conditioned by the
engagement of the student in solving the problem and creating the knowledge model.
Table 2: Self Directed Learning setting
LO FORMAT LO1 – INFOGRAPHIC
RESUME
INSTRUCTIONAL
MEDIA AND
INFORMATION
DISPLAY
INSTRUCTIONAL
STRATEGY AND
CONTENTS
SEQUENCE
INTERACTIONS
AND LEARNER
CONTROL
A collection of graphic
organizers integrates
different media in simple
diagrams: text, images,
symbols and schemas.
The information is
presented with a topiccentered
approach in a
linear way using a comic
style format that provides
contextual comments and
captions for the diagrams.
The learner is invited to a
step by step construction of
the food chain model
through a self directed
learning strategy without
external feedback or
guidance.
Learner is allowed to
review topics and selfcontrol
the pace of the
studying.
SELF-DIRECTED LEARNING (Setting A)
LO2 – STEP BY STEP
MULTIMEDIA
Step by step graphic
organizers are presented
on a screen integrating, in
an animation, text,
images, symbols,
schemas and audio.
The information is
presented with a topiccentered
approach in a
linear way. Comments are
provided in a cartoon
style by the virtual tutor
narration that provides
contextual comments and
captions for the diagrams.
The learner is invited to a
step by step construction
of the food chain model
through a self-directed
learning strategy
supported by a constant
guidance.
Learner is allowed to review
topics and control the pace
of the narrated animation.
LO3 – LEARNING-
GAME
Interactive graphic
organizers are
presented on a screen
integrating, in an
animation, text,
images, symbols,
schemas and audio..
The information is
presented with a taskcentered
approach by
the virtual tutor narration
in a cartoon style. He
provides contextual
comments, negative and
positive feedbacks.
The implemented
Learning-Games consist
of two classification
activities for animal diet
and trophic levels topics
and three rollover/drag
n’ drop exercises for
active construction of
food chains. Learner is
allowed to review topics,
control the pace of the
narrated animation and
perform the problems
tasks several times.
According to the LO formats, we defined the ML principles to be respected as follow in Table 3.
Table 3: Multimedia Learning principles and LOs formats
LOs formats
Goal Multimedia Learning LO1
LO2
LO3
Principles
Infographic Step by Step Learning
Resume Multimedia
game
Reducing
Coherence
Extraneous
Signaling
Processing
Redundancy
Spatial contiguity
Temporal Contiguity
947

Managing Essential
Processing
Fostering
Generative
Processing
Fabio Serenelli et al.
Segmenting
Pre-training
Modality
LOs formats
Multimedia
Personalization
Voice
Image
Interactivity
4.2 Cooperative Learning Setting (B) and Teacher-Directed Learning Setting (C)
The second part of the study aimed to identify the best use of multimedia contents in classroom
comparing 3 learning settings: Self-Directed Learning (A), Cooperative Learning (B) and Teacher-
Directed Learning (C) as summarized in Table 4.
Table 4: Cooperative Learning and Teacher-Directed Learning- For Setting B we decided to deliver
the LO3 type (Learning-Game) but used in pair. The dyads share the same XO laptop and a
earphone kit
COOPERATIVE LEARNING
(Setting B)
TEACHER-DIRECTED LEARNING (Setting C)
LO DESCRIPTION LO3 - Learning-Game in dyad LO4 – Slideshow
INSTRUCTIONAL
STRATEGY AND
CONTENTS
SEQUENCE
INTERACTIONS AND
LEARNER CONTROL
Both the subjects are invited to
pay attention to the main
concepts and to discuss and
negotiate with the mate: learning
timing, game strategies, problem
solving techniques and tutor
feedback.
Learner-Computer Interactions
are the same as in the LO3/Self-
Directed Learning setting adding
the keyboard and mouse sharing
during the learning experience.
The teacher uses a powerpoint presentation
from the basic concepts to the whole infographic
scheme. The slideshow is projected in the
classroom (1,5mx1m). He/She asks the kids to
read together the texts, presents examples of
categories, stimulates the group to name the
right organisms and then summarizes the topic.
The progress is managed by the teacher; after a
rapid check of the main concepts
comprehension he just goes on with the next
instructional chunk. Learners interact with the
teacher answering to stimulations, asking for
repetitions and extra explanations.
In Setting C we used a PowerPoint presentation based on LO2 with the same linear topic centered
strategy. The digital step by step presentation uses the same diagrams as in LO1/LO2/LO3 and the
audio track is replaced by an on-screen text collectively read by the teacher and the class.
5. Evaluation
In order to measure the impact of content design format and learning setting we identified 5
instructional objectives using a content/performance matrix (Table 1), and evaluated the outcomes
using 4 tests: Retention, Comprehension, Problem Solving Transfer, Delayed Problem Solving
Transfer (the same problem solving test evaluates one week later the ability to apply the food chain
model learnt with the LOs).
Table 5: Retention Test - checks the ability to remember the most important parts of the material
TYPE OF
FACTS CONCEPTS
KNOWLEDGE
TASK REMEMBER REMEMBER
LEARNING
OBJECTIVE
Remember specific data
and facts
Remember and understand
definitions
948
Remember and understand
definitions

DESCRIPTION
TEST
Recall the names of the
animals previously shown
in the learning material
OPEN QUESTION: lists
all of the animals that you
remember presented in
the lesson
Fabio Serenelli et al.
Animal Diet: define the
meaning of these classes:
herbivores / omnivores /
carnivores; producers /
consumers / decomposers
MULTIPLE ANSWER: 15
true/false questions
Trophic Levels: define the
meaning of these classes:
producers / consumers /
decomposers
MULTIPLE ANSWER: 15
true/false questions
Table 6: Comprehension transfer test - aims to measure the ability of classifying new examples and
the comprehension levels of the main concepts presented
TYPE OF
KNOWLEDGE
CONCEPTS
TASK APPLY
LEARNING
OBJECTIVE
DESCRIPTION
TEST
Apply and transfer the concepts
classifying new, unknown examples
Classify animals and other living beings
belonging to unknown ecosystems
applying new concepts (Animal Diet)
CLASSIFICATION: “write the names of
animals in the correct category in the
Table below”
Apply and transfer the concepts classifying
new, unknown examples
Classify animals and other living beings
belonging to unknown ecosystems
applying new concepts (Trophic Levels)
CLASSIFICATION: “write the names of
animals in the correct category in the
Table below”
Table 7: Problem Solving Transfer test - asks the student to transfer the food chain model to unknown
environmental contexts performing a few problem solving tasks
TYPE OF
KNOWLEDGE
PROCESSES
TASK APPLY FIND
LEARNING Solve a problem by relating the acquired Reorganize existing concepts to derive a
OBJECTIVE
concepts in the right way
new abstract scheme
Food Chain: use concepts of animal diet Food Chain: represent the food chain
DESCRIPTION and trophic levels for the construction of
new food chains
process in a new abstract scheme
Build a food chain from the following Complete the scheme of a food chain,
TEST organisms: “for each element/link insert matching abstract concepts for animal diet
the correct name”
and trophic levels
At the end of the test session we also administered a User Experience (UX) Questionnaire - a
qualitative tool built to discover relationships between students preferences, satisfactory learning
experiences and learning achievements. Its items are grouped and categorized in:
Estimated prior knowledge
Instructional material clarity
Instructional material usability
Self-efficacy perception after the learning experience
Engagement and motivation related to the LO and the setting
949

6. Results and analysis
Fabio Serenelli et al.
Here we present some results, even if the analysis phase is still ongoing. In fact we still have to run
the evaluation of the UX questionnaires, and the correlation between learning performances and
students engagement. In our further analysis we’ll proceed splitting the sample into two age groups:
11-12 years old subjects “pre-adolescents” attending the 6th grade of primary school; and 13-14
years old “adolescents” attending the 6th grade of primary school (that means that these pupils has
been repeating one or two years during their school curriculum). Hereinafter some preliminary
considerations are following to comment the results of the two main tests (Comprehension Test and
Problem Solving Test; Transfer Test provides their mean general measure). We decided not to
consider the Retention Test (pr=0.7629; see Table 8***) and the comparison between Problem
Solving Test and the Problem Solving Delayed Test.
Table 8: Analisys of variance
Df Sum Sq Mean Sq F value Pr(>F)
Retention 4 0.1138 0.028458 2.1348 0.07629***
Comprehension 4 0.550 0.13750 3.5667 0.007286
Problem_Solving 4 1.4106 0.35264 4.8926 0.0007658
Transfer 4 0.7045 0.17612 4.0526 0.003207
Table 9: Performance averages relating to the setting and to the LO
CS1=
CS2=
CS3=
CS4=
CS5=
LO1/SET-A LO2/SET-A LO3/SET-A LO3/SET-B LO4/SET-C
Comprehension 0,44 0,530 0,535 0,475 0,442
Problem_Solving 0,365 0,562 0,467 0,513 0,50
Transfer 0,403 0,546 0,501 0,494 0,473
Figure 5: Average performance for each experimental condition (CS)
Comprehension Test: performances on concepts understanding tasks and concepts application tasks
are higher in Self-Directed Learning using the laptop (Setting A), regardless of LO formats. In fact the
results between the Learning-Game and the Multimedia Step by Step (LO2 and LO3) are comparable.
Probably the use of the laptop with earphones allows a strong reduction from distractions in the
classroom and increases the focus on relevant information. Both the Teacher-Directed (LO4/Setting
C) and the Infographic Resume (LO1/Setting A) conditions are not recommended to study the food
950

Fabio Serenelli et al.
chain. Contrary to initial assumptions, the Cooperative Setting in dyads doesn’t seem to improve the
understanding of the concepts rather than the Self-directed Setting with Learning-Game (LO3/Setting
A vs B).
Figure 6: Average performance on each test
Problem Solving Test: the paper version of the material (LO1) is definitely not recommended for this
kind of tasks, while LO2 - Multimedia Step by Step seems to be the best solution to improve the
Problem Solving skill (learning objectives are: “solve problems transferring the knowledge in new
context” and “conceptual model abstraction”).
Moreover it's interesting to point out that the Learning-Game (LO3) gives significantly better results
adopting a cooperative learning strategy (Setting B) than a self directed one (A). This outcome could
result from gains in understanding through discussion, or from peer influence of more skilled students
on the others.
Transfer Test: (average of Comprehension Test and Problem Solving Test results) offers us a
task/performance synthesis of the five experimental conditions in relation to the types of knowledge
identified with the CPM. Contrary to initial assumptions – it seems that neither the physical presence
of the teacher (Setting C) nor the cooperative learning (Setting B) favor benefits on learning;
otherwise the best degree of effectiveness seems to be achieved choosing the Self-Directed Learning
(Setting A) combined with a linear, strongly driven multimedia content as LO2 - Multimedia Step-by-
Step that limits interactivity to the control of learning pace. The LO3 - Learning-Game format however
performs quite well without any difference if used individually or in pairs.
References
Beilin, H. (1992) Piaget's enduring contribution to developmental psychology. Developmental Psychology, 28, pp
191-204.
Calvani, A. (2009) Teorie dell’istruzione e carico cognitivo, Trento, Erickson.
Clark, R. C. (2007) Developing technical training: a structured approach for developing ... (p 269). John Wiley
and Sons.
Clark, R. C. and Mayer, R. E. (2003) ELearning and the Science of Instruction. Jossey-Bass Pfeiffer, San
Francisco.
Clark, R. E. (2003) "What Works in Distance Learning: Instructional Strategies". In O'Neil, H.F. (Ed.), What Works
In Distance Learning (p 26). Los Angeles: Center for the Study of Evaluation.
Mammarella, N., Cornoldi, C. and Pazzaglia, F. (2005). Psicologia dell’apprendimento multimediale. Il Mulino.
951

Fabio Serenelli et al.
Mangiatordi, A., Pischetola, M. (2010) "Sustainable Innovation Strategies in Education: OLPC Case Studies in
Ethiopia and Uruguay". IM. Lytras et al., Organizational, Business, and Technological Aspects of the
Knowledge Society, pages 94-104, Springer.
Mayer, R. E. (2005) "Cognitive Theory of Multimedia Learning". In Mayer. R.E. (Ed.), The Cambridge handbook
of multimedia learning (pp 31–48). Cambridge University Press.
Mayer, R. E. (2009) Multimedia learning (2nd ed), New York, Cambridge University Press.
Mayer, R. E. and Chandler, P. (2001) "When learning is just a click away: Does simple user interaction foster
deeper understanding of multimedia messages?", Journal of Educational Psychology, Vol 93(2), Jun, 390-
397.
Merrill, M. D. (1994) Instructional Design Theory. course syllabus (p 484). Educational Technology Publication.
Merrill, M. D. (2002) A pebble-in-the-pond model for instructional design. Performance Improvement. 41(7), pp
39-44.
Merrill, M. D. (2007) First principles of instruction: a synthesis. In Trends and Issues in Instructional Design and
Technology, 2nd Edition. R. A. Reiser and J. V. Dempsey. Upper Saddle River, NJ, Merrill/Prentice Hall. 2,
pp. 62-7
Merrill, M. D. (2009) "First Principles of Instruction". In Reigeluth, C.M. and Carr-Chellman, A. A. (Eds.),
Instructional design Theories And Models Vol 3 Building A Common Knowledge Base (Vol. III, pp. 1-19).
Rutledge.
Mosley, P. (2005). A taxonomy for learning object technology. British Journal of Educational Technology, 36(3),
pp 702-703.
Paivio, A. (1971) Imagery and verbal processes, New York, Holt, Rinehart, and Winston.
Serenelli, F. and Mangiatordi, A. (2010) "The 'One Laptop Per Child' XO laptop as a PLE: a cognitive artifact
beyond hardware and software", PLE Conference proceedings, Cornellá, Barcelona.
Sweller, J. (1988) Cognitive load during problem solving: Effects on learning, Cognitive Science 12 (2), pp 257–
285
952

Designing a U-Learning Course Platform for the Identified
Teacher Training Needs
Nazime Tuncay and Hüseyin Uzunboylu,
Near East University, Nicosia, North Cyprus
nazime.tuncay@gmail.com
huzunboylu@neu.edu.tr
Abstract: Teacher T raining has und ergone m ajor c hanges i n r ecent y ears, w ith t he dev elopment of di gital
information t ransfer, v irtual learning environments(VLE) and s imulations i ntegrated i n learning management
systems having a significant effect on it. Online resources like Edu 2.0 and Second Life (SL) encourage a sense
of c ollaboration and community w hich now needs to b e embraced t hrough t eaching and l earning pr actice.
Researchers, as a r esult of 4 years research study of global e-learning training needs in 25 countries analyzed
the needs of the teachers’ e-education requirements and their preferred e-learner environments. Although, there
were some common needs like “knowledge and skill of preparing e-quizzes”, they found that there was a divide
among teachers’ s kills and knowledge’s. Hence, t he n ecessary s teps t o d evelop proper I CT i nfrastructure
required f or e -Learning are taken. As a r esult of al l t he f indings in t he literature, courses ar e d ecided t o be
delivered to bridge these. A global strategy is decided for moving towards and encouraging teachers towards
deploying e-Learning for their courses. Syllabus is prepared, online evaluation is required and suggestions are
taken from experts coming from different walks of life, from 25 countries. This process ended in 4 months. After
having rewritten the syllabus, course material is prepared and carefully proof read by English teachers and eeducation
experts. The course environment is used by 51 experts and is redesigned by the researchers
according to their suggestions. After 6 months of work, the last form of VirtualNeu (Virtual Near East University
Portal) is published. Encouragement and pr oper incentive t o English Teachers to par ticipate i n the e -training
programs were provided by the ministry of education in North Cyprus. 80 Teachers have registered to the system
and courses will start on 7 September 2011. Researchers mission for the next two years, 4 years and beyond is
clear and unequivocal; like the last four years: Preparing a portal and making it universally accessible and useful
for teachers; delivering e-courses for their specified training needs and, helping them to be more innovative and
more creative individuals who change and improve the world education.
Keywords: Edu 2.0, Second Life, U-education, VirtualNeu, Training Needs of Teachers
1. Introduction
1.1 U-Learning (Ubiquitous Learning)
U-Learning is becoming a necessity of the modern education system. The ubiquitous learning
environment provides an interoperable, pervasive, and seamless learning architecture to connect,
integrate, and share three major dimensions of learning resources: learning collaborators, learning
contents, and learning services (Chang&Shu,2005, Haruo, Kiyoharu, etc.,2003, Cheng, etc., 2005).
The main characteristics of ubiquitous learning are (Chen et al., 2002; Curtis et al., 2002):
Permanency: Learners can never lose their work unless it is purposefully deleted. In addition, all
the learning processes are recorded continuously in every day.
Accessibility: Learners ha ve ac cess t o their d ocuments, da ta, or videos f rom anywhere. T hat
information is provided based on their requests. Therefore, the learning involved is self-directed.
Immediacy: Wherever learners are, they can get any information immediately. Therefore learners
can s olve pr oblems qui ckly. O therwise, t he learner m ay r ecord t he qu estions and look f or t he
answer later.
Interactivity: Learners can interact with experts, teachers, or peers in the form of synchronies or
asynchronous c ommunication. H ence, the ex perts are m ore r eachable and t he k nowledge i s
more available.
Ubiquitous learning is characterized by providing intuitive ways for identifying right collaborators,
right c ontents an d r ight s ervices i n t he r ight pl ace at t he r ight t ime bas ed on s tudents s urrounding
context such as where and when the students are (time and space), what the learning resources and
services av ailable f or the s tudents, and who ar e the l earning c ollaborators that match the s tudents’
needs (Zhang, etc., 2004.; Ogata, etc., 2005; Takahata, etc.; 2004)
953

1.2 VLE (Virtual Learning Environments)
Nazime Tuncay and Hüseyin Uzunboylu
VLE is an important part of websites. When considering websites, a set of web pages does not
constitute a virtual learning environment unless there is social interaction about or around the
information (Dillenbourg, 2000). Based on research findings, Warburton (2009) explained that the
immersive nature of the virtual world, crossing physical, social and cultural dimensions, can provide a
compelling educational experience, particularly in relation to simulation and role-playing activities. For
these teachers, Virtual Learning Environments can be frightening places in which their expertise is
exposed to challenge and students can assume greater power and control over what is taught, how
and when (Greener 2008; Greener 2008). Virtual agents take an important place in virtual
environments. Virtual agents provide an alternative method for supporting navigation in virtual
environments (Ibanez& Delgado-Mata, 2011). For instance, virtual animals can give directions or
transport users to locations (van Dijk et al., 2001). Characters can also orient the user, point to
interesting objects and automatically direct their view (Wernert and Hanson, 1999). Characters can
provide information to the user by either acting as presenters (Nijholt, 2006) or storytellers ( El-Nasr
et. al., 2008).
1.3 SL (Second Life)
SL is being used quite often in VLE in the recent years. Research on educational uses and potential
of v irtual w orlds of ten f ocuses on t heir a bility t o s upport c ollaborative, c reative, par ticipatory a nd
exploratory f orms o f t eaching a nd l earning, as hi ghlighted i n m ost papers of t he af orementioned
issues of the BJET and ALT-J journals, or by the diverse set of educational practices and tools found
in SL and detailed by the Muvenation project (García & Serrano, 2010; García, 2010). Consistent with
other literature on staff and student perceptions of SL, indications to date are that students find SL a
demanding but w orthwhile learning en vironment and t hat t he virtual patient i n par ticular i s a very
useful adj unct t o ot her f orms o f c linical t raining be t hat i n t he c lassroom or t he l ive pa tient s etting
(Heaney,etc.,2011). Previously, Jennings & Collins (2008) have conducted an observational survey of
educational institutions in SL.
1.4 Previous researches
Youssef ( 2005) made a r esearch on ICT and hi gher educ ation teachers training. Uzunboylu ( 2007)
has c arried ou t a s tudy t o det ermine t he at titudes of English l anguage t eachers toward on line
education. He found t hat there ex isted s ignificant difference i n attitude t owards onl ine education
based upon teaching experience, school location, and use of e-mail. Uzunboylu and Sor (2008) have
done a study about the e-learning training needs of primary school students in North Cyprus. In their
study, t hey have c oncluded t hat: T he pr imary s chool t eachers had e-learning training ne eds. T hey
suggested that the school community should prepare an action plan collaboratively with the educators
for motivating teachers in an e-learning training program before actually delivering e-learning training.
In 2008, Tuncay and Uzunboylu have done a study about e-learning training needs of vocational high
school students in North Cyprus. In their study, they have concluded that: The vocational high school
teachers had three types of e-learning training needs: “Technology-based E-learning Training Needs”,
“Web-based E -learning T raining N eeds” an d “ Administrative E -learning T raining N eeds”. I n 20 09,
Tuncay, Stanescu and Uzunboylu, have delivered a research study in four countries and considered
621 teachers’ training needs: 66% of the Romanian teachers said their training needs refer to "mobile
technology" a nd "wireless technologies"; 5 9% of t he E nglish t eachers ans wered t hat t heir t raining
needs concern the "tools for preparing a webpage”; 78% of the Turkish teachers answered that their
training needs refer to "Skill of attaining a certificate programs"; 82% of the Cypriot teachers answered
that their training needs relate to "Learning Management Systems" (see Figure 1).
Hence, i n t he l ast 4 years s everal r esearches ar e d elivered t o f ind s econdary s chool teachers e -
learning training needs. Such a study delivered in Cyprus was presented in ECEL 2008 Conference
(A Research Study about “e-Learning Training Needs” of Vocational High School Teachers in North
Cyprus), and a study on different countries secondary school e-learning training needs (Do Teachers
e-Learning Needs Differentiate among Countries?) was presented in ECEL 2009, in Bari. There is a
digital di vide am ong t eachers ( Youssef, 2005 ; Uzunboylu&Tuncay, 2 009). If w e w ant t o br idge t he
digital divide between teachers, we can help those who are not using these technologies (Youssef,
2005) T echnology plays an important r ole in ed ucation l ife. V ideo and o nline c ommunication
technologies can overcome the obstacles in professional development of teachers (Eastmond, 1998,
in Krammer, etc. 2006).
954

Nazime Tuncay and Hüseyin Uzunboylu
Figure 1: Teachers Training Needs Pyramid in Four Countries (Tuncay, etc., 2009)
However, it is not the videos that create rather learning tasks or activities involving videos and they
can be used to pr omote learning via ap propriate accompanying d irections an d impulses ( Krammer,
etc. al . 2006). A s a c onsequence of t echnological c hallenges, q uality of i nstruction h as bec ome a
central t opic i n t he di scussion bet ween educ ators, r esearchers, and pol icymakers ( Tuncay&
Uzunboylu, 20 10a). T his m etaphor of t echnology bringing p eople f rom dar kness i nto l ight c an be
applied t o m any ot her c ontexts w here pr ofessionals i n t he f ield of educ ation ar e gi ving r ather t han
taking f rom t he dev eloping w orld a nd of fering ho pe t hat we c an i ndeed c reate a w orld of s hared
republications and i nternational u nity, r ather t han di vision, in t he f uture ( Breen, 2006) . In or der t o
achieve efficiency in learning, all categories of actors working within educational, learning and training
contexts need to comprehend the general imagine of learning, built the puzzle, and apply adequately
the l earning s olutions ( Stanescu, I .A, 2008). Among m ost i mportant r easons of s tudents f or not
preferring distance education course was its not being “face to face” (Tuncay&Uzunboylu, 2010b).
According t o t eachers’ c ommon ans wers and ex pectations, a v irtual 3 D aided, u-learning course
platform (VirtualNeu) i s dev eloped ( http://virtualneu.edu20.org) b y using t he E du2.0 l earning
management s ystem. T his en vironment i s i ntegrated i n SL where learners c an en ter as avatars.
Possible metaphors in this environment are considered and a research study is delivered. The effect
of e-learning metaphors in designing a u-learning course is investigated and (Metaphors in E-
Learning) were presented in ECEL 2010 Conference, in Portugal. Researchers’ mission is preparing a
portal and making it uni versally ac cessible and useful for teachers; de livering e -courses f or t heir
specified training needs and, helping them to be more innovative and more creative individuals who
change and improve the world education.
1.5 Purpose
E-Learning Education and Management (ELEM) course was decided by the researchers to be
delivered to a sample group of English Teachers. Purpose of the PhD study is to compare the English
Teachers Perception, Attitude and Success in Online ELEM Course & Blended ELEM Course. For
this a u-learning course platform was needed. A platform with 3D virtual classrooms, with speaking
and moving avatars, with videos, e-portfolio applications, with chat rooms, etc. was needed. It was not
possible t o h ave a ll t hese f acilities with j ust on e learning m anagement s ystem. H ence E du 2. 0,
Second Li fe, V oki, S creencast, J ing, Wallwisher, Wordle, YouTube, GoAnimate, . .etc. f ree onl ine
programs; as well as programming knowledge like HTML, Photoshop, JavaScript, PhP,..etc. are used
in VirtualNeu portal.
Purpose of this pap er is t o de liver t he Designing a U-Learning C ourse Platform for t he I dentified
Teacher Training Needs. This study aims to start a global approach to bridge the e-learning training
needs of teachers by delivering e-courses (electronic courses). For this purpose, courses are
planned, and virtual u-learning course platform “VirtualNeu” (http://virtualneu.edu20.org) is designed.
This study (which is first part of the PhD study) tries to answer the following research questions:
How do t he t eachers per ceive t he us e of e -learning/sharing portal i n educational ac tivities i n
terms of its effects on their motivation towards the educational activities?
955

Nazime Tuncay and Hüseyin Uzunboylu
How do the teachers perceive the usefulness of the e-learning/sharing portal?
How do the students and the teachers perceive the ease of use of the e-learning/sharing portal?
What ar e t he ad vantages and d isadvantages of us ing e -learning/sharing p ortal i n ed ucational
activities from the teachers point of view?
What are the suggestions of teachers about use of VirtualNeu?
What are the needs of English teachers that ELEM course would address, i.e. what information
should the VirtualNeu platform contain?
What are the aims and objectives that should the ELEM course syllabus address such that it can
be applied not only to the sample group teachers but to a more global group of teachers?
Is the content of ELEM course notes acceptable as an international context?
What facilities and menus should VirtualNeu contain?
A questionnaire is developed for each of the issues above and results are derived.
1.6 Limitations
The following limitations are resided in this study:
114 academicians from 12 different countries participated in the online survey.
E-Learning Training Needs Interviews are limited with 40 English Teachers.
Content evaluation survey is limited with 51 experts.
“ELEM Course Expectations Online Survey” is limited with 70 English teachers.
Measurement Tools Evaluation is limited with 68 experts.
This study was conducted with the volunteer participants.
All the research studies are limited with the researcher’s own budget.
2. Method
2.1 Population
114 academicians from 12 d ifferent c ountries p articipated t o the on line s urvey. Over 500 o nline
questionnaires w ere di stributed to experts f rom 25 c ountries ( Australia, B ulgaria, C anada, China,
Colombia, Cyprus, Czech, Dublin, E gypt, England, Finland, F rance, Republic, G reece, Germany,
Green La nd, Hong K ong, I srael, Netherlands, Cyprus, Norway, Portugal, Turkey, R omania, USA).
Announcements were m ade through m inistry of educ ation an d 80 E nglish teachers have r egistered
for the ELEM Course. 40 English teachers from North Cyprus have participated in E-Learning Training
Need Interviews. 51 Experts from 6 (Cyprus, Turkey, Romania, England, German, Greece) different
countries p articipated i n c ontent ev aluation s urvey. 70 English t eachers hav e par ticipated i n E-
Learning Education an d Mana gement E xpectations O nline S urvey. 68 ex perts ev aluated t he
measurement tools of the ELEM course.
2.2 Instruments
The online questionnaire about Syllabus was posted on Surveymonkey.com. In order to evaluate the
items in the qu estionnaire, ex perts evaluation (n = 1 9) was r equired. Experts g roup f rom education
technologist evaluated the data gathering scale both individually and collaboratively. Under the
suggestions of experts, the necessary corrections were done to the draft form of the questionnaire.
Hence, the content validity was maintained by the help of the educational technologist experts. All the
experts’ ev aluations an d s uggestions ar e t aken ov er i n t he dr aft f orm of t he ques tionnaire a nd
afterwards the necessary corrections were made. The Instruments used are:
2.2.1 E-Learning Training Needs Interviews
One to o ne interviews ar e delivered t o 40 English teachers. In these i nterviews they are wanted to
express their own needs in their own words and they are wanted to suggest solutions to these.
956

2.2.2 Syllabus Evaluation Online Survey
Nazime Tuncay and Hüseyin Uzunboylu
This survey composed of 97 items, where experts wanted comments from the participants and asked
them to choose the suitable scales for themselves, in each item of the questionnaire. The scales were
arranged as: “Strongly Agree”, “Agree”, “Neither Agree nor Disagree”, “Disagree” and “Strongly
Disagree”. 114 academicians participated in this. The survey contained questions like:
Do you agree with the aims of section 1.1?
What are your comments about the objectives of module 2?
2.2.3 Environment Evaluation Online Survey
This survey composed of 42 items, where the scales were arranged as: “Should be Included”, “May
be Included” and “Should not be Included”. 51 experts participated in this.
The survey contained questions like:
What do you think about Home Menu?
What do you think about Classes Menu?
What do you think about Portfolio Menu?
2.2.4 Content Evaluation Survey
This survey composed of 30 items, where the scales were arranged as: “Should be Included”, “May
be Included” and “Should not be Included”. 51 experts participated in this.
The survey contained questions like:
What have to be included in Module 1?
What is not necessary to be included in Module 2?
Does the messages given in educational videos clear?
2.2.5 E-Learning Education and Management Expectations Online Survey
The scales were arranged as: “Excellent”, “Quite Good”, “Good”, “Basic”, “Needs to be improved”. 70
experts participated in this.
The survey contained questions like:
How would you describe your skill of using the Windows Operating System?
How w ould you d escribe your s kill of pr eparing s lide s hows by using m ultimedia pr esentation
software (ex. Microsoft PowerPoint)?
and some questions about availability of the sources like
Where can you have access to the internet?
2.2.6 Measurement Tool
A m easurement t ool c onsisting of 50 q uestions was de veloped by r esearchers. 68 ex perts
participated i n t his. A pr ofessional he lp f orm 5 measurement and ev aluation expert was taken f or
validity of test items. This research study is not a linear process: it is cyclical and iterative and it is
comprised of four stages: planning, acting, developing, and reflecting (Martler, 2008). The stages of
this research was planned parallel with the stages of action will be explained below (see Figure 2).
957

Nazime Tuncay and Hüseyin Uzunboylu
Figure 2: Stages of an Action Research (Martler, 2008)
The portal was developed according to the following theories:
Constructivist Learning theory,
Intrinsic motivation theory,
Theory of reasoned action,
Simulative reality theory,
Neurophysiological theory,
Theory of multiple intelligences
2.3 Process of data collection
“Syllabus Evaluation” Online Survey was first distributed online in January 2011, second remark was
distributed in February 2011 and later third time it was distributed in March. By April, a total of 114
questionnaires were collected. “Environment Evaluation” Online Survey was first distributed on April
and after a second remark was made on May, a total of 78 questionnaires were collected. “E-Learning
Education and M anagement E xpectations” Online S urvey is s till be ing c arried on; u ntil n ow 56
teachers’ questionnaires were collected. The last survey was very challenging. Since teachers
summer hol iday has s tarted, t hey are not r eading t heir em ails regularly; t he e-questionnaires ar e
distributed via SMS to their mobile phones. And, second remarks are taking place now.
2.4 Data analysis
Descriptive statistics were used to analyze and to report the data gained from the questionnaires. The
frequencies and percentages were used for data analysis. Also, SPSS 17 was used.
3. Results and Conclusion
This section will b e ex plained in six sub-sections: Training N eeds, SL Meetings, Online S yllabus,
Course Content, E-Learning Education and Management Expectations Online Survey, Measurement
Tool and Some of the First Impressions about VirtualNeu.
958

3.1 Training Needs
Nazime Tuncay and Hüseyin Uzunboylu
40 teachers from 40 different schools in North Cyprus are interviewed. They expressed that they have
training needs on several subjects, in which can be collected in five main topics:
Calculating exam grades using Microsoft Excel
Preparing good presentations using Microsoft Word
Searching through internet
Using Moodle
Designing Web Pages
3.2 SL Meetings
Meetings with Linden People were arranged in SL (See Figure 3). Aim was to have information about
SL and it usages.
Figure 3 SL Meetings
The comments were that SL:
“Was commonly used in nursery courses and language courses”.
“Is an entertaining and social environment”.
“It is like real life, so you do not need to attain to face-to-face courses”.
“The courses will be more entertaining if it is used in teaching”.3.3 Online Syllabus
The comments and suggestions to the syllabus were very helpful in reshaping it (see Figure 4). There
were many suggestions of additional course materials as well as suggestions for additional context.
Some of the useful suggestions were: Uploading User Manuals for SL and VirtualNeu and uploading
videos. Figure 4 shows an example of the comments of the participants.
959

Figure 4 Response Screen
Nazime Tuncay and Hüseyin Uzunboylu
60 of 62 aims in the Syllabus were chosen as “Strongly Agree” and “Agree” and 2 of the aims were
rewritten, Figure 5 illustrates an example of the participants’ answers.
Figure 5 Academicians Evaluation
3.3 Avatars
According to teachers wishes four avatars are created. Teachers were in the need of virtual people
helping t hem i n t heir on line c ourses. S o, i n t his w ay t hey would n ot f eel t hemselves al one. T hese
avatars ar e: T eacherNazime, Alice, P aul an d Z ac. TeacherNazime was t he main c haracter. T hey
were going to meet with her in SecondLife in the VirtualNeu Teacher Training Center.
960

Figure 6: VirtualNeu Teacher Training Center
Nazime Tuncay and Hüseyin Uzunboylu
Figure 7 Activity Place in VirtualNeu Teacher Training Center
Virtual Teacher Training Center is created in Second Life. Its number is 103 at Pibgorn region. Boxes
for students were created (see Figure 6). They will be used in different activities in the courses (see
Figure 7).
Interactive boards were inserted into the centre, where changing URL of this board is possible. It was
designed such that students can write their URLs, Zoom, goes back and goes forward. Three
assistants of VirtualNeu were created by using Voki. Paul’s is a “teaching assistant”. Alice is a
“technical assistant” and Zac is an “activity assistant” (see Figure 8).
Figure 8 Assistants of VirtualNeu
961

Nazime Tuncay and Hüseyin Uzunboylu
After building an d dec orating V irtualNeu T eacher T raining C enter, t he c ourse c ontent was i nserted
into t he s ystem. During r egistration, aut omatic av atars ar e gi ven t o t he r egistered s tudents ( see
Figure 9).
Figure 9 Registered “Students”
3.4 Course Content
3.4.1 Modules
After having rewritten the Syllabus, the content was written for each one of the 5 modules; these are
checked by 5 experts from Turkey and Cyprus and proof read by 2 English teachers. The necessary
corrections were made and they are uploaded to the system as pdf files (see Figure 10).
962

Figure 10 E-Learning Course Notes
3.4.2 Resources
Nazime Tuncay and Hüseyin Uzunboylu
E-books and videos related to the modules are inserted to the system (See Figure 11).
Figure 11 Resources of “Evaluation Module”
3.4.3 Videos
Videos were recorded by Jing (a free video recorder program) and uploaded to YouTube and linked to
VirtualNeu (see Figure 12)
963

Figure 12 An Example of Recorded Videos
3.4.4 User Manuals
Nazime Tuncay and Hüseyin Uzunboylu
User Manuals for VirtualNeu and Edu 2.0 are written following the suggestions of the experts. Figure
7 shows an example of a user manual written for the visitors of the system (see Figure 13).
Figure 13 User Manual
3.4.5 U-Education Environment
U-Education en vironment i s des igned b y t he h elp of t he web d esigners ( see F igure 14) an d as a
result of the teachers and experts evaluations the last form is published, which is updated at regular
periods. It contained the following sections:
964

Figure 14 “VirtualNeu”
3.4.6 Course Content Evaluation
Nazime Tuncay and Hüseyin Uzunboylu
“Content Evaluation Survey” which was posted on SurveyMonkey.com is filled by 114 academicians
(see Figure 15). In Figure 14, orange lines shows number of people whom chose “should be
included”; blue lines shows number of people whom chose “may be included” and purple lines shows
the number of people whom chose “should not be included”. As it can be clearly seen from the Figure
9, onl y a s mall a mount of par ticipants ans wered t hat G roups menu “ may not be i ncluded.” T hus,
according to the great percentage of participants’ answers, no changes are done on the menus. On
the ot her hand , s ome corrections were wanted f or t he v ideos an d us er manual i tems; s o t hey are
corrected.
965

Figure 15 VirtualNeu General Menus
Nazime Tuncay and Hüseyin Uzunboylu
3.5 E-Learning Education and Management Expectations Online Survey
This is an on-going survey. Until now 70 questionnaires are filled. Main teachers’ expectations from
the course and the subjects that they want to learn are listed below.
Teachers emphasized that their expectations from this course are:
“At the end of this course I want to be able to prepare a website which all my students can use
and be a member of it. They will use this website to publish their homework assignments and to
interact with their friends.”
“The f undamental an d es sential use of on line learning, h ow s tudents ar e a ble t o us e v arious
modes of interactive learning tools to stimulate cognitive ability and gain an insight to the modern
hi-speed technological world of easily obtaining and learning information.”
“Being ab le t o us e d ifferent r esources f or educ ation and be ing ab le t o m anage c lass b y us ing
different education methods. Also, to get an idea about interactive learning.”
Teachers expressed that they are keen on learning the following subjects:
“Different w ays of us ing t he c omputer t o t each four skills of English t hat is L istening, Reading,
Writing, Speaking.”
“How to teach pronunciation ,how to teach drama, How to teach enjoyable grammar, to know how
to take online lessons, to use internet and sources”
“How t o attract m y s tudents' at tention more t o m y l essons a nd how to m ake t hem abl e t o
communicate in a foreign language properly in such a modern age we are in.”
3.6 Measurement Tool
Item di fficulty a nd item di scrimination i ndexes were det ermined b y item anal yzing i n t he end of
preliminary study. Some items which had low item difficulty and discrimination indexes were revised.
After the changes, test was performed and item analyses were done again, in May 2011. The test is
applied t o 68 students. According t o the obtained dat a, the r eliability c oefficient of t he t est was
calculated as 0, 94 by Cornbrash’s alpha coefficient method. As a result of these data, it is
determined that the test was useful.
3.7 Some of the First Impressions about VirtualNeu
Not all of the students finished writing their first impressions about VirtualNeu, however; it was seen
that i n gen eral t hey h ave pos itive at titude t owards V irtualNeu. Some o f t he first I mpressions o f
VirtualNeu Students are:
966

Nazime Tuncay and Hüseyin Uzunboylu
Aycan: “VirtualNeu will be very beneficial for us to learn more about the technology and
it will help us to improve our ways of teaching. I believe that, it is a good opportunity for
us to improve ourselves and discover new ways about teaching. The course and the
content are very good. Thank you for giving us a chance to be a part of this course.”
Begüm: “I think it'll be very useful because there are a lot of things that we can use in
our lessons and in my opinion children's attention is going to be higher when I use this
programme.”
Derya: “My first impression about VirtualNeu is positive. I think it'll gain us more e-skill
and we'll recognize more e-device. I'd like to thanks Teacher Nazime.”
Fatma: “Positive. I hope to learn new things and use them with my students. I am
especially excited about Second Life.”
Gamze: “I was not positive about the course at all at the beginning. Now it slightly started
changing. I started the course thinking that whatever I gain is something good for me. I
think it is very early to say whether or not I did well with coming to the course. I will find
out once everything is done. I need to compare the past and the presence thinking what I
wasn’t able to do and what I can do now and most of all what I can bring to the language
classes in conclusion”
Gulsen: “My first impression about Virtual NEU was very positive, I was looking forward
in joining this group and meeting new people, especially other collogues that work at
various areas and regions on the island with different ages and groups. It makes me
believe that it’s like an in-service course which should regularly happen in order to help
us language teachers integrate special aspects and innovated technologies into our
teaching. I would like to thank the creators and advisor of this course.”
Hale: “It is not bad, though it seems educative. Through time, I can make sure about
myself and then share :))) thanks”
Medine: “My first impression about VirtualNeu was highly positive and still is. I would like
to learn and use new information technologies in my classes. There is one issue that ı
am not very clear about and that is, ' Will we be able to log in as we do now after we
complete this course?”
Meys: “My first impression of the course was that, I thought it would be good advantage
of renewing me in education wise and developing me in a subject "e-learning" in which I
was not familiar with that much. I was glad that I was told about the course and that I had
the chance to participate in the course. I had did not encounter any problems regarding
the online classroom. I found the Second Life classroom amazing. It was something very
new to me but caught my attention straight away. I knew that the course would go
smoothly and without troubles. I thought this because the class in the Second Life world
was done and prepared with a lot effort, so this made me sure once again that this
course would be very useful in my life in developing me. To sum up I am glad to be part
of this course.”
Meryem: “Communication throughout technology in education is getting more and more
important for the past years. I hope in our country one day we will be using e-education
at our schools! So this course definitely seems to correspond our needs in this manner.”
Musteyde: “My first impression was , . I like that very much and I believe that my
students and I would benefit from it a lot. Thanks Nazime and others who put such effort
to produce this.”
Münevver: “I must say that I am really excited about this course but, I need to use
VirtualNeu a bit more to evaluate; it looks useful and informative though :)”
Narin: “My first impression is highly positive about Virtual NEU. For now, ı must say
that ı need to work on it more but ı believe, as the time pass; I will be more effective
using it.”
Nazife: “My first impressions of the course were very positive. I think it will be very
useful. I hope, I learn a lot of things about e- education. I would like to thanks Nazime
Teacher.”
967

Nazime Tuncay and Hüseyin Uzunboylu
Nesibe: “I am glad there is such a course to attend. I want to congratulate the team
working on the program and e-learning. I have not used such an online program to teach
anything but some online sources. For primary school students online puzzles tests etc.
really helped me before to keep student’s interest high. VirtualNeu homepage is very
colorful and dazzling, just like an advertisement. That is the point confused me at the
very beginning. I feel the need of the real guidance. There are too many stimuli on the
same long page. I believe, as the time passes all the participants will get used to it. I
guess every participants needs to involve and be interactive which means (sometimes)
spending more time than a face-to-face classroom. However, it would be in the comfort
of your home. I am glad I am participating and wishing fun for everyone.”
Havva: “In my opinion, VirtualNeu is a creative programme. I become excited while using
this programme and believe that my students will like it"
Oytun: “First of all I would like to thank Nazime Tuncay for offering me a place in this
project. My first impression about this platform was very very positive. The creator does
everything very user friendly for us so it very easy to use and follow the instructions. I
was looking forward to meet new people and attend online courses. It will be great
experience for me. I am glad to be here thanks again.”
Ozge: “I believe VirtualNeu will make us much more relaxed and enthusiastic about
using technology in our lessons. It will definitely help us be more creative. I am looking
forward to learning new things. Thank you”
Seden: My first impression is very excited. I think we are goıng to learn more important
and creative things. I think TEACHER Nazime is very helpful when we ask lots of
questions. It is very interesting to take part in second life. Thank you for everything and
good luck for everyone best wishes.
Yıldız: “I really enjoy following the VirtualNeu website every day. Getting announcements
about each new feature of this virtual platform every second is remarkable. I think that
this is a perfect virtual learning environment for teachers who want to broaden their
horizons about e-education and e-skills. Thus ELT teachers will be able to integrate
computer assisted language learning in their lessons. I also believe that by using virtual
learning websites, students will take greater responsibility in the language learning
process by studying at their own pace without time restrictions. Thus e-education
combined with online education will have benefits including greater efficiency in lessons
and improved student achievement and graduation rates.”
4. Discussion and conclusion:
This paper describes an initiative in teacher education which has undergone significant piloting in an
international c ontext. A s a r esult of t raining n eeds interviews, SL m eetings r esults, online s yllabus
evaluations, course content evaluations, e-learning education evaluations and management
expectations online s urvey and m easurement t ool VirtualNeu portal is c reated. I t includes Virtual
Teacher Training Center (created in Second Life) and Teacher Share Portal (via ScreenCast).
Thus, Training Needs of teachers are analysed; “VirtualNeu” is designed; syllabus is written; modules
are written and uploaded into system; now it’s ready to deliver ELEM Courses. The ELEM course is
announced at E ducation, Youth a nd S port M inistry web page o n J une 2 011 ( see F igure 16). T he
registration of the students to the system is an on-going process and will be ended at 1 July 2011.
Figure 16 Announcement on www.mebnet.net
968

Nazime Tuncay and Hüseyin Uzunboylu
This r esearch s tudies ar e i mportant s ince a g lobal e -learning c ourse m aterial based on t oday’s
teachers’ e -learning training needs w as pr epared. Following y ear; t eacher training c ourses will be
delivered via another VLE to English teachers in Turkey, England and Romania.
Collaboration with o ther r esearchers f rom ot her c ountries i s n ecessary f or t he g lobal p urpose of
fulfilling teacher training needs. T hus presentation of these ideas in several different conferences in
different c ountries is nec essary. R esearchers s tart with s mall s teps t o m ake bi g dr eams r eal. First
step i s c ompleted i n f our years; s tep by s tep i n c ountries t eachers e -learning t raining needs ar e
investigated c ollaboratively w ith ot her r esearchers. T he s econd s tep i s des igning u -learning c ourse
platform and t o del iver r esearches t o i mprove i t i n a uni versal l evel. T he t hird s tep i s t o de liver
significant p iloting i n an international c ontext a nd t he f ourth i s t o deliver c ourses t o t eachers f rom
Romania, T urkey, Greece, England and etc. Researchers’ dream is to bridge the e-learning divides
among t eachers. The dr eam begi ns with teachers who be lieve i n you and t eachers ar e t he ones
who will take massive action to make those dreams come true.
5. Recommendations
Further researchers for improving teachers’ e-learning skills and knowledge, answers to the questions
should be sought:
How can the available technologies be used more effectively to teach the e-learning modules?
How c an t he e -learning modules be assessed m ore ef fectively t o e nsure t he t eachers ha ve
reached the specified outcomes of the modules?
How can one adopt lessons learned from the modules to the classes?
How can more guidance and instructions for teachers in e-courses be included?
How can all these be distributed to other countries more easily?
References
Breen, P.(2006). Coming out of the Darkness of the Past, Turkish Online Journal of Distance Education, 7(4).
Chang, C. Y. & Sheu, J. P. (2002) “Design and Implementation of Ad Hoc Classroom and e-Schoolbag Systems
for Ubiquitous Learning”, Paper presented at the IEEE International Workshop on Wireless and Mobile
Technologies in Education, 29-30.
Chen et al., Curtis et al., (2002). In Young Scientific Research, Japan Society for the Promotion of Science, (2,
15700516).
Cheng, Z., Shengguo, S., Kansen, M., Huang, T. & Aiguo, H. (2005). “A Personalized Ubiquitous Education
Support Environment by Comparing Learning Instructional”, Paper presented at the 19th International
Conference on Advanced Information Networking and Applications, 28-30.
Dillenbourg, P. (2000). Virtual Learning Environments, [online], EUN conference 2000, [Online],
http://tecfa.unige.ch/tecfa/publicat/dil-papers-2/Dil.7.5.18.pdf [10 February 2009]
El-Nasr, MS, Vasilakos AV and Robinson J. (2008).Process Drama in the Virtual World- A Survey. International
Journal of Arts and Technology, 1(1), 13-33.
García, M. Pérez, & Serrano, J. Alamo (2010, forthcoming – Eds.). Collection of Educational Tools for Second
Life, Brussels: Menon Network.
García, M. Pérez (2010, forthcoming – Ed.), Pedagogical Practices for Virtual Worlds, (Forthcoming 2010),
Brussels: Menon Network.
Greener, A. R., S. L. Greener, et al. (2008). Strategic Learners at a Distance. 3rd International Conference on
ELearning (ICEL08) Cape Town, SA, Academic Conferences Ltd.
Greener, S. L. (2008). Exploring Readiness for Online Learning, University of Brighton, and School of Education.
Greener, S. L. (2008). Identity crisis: who is teaching whom online? European Conference on ELearning (ECEL)
2009. Agia Napa, Cyprus.
Greener, S. L. (2008) "Self-aware and Self-directed: Student Conceptions of Blended Learning." Merlot Journal of
Online Learning and Teaching
Heaney,R., Timmins,V-S.,Booth,P. & Dawes, J. (2011).Student Perceptions of a Second Life® Virtual Patient to
Complement More Traditional Forms of Clinical Education, Proceedings of the 9 th European Conference
on e-Learning Instituto Superior de Engenharia do Porto Portugal 4-5 November 2010
Haruo,N., Kiyoharu, P. H., Yasufumi, K., & Shiho, M.(2003) “ Designing Ubiquitous and Universal Learning
Situations: Integrating Textbooks and Mobile Devices”, Paper presented at the 19th Annual conference on
Distance Teaching and Learning, 13-15.
Ibanez,J.& Delgado-Mata, C.(2011).Lessons From Research on Interaction With Virtual Enviroments. Journal of
Network and Computer Applications, 34(2011), 268-281.
Jennings, N.; Collins, C. (2008). Virtual or Virtually U: Educational Institutions in Second Life, International
Journal of Social Sciences, 2 (3), 180-186.
969

Nazime Tuncay and Hüseyin Uzunboylu
Krammer, K., Ratzka, N., Klieme, E. Lipowsky, F. Pauli, C. Reusser,K.(2006). Learning with Classroom videos:
Conception and first results of an online teacher-training program. ZDM, 38(5). 422-432.
Martler,C. (2008). Action Research: Teachers as Researchers in the Classroom. Thousand Oaks: Sega
Publicasions.
Nijholt A. (2006). Towards the Automatic Generation of Virtual Presenter Agents. Informing Science, 9, 97-115.
Ogata, H. & Yano,Y. (2004). “Context-Aware Support for Computer-Supported Ubiquitous Learning”, Paper
presented at the 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education.
Stanescu, I.A. (2008). New ELearning Frontiers: MLearning and GLearning. The 4th International Scientific
Conference eLearning and Software for Education, Bucharest, April 17-18, 2008.
Takahata,M.,Shiraki,K.,Sakane,T. & Takebayashi,Y.(2004). “Sound Feedback for Powerful Karate Training”,
Paper presented at the International Conference on New Interfaces for Musical Expression.
Tuncay, N. & Uzunboylu, H. (2010a). Walking in two words, from e-learning paradise to technologically locked in.
Cypriot Journal of Educational Sciences.5(2010):271-281.
Tuncay,N. &Uzunboylu, H.(2010b). “Anxiety and Resistance in Distance Learning”. Cypriot Journal of
Educational Sciences. 5(2010). 142-150.
Uzunboylu, H. (2007) “Teacher Attitudes Toward Online Education Following an Online In-service Program”,
International Journal on E-Learning, Vol 6, No.2, pp. 267-277.
Uzunboylu, H., Tuncay, N. (2009) “E-Learning Divides in North Cyprus”, Asia Pacific Education Review. DOI
10.1007/s12564-009-9019-y
Van Dijk B., op den Akker R., Nijholt A., Zwiers J. Navigation assistance in Virtual Worlds. In: Boyd E., Cohen, E.,
Zaliwski AJ., editors. Proceedings of 2001 informing science conference. Kralow, Poland: June, 518-526.
Warburton, S. (2009) “Second Life in higher education: Assessing the potential for and the barriers to deploying
virtual worlds in learning and teaching”, British Journal of Educational Technology, 40(3), 414-426.
Wernert EA, Hanson AJ, A Framework for Assisted Exploration with Colloboration. In:VIS’99: Proceedings of the
conference on visualization’99. Los Alamitos. CA, USA: IEEE Computer Society Press; 1999. 241-248.
Youssef, A. B. (2005), “Information and Communication Technologies and Higher Education Teachers Training”,
[online], Paris-Sud University, http://www.uoc.edu/rusc/5/1/dt/eng/benyoussef_ ragni.pdf.
Zhang, G., Jin, Q. & Lin, M. (2005). “ A Framework of Social Interaction Support for Ubiquitous Learning”, Paper
presented at the 19th International Conference of Advanced Information Networking and Applications
970

Work
in
Progress
Papers
971

972

Someone to Talk to – Using Automated Characters to
Support Simulated Learning Activities
Liz Falconer and Manuel Frutos-Perez
University of the West of England, Bristol, UK
Liz.Falconer@uwe.ac.uk
Manuel.Frutos-Perez@uwe.ac.uk
Abstract: The University of the West of England (UWE) has a large number of students who will pursue
subsequent careers in a wide range of professional fields such as engineering, law, business, nursing, teaching,
psychology, criminology and design. An important part of that education is the ability to relate theory to practice
(Barrett 2011), and developments in technology over the past years have now created opportunities to enable
students to experience simulations of events and situations that are difficult, unethical or impossible to organise
in the real world, before they put their skills into practice in the real world (Newland 2008). Virtual worlds are
proving to be popular and effective environments at UWE for simulations of a range of experiences, such as
accident investigations, risk assessments, business ethics cases, psychotherapy practice and sociological
experiments. However, as the number of students undertaking these simulations increases, so the call on tutor
time will significantly increase. These simulations require to be scalable, to enable their potential for study by
large cohorts of students. This year we have experimented with automated non-player characters, also known as
“bots,” to enable students to undertake some dialogue during the simulated scenarios without the need for a
number of tutors to be available to take particular roles. The bots are currently unsophisticated keyword
recognition systems, but even these have proven to have some value in two of the simulations; the accident
investigation and the risk assessment, where students were able to gather information from characters they could
“talk” to, making more realistic the experience of exploring the environment where the simulations were taking
place. This paper discusses the results of student feedback, evaluations of these simulations and prototype
development for the next generation bots that we want to implement in future learning simulations based on the
findings of the evaluations.
Keywords: learning simulations, non-player characters, professional practice, virtual worlds
1. The environmental health scenario
Virtual worlds are three-dimensional representations of environments, either realistic or imaginary,
where users can participate in a variety of ways from game play to social interaction. Users of virtual
worlds create a virtual self, an avatar, which represents them within the displayed environment. The
virtual world of Second Life® is a geographically large, customisable environment. Users of Second
Life® can manipulate and build their own environment using inert components or user programmed
interactive scripts contained within objects.
MSc Environmental Health students at UWE undertake an accident investigation and risk assessment
in realistic industrial sets in the virtual world Second Life®, as part of an underpinning theoretical
module that covers risk evaluation theory. Half of the cohort witness the accident in the virtual world
and then act as witnesses for the other half. Once they have been interviewed by their colleagues, the
witnesses undertake a simulated risk assessment in neighbouring premises whilst the accident
investigators carry on with the investigation by inspecting the premises, picking up clues from objects,
taking photographs and interviewing a simple automated bot that takes the role of one of the workers
involved in the accident. The risk assessors also have the opportunity to interview a bot taking the
role of a worker in the company during their inspection. At the conclusion of the exercise the students
are asked to complete an evaluation questionnaire and take part in a “cafe style” forum to discuss
their reactions to their risk studies.
The next section of this paper is a summary of the part of the results of the evaluation undertaken in
April 2011 that refers to bots and personal interactions.
2. Student feedback
Generally the students were interested in the idea of undertaking an accident investigation or risk
assessment in a virtual world when they first joined the module in September 2010. None of the
students had ever used a virtual world before, but most were keen to try it out. Some weren’t though,
as can be seen from Figure 1 below. However, Figure 1 also shows how their views changed over the
course of the exercises.
973

Liz Falconer and Manuel Frutos-Perez
Figure 1: Changing views before and after the exercises
The change of view in the students who were trepidatious at first is summed up by the following
comment: “I like that it simulated real life. I was a bit apprehensive at first because I do not particularly
like 'video game' type things but I found it a very useful exercise. I like the visuals in second life and
how real everything looked. I like that you could meet with friends and have a chat, as opposed to
regular online chatting and it’s just you and the screen. You sort of feel like you are real people in a
fake world.”
Figure 2 below shows the students’ responses to questions about the learning outcome of the
exercises, and demonstrates how they felt very positive about their learning experiences overall. The
following comment illustrates the reality aspect of the exercises: “It is visual and a more realistic way
of learning as I felt like the actual person doing the investigation.”
Figure 2: Students’ views of their learning outcomes
974

Liz Falconer and Manuel Frutos-Perez
Their comments regarding the activity of interviewing the automated bot do demonstrate that it was a
part of the exercise that they enjoyed, but the restrictions of a simple keyword-matching bot are also
apparent:
“It has been fun using second life and I like the level of interaction you can have with the
people and objects around you. This was helpful for the investigation.”
“It is a new and novel way of learning, something I’m very open to than the normal read,
read, read and regurgitate.”
“It was interactive, something different, was fun and interesting. Better than role play.”
“Talking to the bot was the weakest point. It lacked the sophistication (obviously) of the
manager interview and therefore was the least realistic. But it certainly fulfilled a
purpose.”
“It could sometimes be difficult to see or interact/find things. The bots were not able to
interact in a cohesive way.”
“Interviewing the bot I found limited. I would have thought about how to phrase my
questions but the limited scope of the understanding of the bot meant that I knew the
most effective way was to simply state the key words - the experience lost realism at this
point. More interactions such as reporting to a receptionist and signing a visitor’s book
when meeting the manager may have helped produce a more realistic feel.”
“I felt restricted by the limited keywords with Buddy Norton (the bot) gave. Also I would
have preferred talking as compared to typing in the interview.”
These comments demonstrate the need for a more sophisticated approach to the design of bots.
Advanced bots could take a more active part in the kinds of interactive scenarios that we have
described above. The next section of this paper describes our current development work on advanced
bots.
3. Designing advanced bots and interfaces
The need to design professional practice simulation learning activities that are sustainable from a
teaching resource perspective has led us to investigate further the development of more complex
automated characters. These characters can be designed to have certain specific knowledge and to
be able to query online services to access information. They can also be developed to display certain
personality traits. This advanced automation helps to maintain a degree of realism of the learning
simulations (simulating social interaction) while adding flexibility to the experience (learners’
interaction can take place at any time, any date) and helping to control the workload on teaching staff
(Schalkoff 2011). The use of advanced automated characters will make it possible to design learning
simulations for large cohorts of students. It also affords new learning support opportunities; students
could attempt to interview the automated character several times over a period of time to perfect their
technique to obtain information; students could interview the automated character via a variety of
devices or online domains; and their interaction could be automatically translated to a foreign
language of their choice.
We have now developed a working “proof of concept” complex bot. The development process started
with a scoping phase to identify requirements and map possible technical solutions against those
requirements. Artificial Intelligence Markup Language (AIML) was chosen as the main vehicle to
develop the conversational elements of the system that would allow for the automation of
conversations between the bot and the students (Dybala 2010). AIML as a structured computing
language allows for the processing of natural language interaction. It also makes it possible to
programme bot responses to be varied (they can be linguistic responses, hyperlinks, web page
extracts and information feeds from web services) and also for the randomisation of the response
given by the bot to a particular question. Different products already developed by third party providers
were identified as being able to support different sections of the functional architecture: Discourse
(Daden Limited), Sitepal (Oddcast Inc), Translate API (Google Inc), Yeast Templates (University of La
Rioja) and Virtual Keyboard Interface (GreyWyvern Inc).
The current working prototype is an integration of all the above technologies. We have developed a
number of scripts that control how those technologies interact with each other. We have also
developed a number of interfaces for students to interact with the bots, so that conversations with the
975

Liz Falconer and Manuel Frutos-Perez
bots can be held in the virtual world Second Life, but also via a web-enabled mobile device as shown
in Figure 3 below or via a web browser on a computer.
Figure 3: Smartphone browser interface of the advanced bot prototype
The communication architecture that runs the bot prototype is quite complex, particularly for the full
browser interface. We have aimed at designing an interface that appears clear and intuitive (Ju &
Leifer 2008) as shown in Figure 4 below, where all features have been automated apart from the
students’ language choices.
Figure 4: Full browser interface of the advanced bot prototype
4. Further work
The success of the prototype development allows us to move to the next phase of development that
will focus on the sophistication of the conversational abilities of the bot, capturing and processing of
questions that the bot is unable to answer, linking the information bank that drives the conversation to
relevant external services and feeds, and developing distinct instances of the bots to suit particular
characters in the learning simulations.
976

References
Liz Falconer and Manuel Frutos-Perez
Barrett, T. (2011). New approaches to problem-based learning: revitalising your practice in higher education.
Routledge, Oxford.
Dybala, P. (2010). Evaluating subjective aspects of HCI on an example of a non-task oriented conversational
system. International journal on artificial intelligence tools, 19 (6), 819-856.
Ju, W. and Leifer, L. (2008). The Design of Implicit Interactions: Making Interactive Systems Less Obnoxious.
Design Issues, 24 (3), 72-84.
Newland, B. (2009). Innovative eLearning with e-Resources. Bournemouth University Press, Bournemouth.
Schalkoff, R. (2011). Intelligent systems: principles, paradigms and pragmatics. Jones and Bartlett, London.
977

Extreme Scaffolding in the Teaching and Learning of
Programming Languages
Dan-Adrian German
Indiana University School of Informatics and Computing, Bloomington, USA
dgerman@indiana.edu
Abstract: We report on the design and implementation of an e-learning framework that can be used to produce
tutoring modules with an arbitrary degree of scaffolding. Each module teaches by problem-solving; instant
feedback is built in the scaffolding and provides the learner with the reinforcement (negative or positive) that
conditions learning. Although the tutoring system guides the learner's actions, it does not restrict them
excessively and significant number of degrees of freedom remain. This encourages the learners (as they
prioritize their problem-solving steps) to become active constructors of knowledge through direct experimentation.
By analogy with the principles of extreme (or agile) programming, we call the adjustable scaffolding technique
used in our system "extreme" because it too can be set up in micro-increments and tunes the learners into the
same behavioral pattern as they analyze, design, test, integrate and deploy their problem-solving steps.
Keywords: extreme scaffolding, agile instruction
1. Introduction
Since most learning, at least in its early stages, is observational, well-chosen examples are a
necessary (though not sufficient) condition for effective teaching. To be a successful learner one
needs to practice, and internalize those very examples. But students aren't always self-regulated
learners. They may not be aware what they do or do not understand. They sometimes think they get it
when they really don't. Learning to program well requires a lot of determined practice and without
evidence of learning there can be no legitimate claim of teaching. No matter how wonderfully crafted
the examples presented in a lecture, it is not uncommon for students to come and say: "I understand
everything when I watch you develop a program in class, but when I go and try to write a program by
myself I stare at the blank computer screen and don't know where to start." This often leads to panic
and frustration. Just giving examples, no matter how good, is not enough. For teaching to be effective
the learner needs to assume initiative and responsibility. In this paper we present an e-learning
framework that can assist in producing tutoring modules with an arbitrary degree of scaffolding. The
tutoring modules generated by our framework facilitate the necessary transfer of initiative and
responsibility from teacher to students.
2. Framework
We will introduce the framework through an example. We start from its English-language description:
Compute the greatest common-divisor of two nonnegative integers p and q as follows: If q is 0, the
answer is p. If not, divide p by q and take the remainder r. The answer is the greatest common divisor
of q and r.
This can be a problem already discussed in class or a problem that the students should now be able
to solve with little or no difficulty. In preparing a tutoring module for this program the instructor starts
from an annotated model solution (see Figure 1, code in Java).
Figure 1: Solution for the greatest common divisor problem, marked for tutoring module generation.
In this example the annotations consist of backquotes. The conversion program produces a tutoring
module as shown in Figure 2. Marked characters are extracted and listed in lexicographic order at the
bottom of the screen. The student’s task is to reconstruct the program given a set of constraints
(timed test, maximum number of allowable mistakes, etc). Characters listed at the bottom can be
978

Dan-Adrian German
dragged with the mouse and dropped anywhere on the screen; they maintain their color until placed in
the right location, when they turn into white (or the background color) as an indication that they belong
there. In Figure 2 one of the characters (you can see it’s missing from the bottom) has already been
placed correctly; obviously, there are two other locations where it would’ve worked just as well.
Figure 2: Tutoring module for the template shown in Figure 1, ready for student interaction.
3. Model
The tutoring modules produced by our framework resemble CLOZE exercises (Taylor 1953) of Gestalt
theory descent; traditional CLOZE exercises are frequently used as after-reading activities for English
as a Second Language learners in the mainstream classroom (Gibbons 2002). The interactive nature
of our templates fully situates them within the scope of the Cognitive Apprenticeship model (Collins
1991). The generating framework could then be said to provide concrete implementations of the
Extreme Apprenticeship method (Vihavainen 2011). In working with a tutoring module students (a)
work by doing, (b) receive continuous feedback and (c) work at their own pace (no compromise,
unless the module is set up for a test). We can easily distinguish in our tutoring modules the three
phases of the Extreme Apprenticeship method: modeling, scaffolding and fading. The annotated
solution represents the model, the annotation indicates the scaffolding and the fading is built-in (see
the first letter in the screen shot in Figure 2). Notice that the automatic fading leaves behind a zone of
proximal development (Vygotsky 1978) that is constantly evolving and entirely determined by the
specific steps that had been taken by the learner.
4. Data
In his book (Willingham 2010) cognitive scientist Daniel Willingham states a series of principles with
far-reaching implications for teaching: the brain is not designed for thinking it is actually designed to
save us from having to think too often (nevertheless we find successful thinking pleasurable, so we
seek out opportunities to think but we are very selective in doing so); thinking skills and knowledge
are bound together (factual knowledge must precede skill); memory is the residue of thought
(students will not remember your lecture, they will remember their thoughts during your lecture);
understanding is remembering in disguise; practices enables further learning, etc. These principles
seemed to back us up, but to measure the impact of a tutoring module we set up and conducted the
following experiment.
A group of 59 students had their weekly labs on Thursday and Friday. The labs were structured and
conducted in a traditional manner (examples presented by lab instructors, combined with individual
time spent on computer by the students). The lab assignment was to produce a short program due
Sunday night. Late submissions were acceptable (with no penalty) until Monday evening. On Monday
at 2:30pm, in lecture, a pop quiz was given in class asking the students to write the shortest program
that matched the specifications of the lab assignment that was due that evening (or the night before).
The duration of the pop quiz was 15 minutes, the tests were collected and archived as pre-test data.
A tutoring module for the solution to the pop quiz was then presented in lecture (this was the students’
979

Dan-Adrian German
first exposure to the tool) and posted on-line. Tuesday morning a message was sent to the students
with the URL of the tutoring module demonstrated in class and students were asked to work with the
template once just so we can ask them if they liked it or not. A reminder was sent on Wednesday
morning. There was no other implicit or explicit incentive to review/study the program other than
through the requested interaction with the interactive template (the tutoring module presented on
Monday in class). On Wednesday at 2:30pm in lecture we repeated the quiz from Monday, collected
the tests after 15 minutes and archived them as post-test data. Both quizzes were closed book. After
the second quiz we asked a few open-ended questions (which is what students had expected in the
first place) such as: did you work with the interactive template as we asked you to, did you have a
positive experience with the tutoring module (if so, what was the single most useful aspect, if not why
not), did you do better than last time or not, and so on. The answers to these questions were
anonymous, they were collected and archived with the post-test data.
Quizzes were then graded and grouped into categories as shown in Table 1.
Table 1: Pre- and post-test results
Categories Monday lecture Wednesday lecture
Wrong or no idea 27 45% 9 15%
Some idea, in the right direction 19 32% 21 35%
Recognizable (but unfinished) solution 11 18% 10 17%
Complete, correct solution 2 3% 19 32%
5. Conclusions
Significant improvement can be seen in the table: on Monday 45% of the students are hopelessly lost.
Wednesday, after being asked to work with the tutoring module once, a larger percentage (49%) are
able to produce a recognizable solution (with two thirds of these students actually providing a
complete, correct solution). We don’t know how much time students actually spent playing with the
template, and what else they did besides that. We didn’t grade, review or otherwise bring up the lab
assignment in e-mail, blogs, on-line forums, individual and/or office hours communication with the
students during this time (from Monday to Wednesday). The motivation presented to the students for
completing a tutoring module was that they were going to provide usability feedback to us on the tool.
In written, open-ended anonymous comments 8% (5 students) indicated they didn’t work with the
template for whatever reasons (didn’t have time, confused about location on the web or their browser
didn’t seem to work, intended to but forgot etc.) Of the remaining 54 students 76% (41 students)
indicated that they had found it useful in some respect and enjoyed interacting with it. Half of these
cited the ability to actively create by guided exploration as the single most useful instructional aspect.
The remaining half was evenly split between the built-in interactivity and the actual scaffolding per se.
There is evidence that working with a tutoring module builds endurance: "I had to look at [it] over and
over to figure it out," wrote one student. "It forced me to spend time [on task]" wrote another. And as
they learn to better monitor their own understanding students also tend to become more confident: "[i]t
made me slow down [and] made me think about each part [...] the things I could complete I was proud
of and the things I [thought I] couldn't turned out to be easy to figure out eventually."
6. Directions for future work
The system can already support variable-length scaffolding and can easily record/replay the steps
taken by the user. Better database support, adaptive-testing (Wainer 2000) and team-based learning
(TBL) capabilities along with alternative interface support (free typing) is being added. Experiments
have been designed (but not yet carried out) to determine the most suitable application for individual
and group work. The ability to record learners’ steps solves the individual accountability problem in a
TBL setting.
Acknowledgments
I would like to thank Joan Middendorf (former Associate Director of Campus Instructional Consulting
on the Bloomington campus) for expert advice and guidance; Suzanne Menzel and John F. Duncan
from the Division U Collegium for Excellence in Undergraduate Teaching for their constructive
criticism and comments on an earlier draft; and Kathryn Propst from the Center of Innovative
Teaching and Learning for her very efficient steering of the “New Technologies New Pedagogies”
group meetings.
980

References
Dan-Adrian German
Collins, A., Brown, J.S., and Holum, A. (1991) “Cognitive apprenticeship: making thinking visible”, American
Educator, Winter, pp. 38–46.
Gibbons, P. (2002) Scaffolding Language, Scaffolding Learning—Teaching Second Language Learners in the
Mainstream Classroom, Heinemann, Portsmouth, NH.
Taylor, W.L. (1953) "Cloze procedure: A new tool for measuring readability." Journalism Quarterly, Vol. 30, pp.
415–433.
Vihavainen, A., Paksula, M. and Luukkainen, M. (2011) “Extreme Apprenticeship Method in Teaching
Programming for Beginners”, SIGCSE ’11: Proceedings of the 42nd ACM technical symposium on
Computer science education, ACM, New York.
Vygotsky, L.S. (1978) “Interaction Between Learning and Development”, Mind in Society: The Development of
Higher Psychological Processes, Harvard University Press, Cambridge, MA
Wainer, H., and Mislevy, R.J. (2000). “Item response theory, calibration, and estimation.” In Wainer, H. (Ed.)
Computerized Adaptive Testing: A Primer. Lawrence Erlbaum Associates, Mahwah, NJ.
Willingham, D. T. (2010) Why Don’t Students Like School: A Cognitive Scientist Answers Questions About How
the Mind Works and What It Means for the Classroom, Jossey-Bass.
981

Benefits and Barriers: Applying eLearning in the Context of
Organisational Change to Improve the Learning
Experience for Mature, Part-Time Students
Simon McGinnes
Trinity College Dublin, Dublin Ireland
simon.mcginnes@tcd.ie
Abstract: This paper reports on innovations in an evening degree programme in Information Systems at Trinity
College Dublin. The students are mature, experienced professionals who value the collaborative nature of their
programme, yet struggle to find time for learning. The aim is to increase choice over location, time and pace of
learning. Innovations include the adoption of a virtual learning environment and introduction of moderated
discussion groups, helping students to feel connected and to assist one another. Online, interactive coursework
modules and podcast lectures allow students to work at their own pace. Secure intranets provide useful
information for students and staff. Corresponding organisational and administrative changes are also required,
including eLearning support and training for staff and students, coursework scheduling to avoid bottlenecks and
reduce stress, and introduction of choice over content. Overall, the technological changes have been relatively
straightforward, but changing the organisation and its entrenched work methods has proven harder. The results
suggest that asynchronous eLearning can help mature students, but full benefit is likely to be felt only if required
organisational and cultural adjustments can also be made.
Keywords: eLearning; mature students; part-time study; lifelong learning; higher education; information systems
education; organisational change
1. Introduction
Growing numbers of mature students are entering higher education, yet the teaching and
administrative practices in established institutions may be ill-prepared to meet their needs. The
requirements of mature students, particularly when studying part-time, differ from those of
conventional full-time students (Vryonides and Vitsilakis, 2008). The average age of students entering
Trinity College’s evening B.Sc (Hons) and Diploma courses in Information Systems (IS) is 31 (TCD,
2011), which is coincidentally also the most popular age to start a family. Many students have busy
jobs with extended working hours and long daily commutes. The course has an onerous schedule:
three hours each night, three nights per week, for four years. Because of work, family and travel
pressures, students can struggle to meet the attendance requirements. The situation for female
mature students is particularly difficult, since many juggle multiple roles and try to carve out study time
from an already-busy schedule (Vryonides and Vitsilakis, 2008). Enrolments in the programme have
increased in recent years, but the overall proportion of females has declined for over a decade.
The IS students are generally eager and motivated to learn (Bowman and Kearns, 2008). Most are
experienced IT professionals; they learn from each other and value the opportunity for collaboration
with their peers. In some ways, the conventional idea of the university as a source of knowledge is
inverted, because the students bring cutting-edge knowledge and experience into the College for the
benefit of others. Yet the students face significant challenges in completing the course, especially as
job insecurity and traffic congestion have grown in recent years. The contact hours are reduced to fit
the evening format, so there is a substantial amount of syllabus to fit into the time available. Moreover,
the College’s administration is geared towards day students; evening students find it difficult to access
services when they attend at night.
This paper outlines measures being taken in the IS programme to help evening students participate
more successfully and with less stress, particularly in the light of recent technological and social
developments.
2. Why change?
eLearning is helpful for mature students (Hart et al., 2005), particularly because of the flexibility it
affords (Bowman and Kearns, 2008, AFLF, 2007). The aim in the IS programme is to use eLearning
to help students learn at a time and place of their own choosing. We also want to let students study
material that is useful to them, at their own pace. The current syllabus and schedule are inflexible;
students learn only the prescribed material, at the prescribed pace. But students in the IS programme
vary greatly in background, knowledge and roles, and this trend is increasing as the variety of roles in
982

Simon McGinnes
professional IT grows. We want to reflect this diversity by offering students choice, according to their
level of prior knowledge, interests and career requirements. An overarching aim is to structure the
program and services around students’ needs rather than those of the institution.
A blended learning approach has been shown to be effective for mature students (Jennings, 2005,
Bowman and Kearns, 2008). Synchronous and interactive teaching styles, including group sessions,
live meetings and lectures, can fruitfully be combined with asynchronous methods such as blogging,
wikis and podcasts (Billhardt, 2010). But at Trinity College there are barriers to the use of blended
learning. As a traditional bricks-and-mortar institution, the College is emphatically not in the business
of distance learning; the tradition of face-to-face teaching is valued, and services are structured to
support conventional teaching methods (predominantly lecturing). The evening IS programme goes
against the grain because it facilitates learning by adults in an unconventional course structure. While
the College endorses lifelong learning, this does not necessarily translate into structures and services
that facilitate it. Nevertheless, the goal in the IS programme is not to dispense with conventional
teaching methods, but to introduce blended learning whilst retaining the all-important “Trinity
experience”.
3. Prior work
Historically the IS programme has been delivered in much the same way as daytime degree courses:
face-to-face lecturing, often using whiteboard and PowerPoint, with occasional use of more innovative
teaching methods. However, various experiments have been made with eLearning (Jennings, 2005,
Mullally et al., 2006). In one, online synchronous delivery (OSD) was attempted. Lectures were
broadcast live on the web at set times; students participated via online chat, mainly from home. Owing
to bandwidth limitations the lectures were delivered in audio only. The results were disappointing;
students considered the experience inferior to traditional face-to-face lectures, and complained of a
sense of isolation. OSD was seen as a pale imitation of the “real thing”: traditional lectures (Jennings
et al., 2007). This outcome tallies with research which suggests that student engagement in
eLearning can be poor and the results unsatisfactory (Billhardt, 2010). However, the online lectures
were recorded for later use by students; this led to the insight that asynchronous learning could be
useful in this context since the students could use the recorded lectures where and when they chose
(Jennings, 2005).
Another insight was that technical infrastructure was crucial; in this case, network bandwidth was
insufficient to support video. This experience highlights the relatively expensive and complex nature of
eLearning for providers. In conventional, low-cost, low-dependency university teaching, each module
is handled by a single lecturer acting individually. In contrast, eLearning is a morecomplicated
undertaking; it requires coordinated effort on planning, logistics, preparation, training, technology,
teamwork and support.
4. Actions taken
A series of linked innovations has been introduced using an informal action research approach,
building on prior efforts (Jennings et al., 2007, McGinnes and Dowling, 2010). The aim has been to
increase flexibility and choice at minimum cost, whilst reinforcing the programme’s valuable
collaborative nature. The impact of changes has been assessed over a three-year period using
participant observation and student questionnaires. The innovations are outlined below.
First, we adopted a single programme-wide virtual learning environment (WebCT). While this
particular environment may not be optimal for every purpose, there are advantages in taking a
programme-wide approach. For example, support and training can be streamlined and a single focus
is provided for student interaction. Discussion groups have been established, allowing students who
work separately and at odd hours to feel connected and to assist one another (Jennings, 2005,
Billhardt, 2010). Online, self-paced coursework modules are provided in practical subjects, such as
programming; students can repeat or skip sections as required. This reflects the fact that many
students already have expertise, and supports the way many mature students are thought to learn
best: by seeing and doing, interacting with material and trying out ideas.
Lecturers are encouraged to experiment with eLearning, and a trained administrator provides training
and support to staff and students; arrangements have been revised so that support is available out of
office hours. As a pilot, selected lectures have been made available as podcasts which students can
983

Simon McGinnes
view on demand. OSD has not been used, since it is demanding of resources and previous results
were disappointing.
An enlarged website offers a variety of information including how-to guides, FAQs, news, course
calendar, module descriptors with syllabus, texts and coursework requirements, and online video.
This information is designed to empower part-time students (and part-time lecturers) who are present
on campus only infrequently and therefore less able to communicate informally with other students
and staff.
Coursework is scheduled to avoid bottlenecks and reduce stress when assignments are due. The
introduction of a small number of elective modules is intended to give students some choice over the
subjects they study. Each student is allocated a personal tutor, who provides advice and support in
case of ill-health, family, financial or course-related problems. The introduction of student surveys and
class representatives helps provide feedback about the innovations, which is incorporated into a
continuous improvement cycle.
5. Results
The innovations have largely been well-received; student feedback and external examiner comments
have in general been positive. Where negative comments have been received, they have been
addressed quickly whenever possible. However, structural change has been more challenging than
we hoped. We have learned that it can be risky to raise student expectations too much; for example,
the late withdrawal of a promised elective module by the School (on cost-saving grounds) prompted
strong dissatisfaction amongst students.
Although the institution proclaims its support for lifelong learning, this does not necessarily translate
into acceptance of, or support for, eLearning at a departmental level. The introduction of eLearning
technology itself has been relatively simple. However, ongoing efforts are necessary to alter mindsets
and practices amongst academics, administrators and students. Trinity College’s culture and
processes are often unfavourable for mature, part-time students, but they are extremely entrenched
and resist change (the College is over 400 years old). Unhelpful perceptions are encountered; many
academics and administrators see the student body only as a uniform group of full-time day (i.e. nonmature)
students; the “job of a lecturer is to lecture”, and so on. Effecting change is therefore an
exhaustive process of winning over individuals and finding workarounds where change is difficult.
Administrators have little incentive to alter processes, since change is seen as disruptive and costly;
tradition or precedent are often invoked as reasons for keeping things as they are.
It can be problematic also to motivate academics to engage in eLearning, since they have little
incentive; performance is assessed primarily through research and publication, and so investing
additional time in teaching can be viewed as wasted effort. Some have entrenched attitudes to
eLearning, perhaps borne of prior experience with earlier technologies. One senior academic made
his opposition explicit by protesting that “eLearning doesn’t work”. This experience seems to mirror
that in other established institutions (Vryonides and Vitsilakis, 2008).
Perhaps the most compelling insight which has emerged from this project is that eLearning is a team
effort and feasible only when conducted on a sufficiently large scale. Unlike conventional university
teaching, which is typically a one-person effort and inherently flexible, eLearning requires coordinated
work by academics, administrators and support staff. Isolated eLearning initiatives by individual
lecturers are time-consuming and costly, even if the technical resources are available, and difficult to
carry out successfully. This requirement for team collaboration cuts across the institution’s
management structures, which are organised along functional lines, and is therefore hard to achieve
without active management support (Jennings, 2005).
6. Conclusions and recommendations
Flexible learning is undoubtedly helpful for mature students. Our progress has been slow but we have
been encouraged by the limited successes to date with eLearning and associated improvements in
the IS programme. Our overarching experience has been that it is non-technological factors—cultural,
perceptual and organisational—which are the major barriers to eLearning, particularly in a wellestablished
university where tradition is prized (McGinnes and Dowling, 2010). A multi-pronged
approach to change is required, encompassing management, processes, assumptions, regulations,
services, methods, pedagogy and technology.
984

References
Simon McGinnes
AFLF 2007. ELearning Benchmarking Project: eLearning for mature aged learners. Australian Flexible Learning
Framework.
Billhardt, B. 2010. What Does Mature eLearning Look Like? Training and Development in Australia, Vol 37, No.
3, pp 21.
Bowman, K. & Kearns, P. 2008. eLearning for the mature age worker. Australian Flexible Learning Framework
(AFLF). National Centre for Vocational Education Research (NCVER), Australia. Department of Education,
Science and Training (DEST).
Hart, M., Burgess, J. & Betts, H. 2005. The role of eLearning in accelerating part-time student progression. In:
REMENYI, D. (ed.) 4th European Conference on eLearning. Royal Netherlands Academy of Arts and
Sciences, Amsterdam: Academic Conferences Limited.
Jennings, A. 2005. Implementing an Integrated Web-Based Synchronous eLearning Collaboration Platform at
Tertiary Level for Part-Time Mature Evening Students. MSc Thesis, Trinity College, University of Dublin.
Jennings, A., Mullally, A., O’Connor, C., Dolan, D., Parkinson, A. & Redmond, J. A. 2007. Is the Jury Still Out on
“Blended Learning”? Web Information Systems and Technologies, Vol, No., pp 355-366.
McGinnes, S. & Dowling, F. 2010. Overcoming Time, Place and Pace: Using Technology-Enhanced Teaching to
Support Mature Evening Degree Students. In: O’MAHONY, C. (ed.) Flexible Learning: Proceedings of the
Fourth Annual Conference of the National Academy for Integration of Research, Teaching and Learning
(NAIRTL). Royal College of Surgeons, Dublin, Ireland: NAIRTL.
Mullally, A., Jennings, A., O’Connor, C., Dolan, D., Parkinson, A. & Redmond, J. 2006. Use of a Web-based
Teaching Collaborative Platform at Third Level: A Qualified Success? Advances in Computer, Information,
and Systems Sciences, and Engineering, Vol, No., pp 373-379.
TCD. 2011. BSc (Hons) Information Systems Course website [Online]. Available:
http://www.scss.tcd.ie/undergraduate/bscis/index.htm [Accessed].
Vryonides, M. & Vitsilakis, C. 2008. Widening participation in postgraduate studies in Greece: mature working
women attending an eLearning programme. Journal of Education Policy, Vol 23, No. 3, pp 199-208.
985

986

Posters
with
987

988

Investigating Student Engagement With an Electronically
Delivered Simulation of Professional Practice
Olivia Billingham
University of the West of England, Bristol, UK
olivia2.billingham@uwe.ac.uk
Abstract: Electronically delivered simulations of professional practise enable educators to place students in
vocationally relevant situations or to experience events that would be impossible, or extremely difficult or costly to
replicate without recent advances in technology. The Simulations in Higher Education (SHE) initiative (E-learning
development unit) at the University of the West of England (UWE) is a focus for information and communication
technology (ICT) supported simulations that are being developed across the university. The research presented
here will focus on SIMulations in Transactional Activities (SIMITA), an online transactional learning environment
(TLE) developed at UWE as part of the SHE initiative. It is widely acknowledged that an engaged student stands
to attain more academically than their disengaged counterparts as student engagement has been positively
related to academic outcomes in many studies. In addition to the academic benefits, an engaged student is more
likely to enjoy a better quality learning experience overall. In order that students can benefit fully from the
enhanced learning experience that an electronically delivered simulation can provide, they must engage with that
simulation. A pilot evaluation of SIMITA revealed that students readily engaged with the simulation. The pilot
considered system statistics of usage, student feedback and the willingness of students to participate in noncredit
bearing activities. Following on from these preliminary findings, this poster will present the results of an
investigation that utilised self-report questionnaires, a focus group and observations of participation in an attempt
to explore student engagement with SIMITA. Further, the investigation considered other factors that might
influence the students’ engagement with the simulation. The findings presented will inform future iterations of this
investigation with SIMITA and also other simulations that form part of the SHE initiative. This research will be of
interest to educators who are considering or interested in blending online simulations into the curriculum or to
educators who are interested in student engagement with electronic methods of teaching.
Keywords: simulation, student engagement, electronically delivered simulation
1. Introduction
Electronically delivered simulations enable students to put classroom theory into practise and gain
invaluable experience of dealing with relevant, work place situations before coming face-to-face with
them in a real life work place. When using an electronically delivered simulation, students can make
decisions and mistakes in a ‘safe’ environment and can explore the consequences of their actions
with no real world repercussions. Simulations can assist students in bridging the gap between
classroom theories and real world practise before they enter the work place.
SIMITA, the simulation that has formed the basis for this initial study, currently supports 23 final year
undergraduate students studying the legal process module as part of their law degree. Students work
in firms of 4 students on a civil case scenario. Each firm handles a claimant or defendant action to the
point of negotiation and settlement with their opposing firm. Students are also able to interview clients
or seek support from their advisors (lecturers). SIMITA provides these students with the opportunity
to practise case handling skills utilising technology that feels and behaves like that of a modern
professional office. Students can store, generate, edit and update case documents and resources,
email clients and opposing firms and organise their case work between members of the firm by
assigning tasks and using the discussion area. SIMITA enables students to take an active role in
learning about legal processes.
In addition to the benefits that it offers the students, SIMITA enables the module tutors to easily see
the student’s progress in real time. This means that they can offer guidance at appropriate points in
the process and identify and alert students to mistakes as they happen. This provides the students
with a richer learning experience as they can explore and understand their mistakes at the time of
making them rather than several months later at the end of the module.
The definition of student engagement is complex and broad (Deneen, L 2010) and what constitutes
student engagement may vary between studies and research groups depending on their agenda and
objectives. Despite this variance, Trowler and Trowler have concluded that ‘the value of student
engagement is no longer questioned’ (Trowler and Trowler 2010). This is because it has been
demonstrated in many studies that student engagement is positively related to academic outcomes
including; high quality learning outcomes ((Krause and Coates 2008 cited by Bryson and Hardy
989

Olivia Billingham
2010); gains in critical thinking (Pike and Kuh 2005 cited by Bryson and Hardy 2010) and deeper
approaches to learning (Zimitat and Horstmanshof 2007 cited by Bryson and Hardy 2010).
To shape this initial investigation I have used the Hu and Kuh definition of engagement as: ‘the quality
of effort students themselves devote to educationally purposeful activities that contribute directly to
desired outcomes’ (Hu and Kuh (2001, p. 3) cited in Krause and Coates 2008).
Advancements in information and communication technologies present new opportunities to think
about student engagement and how it can be fostered in online settings (Krause 2005). Electronically
delivered educational simulations like SIMITA are an example of just such an opportunity.
This work in progress will present findings from an initial investigation into student engagement with
SIMITA.
2. Methods
Participants were 23 final year undergraduate students studying the legal process module as part of
their law degree. Eighteen students completed a self-report questionnaire and four students were
invited to participate in a focus group.
Patterns of student participation were measured by recording the number of times that SIMITA was
accessed by a unique user during the period 1 st January 2011 until 16 th February 2011. The students
had already been using SIMITA for a few months when observations began.
Students were asked to complete a self-report questionnaire containing scale or open-ended items
that explored factors that might act to influence the students’ engagement with SIMITA. These
included their initial reaction to the simulation, student perceptions of the value and usefulness of the
simulation and also the motivational orientation of the students.
The data will be analysed using Microsoft excel and SPSS statistics 19.
3. Findings
Preliminary investigation during SIMITAs pilot phase in 2009 revealed that students readily
participated with the simulation and student feedback was highly favourable (Falconer, Frutos-Perez
2009). Students accessed the TLE 7 days per week and throughout a 24 hour period except between
3am and 5am. Students also eagerly engaged in non-credit bearing activities such as designing their
firms’ website.
System statistics were collected as part of this investigation and showed that this student cohort
exhibited very similar patterns of participation to that reported with the pilot student cohort. The
students have accessed SIMITA 7 days per week and at all hours of the day except between 05.00
and 06.59 (Figure 1. Usage peaks on a Monday and during the day between 10.00 and 16.00 (Table.
1). In total, SIMITA was accessed 3133 times over the 6 week period that was observed. That
equates to 136 times per student on average.
Figure 1 shows the percent of times that SIMITA was accessed by a unique user each hour of the day
during the period observed from 1st January 2011 until 16 th February 2011. Table 1 shows the
number of times (hits) and the percent of times that SIMITA was accessed by a unique user each day
of the week during the period observed from 1st January 2011 until 16 th February 2011.
Analysis of the questionnaire and focus group data will provide a greater understanding of what these
observations of participation mean in terms of student engagement.
4. Conclusions
The patterns of participation exhibited by both this and the pilot cohort of students suggest that in
terms of participation at least, the students are engaging with SIMITA. As these participation patterns
have been presented by two independent student cohorts we can suppose that they are a function of
using SIMITA and not peculiar to the pilot student cohort.
990

Olivia Billingham
Figure 1: Hour of the day access to SIMITA
Table 1: Day of the week access to SIMITA
Day of week Hits Percent
SUN 282 9%
MON 926 36%
TUE 493 16%
WED 463 15%
THU 412 13%
FRI 366 12%
SAT 191 6%
Given that successfully engaging students clearly impacts on their learning outcomes and the
impressive levels of participation exhibited by students during both the pilot and this initial study, this
work in progress will present findings thus far, that explore or investigate the ‘student engagement’
construct in relation to this simulation. This will include consideration of the following factors that may
influence engagement with SIMITA:
Initial reaction to the simulation
Motivational orientation of users
Student perceptions of the value or usefulness of the simulation
Student perceptions of whether the simulation has helped them to learn
Student perceptions of their own self-efficacy
Acknowledgements
This research was funded by an Early Career Researcher Starter Grant from the University of the
West of England, UK. Thanks to Manuel Frutos-Perez, E-learning Development Unit, University of the
West of England, for collecting the system statistics.
References
Bryson, C., Hardy, C., (2010). Why does student engagement matter? Available:
http://www.nusconnect.org.uk/pageassets/campaigns/highereducation/student-engagementhub/resources/articles/SE-A01-BrysonHand.pdf
[03.06.11].
991

Olivia Billingham
Deneen, L., (2010). What is student engagement, anyway? Educause quarterly, 33:1. Available:
http://www.educause.edu/EDUCAUSE+Quarterly/EDUCAUSEQuarterlyMagazineVolum/WhatemIsemStude
ntEngagementAny/199393 [03.06.11].
e-Learning development Unit, Simulations in Higher Education. Available: http://www.uwe.ac.uk/elearning/she/
[01.06.11].
Falconer, L., Frutos-Perez, M., (2009). Online Simulation of Real Life Experiences; the Educational Potential.
Proceedings of ED-MEDIA 2009: The World Conference on Educational Multi-media, Hypermedia and
Telecommunications, June 22 nd - 26 th 2009. Honolulu, Hawaii. Available:
http://www.editlib.org/j/EDMEDIA/v/2009/n/1 [06.04.11].
Krause, KL., Coates, H., (2008) Students’ Engagement in First-year University. Available:
http://www98.griffith.edu.au/dspace/bitstream/10072/26304/1/53553_1.pdf [07.06.11].
Krause, KL., (2005) Engaged, inert or otherwise occupied? Deconstructing the 21st century undergraduate
student. In: Sharing Scholarship in Learning and Teaching: Engaging Students. James Cook University
Symposium 2005, James Cook University, Townsville/Cairns, Queensland, 21-22 September 2005.
Available: http://www.griffith.edu.au/__data/assets/pdf_file/0011/37496/JCUKeynote2005.pdf [07.06.11].
Trowler, V., Trowler, P., (2010). Student Engagement Evidence Summary. The Higher Education Academy.
Available:
http://www.heacademy.ac.uk/assets/York/documents/ourwork/studentengagement/StudentEngagementEvid
enceSummary.pdf [02.06.11]
992

Instrumental Distance Learning in Higher Music Education
Karin Levinsen 1 , Rikke Orngreen 1 , Mie Buhl 1 , Marianne Løkke Jakobsen 2 and
Jesper Andersen 2
1 Danish School of Education, Aarhus University, Denmark
2 The Royal Danish Academy of Music, Denmark
kale@dpu.dk
rior@dpu.dk
mib@dpu.dk
marianne.jakobsen@dkdm.dk
jesper.andersen@dkdm.dk
Abstract: This brief paper presents a research proposal based on an investigation of the complex challenges
and potentials of using video conferencing in connection with teaching and learning processes in the domain of
higher music education. A historical outline of the research and development project is provided, followed by the
preliminary findings of the initial activities. These findings have led to further research questions that would be
relevant to explore in future.
Keywords: eLearning, video conferencing, multimodality, teaching, design for learning
1. The project
In 2009 the Royal Danish Academy of Music (RDAM) launched an innovative development project on
distance learning in which instrumental teaching takes place in a new video conference environment
featuring advanced technological equipment. The aim is to develop a sustainable teaching practice at
RDAM that provides excellent competence building opportunities for teachers and technicians.
RDAM provides instruction in a broad range of types and styles of music and the majority of courses
offered are aimed at performance at the highest level. In addition to instrumentalist and vocalist
programmes, RDAM offers specialized courses within music teaching, composition, conducting,
church music, and recording. RDAM also offers a three-year bachelor programme and a two-year
master's programme in which students specialize in a particular field. Following their master's degree,
students may continue to an advanced two-year postgraduate diploma programme, which concludes
with a public debut concert.
RDAM sees video conferencing as a means to expand the academy’s international cooperation, to
share expertise and to establish networks and new performance spaces for joint high profile master
class teaching. Therefore, in connection with the project, RDAM relies extensively on international
cooperation with partners such as the Manhattan School of Music (MSM), the Sydney Conservatory
of Music and the Eastman School of Music, University of Rochester
The research perspective of the project was established in early 2011, when RDAM contacted the
research programme ICT and media in a learning perspective at Aarhus University (AU), Faculty of
Arts. The combined research and development project rests on the assumption that implementation of
video conferencing as a teaching methodology is a process of organizational change and adjustment
that challenges traditional pedagogy and the everyday practices of students and teachers.
The project is in an early stage. In May 2011, RDAM submitted an application to the Danish Ministry
of Culture for additional support to further the project. The goal is large-scale application in early
2012. However, the initial activities have already produced areas of interest for further studies and
experiments.
2. The research literature and key concepts
Literature on video conferencing relates primarily to surgery and nursing, areas that depend on high
visual quality. Marrow et al. (2002) argue that video conferencing scaffolds students’ reflexivity and
ability to solve problems. Maruping & Agarwal (2009) list advantages such as immediate feedback, a
repertoire of multimodal sign systems and sense modalities, synchronous communication and social
closeness. One large, solitary quantitative study finds that video conferencing is pedagogically neutral
compared to traditional teaching (Cavanaugh 2001). Hedestig and Kaptelinin (2005) find that
breakdowns are unavoidable. They argue that rather than trying to eliminate breakdowns, video
993

Karin Levinsen et al.
conference organizers and technicians should try to overcome them in order to preserve the virtual
environment as a space where humans can be and interact rather than a place or a configuration of
individuals and artifacts. Even with the present speed of transmission, delay affects dialogue functions
such as turn-taking, sequencing and repair (Ruhleder & Jordan 2001, Marrow et al. 2002), which in
turn affect the experience of trust and telepresence (Ruhleder & Jordan 2001). People can learn to
compensate but novices and sporadic users may find this difficult. Ruhleder & Jordan found that
people try to avoid situations affected by delay. Hedestig & Kaptelinin (2005) found that in most
cases, the technician becomes the facilitator who handles both technical and collaborative
breakdowns. According to our initial observations, these findings seem to be general to video
conferencing no matter the subject.
The literature concerning music education is scarce and represents mainly case reports rather than
research. Only few titles address music education at a level comparable to the music academies. This
is confirmed in a recent study – Vi r Music - presented at the Distance Learning Conference at RDAM
in august 2011 (Nissi, 2011) However, recent quantitative studies provide interesting examples of
typical behaviours in music education (Orman & Whitaker 2010). The literature review shows how
time in one-on-one sessions is spent on the teacher talking, playing (modelling) and coaching verbally
and with non-verbal gestures. The analysis of video conference classes examined: “sequential
patterns of instruction, focus of attention, amount and type of performance, eye contact, and other
nonverbal behaviors” (ibid, p. 4). Though there are many other nuances to the study, it is interesting
that the researchers found that students performed/played more frequently (22%) and that eye
contact increased during distance lessons.
3. Initial activities
New York set-up Copenhagen set-up
Danish technician at work
Figure 1: From the March workshop between RDAM, Denmark and MSM, New York
During 2010 the project explored possible ways of using instrumental/vocal distance learning in
experimental set-ups. For example, a teacher from RDAM directed a string quartet in Beijing. These
activities were financed by the Centre for Development of Human Resources and Quality
994

Karin Levinsen et al.
Management as a competence development project aimed at faculty members. This project led to
RDAM investing in the advanced equipment necessary to ensure the quality of especially sound
transmission demanded by high-level music distance learning. Accordingly, RDAM’s management
designated distance learning as a priority development area in the coming years and they expect the
project to link art communication, education, pedagogy and scientific research in a unique way.
In March 2011 RDAM hosted a seminar on distance learning in collaboration with MSM. This seminar
marked the peak so far of the test phase for the implementation of the new technology as the
backbone of a teaching environment and a technological setting.
In addition to the teaching calls between New York and Copenhagen, there were many technical calls
to test sound, network technology, and the collaboration between the distributed technological teams.
The seminar provided the scene for the initial observations of the researchers from Aarhus University.
Each session was followed by discussions amongst teachers, students and researchers. Apart from
the immediate surprise of the test participants at how comfortable they felt with the video conference
situation, the discussion revolved on the pedagogical and interpersonal challenges that arise when
teachers’ and students’ interaction is mediated through technology. But there were also thorough talks
about the technical limitations/opportunities connected to the issue.
There is a large body of research that addresses the relation between video conferencing, teaching
and learning. However, the literature does not address the special needs and circumstances that are
relevant to high level music education from a pedagogical perspective. In August 2011 RDAM hosted
an international conference to contribute to creating international dialogue and supporting knowledge
sharing. The conference also provided an opportunity for the researchers from Aarhus University to
expand their knowledge of the field and to present the research project internationally.
4. Initial findings
The majority of the participants (teachers and students) at the March seminar had never tried this kind
of dedicated video conference set-up before, whereas a few had tried Skype as a teaching platform.
They all expressed that they perceived a potential and that it worked surprisingly well. However, it
became obvious that available space in terms of arrangement, distance and visual direction are
factors of importance as are monitor size and position. The way the musicians in an ensemble are
situated in relation to each other within the available space affects their experience of closeness.
Vocal soloists differ from instrumental soloists as they use the body as their instrument. It is important
that the teacher can observe both the face and the full body. This affects transmission of sound and
image, light settings and the quality and size of screens. Different constellations have to be explored
in various situations as instrumental or vocal soloists and ensembles create different challenges and
needs for robust video conferencing pedagogy and technological handling.
The participants experienced that telepresence depends on collaboration between technicians at both
ends as well as between musicians and technicians and that the teachers must be able to instruct the
technicians about their specific needs. For example, an unacknowledged difference in sound
experience may lead the teacher to instruct students to adjust their technique or
expression/interpretation of the music in ways that may appear “wrong” in an analogue performance.
This points to the need to explore means of expression that can mediate the pedagogical intention of
a specific session and allow transfer from pedagogical needs to operational technological solutions.
The RDAM teachers had to let go of physical interaction such as conducting, counting the beat or
playing simultaneously due to the slight delay that persists despite extremely high bandwidth. The
teachers modified their practice and adjusted to the conditions of the technology by verbalizing their
body language. Finally the teachers experienced that the students may feel uncomfortable if they do
not know their teacher well or if the set-up makes it difficult for them to see and sense the teacher.
They also want to know who else is present in order to relax and experience telepresence. The
challenge that needs to be explored here is that the communication between teacher and learner
changes modality. In order to develop a robust pedagogy it is important to know the consequences of
changes in modality as they may open up to new possibilities as well as losses as not all
communication is fit for remediation.
In the discussions, the teachers proposed solutions to these challenges. Some solutions apply to the
technology used and the physical set-up, some to pedagogy and yet others to the need to create trust
995

Karin Levinsen et al.
in order to support telepresence. However, this is a complex field and it is not obvious where the limits
between technology, human interaction, pedagogy and mediation are to be drawn. This is the major
research challenge of the project.
5. Future perspectives
Based on these findings the shared goal is to produce knowledge that may enhance the competence
building of teachers and technicians and develop adequate pedagogical theory and practice for video
conference-based music teaching. Methodologically, the project explores how and at what costs and
benefits video conferencing affects teaching and learning processes and investigates whether the
technology allows for new educational forms, for example, peer-to-peer activities among students,
which may expand the educational repertoire.
Due to the nature of the field and the lack of knowledge, future research will be conducted as action
research in which researchers collaborate with the actors in the field where new methodologies are
being developed and new technology is being applied. This means that the research takes its point of
departure in the context and the practice and needs of the participants in the project.
In the autumn of 2011 the project will be further evolved regarding:
Identification of areas suitable for deep empirical studies
Formulation of research question(s)
Research design
The establishment of an international consortium of partners
Already at this stage, RDAM and the researchers involved are convinced that the project will
contribute not only to the implementation of video conferencing at RDAM but also to the development
of general principles of pedagogy for high level music education and teacher competence building in
the globalized community of high performing academies of music. However, the research may
additionally be expected to contribute to a wider area of video conference pedagogy, as the
multimodality and sensitivity that are key issues for conservatories may be explored in ways that may
enhance other uses of video conferencing.
References
Augestad & Lindsetmo (2009) “Overcoming Distance: Video-Conferencing as a Clinical and Educational Tool
Among Surgeons”, World Journal of Surgery 33, pp1356–1365.
Cavanaugh (2001) “The Effectiveness of Interactive Distance Education Technologies in K-12 Learning: A Meta-
Analysis”, International Journal of Educational Telecommunications 7(1), pp73-88.
Dammers (2009) ”Utilizing Internet–Based Vidoconferencing for Instrumental Music Lessons”. Update 28(1), 17-
24.
Hedestig & Kaptelinin (2005) “Facilitator's Roles in a Videoconference Learning Environment”, Information
Systems Frontiers 7(1), pp71–83.
Marrow, Hollyoke, Hamer & Kenrick (2002) “Clinical supervision using video-conferencing technology: a reflective
account”, Journal of Nursing Management 10, pp275–282.
Maruping & Agarwal (2004) “Managing team interpersonal processes through technology: a task-technology fit
perspective”, Journal of Applied Psychology 89, pp 975–990.
Nissi, O. (2011) Presentation of the Vi-r-Music project at Distance Learning Conference, RDAM August 2011.
See also http://virmusicfinalreport.blogspot.com/p/elearning.html
Orman & Whitaker (2010) “Time Usage during Face-to-Face and Synchronous Distance Music Lessons”, The
American Journal of Distance Education 24, pp 92–103.
Ruhleder & Jordan (2001) “Co-Constructing Non-Mutual Realities: Delay-Generated Trouble in Distributed
Interaction”, Computer Supported Cooperative Work 10, pp 113–138.
996

Reflections on Academic Blogging as a Vehicle for
Professional Development
Peps Mccrea
University of Brighton, UK
p.j.mccrea@brighton.ac.uk
Abstract: This paper is a work-in-progress: it outlines the earliest stages of a larger inquiry into the affordances
of academic blogging as a vehicle for professional development. During this time I have been immersing myself
in the online environment and grappling to figure out what academic blogging means to me. To offer the reader a
rich insight into this experience I present a reflective account together with an analysis of these formative
experiences of social media. Higher Education (HE) in the Western world is currently grappling to position itself in
a shifting landscape of economic rationalisation, too-fast-to-keep-up technological innovation, and escalating
marketisation (OLTF 2011). Blended learning and social media are being explored as potential solutions to these
pressures. In this paper I explore the potential opportunities and challenges afforded by academic blogging for
professional development. The concepts I have used as a framework for discussion are: exposure, engagement
and networking. It is argued that these features combined with the non-hierarchical structure of social media
(Siemens and Weller 2011) mean that academic blogging has the potential to create a self-organising, highly
responsive and digitally reflexive staff, for a relatively low cost. However, not all academics or institutions will be
comfortable with the time investment or risk associated with writing in the public domain. However, if institutions
refuse to accept blogging as a form of scholarly activity then academics may struggle to position themselves as
public intellectuals in the digital age (Kirkup 2010).
Keywords: academic scholarship, professional development, online identity, social media, blogging
1. Introduction
This paper aims to capture the emerging experiences of my inquiry into the affordances of academic
blogging as a vehicle for professional development. It is composed of a reflective account together
with a brief and somewhat superficial analysis of my formative experiences of social media. I hope
that this approach will offer the reader an authentic and rich insight into this newish area of scholarly
activity (Hayler 2010). Comments on the narrative in Mccrea (2011) such as 'wondering if you
somehow read my mind about tweeting, blogging etc.' and 'best insight into power of blogging, Twitter
and network I have ever read' could be see to add some credibility to the experiences described.
This paper comes at a time when Higher Education (HE) in the West is grappling to position itself in a
shifting landscape of economic rationalisation, too-fast-to-keep-up technological innovation, and
escalating marketisation (OLTF 2011). As blended learning becomes increasingly attended to as a
panaceaic response to these pressures, the demand for digitally literate academic capacity grows. I
suggest that blogging challenges existing rubrics for professional development in HE, and offers a
exposed, engaged and networked experience that has the potential to foster robust levels of digital
literacy over time.
One day someone convinces you to try Twitter. You don’t like it. But you persist. You
respect your friend and they say it is not like Facebook. You upload your best profile pic
and fill in your Bio. You only have 160 characters. Who are you? Your professional self,
your personal self, a mixture, or the whole bag? You have your first online identity crisis.
A couple of weeks of frustration go by and you start to follow people who are of interest
to you. They are ‘tweeting’ some interesting stuff and even seem to be having
conversations. How do they find the time? Do they not have jobs? You are becoming
increasingly aware that you are watching but not contributing. You are ‘lurking’.
Most of the people you follow blog when they want to say something substantial. And
then tweet about it. And then other people you follow comment. And then tweet about it.
You consider your first tweet. The big one. Oh how they will judge you. You type it,
delete, retype, and then leave it for another day.
Your first few tweets come and go. Not much happens. Your friends congratulate you.
You are a bit disappointed. You comment on a few blogs. They reply! You feel a little
excited. You look around, but no-one’s watching.
By now you have a good handle on Twitter, and you use it as a kind of personalised
newsfeed. You are up to the minute in your area of interest and feel a greater sense of
997

Peps Mccrea
professional engagement. You start to share some things you have come across, even
have a few conversations with the people you follow. You want more.
2. Engaged
My first suggestion is that when participating through social media, the combination of affordances
can lead to a heightened sense of academic engagement. These affordances include:
Asynchronous communication enabling both a high degree of time-to-reflect flexibility and
potentially permanently open conversations
Blogging as a creative endeavour offering an active, satisfying and sustainable form of
professional development (Mackey and Evans 2011)
Ideas-oriented conversation providing an intellectually fertile environment in contrast to
predominantly outcomes-oriented organisational set-ups where dialogue can often be tempered
by managerial and political interference
How does this 'fit' with current institutional dialogues? Davies and Merchant (2007) argue that social
media enriches offline relationships: that in addition to fundamentally changing how we write and
communicate, it also changes how and with who we interact, including those with which we share our
working days. In an unexpected twist, could this change in local practice be its most significant impact
of academic blogging in HE?
You find Wordpress and spend days choosing a title. You consider your first post. What
story do you have the authority to tell? There are so many people blogging. How can you
contribute to the conversation? It’s a bit daunting. But then, so was Twitter initially.
You come across one blog post that really gets you going. You comment but there is
more you want to say. You realise that this is your first blog post. You want to be a bigger
part of this conversation.
It takes ages. Your best writing. You click ‘publish’. Not much happens. You exposed
your thinking to the world and nothing happened! You find yourself seeing things in terms
of your next blog post. Someone says something interesting at lunch and you make a
note in your iPhone.
You write a few more posts. One is actually quite interesting. This time when you tweet
about it, @timbuckteeth picks it up and tweets about it. Suddenly you are exposed to 10
000 people! Two of them comment on your post. A bunch of people follow you. And then
a couple more next day. Cool.
3. Exposed
My second suggestion is that academics who contribute via social media encounter a form of
increased conceptual exposure that is not normally afforded by contemporary professional
development. Combining norms of periodic self-disclosure and risk-taking with limited audience
control (Davies and Merchant 2007), blogging has compelled me to regularly and publically explicate
my self, my thinking and my position on live as well as age-old issues.
However, with exposure comes vulnerability. Recent Twitter flurries (think footballer) have highlighted
how social media can destroy professional reputations within hours. Krikup (2010) suggests that it
may be safer to practice a genre of writing that is less likely to bring you into potential conflict with
your employer. Public exposure is not something all academics or institutions will be comfortable,
particularly if practising any form of institutional critique.
You begin to rethink how you blog. The academic style of writing. You think about your
use of social media. Is it okay to check Twitter at work? You begin to think about how
much you are learning. You wonder why more people aren’t doing this.
You write a blog post about a paper you have read. The author comments on your blog
post. You reply. You are having a conversation with someone you read! Wow. Why aren’t
you having more of these conversations at work? You feel engaged in academic thinking.
All the time.
You wonder about blogging as a form of scholarly activity. You begin to see commenting
as a kind of peer review and blogging as a kind of open access reporting. You think
998

Peps Mccrea
about how blogging has changed how and who you read. You wonder if this is what they
mean by an ‘online community’.
You find yourself not only questioning academic reporting, but learning in the digital age.
You find yourself equipped with a new vocabulary. You use #hashtags with fluency. You
feel an affinity with Connectivism. You think about the idea of learning existing within
technology. You think about how much you think.
You think about increasing economic pressures and the marketisation of Higher
Education. You think about digital literacy and digital collaboration. You think about your
colleagues who are still living in the unblended world. You begin to see yourself
differently. You wonder about the future. You wonder about how you can contribute
more.
You find yourself wondering about institutional practices. About informal learning. And
about education. Again, and again and again. You are engaged. You are exposed. You
are networked. You are learning.
4. Networked
My third suggestion is that due to its lack of geographical and chronological limitations, open access
to people with similar interests, and ease of making and breaking ties, academic blogging allows you
to become part of a personalised network. There are advantages to this, including rapid access to a
vast range of resources and expertise - I had over 10 responses from Professors and Technologists
in the hours after posting this draft narrative on my blog (Mccrea 2011).
Kjellberg (2010) asks whether the most significant benefit is the feeling of being part of something
bigger: the opportunity to connect to a specific context. However, I wonder if its homophilic
affordances could also be social media's greatest weakness, through diminished social diversity.
5. Conclusion
In conclusion, I suggest that the combination of these affordances offers a potentially rich and
transformative vehicle for academic professional development. It allows individuals to make sense of
the increasingly blurred boundaries between online and offline, between formal and informal, between
traditional and open scholarship (Pachler and Daly 2009). However, this comes with an investment in
time that not all will be comfortable with.
At an institutional level social media deserves serious attention - due to its non-hierarchical
configuration (Siemens and Weller 2011) it has the potential to create a self-organising, highly
responsive and digitally reflexive staff, for a relatively low cost, albeit with a new set of complications,
as academic thinking and practice stray further into public transparency. As Kirkup (2010) argues: if
institutions refuse to accept blogging as a form of scholarly activity then academics will struggle to
position themselves as public intellectuals in the digital age.
References
Davies, J and Merchant, G (2007). Looking from the inside out: academic blogging as new literacy. Lankshear,
C and Knobel, M (eds.) A new literacies sampler. New York, Peter Lang, 167-19
Hayler, M (2010) Autoethnography: making memory methodology. Research in Education (R.Ed), 3(1), 5-9
Kirkup, G (2010) Academic blogging: academic practice and academic identity. London Review of Education,
8(1), 75-84
Kjellberg S (2010) I am a blogging researcher: Motivations for blogging in a scholarly context. First Monday,
15(8)
Mackey, J and Evans, T (2011) Interconnecting networks of practice for professional learning. International
Review of Research in Open and Distance Learning, 12(3), 1-18
Mccrea, P (2011) Unconventional approach to a paper about blogging. Learnerosity [Blog],
[accessed 01/06/11]
OLTF (2011) Collaborate to compete: seizing the opportunity of online learning for UK higher
education. Online [accessed 06/03/11]
Pachler, N and Daly, C (2009) Narrative and learning with Web 2.0 technologies: towards a research agenda.
Journal of Computer Assisted Learning, 25(1). p 6-18
Siemens, G and Weller, M (2011) Higher education and the promises and perils of social networks. Revista de
Universidad y Sociedad del Conocimiento (RUSC), 8(1), 164–17
999

A Framework for Understanding Online Learning
Communities
Sónia Sousa 1 , David Ribeiro Lamas 1 , José Braga de Vasconcelos 2 and Ilya
Shmorgun 1
1
Institute of Informatics, Tallinn University, Estonia
2
Faculdade de Ciência e Tecnologia, Universidade Fernando Pessoa, Porto,
Portugal
sonia.sousa@tlu.ee
david.lamas@tlu.ee
jvasco@ufp.edu.pt
ilya.shmorgun@gmail.com
Abstract: This paper attempts to provide a deeply understand of how a community are developed in an online
learning context. This research was developed as a part of a broader research purpose that aims to deeper
understand how does trust relate to the activity patterns of the online learning contexts. Results gather there from
supported a survey design on relate trust with online activity patterns as well supported the development and
implementation of an ontology that aims to facilitate a systematic recording of online learning community
manifestations as an effort to understand their life cycle. The relevance of this paper is grounded in the changes
that are taking place in today learning and teaching contexts. Among other relevant aspects a key features of this
conceptualization – on “what are online learning communities” – is to provide a broader perspective and
understand on possible potential effects that those changes can have in our daily relationships, in the way we
seek and acquire information, or even in the way we teach and learn.
Keywords: online learning communities, concept maps, community development
1. Introduction
Our social relations with the online media in general or to any technological medium ever since
enabled the connection of people with corresponding interests, regardless time or space restrictions
and always allowed alternative forms of communication for people who already know each other
primarily in real life, also served as support to a variety of social and professional goals, set a ground
for flourishing social networking and collaboration. That is why community is quite possibly the most
over-used word in the Net industry nowadays, community is the ability to connect with people who
have similar interest, it may well be the key to the digital world, but the term has been diluted and
debased to describe even the most tenuous connections, the minimal interaction. More, as the
Internet has afforded a proliferation of community building tools and organizations, communities
assumed the participation factor for granted, leading some online community initiatives to emptiness
and dissent. What points to the fact that although firmly routed into Internet’s tradition, online
communities are inherently fragile which leads to the necessity of understanding what fosters the
success of an online communities. This is especially within a teaching and learning community
environment, where learning continuous to be an essential social process, despite the tendency of
some of us to shut ourselves away and sit in Rodinesque isolation. The success of the learning
outcomes depends strongly on the online learning environment’s ability to support a sense of
community. On the other hand, it is now a widely belief that communities, societies and culture as a
whole are tailored by the diversity of individuals, who contribute to the intellectual climate and
technological infrastructure of society, rather than the effects of media itself.
2. Fundamental notions of building community-learning environments
The development of communities has been an aspect of the Internet since it beginning, and it has
ever since enabled the connection of people with corresponding interests, regardless of time and
space restrictions. And, in spite of in the beginning the Internet was know or seen as a mere
repository of information and data where online community members does not necessarily implied a
strong bond among the community, that has changed with the increased availability of user-generated
content mechanisms and with the growth of social networking services. The Internet become the hub
of socialization; become the logical extension of our human tendencies toward togetherness, that
have been tailored our society and our cultures. Those reflected tendencies towards an individualcentered
approach whereas group-centered activities, creating context where each individual
contributes to the intellectual climate and technological infrastructure of society, rather than the effects
1000

Sónia Sousa et al.
of media itself. Online learning communities one of the examples of such phenomena usually built
upon multidisciplinary and innovative collaborative communities. Our social relations with online
media in general or any technological medium ever since enabled the connection of people with
corresponding interests, regardless of time or space restrictions and always allowed alternative forms
of communication for people who already know each other primarily in real life, also served as support
to a variety of social and professional goals, set a ground for flourishing social networking and
collaboration.
In that perspective communities, societies and culture as a whole are tailored by the diversity of
individuals, and each one contribute to the intellectual climate and technological infrastructure of
society, rather than the effects of media itself. Notions that are connected with the community
development process as the duality between the human need for participation, based on their daily
social experiences and on the reunification of those experiences, i.e describe the learning performed
in terms of participation and reification (Lave and Wenger, 1991; Wegner and Snyder, 2000). From
that point of view and according to Lave and Wenger (1991) there is no such thing as "learning" suis
generis, but only changing participation in culturally design settings of everyday life. And this
participation process occurs as a process of changing understanding in practice that is learning. In
other words learning emerges from the duality between the human need for participation, their daily
social experiences and on the reunification of those experiences, i.e the learning is performed in
terms of participation and reification processes as Wegner and Snyder (2000) describes it.
An online community then, can be seen as a facilitator of the knowledge construction process. As
online communities and their environments can provide the needed resources for their members to
learn, can facilitate the learning process, as in this virtual spaces learning occurs as part of each
individual network and their shared relations and interactions (i.e. results within a specific social
context and results from their interactions with their media artifacts) (Pudelko, 2003). Following that
idea, the idea that a virtual learning community-environment can assume an important part of toady’s
virtual learning process, as a learning communities per si as their participants learning while act in
community. In other words a virtual community beside provide members a space for participation also
provide the necessary tools and support for knowledge construction environment, and that is what
shapes individual development and learning (Paavola et.al., 2004; Bandura, 1969; Lave and Wenger,
1991; Brown and Duguid 2000). The main rational of this paper is built on the belief that by
understanding online learning communities we will be able to better foster their inherent formal and
informal learning processes in a number of contexts. This paper describes and discusses the
development and implementation of a research framework designed towards a deeper understanding
of what are online learning communities and how they are seen nowadays within the education
contexts.
3. Research approach
The framework designed here is based on a design-based research approach which is focus on
characterizing on the understanding of community development and influence in all its complexity,
when in online-learning situations, focusing especially in ways to potentiate people’s learning activities
that demands co-working actions and collective learning and sharing activities. This research
approach is focused on a participatory approach design. The major contributions of the research
herein depicted are twofold: On one hand, we identify the main components of community-learning
environments, built on top of an extensive literature review. On the other hand, we outline a
collaborative research strategy to explore people’s perspective of community-learning environments.
This approach included two distinct participatory design sessions: one session deployed with
eLearning expertise and another session deployed with eLearning practitioners. Both participants
were given a focus question: what are online learning communities – and to both were asked to
design a concept map that answer that question. This concept maps provided preliminary exploratory
insights that leaded to the development of overall concept map that defines an online learning
community.
4. Results discussion
Results gathered indicate similarities between experts and non-experts views of online learning
communities, especially when representing the teacher, learner and technology role within the
community space. But, there were as well divergent views in some on how these roles interconnected
with each other and how they are represented within the concept map structure. Also, we argue that
most virtual communities can be considering as learning communities. From that point of vie makes
1001

Sónia Sousa et al.
little sense referring to online learning communities, but more as online communities. Although not all
communities can be classified as emerging communities of practice, as Wegner and Snyder (2000)
argues, since not all involve same learning goal activities and therefore provides distinct ways of
learning. For example there are those that are developed on a based of a common interest, for
example on a relationship based or just for knowledge exchange purpose. There are those who are
formed for answer a specific goal or purpose, to solve a project or solve a specific problem. Or, there
are those who are formed to serve specific education contexts, like support learning or teaching
activity in a formal learning context. Agreeing that there are three main points that define a online
community: (1) community goals, associated with the activity that will involve the construction of
knowledge, (2) their level of engagement and (3) their evolution and sustainability through time.
4.1 eLearning practitioners view
The concept map seen on Figure 1 was created based on feedback of eLearning practitioners. Their
ideas were collected and a concept map created to illustrate their understanding of the domain. The
primary concepts have been marked with yellow. These are: online learning communities, learners,
teachers, Internet and networks. The same primary concepts are present in the other concept maps
as well, which represent the understanding of the domain from the perspective of experts and
researchers.
Figure 1: The eLearning practitioners view on what are online learning communities
4.2 eLearning expert view
The concept map seen on Figure 2 demonstrates the experts’ understanding of the domain of online
learning communities. Here the same primary concepts are marked with yellow as well.
4.3 eLearning researcher view
The concept map from Figure 3 demonstrates the research’s understanding of eLearning. The same
primary concepts are identified although the diagram offers a slightly different perspective on the topic
than the one seen on Figure 1 and Figure 2.
4.4 The overall view
Based on the Figures 1, 2 and 3 an overall view concept map was created, see figure 4. The creation
of the overview concept map was conducted according to the following procedure:
1002

Sónia Sousa et al.
The concepts on the each original concept map were assigned a unique color. Thus the concepts
on Figure 1 were assigned a red color, the concepts on Figure 2 – a blue color, and the concepts
on Figure 3 – a yellow color.
The three original concept maps were then copied onto a single canvas and duplicate concepts,
found on the original maps, were merged together.
As the duplicate concepts were found and merged, they were assigned a new color. The merging
of yellow and blue concepts was marked as green; red and yellow concepts were marked as
orange; blue and red were marked as purple; red, yellow and blue were marked as white.
Figure 2: The eLearning experts view on what are online learning communities
Figure 3: The eLearning researcher view on what are online learning communities
1003

Sónia Sousa et al.
Figure 4: The eLearning practitioners, experts and researcher view on what are online learning
communities
1004

5. Closing remarks
Sónia Sousa et al.
As a result of this procedure the three original concept maps were put together and all the duplicate
concepts eliminated. The produced concept map is rather dense, as a lot of concepts needed to be
put together on a single canvas.
The purpose of this new concept map is to fully demonstrate the understanding of the eLearning
domain by the three initial groups: the practitioners, the experts in the field of eLearning and the
researcher. The concepts marked as green, orange, purple and white stand out visually and signify
the concepts that two or more initial target groups share, when describing their understanding of
online learning communities.
In sum, input gathered from the present research approach provided preliminary exploratory insights
that helped on the design of a survey questions that relates trust with the online learners activity
patterns.
References
Bandura A. (1969). Social Learning theory of identificatory process. Goslin, David A. Handbook of socialization
theory and research / edited by David A. Goslin Rand McNally, Chicago.
Brown, J. S., & Duguid, P. (2000). The Social Life of Information. Boston, Massachusetts: Harvard Business
School Press.
Henri, F., and Pudelko, B. (2003). Understanding and analysing activity and learning in virtual communities.
Journal of Computer Assisted Learning (19), 474-487.
Lave, J. & Wenger, E. (1991), Situated learning: Legitimate peripheral participation, Cambridge University Press ,
New York .
Paavola, S., Lipponen, L. and Hakkarainen, K. (2004) Models of Innovative knowledge Communities and Three
Metaphors of Learning. Review of educational research 2004. Vol. 74, 4 (pp. 557-576)
Wenger, E., and Snyder, W (2000) Communities of Practice; the organisational frontier Harvard Business Review
Jan-Feb, pps 139-145.
1005

Trust in Distributed Personal Learning Environments: The
Case Study of LePress PLE
Sónia Sousa, David Ribeiro Lamas and Vladimir Tomberg
Institute of Informatics, Tallinn University, Estonia
sonia.sousa@tlu.ee
david.lamas@tlu.ee
vtomberg@tlu.ee
Abstract: This paper reports a research effort to improve the learning process through the use of blogging. It
main rational is built on the possibility of eliciting a set of potential trust effect in the design of a particular online
learning space, called LePress. The LePress is a WordPress plug-in that aims to bring assignment related
workflow management and context specific semantic data exchange to WordPress. It describes the aplication of
a mixing method approach, which, interconnects two research works results, and is based on the observation of
online learning communities in action. One research refers to a development of a blog extension prototype called
LePress. The other refers to the results gather from a survey deployed to better understand the effects and
influence of trust people’s online activity and sharing patterns. The innovative nature of this research can be
characterized by the elicitation of a set of potential effects of trust in the design of particular software tool which is
used for enhancing learning and problem solving practices in working life contexts. It relevance is grounded in
the changes that are taking place nowadays in the education contexts. Helps bordering educators and
researchers’ perspectives on the possible potential effects that those changes can bring to our daily learning
activities.
Keywords: trust, blogs, learning flow, WordPress, personal learning environments
1. Introduction
In the early nineties, newsgroups were the primary driver force of the Internet communities, but since
then various other means of establishing online communities has proliferated.
In the beginning the establishment of online communities took the participation factor for granted,
leading some of these initiatives to emptiness and dissent. As this view of the Internet as a mere
repository of information and data, not necessarily implied a strong bond among community
members.
An example of that is the email distribution list, which may have hundreds of members, but the
communication-taking place in there may be merely informational. It members may remain relative
strangers and the membership turnover could even be high.
But, that has changed with the increased availability of user-generated content mechanisms and the
growth of social networking services.
Nowadays, the Internet has become the hub of socialization. In other words it become a logical
extension of our human tendencies toward togetherness. That has been tailoring our society and our
culture. As it enable the possibility to create contexts where, each individual or group, contributes to
the intellectual climate and technological infrastructure of society, rather than the effects of media
itself.
These online communication mediums have become a supplemental form of communication between
people who know each other primarily in real life supporting a variety of social and professional goals.
1.1 Community building learning environments
Although this community-building context is nothing new, as It exists longer before the Internet
inception. The Internet, however had contributed far most for it proliferation.
As well the social networking concept have been around much longer than the Internet or even mass
communication. People have always been social beings and our ability to work together in groups,
creating value that is greater than the sum of its parts, is one of our greatest assets.
1006

Sónia Sousa et al.
Again the current Web (which is a much different entity than the Web of a decade ago) has change
that as well. As it created the possibility of everyone develops content and to interact with other’s
without time or space frontiers. Made possible a new form of user empowering.
Most social network services are now web based and also provide means for users to interact over
the Internet. Social networking services, or online social services, are now focused on building and
reflecting social networks concepts or social relationship concepts through the Internet.
All those possibilities had changed the education, as we know it. Changed the way teacher/student or
student/student relate and learn. The learning online possibility is now seen as a natural process.
2. Research approach
The innovative nature of this project can be characterized by the elicitation of a set of potential trust
effects in the design of particular software tools for enhancing learning and problem solving practices
in working life contexts.
It focuses on the observation of possible interrelations between learners’ trust in these online
environments and their attitudes towards openness and sharing in a blog-based personal learning
environment LePress. By understanding this relation, we will be able to contribute to the design
decisions regarding the LePress. We believe that this understanding can, then, lead to the increase of
people’s participation within these contexts.
We initiate, then, our work towards understanding the interrelation between trust and people’s
attitudes towards sharing by first, analyzing a blog plug-in prototype called LePress.
Then observe and elicit a set of potential trust effects in the design of this particular software tool.
Aiming, in the end, to contribute for enhancing some trust related working life contexts that leads to
improve people’s participation within this communication mediums.
2.1 Trust effects in Personal Learning Environments
Learning, nowadays, can no longer be seen as a passive activity, it becomes an autonomous activity,
where learners are openly responsible for the learning processes as well as for actively participating
towards it.
Communicating tools that are present nowadays show a clear tendency to togetherness and towards
a strong and identifiable need to communicate, interact and exchange information.
These tools had transformed the Internet into a hub of socialization and are more then ever
representing the logical extension of our human tendencies toward togetherness.
These tools are somehow tailoring our society and culture in general. These technologically enhanced
social contexts represent new identities that are being formed and evolving individually or collectively
and each one is tailored by each individual’s diversity (Putnam, 2007).
Concepts, where trust, plays foremost a role with ubiquitous importance and relevance. As those
environments demands for highly motivated people. People that is capable of self-actualization. Also,
those are individuals that need to feel safe and accepted in their relationship space in order for them
to be willing to participate and engage in a mutual dialogue.
Trust affects one’s predisposition to interact with each other. Trust, for instance, can shape our
willingness to rely on others and our ability to believe that each other’s actions will eventually lead to
expected results. Or it can influence our beliefs, attitudes and behaviors towards learning and sharing.
Trust, is a key element for provide cooperation and collaboration practices among individuals
(McAllister, 1995; M. Bachrach and D. Gambetta, 2001; Tschannen-Moran, 2001; D.Gambetta, 1998;
Dasgupta, 2000).
1007

2.2 LePress
Sónia Sousa et al.
Personal Learning Environments (PLE) tend to lead to the facilitation of learner-based constructivist
learning processes and tend to promote the usage of open resources, and Web 2.0 tools in opposition
to the teacher-centric tactics enabled by typical, Web 1.0 associated LMS.
Blogs it is other so-called Web 2.0 tools that prove to be a suitable building block of more learnercentered
learning environments. And it has become, especially in the last five years, one of the major
trends in the domain of technology-enhanced learning.
Whereas, this trends towards blogging, clearly relates to the simplicity of publishing, reading and
discussing blogs’ mechanisms. We still miss relevant functionalities in these tools, which hinder their
systematic adoption in educational settings.
While many teaching and learning tasks are easy to implement in a blog-based PLE, this type of tools
still lacks some of the important features that made traditional Learning Management Systems
efficient for both teachers and learners.
For instance, it is quite difficult to manage assignment related workflows and to promote semantic
data exchange between multiple blog instances.
LePress 1 is a learning flow management extension for WordPress blog engine (Tomberg, Laanpere
and Lamas, 2010). LePress is software plug-in for popular WordPress blogging platform. As its name
implies (LePress is acronym for Learning WordPress) LePress is intended for adding specific learning
functionalities into WordPress. It main feature is on its design, that is focus on provide a usable
Course Coordination Space which provides a natural connection between formal university courses
and PLE.
LePress can be installed on multiuser WordPress installations where all existing users can get
benefits of participating in coursework. But main unique feature of LePress is a possibility to connect
in framework of learning courses the independently installed instances of WordPress blogs. This
allows an owner of external WordPress blog who already uses it as e-Portfolio or as a reflection tool
to instantly join into the courses announced in another blog, he or she can install LePress plug-in and
immediately register herself in open courses.
LePress is based on three main guidelines: (1) ensuring teachers and learner’s a natural and simple
learning flow; (2) ensuring basic features sets, while leaving extended functionality transparent and
ready to use when needed; (3) enabling maximum archive features by ensuring a blog architecture
reuse; (4) or ensuring that no feature is implemented if doesn’t carry substantial added value.
It general architecture, see figure 1, includes three layers: the User, The Learning Flow, and the
Learning Content Layers.
Users interact with their blogs on User Layer. In case if user activity has any course-related load, it
begins to be controlled by LePress.
LePress acts on Learning Flow layer by connecting separately distributed instances of WordPress
installations. LePress provides solutions for interblog communications and accompanying of blog
artefacts like posts, comments and categories with specific learning-related data.
On the down Learning Content Layer arranged resource providers. They supply media for using it in
blog posts to make learning process more reach and interactive.
LePress non-destructive workflow approach for blog users, allows the users to continue work in their
blogs as usual, posting messages and commenting other’s posts. Additionally, LePress helps
teachers to maintain learning oriented tasks like course categories and learning posts and
assessments.
1 Available at http://wordpress.org/extend/plugins/lepress-teacher/
1008

Sónia Sousa et al.
Figure 1: LePress provides assessment workflow in layered architecture of modern PLE
Basic features of LePress were achieved by assuming that a course is a collection of learning
activities, which happened on and during predetermined moments in time (figure 2).
Blog postings activities in LePress might be relate to such elements as learning materials,
discussions, assignments, and assessments. Those are performed in an environment, in which both
teacher and students also take parts of a given course.
Figure 2: Mapping the entities of blogs to concepts of LePress courses
For example LePress plug-in implements a workflow possibility for formal learning course assessment
of teacher of students’ home works. As well allows teachers to create courses on the base of their
WordPress blogs and register students in them.
Also in addition to various ways of using blogs in course works (March, 2007), (Attwell, 2007) in
LePress the teacher can also produce special posts that are considered by system as assignments
for homework. In this case students are automatically notified about new assignments and deadlines
and can publish their homeworks in the own blogs. Because these posts will be automatically
gathered by LePress and delivered to dedicated class-book of the teacher, then the teacher in return
can assess these submissions, grade them and sent personal feedback.
1009

Sónia Sousa et al.
3. Elicitation of a set of potential trust effects in the design of LePress
A survey was deployed to better understand the effects and influence of trust people’s online activity
and sharing patterns. Results gather takes in consideration the effects of trust in an online social
context, providing guidelines for effective online community design, development and assessment.
This survey was randomly conducted on 340 individuals (in Portugal and in Cape Verde). Results
showed they are almost equally distributed among gender (53.2% male, 46.8% female). From those
46.5% are higher education students, 41.2% are teachers of various levels of instruction, and 12.3%
have other occupations.
This survey assesses people’s trust behaviors, attitudes and beliefs and their predisposition to share
information. This regards people’s sharing information patterns in distinct natures, (from personal
status to generic music files and other documents) with third parties with whom they maintained
relations of several degrees (from family and friends to total strangers).
So far, the survey results indicated a relevant connection between the predisposition of trust and the
degree of the relation with the third party as this sample's individuals were more willing to share with,
thus trusting, their family and friends than total strangers.
Further, the difference between the willingness to trust family & friends and total strangers was also
higher when asked about sharing their personal status and information than when asked about
sharing generic items.
3.1 Observed roles and perceptions towards trust and sharing attitudes
An important advantage of using blogs in education, according to Du and Wagner (2005) is the ability
fostered by blogs to reinforce individual accountability.
This is accomplished in three ways:
Non-anonymity — personal responsibility of students to progress;
Individualized feedback — embedded in blog capabilities for receiving feedback from teacher and
students;
Benchmarking and self-assessment — possibility for student to compare own work with works of
other students.
Current online social networking services are characterized by the presence of features that allows
individuals to (Weaver and Morrison, 2008):
Construct a public or semi-public profile within a bounded system;
Articulate a list of other users with whom they share a connection; and normally
View and traverse their list of connections and those made by others within the system.
3.1.1 Eliciting potential trust implications
Trust integrates many facets that directly and indirectly influence the relationship process in this
online technological oriented context.
Althought LePress (Tomberg et al, 2010) addresses the perceptions and roles of users, it
accomplishes it in a very open and unrestrictive way, which might potentially induce trust related
concerns within its users, according to the results of our survey.
From a trust perspective, LePress’s non-anonymity can be seen as a feature that fosters people’s
predisposition to engage in a relationship, or construct a public or semi-public profile however, it fails
to afford participant control over what is shared with whom and how.
As such, and in the light improving LePress’s design based on the results of our studies on the
relation between trust and online learning communities, our recommendation is that LePress should in
fact allow its users a fine level of control of what is shared with whom and under what conditions so
1010

Sónia Sousa et al.
that they can more easily share information or provide feedback in environment perceived as trustful
and secure.
An environment where users not only feel supported by others when engaging in their learning
activities (Du and Wagner, 2005) but an environment where they have more control, for example, with
whom they share an assignment comment or grade.
A LePress user that way can choose with whom they share their personal information, like resources,
assignments, their assessment workflow or personal feedback send by the teacher, which is only
partially the case in the current version of this PLE tool.
4. Closing remarks
Everyday, and often without any reflection, we place our trust in people and in services those people
provide. For the most part, the trust we place in others and in the actions of others are taken
unreflective. However, there are occasions that trust bond is weakened and in which we have to give
more thought to evaluating the risk of some trust assignment. In these occasions people's perceptions
and behaviors could be influenced by the nature of the relationship outcome.
Conclusions lead to believe that without a strategic driven analysis, and without considering it
appropriate factors, with it's suitable combinations of situation and time, is difficult to proceed in a
deep and further analysis in how those factors can support different learning and working life
contexts.
For designing LePress course environment, the most important and promising is student’s preference
of use learning resources available on the public Web and readiness of students to use for distributed
PLE tools like LePress.
As well, results point out that trust is an important factor in the learning process. An online learning
environment should facilitate the learner in determining the quality of the relationship. Learners’
attitudes are influenced by their perception of others. Perceiving if other person behavior helps them
to be more willing to share, especially if they are friendly, transparent and honest.
References
Attwell, G. (2008) “The Social Impact of Personal Learning Environments”. In : Connected Minds, Emerging
Cultures: Cybercultures in Online Learning. Information Age Publishing, Incorporated.
Bachrach M. and Gambetta D. (2001) “Trust as Type Detection,” in Trust and deception in virtual societies, C.
Castelfranchi, Ed. Kluwer Academic Publishers, pp. 1–22.
Dasgupta P. (2000) “Trust as a commodity,” in Trust: Making and Breaking Cooperative Relations, D. Gameta
and B. Blackwell, Eds. Oxford: electronic edition, Department of Sociology, University of Oxford, pp. 49–
72[Online]. Available: http://www.sociology.ox.ac.uk/papers/ dasgupta49-72.pdf
Du H. and Wagner C. (2005) “Learning with Weblogs: An Empirical Investigation”. In : 38th Hawaii International
Conference on System Sciences (HICSS), pp.7b-7b.
Gambetta D. (1998) “Trust making and breaking cooperative relations,” in Can we trust trust?, D. Gambetta, Ed.
Basil Blackwell, pp. 213–237.
March, T. (2007): Revisiting WebQuests in a Web 2 World. How developments in technology and pedagogy
combine to scaffold personal learning, Interactive Educational Multimedia (Band 15)
McAllister D. J. (1995) “Affect and cognition-based trust as foundations for interpersonal cooperation in
organizations,” The Academy of Management Journal, vol. 38, no. 1, pp. 24–59.
Putnam. E. (2007) “Diversity and community in the twenty-first century. The 2006 Johan Skytte prize lecture.
Scandinavian Political Studies.” Available at http://www.utoronto.ca/ethnicstudies/Putnam.pdf.
Tschannen-Moran M. (2001) “Collaboration and the need for trust,” Journal of Educational Administration, vol.
39, no. 4, pp. 308–331.
Tomberg, V., Laanpere, M. and Lamas, D. (2010). Learning Flow Management and Semantic Data Exchange
between Blog-based Personal Learning Environments. G. Leitner, M. Hitz, and A. Holzinger (Eds.). HCI in
Work & Learning, Life & Leisure - USAB 2010 (340 - 352). Berlin: Springer Verlag
Weaver A. C. and Morrison B. B. (2008) “Social networking,” IEEE Computer, vol. 41, no. 2, pp. 97–100.
1011

Breaking Down Barriers: Development of a Wiki Based
Module to Enhance the International Learning Experience
Karen Strickland, Liz Adamson, Carolyn Blight and Wendy McInally
Edinburgh Napier University, Edinburgh, UK
k.strickland@napier.ac.uk
l.adamson@napier.ac.uk
c.blight@napier.ac.uk
w.mcinally@napier.ac.uk
Abstract: This paper reports on the pedagogic design and development of a module which aims to provide an
international learning experience to undergraduate nursing students in a meaningful way, using a wiki as a
shared learning space with international partner institutions. A shortage of healthcare professionals has led to
increased demand for preparing undergraduate nursing students for global mobility, the provision of an authentic
international learning experience in the undergraduate programme is therefore essential. Coupled with social and
economic constraints the traditional model of overseas clinical placements has become increasingly challenging
to achieve. The project team therefore sought to develop a module which offered an authentic international
learning experience which overcomes such barriers and widens the accessibility of the international learning
experience to all students. Wikis are increasingly being used as learning spaces in academia as they provide a
student led collaborative learning experience and have been seen to promote active student engagement in
learning. Wikis provide the opportunity for partner institutions to provide shared learning experiences between
students thus providing the opportunity for students to engage with their overseas peers in specific learning
activities. This paper will report on the process of the module design and development, such as establishing the
international partnerships necessary for the success of the module as well as the underpinning pedagogical
influences of this mode of learning. In doing so particular focus will be placed on where and when to think beyond
the opportunities offered by institutional VLEs, and the limitations inherent within them, when seeking to
effectively engage online students who are from different institutional cohorts within an international context.
Keywords: internationalisation, wiki, web 2.0, peer learning, student experience
1. Introduction
Internationalising the curriculum is a priority within higher education (Leask 2000). The Bologna
Ministerial meeting committed to a target of 20% of Higher Education (HE) students having an
international learning experience by 2020. Traditionally most international learning experiences in
nursing are focussed on the need for exchange programmes or overseas student placements.
Additionally the 20% target would be challenging to achieve in the nursing sector by international
placements alone. Whilst it is recognised that international placements may be the ideal, there are a
number of social and economic factors which inhibit students ability to undertake such placements
resulting in an inequity of the international learning experience therefore an alternative to the
overseas placement is sought.
The development of this wiki-based module offers one solution to this problem. The paper presents a
rationale for developing an innovative alternative to overseas clinical placements for providing an
international learning experience and developing global citizenship within the nursing profession.
2. Use of technology to enhance learning
Technology enhanced learning approaches offer many advantages including the ability to
communicate with peers and colleagues regardless of geographic location. Whilst many UK higher
education institutions have embraced virtual learning environments (VLE) the opportunities to break
down barriers created by the institutionally focused VLE by using some of the Web 2.0 technologies
offer students a richer and more varied learning experience.
Wikis are increasingly being used as learning spaces in academia as they provide a student led
collaborative learning experience and have been seen to promote active student engagement in
learning (Brown et al, 2010; Choy & Ng, 2007). Wikis provide the opportunity for partner institutions to
provide shared learning experiences between students thus providing the opportunity for students to
engage with their overseas peers in specific learning activities.
An online international module where students can engage in shared learning with their international
peers is an alternative to the traditional overseas placements and will enable more students to have
1012

Karen Strickland et al.
an authentic international learning experience. In addition to this, students who may not be able to
undertake prolonged overseas travel may also be afforded the opportunity to engage in learning
about nursing in an international context. One of the key advantages of the proposed module is that it
widens the accessibility of the international learning experience to those students who are unable to
travel for extended periods for time for overseas placements. This module therefore supports the
University’s Learning, Teaching & Assessment Strategy (2010) key statement numbers one and four.
3. Module development
The module project team consisted of academic staff and representatives from the university’s
International Student Nurses Society. They have developed a module which offers an authentic
international learning experience by providing an opportunity for undergraduate nursing students to
learn about other nursing cultures and to work with students from overseas, using a wiki as the shared
learning space.
The idea of using a wiki as shared learning space was inspired by an existing case study in the
institution’s Learning, Teaching & Assessment Resource Bank where the model has been
successfully used within another faculty. The lecturer who developed the module acted as an advisor
and critical friend throughout the process of this module development. The process of development
involved the team participating in a series of action meetings to explore possible alternatives to the
overseas placement where students also contributed to the discussions and aspects of module
content. The students provided valuable insight into the learner experience.
International partnerships for this module were developed from existing networks. Partners from
Western Carolina University (USA) and Lahti University (Finland) were involved in negotiating the
learning activities to ensure a comparable learning experience for all students.
The module is aimed at undergraduate nursing students from all fields of nursing and will be delivered
at SCQF level 9. The approach complements the Scottish Government (2009) European action plan
on EU engagement and is compatible with EU developments for an integrated health workforce
(European Commission 2008) and contributes to our institutional LTA Strategy aims of being
“ambitious for our students” by enhancing their potential contribution to the global society, thus
enhancing their employability. The value of this project extends beyond the institution by providing
educational benefits to the international partners.
The module aims to:
Provide an authentic international learning experience for undergraduate nurses
Explore the use of a wiki as an appropriate shared learning environment for the international
nursing module
The first cohort of students from Edinburgh Napier, Western Carolina and Finland will commence this
module in September 2011. There will be approximately 10 students from each institution. Edinburgh
Napier students will access this module via the Virtual Learning Environment (VLE) for all module
related communications and each of the learning tasks. The partner institutions will also retain their
own VLE for their students. The student will select a country to focus their international learning
experience upon and will then be instructed on how to access the shared learning space which will be
hosted on an external wiki site (Wikispaces).
The students will focus on key activities in relation to one international country to compare and
contrast the health and social care issues and trends in that country, determine the relationship
between health and social care trends and nursing roles in the other country, evaluate the
implementation of international health and social policy by the national government and critically
reflect on the contribution of nursing to international health and social care systems. The students will
also be asked to produce evidence of their active engagement in discussions with their international
peers on the wiki site.
Necessarily the evaluation will involve the use of both quantitative and qualitative methods. The use
of technology will also feature in the evaluation methods as quantitative feedback and limited
1013

Karen Strickland et al.
qualitative feedback will be sought through the use of an online survey and further in-depth qualitative
evaluations will be sought using Elluminate Live.
The module development was been funded by an institutional Teaching Fellow Development Grant
and additional monies to fund travel to one of the key partner institutions (Western Carolina
University).
References
Brown, G. Quentin-Baxter, M (2010) WikiVet: building a community of practice to support a self-sustaining wiki for
veterinary education. International Journal of Web Based Communities. 6(2): 183-196
Choy, S., Ng, K. (2007) Implementing wiki software for supplementing online learning, Australasian Journal of
Educational Technology. 23(2): 209-226.
Commission for the European Communities (2008) Green paper on the European Workforce for Health. Available
from: http://ec.europa.eu/health/workforce/green_paper/index_en.htm [date accessed 1st June 2011]
Knight, J. (2004) Internationaization remodelled: Definition, approaches, and rationales. Journal of Studies in
International Education. 8 (1): 5- 31
Leask, B. (2000) Internationalisation: Changing contexts and their implications for teaching , learning and
assessment
Available from: http://www.ascilite.org.au/aset-archives/confs/aset-herdsa2000/procs/leask1.html [accessed
10.6.11]
Thompson, K. Boore, J., Deeny, P. (2000) A comparison of an experience for nursing students in developed and
developing countries. International Journal of Nursing Studies. 37: 481-492.
Williamson, M., Harrison, L. (2010) Providing culturally appropriate care: A literature review. International Journal
of Nursing Studies. 47: 761-769.
1014

eLearning in German Higher Education: Technology
Implementation as a Challenge for Organizational Change
Novita Yulianti, Michael Lund and Georg Müller-Christ
Bremen University, Bremen, Germany
novita@uni-bremen.de
mlund@informatik.uni-bremen.de
gmc@uni-bremen.de
Abstract: In this research, the implementation of eLearning services is observed from an organizational
viewpoint. It attempts to find out organizational and personal challenges that higher education has to overcome
when eLearning is to be implemented in teaching and learning activities. The current situation shows that most
higher-education institutions have established eLearning services so that eLearning services are now available
for all students and teaching staff. However, only basic services are in use. The administration of universities
shows that the use is often inefficient and not fully implemented in everyday teaching practice. Moreover, the full
potential of the eLearning services that could make teaching more effective and interactive are neither
acknowledged nor employed. This paper focuses on analyzing the main barriers that hinder teaching staff from
adopting a new culture of teaching based on digital media and becoming familiarized with eLearning services. A
preliminary study was carried out by interviewing experts and staff from an eLearning service unit in order to gain
insight into the drivers and barriers faced during the introduction and implementation of eLearning services. A
case study will be conducted focusing on higher-education institutions in Germany whose main form of teaching
is on-campus and face-to-face, rather than on open and distanceLearning institutions that encounter different
strategic issues in applying eLearning. A focus group comprising of members from all faculties will be formed
which is intended to enable the adaptation of the learning environment to the behavior and needs of the teaching
staff. However, this participatory challenge also drives the teaching staff to create a new culture of teaching by
using eLearning services. Finally, this research into eLearning implementation in higher education institutions in
Germany also reveals the current organizational problems in the system. In this way, the implementation of
eLearning becomes a tool for organizational development.
Keywords: eLearning, higher education, staff motivation, organization
1. Introduction
Technology has continued to play a significant role in higher education. Its nature has the potential to
alleviate some of the problems faced in higher education and improve the quality of teaching through
the production and transmission of knowledge. Nowadays, most of higher education institutions
provide eLearning services for all students and academic staff. Although basic services are in use, the
administration of universities reveals that the offered eLearning services are not implemented in
everyday teaching practice. Moreover, the potential of the eLearning services that could make
teaching more effective and interactive are neither fully acknowledged nor employed. This paper
focuses on factors within higher education that influence staff in their adoption of eLearning. It begins
by describing the implementation of eLearning in higher education, and then discusses the inhibitors
and facilitators that arise during implementation. Finally, it presents the future work that will be carried
out.
2. eLearning implementation in Higher Education
2.1 Current situation
eLearning becomes increasingly important in higher education institutions. In higher education in
Germany, eLearning—computer- and web-supported teaching and learning—are established and
continually developed over the past few years. Various eLearning formats such as complementary
material for lectures, interactive course programs, virtual classroom courses with tutoring and
collaborative learning, distance lectures, and virtual practical sessions are offered to students in which
the complementary material for lectures is in the format usually offered in higher education and mostly
accessed by students (Bargel, et al, 2008; Kleimann, et al, 2008).
German teaching staff appreciate the use of modern information technology at university, but a lot of
opportunities to make teaching more effective and interactive are still being left unused (Rosenboom,
n.d.). Previous studies show that the integration of eLearning in higher education is so far
disappointing, both in its strategic use as well as at the level of its educational work process.
1015

Novita Yulianti et al.
Research conducted at German universities finds that only one-third of the teaching staff already offer
eLearning courses, with 40 percent of those who complete such a course expressing that they will do
so again (Rosenboom, n.d.). Technology development tends to outreach critical thinking and
pedagogical design in universities, and the sustainable integration of eLearning into higher education
remains a major challenge (Schneckenberg, 2009).
Moreover, it is found that most of students stated that their teachers use eLearning service only for
basic activity such as file upload and download, though many services are offered. This result is
based on a study on students’ feedbacks in early stage of this research by an eLearning service unit,
which is aimed to evaluate the use of eLearning services. It seems that the academic staff do not
apply eLearning tools to design their teaching and learning activities.
2.2 Academic staff as a key factor
Academic staff are identified as playing a major role in the adoption of eLearning in higher education.
As in the traditional frame of teaching, academic staff are the gatekeepers of research and teaching in
higher education. Academic staff define the content of teaching and carry out teaching and
examination of teaching results (Kerres, 2005; Schneckenberg, 2009). Individual perception and
faculty culture also play a main role in the acceptance or rejection of eLearning practice. Academic
staff express concern regarding online education. In particular, they believe that in the future online
dialogues will replace face-to-face interaction.
The motivation and competence of teaching staff regarding the use of eLearning plays a leading role
in the spread of eLearning. Several studies show that the slow adoption of eLearning in higher
education is caused by an adequate level of competence of the majority of academic staff (Kerres,
2005; Werner, 2006; Schmahl, 2008, Schneckenberg, 2009). The development of competence will
have a high impact on faculty behavior only if it is complemented by additional institutional measures
which influence the motivational level of academic staff (Schmahl, 2008). Additionally, the structural
characteristics of higher education as well as cultural barriers are also becoming the underlying
problem for the wider adoption of eLearning and other educational innovations in higher education
(Schneckenberg, 2009).
3. Impact of teaching style on technology implementation
Academic staff develop a teaching style based on their beliefs about what constitutes good teaching,
personal preferences, their abilities, and the norms of their particular discipline. There are different
approaches to teaching depending on the academic discipline, class size, and the individual
preferences of the academic staff. Most teachers teach the way they learn (Brown, 2003). Since many
teachers experience academic success in learning environments that are teacher-centric and rely
heavily on lecturing, it is comprehensible that their preferred style of teaching, at least initially, will be
to repeat what work for them. The way in which any method, whether lecture- or game-, problembased
learning or discussion is used within a learning practice is the choice of the educators and
should be a reflection of their philosophy (Brown, 2003).
It seems that the teaching style of academic staff could be one aspect that influences the adoption of
eLearning services. Therefore, it will be valuable to consider the teaching style of academic staff
when designing eLearning services so that the academic staff can choose the appropriate services
that are most suited to their preference.
4. Result and discussion
In order to gain a better understanding of the aspect that encourages academic staff to integrate
eLearning services into their courses, as well as aspects that hinder the widespread implementation
of eLearning services, a preliminary study is carried out by interviewing experts in an eLearning
service unit. It appears that a need to solve the problems faced in day-to-day teaching practice is one
of the motivating factors for the integration of eLearning services. This factor promotes the use of
eLearning services, especially when the offered service fits the needs of the academic staff. In
addition, the curiosity of academic staff in working with the new practices, which they believe can
support their teaching practices, as well as the initiative to fulfill students’ needs have prompted the
staff to use the offered services.
1016

Novita Yulianti et al.
Several factors are seen as potential inhibitors in higher education. These include time related
matters. For most staff, using eLearning services in their courses means that they should allocate a
greater amount of time. Science culture in the higher education in Germany shows that research is
rewarded more than teaching. Providing good-quality teaching is important on one side, but on the
other, research carries greater prestige among the community. This situation makes the academic
staff allocate more time for research than for teaching. Moreover, the academic staff do not perceive a
clear added value or potential of eLearning services offered, either for themselves or in regard to the
didactical aspect. Additionally, it is the lack of knowledge on how to use eLearning in terms of
technical and didactical aspects that is mentioned as an inhibiting aspect of the integration. It seems
that there is a lack of awareness of the quality and potential that eLearning can achieve, which can be
caused by a strong allegiance to the face-to-face teaching model as well as skepticism about
eLearning particularly on issues of its potential and quality.
Based on the inhibitors discussed above, several potential solutions are proposed as follows:
Bringing support closer to the needs of users by providing a change facilitator or coordinator for
each faculty and establishing a network of change facilitators.
Developing media literacy of academic staff through a customized competence development.
Active information and communication.
Providing support and reward.
5. Summary and future works
It is important to know the real needs, concerns, interests and attitudes of academic staff in order to
be able to offer an appropriate solution. The teaching style of academic staff should be considered
when designing eLearning services so that the academic staff can choose appropriate services that
are suitable to their own style. In order to meet the requirement, the concept of the Participatory
Approach (Schuler and Namioka, 1993) will be adopted and combined with the Concern-Based
Adoption Model (CBAM) (Straub, 2009) and Roger’s diffusion of innovation (Roger, 2003) to formulate
a model of change in higher education which focuses on the needs and behavior of teaching staff.
This work will form a focus group of experts comprising of representatives from several departments
in order to analyze the suggested solution more closely.
References
Bargel, T., Ramm, M. and Multrus, F. (2008) Studiensituation und studentische Orientierungen 10.
Studierendensurvey an Universitäten und Fachhochschulen, Bundesministerium für Bildung und Forschung
(BMBF) Referat Wissenschaftlicher Nachwuchs, wissenschaftliche Weiterbildung, Bonn, Berlin.
Brown, B. L. (2003) “Teaching Style vs. Learning Style”, [online], Educational Resources Information Center
Myths and Realities 26, www.calpro-online.org/eric/textonly/docgen.asp?tbl=mr&ID=117 (accessed on
10/01/2011).
Kerres, M. (2005) Lehrkompetenz für eLearning-Innovationen in der Hochschule Ergebnisse einer explorativen
Studie zu Massnahmen der Entwicklung von eLehrkompetenz. SCIL-Arbeitsbericht 6.
Kleimann, B., Özkilic, M. and Göcks, M. (2008) “Studieren im Web 2.0 Studienbezogene Web und
E Learning Dienste“, [online], HISBUS-Kurzinformation Nr. 21. HIS Hochschul-Informations-System
GmbH, https://hisbus.his.de/hisbus/docs/hisbus21.pdf (accessed on 05/10/2010).
Roger, E. (2003) Diffusion of Innovations, 5th edition, Free Press, New York.
Rosenboom, S. (n.d) “Dozenten nutzen Chancen des digitalen Campus noch nicht aus“, [online],
http://www.microsoft.com/germany/presseservice/news/pressemitteilung.mspx?id=531914 (accessed on
11/02/2011).
Schmahl, J. (2008) ELearning an Hochschulen – Kompetenzentwicklungs-strategien für Hochschullehrende
Ergebnisse von zwei empirischen Untersuchungen zu Anreizen und Hemmnissen der Kompetenzentwicklung
von Hochschullehrenden im Bereich ELearning, Dissertation, Universität Duisburg-Essen.
Schneckenberg, D. (2009) “Understanding the real barriers to technology-enhanced innovation in higher
education”, Educational Research, Vol. 51, No. 4, pp 411 – 424.
Schuler, D. and Namioka, A. (1993) Participatory Design: Principles and Practice, Lawrence Erlbaum Associates,
Inc., Hillsdale, New Jersey.
Straub, E. T. (2009) “Understanding Technology Adoption: Theory and Future Directions for Informal Learning”,
Review of Educational Research, Vol. 79, No. 2, pp 625 – 649.
Werner, B. (2006) “Status des ELearning an deutschen Hochschulen“, [online], http://www.eteaching.org/projekt/fallstudien/Status_des_ELearning.pdf
(accessed on 15/10/2010).
1017