Customized PLE implementations for test-bed scenario ... - ROLE

Consequently, the deliverable deals with important aspects concerning this practice
sharing approach, like privacy and trust, the dimensions of capturing learner
interactions, competence development issues, etc., and proposes an infrastructure to
enable good PLE practice sharing on different kinds of PLE solutions (platforms and
client-sided prototypes).
Community evaluation methodology
The evaluation of this community-based bottom-up approach can be achieved through
very different aspects, which are manifested in ROLE deliverables or declared as next
steps in the project.
On the one hand, the internal deliverable ID6.3/ID6.4 (‘Draft evaluation methodology
from the technical, usability and psycho-pedagogical perspective including guidelines for
the empirical realisation’) summarise generic methodologies (plus factors and methods)
for evaluating the success of information systems, technology acceptance, and
motivation and Community of Practice (CoP) success. Here, also aspects from
evolutionary biology would be of interest. For instance, the usage of PLEs and single
tools could be examined on basis of a Darwinist model, leading to the evaluation of the
evolvability and fitness for purpose of PLE technologies in CoPs. A workshop to be held
in March 2011 (cf. http://augur.wu.ac.at/EFEPLE11/) will address this special topic.
Additionally, trust and privacy are tackled but not explored in detail. Particularly for
capturing interaction data, it is necessary to consider privacy while applying new PLE
technology and generating recommendations require trustful software.
On the other hand, the community-driven PLE approach by WP7 requires also the
evaluation of typical Web 2.0 issues, like the quality of user-given input (Folksonomies),
the accuracy and novelty of PLE recommendations, or the dynamics and structure of
networked collaboration and the CoPs themselves. Furthermore, the PLE practice
sharing approach documented in the deliverable D7.2 enables the evaluation on the
basis of concrete data available in the pattern repository, e.g. by analysing the number
of activity patterns shared, explicit user feedback on activities and activity outcomes,
etc. Regarding these considerations, work has started in the second year of the ROLE
project. First results have been published e.g. at the Workshop on Recommender
Systems for Technology Enhanced Learning (Manouselis et al., 2010) and the
EFEPLE’11 Workshop (see above).
Test-bed validation strategy
In order to validate the community considerations of ROLE, the following questions are
supposed to be answered by the test-bed responsible and/or selected learners:
• How can learning activities be characterised in the test-bed? Is learning restricted
by the boundaries of an organisation? How is it restricted? Describe the
restrictions given by the boundaries of an organisation! Is learning restricted by
the boundaries of a context? How is it restricted? Describe the boundaries given
by the context! Do learners elaborate the outcomes on their own or do they have
to collaborate with others? Are they involved in passive lecturing activities, or are
Page 14 of 102