July 2006 Volume 9 Number 3 - CiteSeerX

Numerous studies have demonstrated that students benefit significantly from peer assessment; however, some
students have revealed that peers may lack the ability to evaluate each others’ work or not take their role
seriously, allowing friendships, entertainment value, etc., to influence the marks they give to peer coursework.
Furthermore, students often lack experience in peer assessment and encounter difficulties in interpreting
assessment criteria. These obstacles often result in subjective and unfair assessments and, thus, students’
reflection and learning effectiveness are not enhanced.
This work presents a novel approach that models collaborative learning as multi-issue agent negotiation using
fuzzy constraints. Agent negotiation (Pruitt, 1981) is an iterative process through which the proposed
methodology aggregates students’ marks to reduce personal bias. In this framework, students define individual
fuzzy membership functions based on their evaluation concepts and agents facilitate student-student negotiations
during the assessment process. By applying this method, agents can reach mutually acceptable agreements that
overcome the unfair assessment as a result of students’ various degree of understanding the assessment criteria.
Thus, the negotiated agreement improves the assessment process and learning effectiveness.
The remainder of this paper is organized as follows. Section 2 introduces the concept of peer assessment. Section
3 describes the overall framework of the assessment agent and its computational model. Then, a walk-through
example is utilized to illustrate the application of the proposed methodology. Section 4 presents the experimental
design, analytical results, and evaluation of the questionnaire results. Finally, Section 5 draws the conclusion.
2. Peer Assessment
Assessment is a learning tool. However, most traditional assessment methods often encourage “surface
learning,” which is characterized by information memorization and comprehension (Sitthiworachart and Joy,
2003). Falchikov defined peer assessment as “the process whereby groups rate their peers” (Falchikov, 2001). It
can include student involvement not only in the final judgement made of student work, but also in the prior
setting of criteria and the selection of evidence of achievement (Biggs, 1999).
Peer assessment is an interactive assessment method that enhances student interpretation and reflection, enabling
instructors to improve their understanding of student performance. This method moves coursework assessment
from instructor-centered to student-centered. Based on previous research (Ford et al., 1993; Liu et al., 2002;
Sitthiworachart and Joy, 2003), students are capable of learning how to criticize the work of their peers and
accept peer criticism, thereby developing their critical thinking skills and self-reinforcement through peer
assessment.
Peer assessment requires cognitive activities such as reviewing, summarizing, clarifying, providing feedback,
diagnosing errors and identifying missing knowledge or deviations (Van Lehn et al., 1995). However, based on
the framework described by Sitthiworachart (Sitthiworachart and Joy, 2003) and Liu (Liu et al., 1999), the peer
assessment process can be divided into four separate stages (Fig. 1). During stage 1, students complete their
assignments on their own time and then submit assignments. During stage 2, peer reviewers assess peer
assignments and then discuss their marks with the other students in their group who marked the same
assignments. After the reviewers generate preliminary scores and feedback (stage 2), each student in the stage 3
evaluates the quality of marks given by the other markers. Finally, during stage 4, the results, which are
aggregates of all marks, are sent to the original author who then revises the original assignment based on peer
feedback. By this process, students typically develop a serious attitude toward their coursework.
Stage 1
Do assignment
Stage 2
Do peer assessment
exercise
Submit the assignment Mark & Feedback
Stage 3
Mark quality of
marking
Figure 1. Peer assessment process
Scores & Feedback
Stage 4
Show results
Thus, peer assessment entails formative work reviewing to provide feedback and summative grading. Moreover,
peer assessment also integrates a feedback mechanism into the peer assessment process. Some studies indicate
that receiving feedback is correlated with effective learning (Van Lehn et al., 1995; Bangert-Drowns et al.,
1991).
17