Teaching and Assessing Soft Skills - MASS - Measuring and ...

trained in the use of the rubrics and normally a sample of responses is assessed by at least
two raters so that inter-rater reliability can be checked.
Performance assessment
“Performance-based assessment is a type of testing that calls for demonstration of
understanding and skill in applied, procedural, or open-ended settings” (Baker, O’Neil, &
Linn, 1993, p. 1210). Performance assessment has been used in judging activities that do not
normally leave an artefact that could be evaluated later. Examples have included gymnastics
and dance, but the act of making an object was also a performance that could be evaluated
separately from any assessment of the object that was produced. Performance assessment
has been extended to a wider range of activities, including science laboratory classes,
medical diagnoses and building brick walls. In these cases, products existed that were
evaluated, but soft skills are more likely to be observed in their execution during
construction of the artefact than in the artefact that is produced. It is the act of producing,
rather than the product, that is being judged.
Performance assessment was characterised by the use of open-ended tasks with responses
constructed by the student rather than being selected from defined options and by a focus
on complex skills in a curriculum context, and were judged in terms of domain specific
criteria rather than against a generalised trait (E. L. Baker et al., 1993, p. 1211). The last of
these claims is a challenge to the generalizability of any inferences that may be drawn from
a performance assessment. One goal of assessment is to judge and provide feedback about a
specific activity, but if that feedback cannot be used in subsequent performances, it is of
little value to the learner.
Teacher/Holistic judgment
Teacher judgment has featured prominently in the literature on assessment and has been
shown to be central to almost all forms of assessment. An obvious exception is the multiplechoice
format, although in that format, judgments are made when the tasks and response
alternatives are chosen.
Teacher judgement has been shown to work well in the school sector where teachers know
students’ attributes well through frequent and close observation. Consistency of judgements
within panels of teachers has been demonstrated. What cannot be shown is that this
consistency extends across school boundaries. This means that students from different
schools may be judged against different standards, and so individual achievement, assessed
using this method, does not provide a basis for broad comparison.
This method is unlikely to transfer to either the VET or higher education sectors where such
close observation in a range of contexts does not occur. A trainer may monitor learners in an
institutional classroom or workshop setting, but is unlikely also to see them in social or
workplace settings. Similarly, workplace supervisors may observe individuals in that context,
but not in others. The bases of judgements made by these raters will be restricted to a
limited range of contexts.
An issue raised in the literature reviewed above is a restriction of the knowledge of
standards and their application to a community of assessment practitioners (Sadler, 1989).
This is shown to have implications for feedback and for a role for students in self-
125