2006 Edition 2 (Issue 144) - Sasmt-savmo.org.za

The next stage was to see what happened when another
29 assessors were asked to use C1-C12 to judge performances.
This time, the video recording consisted of a series
of ten solo performances, each on a different instrument.
There were two groups of assessors:
Group 1: Twelve music teachers and student teachers
specializing in music
Group 2: Seventeen student teachers specializing in subjects
other than music
Some members of this group had shown interest in
music through, for example, joining their college
choir. But none had studied music at school beyond
the age of 16, or taken instrumental lessons since
leaving school.
The assessors were asked to imagine that each performance
was part of a Grade 8 examination, and to assess the
performance as seen and heard without making any allowances,
for example for performers who looked younger.
For each performance there was a double-sided sheet to
be completed. On the first side, the assessor gave a single
mark of up to 30 using the Associated Board of the Royal
Schools of Music’s criterion-referenced classification system
(distinction, merit, pass, and fail) as a guide. On the
second side, the assessor rated the performance on each of
the twelve bipolar constructs using a four-point scale.
The marks given by individuals for performances were converted
into ranks, with the performance given the highest
mark being assigned a rank of one. The constructs were
scored from one to four according to their placing on the
four-point scale. There was a positive correlation between
each of the constructs and the overall rank ranging from r
= 0.4 (C6) to r = 0.7 (C10 and C11) (n = 290). 5 I followed
this up with another statistical technique: multiple regression
analysis. 6 This showed that the constructs accounted
for more than two-thirds of the variance in the ranking
of the performances, for both Group 1 and Group 2. This
indicates that the holistic assessment could be accounted
for in terms of common constructs to a substantial extent.
It is interesting that there is so little difference in the results
for Groups 1 and 2, i.e. that there is little apparent difference
in the holistic assessment according to the extent of
formal musical expertise. This offers tentative support to
the theory that the reliability of holistic assessment stems,
at least partly, from practice in every situation.
We have seen that holistic assessment has advantages over
segmented assessment. It is more musically credible, in
the sense that it is more like assessment made of musical
performance in the real world. In addition, it can be more
reliable, and no more subjective.
This discussion has been possible only because there is
some general understanding of what is meant by ‘performer’
and ‘performance’. We have some idea of what assessment
systems in this field are trying to predict. We can
tell if the marks produced are nonsense.
This is an unusual situation. Much educational assessment
with an outcome of a single mark or grade takes place in
a less certain context. We may know what a performer is,
but do we know what a musician is? Yet we routinely combine
marks obtained for listening, composing, and performing
to give a music GCSE, or A level grade. Is there an
understanding of what a mathematician or a scientist is?
Yet we combine marks to give single grades also in these
subjects.
It is sometimes argued that there is something particularly
difficult about assessment in the arts. Might it not be that
some areas of the arts offer opportunities for particularly
rigorous assessment? If we understand what behaviour
we are trying to measure, then we can tell if the marks
we obtain are sensible. Perhaps those who devise summative
assessment systems for non-arts subjects could learn
something from looking at aspects of the arts.
BIBLIOGRAPHY AND FURTHER READING
FISKE, M. 1977. Relationship of Selected Factors in Trumpet
Performance Adjudication Reliability. Journal of
Research in Music Education. 25(4):256-263
KELLY, G. 1955. The Psychology of Personal Constructs.
New York: Norton.
MILLS, J. 1991. Assessing Musical Performance Musically.
Educational Studies. 17(2):173-181.
MILLS, J. 2005. Music in the School. Oxford: Oxford University
Press.
Reproduced from Music in the School by permission of
Oxford University Press (www.oup.com).
ISBN 0-19-322300-7
Janet Mills is a Research Fellow at the Royal College of Music, London.
She began her career as a secondary school music teacher,
and was a teacher trainer prior to working for
ten years as an HM inspector of Schools. She
works widely in schools, universities and the
community. Her writing includes Music in the
School (OUP 2005), Music in the Primary School
(CUP 2001) and many research articles.
1.
2.
3.
4.
5.
6.
See Mills 2005:156
Fiske 1977
Mills 1991
Kelly 1955
Correlation coefficients (denoted r) can range from 1 (perfect
positive correlation) to 0 (no correlation) to -1 (perfect negative
correlation). So the marks that assessors gave the performance
were influenced most by whether the performance was
clean or sensitive, and least by whether they thought that the
tempo was appropriate.
The multiple regression analysis searched for values a1 to a12
such that a ‘regression’ equation of the form:
Rank = a1C1 + a2C2 +a3C3 + … + a12C12
accounts for as much as possible of the variance in ranks, when
calculated across the 29 x 10 = 290 performances heard. The
regression equation that was calculated here accounts for
71% (n = 290) of the variance in the ranks. The separate figures
for Groups 1 and 2 are 73% (n = 120) and 69% (n = 170)
respectively.
Suid-Afrikaanse Musiek Onderwyser |144 | November 2006