Policy Paper - Drama Improves Lisbon Key Competences in Education

R e s u l t s
Please note: the numbers indicated in the table are average scores in that specific
cluster; it does not mean that all programmes belonging to that cluster have exactly the
same structure, just a very similar pattern.
It is also important to note that all programme leaders, regardless of the topic, the
programme or their nationality, find it most important that the programme develops
the social competences of pupils, and their creativity and thinking skills. Almost all 111
programme leaders scored 5 on both of these goals.
However, taking into account the remaining purposes, the four clusters can be broken
down as follows:
1. Programmes that use educational theatre and drama for learning about a welldefined
theme or topic, e.g. a Theatre in Education programme (55 programmes)
2. Programmes that are less concerned with any of the other three goals, e.g. a series
of communication games (6 programmes)
3. Programmes that strongly focus on drama skills, e.g. improvisation (14 programmes)
4. Programmes that put more emphasis on creating a piece of artistic work, e.g.
students’ theatre (31 programmes)
B.10. External experts’ assessments of drama
descriptions
Two internationally-known and highly respected experts, namely Ms. Cecily O’Neill (UK)
and Ms. Gretta Berghammer (US) were asked to pre-assess all the drama descriptions of
the examined educational theatre and drama programmes. The programme leaders were
not told who the external assessors were. Both of them were requested to assess all 111
programmes against the five key competences, and to estimate their efficacy on a scale
of 1-10.
Detailed statistical analysis of their assessments shows that
• There was a very high correlation among the five competences on the part of both
assessors; thus programmes receiving a high score in one competence tended to
score highly for all competences, while programmes scoring less were scored lower
across all of the competences.
• For this reason we have calculated an “average efficiency score” for all programmes
for both assessors. There is a strongly significant correlation 56 between the two
assessors’ average scores (r = 0.54; p < 0.001), which means that the programmes
56 In statistics, correlation means statistical relationship between two variables. The correlation is expressed
with the r value which is a number between 0 and 1 where 0 means no relationship at all and 1 means
that the two variables are completely identical. The larger the number is, the more connection / similarity it
reveals between the two variables.
have been assessed in a similar way by both experts, on the basis of the written
descriptions. This supports the hypothesis that the assessments are not merely
subjective but also have an objective component.
• While one assessor scored on the full scale (1-10), the other used only the upper
half (5-10). However, although using a different score range, they both distinguished
between better, weaker and average programmes.
• For each drama programme, the assessors were asked to estimate their confidence
in the accuracy of their assessments on a scale of 0-100. The average score of
the programme correlated strikingly highly with confidence in the assessment (A
expert: r = 0.82; p < 0.000; B expert: r = 0.91; p < 0.000). So the more detailed the
description was, the more confident the assessor felt about it and the higher the
score she gave it. It is important to learn the lesson that in case anyone wishes
to present a drama programme to an external assessor (e.g. when applying for a
grant), a detailed description is highly beneficial.
Ms. Cecily O’Neill has submitted a report, which provides a valuable qualitative
complement to the quantitative results. We reproduce extracts from it below.
Estimating the efficacy of so many different drama and Theatre in Education programmes
has been an interesting and challenging task. The descriptions provided by the teachers
and facilitators have varied enormously – from a few lines to several pages, and allotting
a numerical value to the work based on these descriptions was difficult at times. But the
complexity of the research design should ensure that other measures and observations
will produce sound and constructive research findings.
Generally, the results are unsurprising. Because drama is essentially a dialogic form,
drama activities are likely to promote verbal and social interaction. Scores on the first of
the six competencies under investigation – communication in the mother tongue – are
generally positive. In drama, participants listen and respond. They express, appreciate
and interpret their own and others’ feelings and reactions within the safety of hypothetical
situations. There is a freedom to these verbal exchanges that is unlike regular classroom
interactions.
The same is true of the third competence. Participants
in drama negotiate and communicate with each
other, in small and large groups. They adopt
different roles and perspectives, as they explore
different viewpoints in the drama. Collaboration is
essential, as is the need to overcome frustration
and approach complex tasks constructively. It is
almost inevitable that social competence will be
promoted through the work. It is worth pointing
R e s u l t s
70
71