Calibrating tests to the CEFR

The European Association for Quality Language ServicesStep 8: Adjust Decision TableThe revised cut-off scores can be plotted in a new Decision Table as in Table 3. The pictureshown by Table 3 does now look more balanced. Obviously the fictional learner who failed toreach A1 on the test had an exceptionally bad day. If we were to do a sophisticated statisticalanalysis of the test we would no doubt discover that their responses were inconsistent.Otherwise our problem is the lack of “hits” at B1. But it is balanced.0Test0 A1 A2 B1 B2 C1 TotalCriterion (Exams)A1A2B1B2C13 1 1 51 4 1 62 2 42 21 1Total 1 3 7 2 5 0 18Table 3: Decision Table (fictional data) - revisedStep 9: Recalculating the Proportion of Common Classifications.As Table 3 shows, we now have 9 matched classifications – or 50%. This is a very respectableresult.A1 A2 B1 B2 C1 Total3 + 4 + 0 + 2 + 0 = 9Table 4: Table 3 Common classifications to CEFR levelsFinally, let us repeat the procedure with a real example.Table 5 shows how teacher judgements were used to confirm provisional cut-off pointsbetween levels on the Eurocentres Scale of Language Proficiency for an item bank testingsystemic knowledge of German. Both sets of results are presented in terms of levels on theEurocentres scale, not the CEFR. The study is illustrated here only as a publishedmethodological example undertaken in a language school 4 . The results on the two assessmentsare simply plotted against each other in a Microsoft Word table. Again it helps to shade thediagonal line – which is where all the learners would be if the correlation between the twoassessments was absolutely perfect.4 North, B. (2000): Linking Language Assessments: an example in a low stakes context. System 28, 555-577.© EAQUALS: The European Association for Quality Language Services:EAQUALS SECRETARIAT: P.O. Box 95, Budapest, H-1301 HungaryEmail: info@eaquals.org Internet: www.eaquals.orgRegistered Office: Via Torrebianca 18 34132 Trieste, Italy9