Revising the Oral Exam (BOEC) and Linking it to the New ... - ALTE

Revising the Oral Exam (BOEC) and
Linking it to the New Chinese EFL
Proficiency Scales
Hongjiang LI & Wenlong SHI, Beijing Education Examination Authority
Baocheng HAN, Beijing Foreign Studies University
2011-7-ALTE

Outline
• An overview of BEEA
• Background information about BOEC
• BOEC exams: a close examination
• BOEC: next step

1. What is BEEA?
Beijing
Education
Examination
Authority

BEEA is …
• is affiliated to Beijing Municipal Education
Commission.
• is an educational examination service
organization.
• is a not-for-profit organization.
• offers a wide range of examinations.

BEEA …
• has close collaboration with other testing boards.

2. What is BOEC?
• BOEC: Beijing Oral English Certificate.
• It was developed in 2001 to meet the needs of
training and recruiting people from all walks of
life to serve the 2008 Beijing Olympic Games.
• 11 municipal agencies recognize the test result.
• BOEC is now widely accepted both in Beijing and
other provinces.
• 20,000 – 80,000 examinees each year.
• Over 500,000 people have taken the test.

̶
̶
̶
• The certificate of the exam can help testtakers
to
get a better job.
get promoted
demonstrate their personal progress

Examinees by year

Examinees by age

3. BOEC examinations
• It is a 1-to-1 carefully structured conversation
between the candidate and a trained, certified
examiner.
• The genuine interaction measures fluency,
accuracy and communication skills.
• It was launched in 2002, and revised in 2007.
• In 2009, BOEC was completely computerized in
registration, administration and reporting results.

BOEC Levels
LEVEL 3
Advanced Stage
Independent
User
LEVEL 2
Intermediate Stage
Limited User
LEVEL 1
Elementary Stage
Very Limited
User

Examinees by level

BOEC Test Format
Advanced stage
Intermediate stage
LEVEL 3
Elementary stage
LEVEL 1
• Aconversation(2
min) +
• Picture description (2
min) +
• Discussion (1‐2 min)
5‐7 minutes
LEVEL 2
• Aconversation(3
min)
• Topic presentation
(3min)
• Topic discussion (2
min)
8‐10 minutes
• Aconversation(4
min)
• Topic presentation &
discussion (4 min)
• Topic presentation &
discussion (4 min)
12‐15 minutes

1-to-1 structured interview

1-to-1 structured interview

Computerized administration

1-to-1 structured interview

1-to-1 structured interview

1-to-1 structured interview

BOEC examiner system
• Chief examiner
• Advanced examiners
• Examiners (rigorously trained)

Reliability and validity issues
• BOEC chief examiner and advanced examiners
form a board and consider
− academic standards
− examiner training including reliability issues
− validity of the examination system
− review and monitoring the exam

BOEC examiner certification
1. Apply 2. Training
3. Testing 4. Certified as an examiner

BOEC test content
• BOEC elementary: Level 1

BOEC test content
• BOEC Intermediate: Level 2

BOEC test content
• BOEC Advanced: Level 3

̶
̶
̶
̶
BOEC rating procedure
• The examiner works both as interlocutor and rater.
• The examiner gives one holistic score and four
analytical scores against five analytical criteria:
Grammar and vocabulary (range and flexibility, accuracy)
Discourse Management (coherence, extent, relevance)
Pronunciation (stress and rhythm, intonation, individual sounds)
Interaction (initiating and responding, hesitation, turn taking)

• Each criterion is described on an assessment scale
of five main levels with complete descriptors attached
to levels 1, 3 and 5.
• Additional halfway levels (1.5, 2.5, 3.5, 4.5) make
nine band scores in total. The 3.0 band is considered
an adequate performance.
• The 4.5 band and above is considered outstanding.
• All exams are recorded and each examiner will be
reviewed by the advanced examiner.

4. BOEC: Next Step
• Redefine test construct.
• Revise test syllabus.
• Revise test format: integrate listening and
speaking at the advanced stage.
• Map BOEC scores onto NELPS and CEFR
scales.

BOEC: Current Criterion
BOEC Advanced
Level 3
BOEC Intermediate
Level 2
BOEC Elementary
Level 1
College English
Syllabus
High School English
Syllabus
Primary School English
Syllabus

BOEC: Revised Criterion
Level 6
C2
NELPS Levels
Level 5
Level 4
Level 3
Level 2
Level 1
BOEC Advanced
Level 3
BOEC Intermediate
Level 2
BOEC Elementary
Level 1
C1
B2
B1
A2
A1
CEFR Levels