Slides from this presentation - Calico

calico.org

Slides from this presentation - Calico

Thursday, April 16, 2009

Bringing together error

correction and learner corpus

analysis:

A new look at written

corrective feedback in

teaching beginning German

Nina Vyatkina, Joe Cunningham

CALICO 2009

Tempe, AZ, March 14


Thursday, April 16, 2009

Written Corrective

Feedback (WCF)

Research background:

Systematic correction of all student errors

leads to lower error rates (Higgs & Clifford,

1982; Lalande, 1982)

Selective correction of specific error types is

more beneficial than unfocused WCF (Bates et

al., 1993; Ferris, 1995)

Error correction should be eliminated because

it is ineffective and even harmful (Krashen,

1984; Semke, 1984; Truscott, 1996, 2007)

2


Thursday, April 16, 2009

General agreement:

“accuracy in writing matters to academic and

professional audiences” (Ferris, 2006)

Student writers expect to be corrected and value

WCF from their teachers (e.g. Ferris, 1995; Sheen,

2007; Truscott, 1996)

3


Thursday, April 16, 2009

Lacunae to fill:

Expand the empirical research basis in languages

other than English (Ellis et al., 2008)

Examine the relative effects of different feedback

types within a single study (Bitchener, 2008)

Compare feedback impact on student progress in

relation to different error types (Ferris, 2006)

Target beginning learners (Ferris, 2004)

4


Thursday, April 16, 2009

Why a new look at

WCF in German?

Inconclusive research results from studies

conducted 25-27 years ago:

More explicit feedback types are more

beneficial than less explicit feedback types

(Lalande, 1982)

No difference between more explicit and less

explicit feedback types (Semke, 1984)

Teachers and coordinators continue providing

WCF using error correction codes (anecdotal

evidence)

5


Thursday, April 16, 2009

Focus of the study

To what extent does error feedback from

instructors help L2 learner writers to improve

their accuracy in the short run (from one draft of

an essay to the next) and in the long run (from

the beginning to the end of the semester)? (Based

on Ferris, 2006)

Does electronic error-coding help learners more

than paper-and-pencil error-coding ?

6


Thursday, April 16, 2009

Research questions

Do error rates change differently in response to

different error treatments:

...between the two drafts of one essay?

...over the course of a semester?

Are different types and categories of errors

affected differently by error treatment?

7


Thursday, April 16, 2009

The Corpus

Longitudinal Corpus of Learner German

Size: currently >100,000 tokens (the 3rd

consecutive semester of data collection)

Context: tutored instruction in a collegiate setting

Levels: beginning onward

Design: longitudinal / cross-sectional

Data types: written essays (test writing sections;

oral test data; online workbook exercises)

> 30 learner and task variables

8


Thursday, April 16, 2009


Thursday, April 16, 2009

Error annotation

Few learner corpora are tagged for errors

This study: teachers planted electronic error tags

into student essays that served as “clues or

bootstraps” (Wible et al., 2001) for analysis

=> “problem-oriented corpus

annotation” (McEnery et al., 2006)

688 essay drafts written by 111 students errorcoded

10


Thursday, April 16, 2009

Action research

project

7 graduate teaching assistants

Participated in the project that was a part of

the methodology course

Consistently applied one WCF condition,

returned marked essays to students

Coded the raw and the final drafts of essays 1,

3, and 5 with electronic tags and submitted it

to supplement an electronic database (learner

corpus)

11


Thursday, April 16, 2009

Pedagogical

intervention

A multi-section GFL program at a large

Midwestern university

11 sections of 1st and 2nd semester German

5 essays over a 16 week long semester

1st draft: timed task (50 minutes):

1st semester: 70 words/essay

2nd semester: 100 words/essay

typed and saved in a computer lab

2nd draft (revision): typed at home, submitted

electronically

12


Thursday, April 16, 2009

Error treatment

(WCF) types

Direct correction: underlining the errors and

providing the correct form

Coded feedback: indicating the type of error

based on an abbreviated code system

colored electronic codes

paper-and-pencil codes

Uncoded feedback: underlining the errors

without specifying their type (Robb et al., 1986).

13


Thursday, April 16, 2009

Conditions and

participants

ED1; N=27 (Electronic Direct, 1st semester)

EC1; N=23 (Electronic Coding, 1st semester)

EU1; N=16 (Electronic Underline, 1st semester)

EU1H; N=5 (El. Underline Honors, 1st semester)

EC2; N=22 (Electronic Coding, 2nd semester)

PC2; N=18 (Paper Coding, 2nd semester)

14


Thursday, April 16, 2009

The “German Toolbar”


Thursday, April 16, 2009

Direct correction

example

16


Thursday, April 16, 2009

Coded feedback

example

17


Thursday, April 16, 2009

Uncoded feedback

(underlining)

18


Thursday, April 16, 2009

Comparisons

Short-term revision effects (comparison of the 1st

draft and the 2nd draft of essays 1, 3 , 5)

Long-term effects (comparisons between 1st

drafts of essays 1, 3, 5)

7 error categories: Verb, Noun, Word, Word

Order, New Structure, Punc, Spell (response

variables)

Error frequencies were normalized per 100 words

19


Thursday, April 16, 2009

Revision effects:

Essay length

Mean length (# words) N Std. Deviation

Ess.1 Dr.1 84.24 99 17.297

Ess.1 Dr.2 84.81 99 17.35

Ess.3 Dr.1 112.32 68 34.772

Ess.3 Dr.2 111.79 68 33.781

Ess.5 Dr.1 95.86 64 26.409

Ess.5 Dr.2 96.28 64 26.169

20


Thursday, April 16, 2009

Baseline error rate

One-way ANOVA:

First semester groups (ED1, EU1, EC1, EU1H):

no difference in error rates (p


Thursday, April 16, 2009

Short-term

revision effects

All groups significantly reduced the overall

error rate from draft 1 to draft 2

Repeated-measures ANOVA results (p< 0.05)

Essay 1: F(1,93) = 3.948; p=.000; η 2 =.298

Essay 3: F(1,63) = 9.553; p=.000; η 2 =.603

Essay 5: F(1,58) = 1.141; p=.000; η 2 =.663

22


Thursday, April 16, 2009

Short-term

revision effects (cont.)

The change in error rates between drafts was

significantly different between groups on

essays 1 and 5 but not on essay 3

Repeated-measures ANOVA results (p< 0.05)

Essay 1: F(5,93) = 2.334; p=.048; η 2 =.111

Essay 3: F(4,63) = 1.270; p=.291; η 2 =.075

Essay 5: F(5,58) = 5.918; p=.000; η 2 =.338

23


Thursday, April 16, 2009

Revision effects:

Total errors essay 1


Thursday, April 16, 2009

Revision effects:

Total errors essay 3

25


Thursday, April 16, 2009

Revision effects:

Total errors essay 5

26


Thursday, April 16, 2009

Revision effects:

Error categories

1st semester:

ED1 better than EU1 and EC1 for all error

categories (except Word Order)

EC1 and EU1: mixed pattern

2nd semester:

no difference between EC2 and PC2

27


Thursday, April 16, 2009

Revision effects:

Summary

Students in all treatment conditions improve

their accuracy on each second draft in

comparison to the first draft

Direct correction led to greater overall error

correction between drafts than underlining and

coding (1st semester)

Paper coding and electronic coding led to similar

rates of error corrections between drafts (2nd

semester)

28


Thursday, April 16, 2009

Long-term effects

All groups significantly changed their overall

error rate from essay 1 to essay 3 to essay 5 (1st

drafts only, EU1H excluded)

ED1, EC1, EU1: reduced error rates on essay 3

(except EC1) but increased them on essay 5:

F(2,52) = 4.808; p< 0.05; η 2 =.156

EC2 changed similarly to 1st semester groups;

PC2 - steady decrease:

F(2,32) = 7.153; p< 0.05; η 2 =.309

29


Thursday, April 16, 2009

Long-term effects (cont.)

The change in error rates was significantly

different between 2nd semester groups but not

between 1st semester groups

Repeated-measures ANOVA results (p< 0.05)

ED1, EC1, EU1, EU1H:

EC2, PC2:

F(6,114) = 1.354; p> 0.05; η 2 =.067

F(2,32) = 1.028; p< 0.05; η 2 =.391

30


Relative Frequencies

Thursday, April 16, 2009

Change over time

25

20

15

10

5

0

Total Error Rates - 1st Semester

E1 E3 E5

EC1

EU1

ED1

EC1

31


Relative Frequencies

Thursday, April 16, 2009

Change over time

20

18

16

14

12

10

8

6

4

2

0

Total Error Rates - 2nd Semester

E1 E3 E5

EC2

PC2

32


Thursday, April 16, 2009

Long-term effects:

Error categories

First semester (without Honors):

no significant effects in specific error

categories (except “Word”: mixed pattern)

Second semester:

no significant effects in specific error

categories (except “Word” and “Noun”: mixed

pattern)

33


Thursday, April 16, 2009

Long-term effects:

Summary

There was no significant difference between the

effects of direct correction, electronic coding, and

underlining as to changes in error rates over time

(1st semester)

There was significant difference between the

changes in error rates in response to paper

coding and electronic coding over time, but the

pattern of change was mixed and did not show

better results for either condition (2nd semester)

34


Thursday, April 16, 2009

Conclusion

All types of teacher feedback led to accuracy

improvement in the revised essay drafts

Direct correction helped learners to achieve

better accuracy in the revised drafts

No feedback type helped learners to improve

their accuracy better than the other feedback

types in the long run (including electronic

coding!)

35


Thursday, April 16, 2009

Discussion

An uneven developmental pattern has been

shown to be typical for language acquisition in

general

Essay topics may have contributed to different

changes in accuracy rates on different essays

Intervening variables may have caused

differences between EC2 and PC2

36


Thursday, April 16, 2009

Recommendations for

teachers

A sensible feedback mixture should be employed

Direct correction for “untreatable” errors

Coding/underlining for “treatable” errors

(=Ferris, 2006)

Do not spend too much time for indirect coding

Expect improvement between drafts

Take a long term view: new structures will bring

new errors

37


Thursday, April 16, 2009

Directions for future

research

Qualitative analysis of the essays (topics; specific

grammar foci; corrections made by individual

students in response to different feedback types)

New pedagogical experiments:

computer-mediated peer revision work on the

first draft; teachers code/correct only the final

draft (following High et al., 2002)

control group: no feedback on grammatical

accuracy

38


Thursday, April 16, 2009

Acknowledgements

Natalie Aaron

Jenny Baisert

Franziska Bergner

Michael DeHaven

Kristen Reinert

Caroline Smieja

Mickey Waxman

Pia Zwegers

Jennifer Laverentz and Jonathan Perkins of the KU Language and

Technology Center

New Faculty General Research Fund of the University of Kansas

More magazines by this user
Similar magazines