Gibson Ferguson Language Planning and Education Edinburgh ...

52 Language Planning and Education
3.2.1.2 The Rossell and Baker (1996) and Greene (1997) studies
Focusing – somewhat narrowly – on a comparison of TBE with four other program
types (submersion, ESL, ‘structured immersion’ and DBE) with respect to outcomes
in English language, reading in English and mathematics, Rossell and Baker
reviewed over 300 studies, identifying only 72 (25 per cent) as methodologically
acceptable. 17 The key findings were:
• On standardised tests TBE proved superior to submersion in only 22 per cent
of studies in the case of reading, 7 per cent in English language, and 9 per cent
in mathematics.
• In none of the studies comparing TBE with ‘structured immersion’ (N=12) was
there any advantage for TBE in reading, English language or mathematics.
• In the one study comparing TBE with maintenance bilingual education there
was an advantage in reading skills for the TBE program.
Rossell and Baker (1996: 44) conclude that ‘the case for transitional bilingual
education is not based on the soundly derived research evidence that its supporters
claim’. Their preferred program type is English-only ‘structured immersion’,
‘modelled after the Canadian immersion programs’, though they concede that
that type of instructional program is only one of many factors influencing school
achievement.
While these findings are at first sight very unfavourable to bilingual education,
closer scrutiny of the Rossell and Baker (1996) methodology has revealed flaws that
substantially undermine their validity. These include:
1. The use of narrative review, and in particular the raw, unsophisticated ‘votecounting’
technique, which tallies studies for and against TBE. The problem
here is that, given variations between studies in sample size, power and design,
it is not at all clear that they should be equally weighted.
2. Programs are compared by label with little or no exploration of their actual
operationalisation (e.g. Crawford 1997). 18
3. Rossell and Baker (1996) appear to equate Canadian immersion programs
with ‘structured immersion’ programs in the United States, drawing on one as
evidence for the other, when in fact, as pointed out earlier, the two have quite
different goals and serve quite different student populations (Cummins 1998,
2000).
4. Criteria for methodologically acceptable studies are applied inconsistently,
resulting in the inclusion of some studies that should by these same criteria
have been excluded (Greene 1997: 4).
There are good reasons, then, for concluding that the Rossell and Baker review is
flawed, and cannot bear the weight placed on it by critics of BE in public discussion
of Title VII programs.
In a curious parallel to Willig’s re-analysis of the Baker and De Kanter (1981)
review, Greene (1997) re-examined the 72 studies reviewed by Rossell and Baker,