The Effects of Distance Education on K-12 Student Outcomes: A ...

More documents

Recommendations

Info

Instruction 2003 Texas <strong>Education</strong> Agency 2003* 9-11 English, Mathematics, Science, Social Studies State charter * indicates studies yielding multiple effect sizes State test Full program Combination 15-21 -0.014 0.445/- 0.474 Statistical analysis <strong>of</strong> effect sizes <strong>The</strong> test for heterogeneity (Q), based on Hedges and Olkin (1985), was used to determine whether the effect sizes <strong>of</strong> the studies were homogenously distributed, in other words, to learn whether the distribution <strong>of</strong> effect sizes around their mean was what would be expected from sampling error alone (Lipsey & Wilson 2001). <strong>The</strong> Q value for the weighted effect sizes was 1.485, and was considered to be homogeneous, indicating that the variance observed was likely to be due to sampling error. <strong>The</strong>refore, the fixed-effects model was used to estimate variance (Kromrey & Hogarty 2002). Study feature analyses were performed to determine the extent to which student outcomes were moderated by the study variables. Statistical Analysis System (SAS) s<strong>of</strong>tware was used for the analyses. Effect size comparisons were done for the variables: grade level, content area, duration and frequency <strong>of</strong> the distance learning experience, instructional role <strong>of</strong> the distance education, pacing <strong>of</strong> the instruction, role <strong>of</strong> the instructor, timing <strong>of</strong> the interactions, and types <strong>of</strong> interactions, as well as for various study quality and invalidity factors. Results Characteristics <strong>of</strong> the Studies <strong>The</strong> 14 studies included in the analysis yielded 116 independent effect sizes drawn from a combined sample <strong>of</strong> 7561 students whose performance as a result <strong>of</strong> participation in a distance education program was compared to control groups in which students did not participate in distance education. Sixty one percent <strong>of</strong> the study results had sample sizes <strong>of</strong> less than 50, and 16% had sample sizes above 100. All but one <strong>of</strong> the studies included more than one comparison, and the average number <strong>of</strong> comparisons per study was 8, ranging from one to 38. Eighty six percent <strong>of</strong> the studies were organization reports, 7% were published articles, and 7% were dissertations. All <strong>of</strong> the studies were published between 1999 and 2004, with eleven published during 2003 and 2004, and three published from 1999 through 2001. Two studies were published in Canada, and the other twelve were published in the U.S. A range <strong>of</strong> distance learning structures was examined in the literature. Half <strong>of</strong> the studies reported on programs that used asynchronous timing in instruction. Three studies documented a program that used a combination <strong>of</strong> synchronous and asynchronous instruction, one program was delivered synchronously, and the remaining programs did not report on instructional timing. Ten <strong>of</strong> the studies reported results <strong>of</strong> student participation in full year-long distance learning programs, one included data for distance learning courses, and three studies focused on portions <strong>of</strong> courses delivered at a distance for less than a semester. Thirteen studies included data from 12 <strong>of</strong> 32
programs in which students participated approximately five days per week, and the other study did not indicate the frequency <strong>of</strong> student participation. <strong>The</strong> studies encompassed a variety <strong>of</strong> instructional features. <strong>The</strong> bulk <strong>of</strong> the results, 75%, occurred in the secondary grades, 6-12. <strong>The</strong> other results concern elementary age children, in grades 3-5. Results from seven academic content areas were reported. Thirty percent <strong>of</strong> the results came from tests <strong>of</strong> reading ability, followed by mathematics, which accounted for 26% <strong>of</strong> the results. Writing was the subject for 16% <strong>of</strong> the results, science was the topic <strong>of</strong> 14%, and social studies made up 9% <strong>of</strong> the results. Three percent <strong>of</strong> results came from physical education comparisons, and one percent from a test <strong>of</strong> listening. National tests were used to compare outcomes in one study, state tests were used in nine studies, teacher made tests were used in two studies, and one study reported data from both state and national tests. Overall <strong>Effects</strong> <strong>of</strong> K-12 <strong>Distance</strong> <strong>Education</strong> <strong>The</strong> analysis resulted in an overall weighted effect size not significantly different from zero, a result that is in line with the results <strong>of</strong> recent meta-analyses <strong>of</strong> distance education (see Table 1), which tend to show that distance education is as effective as classroom instruction. <strong>The</strong> weighted mean effect size across all results was -0.028, with a standard error <strong>of</strong> 0.045 and a 95% confidence interval from -0.116 to 0.060. <strong>The</strong> average unweighted Cohen’s d was -0.034, and the median effect size was -0.015. <strong>The</strong> effect sizes varied considerably among the studies (see Figure 1). Unweighted effect sizes ranged from -1.158 to 0.597, with a standard deviation <strong>of</strong> 0.157, indicating that some applications <strong>of</strong> distance education appeared to be much better than classroom instruction and others were much worse. However, each <strong>of</strong> the fourteen studies and all except one <strong>of</strong> the 116 outcomes within the studies had individual effect sizes that did not differ significantly from zero. Figure 1. Distribution <strong>of</strong> unweighted effect sizes Number <strong>of</strong> results Distribution <strong>of</strong> unweighted effect sizes -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 Effect size 13 <strong>of</strong> 32 100 80 60 40 20 0
Page 1 and 2: The Effect
Page 3 and 4: The Effect
Page 5 and 6: are inherently interesting and mean
Page 7 and 8: 4. To what extent are results relat
Page 9 and 10: Abstracts in American Educa
Page 11: encompassing zero, is an indication
Page 15 and 16: of invalidity. Non
Page 17 and 18: professional devel
Page 19 and 20: a rather short-sighted view <strong
Page 21 and 22: References References marked with a
Page 23 and 24: Fording, L. (2004). Educati
Page 25 and 26: * Stevens, K. (1999). Two Canadian
Page 27 and 28: 9. Content area (“content”). Un
Page 29 and 30: Other=5. 4. Setting of</str
Page 31 and 32: 6. Statistical Regression (“regre

The Effects of Distance Education on K-12 Student Outcomes: A ...

Create successful ePaper yourself

Delete template?

Save as template?