10.07.2015 Views

e-asTTle linking - technical report.pdf

e-asTTle linking - technical report.pdf

e-asTTle linking - technical report.pdf

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Converting current e-asTTlescores to previous e-asTTle scoresIn order to convert a current e-asTTle scale score above 1280 to a previous e-asTTle scale score,apply the following formula:Where:• denotes the current e-asTTle writing score• denotes the previous e-asTTle writing scoreScores will be less precise after they have been converted. An estimate of error has beencalculated by combining the standard error associated with linking with the measurement errorin the previous and current versions of e-asTTle. The standard error on a score converted fromthe current version of e-asTTle is around 73 previous e-asTTle score points.A current e-asTTle score of 1280 or lower should not be converted.Table 1 Table of scale conversionsPrevious e-asTTle scale score ( ) Previous e-asTTle scale score ( )1250 1283±701275 1306±701300 1330±701325 1354±701350 1378±701375 1401±701400 1425±701425 1449±701450 1473±701475 1496±701500 1520±701525 1544±701550 1568±701575 1591±701600 1615±701625 1639±701650 1663±701675 1686±701700 1710±701725 1734±701750 1758±701775 1781±701800 1805±70Page 2 of 7


Current e-asTTle scoreThis graphic suggests a strong relationship – the data is obviously linear and sits very close to aline of best fit. This linear behaviour is not surprising given that both scales are transformationsof logit-based writing measurement scales. For example, at Year 6 both scales are constructed sothat the mean score is 1500 and the standard deviation is 100 score points.Linking based on double-markingSixty-nine student scripts (responses to e-asTTle prompts) – already marked using the previouse-asTTle writing rubric – were re-marked using the current e-asTTle rubric. This wasimplemented independently by two of the developers of the current e-asTTle writing rubric.These two developers were asked to justify or re-mark any outliers.Three student scripts with previous e-asTTle scores below around 1250 were removed from thedata. This was because the relationship between the previous and current e-asTTle scores forthese scripts was markedly different from that for the remaining scripts. The reason for thisdifference may be due to the current e-asTTle rubric being able to distinguish responses at lowerlevels of writing proficiency than the previous e-asTTle rubric. The resulting data is depicted inFigure 2. For comparative purposes, the data including the removed data points is depicted inFigure 3.Figure 2 The student script data200019001800170016001500140013001200110010001000 1100 1200 1300 1400 1500 1600 1700 1800Previous e-asTTLe scoreMarker 1 Marker 2Page 4 of 7


Current e-asTTle scoreAgain, the linear relationship in this data seems fairly clear. Some of the data sits further from aline of best fit than that depicted in Figure 1 but this is to be expected given that variancebetween markers has been introduced.Figure 3 The student script data including the removed points200019001800170016001500140013001200110010001000 1100 1200 1300 1400 1500 1600 1700 1800Previous e-asTTLe scoreMarker 1 Marker 2It is clear that the data points in Figure 3 which have previous e-asTTle scores less than around1250 are demonstrating a relationship different from the remaining data points.Linking using the pooled dataThe equipercentile links and the script data were pooled and a line of best fit was overlaid on thepooled data. The line of best fit from the pooled data was used to determine the link betweenthe previous and current e-asTTle writing scales. This is depicted in Figure 4.Page 5 of 7


Current e-asTTle scoreFigure 4 The pooled linking data200019001800y = 0.9534x + 94.542R² = 0.8246170016001500140013001200110010001000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000Previous e-asTTle scoreMarkers and normsLinear (Markers and norms)The pooled data agrees closely with its linear model as is indicated by the high R-squared value.An estimate of the standard error associated with linking – around 50 – was estimated byusing the standard error of the intercept parameter in the linear regression performed to findthe link formula.Utility of the marking rubricThe following observations were made by the two developers of the current e-asTTle writingrubric who marked the student scripts used in this linking study. They focus on the utility of thecurrent e-asTTle marking rubric.Page 6 of 7

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!