33. Overall Summary of Teacher feedback from ... - e-asTTle

2 Lavery, L., & Brown, G. T. L.individual students or groups/subgroups ofstudents. Teachers design an asTTle test byselecting the curriculum areas and levels ofdifficulty that they wish to assess. Theseselections are maximised by the asTTle toolto create a 40-minute pencil and paper testconsisting of a mixture of open- and closedresponseitems. Once student responses andscores are entered into the asTTle toolteachers may select a range of reports thatallow them to interpret student performanceby reference to nationally representativenorms, curriculum levels, and curriculumachievement objectives. Specifically, asTTleanswers questions related to (a) how well arestudents doing compared to similar students,(b) how well are students doing on importantachievement objectives, (c) how well studentsare doing compared to curriculumachievement levels, and (d) what are someteaching resources that would assist inimproving students’ performance.MethodologyNew Zealand teachers participated in avariety of workshops around the country runby the asTTle team to write and reviewassessment items for reading and writing.Their task was to write and review assessmentmaterials appropriate to the interests,achievement objectives, and ages of studentsin Years 5-7 learning in curriculum levels 2-4consistent with the principles of theappropriate curriculum document andconsistent with good classroom practice.Once reviewed, items were assembled intotest forms estimated to require 40 minutes forthe majority of students. Two differentstrategies were used in assembling test forms.In reading and writing, each form had anapproximately equal distribution of items ortasks across the Curriculum Levels 2 to 4 andwere administered to students at any of thetarget year levels. In contrast, themathematics test forms, although containingmaterials from all three levels, were designedto have more of a certain level for use in eachof the target year levels. The Year 5 papershad more Level 2 items, the Year 6 papershad more Level 3 items, while the Year 7papers had more Level 4 items. Note that thetest forms as administered were balanced inquite a different fashion to the asTTlecomputer tool which allows teachers tocustom select the difficulty desired regardlessof the year or age of students. For example, ateacher using asTTle can create a test with noor few hard items for a younger or less ablegroup of students and vice versa.Trials and calibrations for asTTleassessments across the reading, writing, andmathematics domains in both languages wereconducted on behalf of asTTle by classes ofstudents in New Zealand schools. Each trialand calibration was administered by teacherswith their own class of students. BetweenOctober 2000 and June 2002 data werecollected for English reading, writing, andmathematics a total of ten times (i.e., six forliteracy and four for numeracy). Trials wereconducted on relatively small samples toidentify dimensions of the items and testforms that might need modification.Calibrations were conducted after the trialswith large nationally representative samplesfor the purpose of establishing New Zealandstudent performance norms. Table 1 showsthat nearly 50,000 students completed thetrials and calibrations for all three Englishsubjects.Table 1Trial and Calibration Student Sample bySubjectSubjectSample SizeReading 24,513Writing 10,377Mathematics 13,397Total 48,287In Maori, there are about 2,500 students inMaori-medium instruction in each of thetarget years. In order to prevent over usage ofthis small number of students, trials andcalibrations were combined so that data werecollected three times between November 2001and June 2002. Note that this report is basedon the feedback and teacher evaluativecomments to the English reading, writing, andmathematics assessments only.