Development of the asTTle Writing Assessment Rubrics for Scoring ...

Development of the asTTle Writing Assessment Rubrics for Scoring ...

Development of the asTTle Writing Assessment Rubrics for Scoring ...

  • No tags were found...

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

Technical Report 6, March 2001Development of the asTTle Writing Assessment Rubrics for ScoringExtended Writing TasksTechnical Report 6, Project asTTle, University of Auckland, 2001Kathryn GlasswellQueensland University of TechnologyJudy ParrUniversity of AucklandMargaret AikmanAuckland College of EducationThis report outlines an assessment procedure for extended texts based on “Writing”components of the English Curriculum. It reviews recent research on scoring such texts,explores the notions of genre and purpose, notes difficulties for New Zealand teachers inthe current documents available to them, and defends the argument that writing is a socialact and the purpose or function of the writing is defined by the context. This means that theform and features of the text may differ according to purpose or that the same form (such asa letter) can serve different purposes. Thus, the scoring rubrics are aligned with thefunction of the writing, and communication and social purpose are overarching concepts.Rubrics are developed for each of these major functions, each containing three metadivisionsfrom which to consider the features of text as related to the purpose and context:rhetorical (audience awareness/ purpose); organisational/structural (content inclusion;coherence – sequencing ideas and linking; and language resources for achieving purpose),and conventional: sentences and words (grammatical conventions, spelling, andpunctuation).Table of ContentsAssessment of Extended Text: The ScoringRubrics ..............................................................1Theoretical Perspective and the Context forDevelopment ................................................1Description of Rubrics .................................4Overview of Process of Development andTrial Implementation.........................................9Method .........................................................9Data Sources.................................................9The Teachers...................................................10Feedback from Teachers Concerning Rubrics 10Ease of Use.................................................11Relevance to NZ Context ...........................12Usefulness for Classroom Teachers ...........12Support for Teachers when Using Rubrics .....13The Writing Tasks...........................................14Summary .........................................................15References.......................................................15Appendix 1 – Scoring Rubrics........................17Appendix 2 – Teacher SummativeEvaluation Questionnaire................................25Appendix 3 – Tips on Scoring ........................27Assessment of Extended Text: The ScoringRubricsTheoretical Perspective and the Context forDevelopmentAssessment of students’ extended writing isa challenging task that presents a range ofissues and problems depending on astakeholder’s vantage point (White, 1994). Thework reported here specifically concernsassessment linked to teaching and learning.The work involved the development of scoringrubrics for teacher assessment of extended textas part of a larger national project, designed toprovide teachers of children in Years 5–7 withassessment tools that can be tailored toteachers’ and schools’ requirements. Theassessment tools are designed to serve a dualpurpose: They are designed to provide teacherswith information about their students,specifically to assist their teaching for increasedlearning outcomes in the areas of reading,1

2 Glasswell, K., Parr, J., & Aikman, M.writing, and numeracy. They are also designedto allow teachers to compare their students withothers using the norms for the assessment tasksselected. These tools can be used when andwhere schools or classroom teachers wish andas often as they find useful. Thus, the task wasto develop rubrics within this context to servesuch purposes.Where assessment is to serve the function ofinforming teaching practice and, subsequently,improving individual students’ learning, suchtools have to be designed with close attentionboth to notions of developmental progressionand to contextual factors. Contextual factorsinclude the present state of knowledge in thefield of written composition and its assessment;the mandated curriculum; the skills in, trainingin, ideas about, and goals for literacy learningheld by teachers doing the assessment;psychometric concerns for sound assessment;and more pragmatic considerations such as theorganisational practices of those who will usethe rubrics. The task was to develop rubricsthat would both reflect and construct teachers’expertise in teaching writing as well as inassessing it.Recent approaches to assessing writingappear to reflect the broader debate about genreversus context. The view that acquiringexpertise in writing is the acquisition of ageneric set of processes that could be appliedacross all writing tasks is reflected in theassessment rubrics developed (e.g., the Six-Trait Analytic Writing Assessment Model,Northwest Regional Educational Laboratory,1998). Functional genre theorists (e.g., Martin,Christie, & Rothery, 1987) emphasise thedefinable text features common across variouscontexts. Social process theorists (e.g.,Berkenkotter, 1993; Chapman, 1999; Freedman& Medway, 1994) argue that genres are highlycontext dependent and that writers access themfor particular social purposes. The theoreticalstance informing our development of scoringrubrics is that writing is a purposeful socialinteraction that can function to accomplishcertain social goals. Further, features of textsare related to the purposes of and contexts forwriting, so scoring rubrics need to reflect this.Although there are differing ways ofclassifying the functions of writing, there tendsto be considerable overlap among the systemsof various authors (e.g., Derewianka, 1990;Knapp & Watkins, 1994; Wing Jan, 1991; Wray& Lewis, 1999). Informed by research – butalso keeping in mind the New Zealand nationalcurriculum, where writing (after Britton,Burgess, Martin, McLeod, & Rosen, 1975) isseen to serve three broad functions: expressive,transactional, and poetic – we viewed writing asserving six major functions or processes (afterKnapp & Watkins, 1994). These are: toexplain; to argue or persuade; to instruct or layout a procedure; to classify, organise, describe,and report information; to inform and entertainthrough imaginative narrative; and to informand entertain through recount. This was thetheoretical stance and resulting framework forthe development of the scoring rubrics.Thus, the choice we made was to follow theconceptualisation of genre as driven byfunctional purpose (Gerot & Wignell, 1994;Martin, 1985) rather than by mode (text form).Even in this choice, we hope there will beramifications for professional development ofteachers. In some teachers’ texts, there appearsto be a certain measure of confusion betweenfunction and mode or text form; indeed, somewriters seem to view them as synonymous. Forexample, in the research that preceded thisreport (Limbrick, Girven, & Keenan, 2000), thetask was to map the New Zealand Englishcurriculum. Under the heading “WritingStrategies” was the aim derived from English inthe New Zealand Curriculum (ENZC) (Ministryof Education, 1994) – namely, to “demonstrateability to record experiences, events, feelingsand ideas in a range of text forms usingappropriate language and text features” (p. 9 indraft). Listed under this aim (see Appendix 1,p. 7) was a mixture of both functions such as torecount, to argue, or to list a procedure and texttypes such as a letter, a diary, or a play.Further confusion over the blurring ofdistinction between text form and function isapparent in classroom practice. Many NewZealand teachers, for example, teach theirchildren how to structure letters in the beliefthat this will be helpful information for them as

Technical Report 6: Writing Assessment Rubrics 3developing writers. The erroneous assumptionhere is that writing a letter is mostly aboutformat and layout (e.g., teachers will often pointout to children the difference between formaland friendly letters in terms of language use, orregister, and layout). In reality, the letter formatis a matter of mode not genre. An over-ridingconsideration for one’s letter writing needs tobe the function or purpose for which a letter isbeing written. Letters may be written to makecomplaints, to argue a point, to recount anevent, to make an explanation, to tell ananecdote, or to advertise a product. In short,letters may have different purposes and, thus,the structuring of these texts and their lexicogrammaticalresources will differ significantly,regardless of the fact that each will still beconsidered a letter in terms of layout andtransmission. So, a letter, it seems, is not just aletter (Gerot, 1995). To teach children texttypes with little regard for purpose or audiencemasks important rhetorical considerations.Teachers who wish children to understandlanguage as a system for representing andtransforming their own worlds will recognisethe need to develop knowledge not just abouttexts but about writing as a purpose-drivencommunicative response to social and culturalcontexts. This concept of communication inresponse to social and cultural contexts is whatwe wish to foreground with the assessmenttasks and the rubric design.In line with others (e.g., Derewianka, 1990;Knapp & Watkins, 1994), we argue that thereare features of text associated with each of itsfunctions or purposes. Specification of these,similarly, draws from a research tradition thatbegan with Nold (1981; see also Parr, 1989;Glasswell, 1999), which views written pieces ashaving rhetorical, structural, lexicogrammatical,and conventional dimensions.A further premise that was important ininforming our design of the scoring rubricsbroadly involved the idea that diagnosticassessment should help teaching to proceed inan individual’s zone of proximal development(Vygotsky, 1978). That is, diagnosticassessment needs not only to specify criteriathat place a student at a particular point, butalso to suggest what is required for that studentto move forward. Thus, one of the criteria forthe rubrics was that they had to specify adevelopmental continuum. English in the NewZealand Curriculum (ENZC) indicated that theschool year groups (Years 5–7) covered by theassessment tools were likely to span threedevelopmental levels (curriculum levels 2–4)within the eight-level curriculum (NB: levels donot correspond to years of schooling). Further,the developmental continuum concept was tooperate within a single task as per Planning andAssessment in English (Ministry of Education,1997).The design approach taken here reflects thegoal of the Assessment Tools for Teaching andLearning (asTTle) Project to have rubrics thatwould be used for assessing responses for arange of tasks associated with each of thefunctions of writing. For example, there maybe several different writing tasks that require astudent to argue or persuade and the rubricsneed to be applicable to each task and acrosslevels. This approach differs significantly fromthat used by large-scale testing organisations(e.g., National Assessment of EducationalProgress (NAEP) (USA), National EducationMonitoring Project (NEMP) (NZ), andEducational Testing Centre (ETC) (Australia))where task-specific rubrics are designed andused. In addition, NAEP and ETC generallydesign tasks specifically for a single year level.In these regimes, performance, therefore, isassessed against a specific task and within onelevel. The asTTle project, as its title suggests,is primarily concerned with assessment tools forteaching and learning. Information aboutrelativity is only one part of it. Thus, the set ofconstraints and the purpose underpinning therubric design makes direct comparison withother writing scoring rubrics difficult.The tasks for extended writing assessment donot vary across the three levels of achievement;rather students’ degree of expertise inresponding to the task varies. In the theoreticalcontext underpinning the design of theserubrics, expertise is seen to exist on acontinuum of developing expertise. Thus, inscoring, decisions are to be made about whereon the continuum of levels the various aspectsof a piece lie, according to the criteria specified.

4 Glasswell, K., Parr, J., & Aikman, M.Further, within each level, the weight ofevidence should allow aspects of the piece ofwriting to be seen as demonstrating basic,proficient, or advanced qualities on the criteriawithin each of the categories.Description of RubricsOur argument, therefore, following others inthe research literature, is that writing is a socialact and the purpose or function of the writing isdefined by the context. This means that theform and features of the text may differaccording to purpose and that the same form(such as a letter) can serve different purposes.Thus, the scoring rubrics are aligned with thefunction of the writing, and communication andsocial purpose are overarching concepts.The three types of writing described inEnglish in the New Zealand Curriculum(ENZC) – namely, expressive, poetic, andtransactional – were seen as too broad adelineation within which to develop workablerubrics. Specified in the writing achievementobjectives within the ENZC document arenarrating, instructing, arguing, recounting,explaining, and providing factual account orreporting. Finally, Drawing from the researchand teaching literature (e.g., Derewianka, 1990;Knapp & Watkins, 1994; Wing Jan, 1991; Wray& Lewis, 1999), we defined writing as servingthese six main functions:• To explain;• To argue or persuade;• To instruct or lay out a procedure;• To classify, organise, describe, and reportinformation;• To inform or entertain through imaginativenarrative;• To inform or entertain through recount.Rubrics were developed for each of thesemajor functions. (Note: There were no tasks inthe trial papers that could be said to call forpersonal interpretive writing or for explanation.Indeed, official documents accompanyingENZC state that expressive writing, as they termit, is not assessable. Our feeling, however, isthat the type of personal writing that reflects onactions and events is able to be assessed.)The scoring rubrics all contain three metadivisionsfrom which to consider the features oftext as related to the purpose and context (seeFigure 1). Within the three meta-divisions,there are seven categories with criterionstatements. The meta-divisions with theircategories are: rhetorical (audience awareness/purpose); organisational/structural (contentinclusion; coherence – sequencing ideas andlinking; and language resources for achievingpurpose); and conventional: sentences andwords (grammatical conventions, spelling, andpunctuation).ContentinclusionsGrammar:sentences andwordsRhetorical concerns:the writer, the context, and the purposeText: content and structureText structureConventionsSpellingLanguageresources forachieving thepurposePunctuationFigure 1. A common framework for all rubrics.Thus, rubrics to score tasks encapsulating thefive major functions were developed, each withcommon meta-divisions. However, criteriawithin the categories of two of the metadivisions(rhetorical and organisational)differed according to function.Criteria in the seven categories under thesemeta-divisions can be scored at levels 2, 3, or 4and, within a level, at basic, proficient, andadvanced according to the weight of evidence.This allows a profile to be constructed for awriter, a profile that may be different acrosswriting functions as well as across aspects(categories) within a function. Our goal was todesign rubrics that had the potential to providedetailed, useful information to classroomteachers. Assessing children’s performance onwriting tasks using the rubrics will allowteachers to view each child’s developmentindividually. Teachers will be able to ascertainan individual child’s profile across thedimensions of text and compare this profile tothe profiles of other students. In some cases,

Technical Report 6: Writing Assessment Rubrics 5children achieving the same “score” will do sofor different reasons.Developmental theorising would support thisnotion of differences in forms as well as levelsof expertise. Our assumption is that while ourachievement goals for children will remainstable (for example, that they will reach Level 4advanced, or the Literacy Task Force’sstatement that “by the age of nine every childwill be able to read, write and do maths”), thepaths they may take to that achievement willdiffer (Clay, 1998; McNaughton, 1995). Whenteachers use the rubrics, they will be able to“map” a child’s development across dimensionsof text, allowing diagnosis of sub-skilldifficulties to be situated within the context of awhole communicative task. Teachers will seewhere each child’s strengths and weaknesses lieand will then be able to use the developmentalcharacteristics provided by the rubrics (basedon ENZC) to plan teaching and learningactivities in each child’s zone(s) of proximaldevelopment. They have a description of howto “up the ante” (Bruner, 1983).This is a powerful feature. Further, the factthat the rubrics relate to broad purposes forwriting rather than individual tasks or textforms, becomes a major strength. Teacherscannot “teach to the test” as the “test” is not astable task. The specifics of the task vary,while the broad communicative functions ofwriting will remain the same. This is ourapproach to teaching writing and to developingskilled writers, and it dovetails with NewZealand teachers’ philosophy of developingliteracy through real and meaningful activities.Following is an explanation related to thecriteria in each of the categories for each of thefunctions of writing. This explanation includesthe major feature of the functional genre, thetext types that may be included, the content andstructural features, and the major languageresources identified with each function.Table 1Explanations for the criteria in the rubric “toexplain”Purpose or function of writingTo explain (excluding commonsense or everydayexplanations)To give an account of how something is formed orhow something works, and provide associatedreasons.To explain the processes involved in, and reasonsfor, mechanical, natural, technological, orsociocultural phenomena.TypesTwo main types of explanation with variations infocus:How does it work? (E.g., How does a pump work?How does Parliament work? How are mountainsformed? How do plants grow?)Why is it so? (Why do some things float? Why doour bodies need food? Why is the ozone layerthinning?)Generic Features (Content)Staging (essential):Introduction: A general statement to establish thepurpose of the text to position the reader (may be inthe form of a title). Identifies the phenomenon to beexplained.Body: Explanation Sequence – an account is givenof how and/or why something occurs or works(focus on giving reasons and making the processunderstandable). Note: Complex explanations mayhave multiple parts or sub-sections. Explanationsmay be part of more complex or substantial texts(e.g., a report on tuatara may include anexplanation section to detail the reproductive cycle– how tuatara reproduce).Text Organisation/ Structure – sequencing andlinkingGenerally organised around a sequence explainingwhy something is so or how it works. Ordering islogical and links between aspects of thephenomenon (e.g., sequence or parts) and theirassociated reasons/functions are evident (use ofconjunctions of time and cause and effect).Organising devices such as paragraphs assistwriters to structure related aspects into themedgroups. Links between paragraphs assist increating cohesion and relevance.Language Resources for TextFocus on generic (non) human participants (thewind, glaciers, tuatara, computers).Precise, descriptive, factual language (verbs,adverbials, adjectivals, nouns) employed to givedetail to the explanation and causal circumstances.Technical language related to the topic (whereappropriate) adds authority to the text and writer.Explanations generally employ declarative moodchoices to make statements of fact and offerreasons for and explanations of the phenomena.Verbs: Mainly action verbs that tell of actions (andbehaviours – depending on the field). Someexisting and relational verbs assist in establishingexplanations.Tenses are commonly timeless present tense(evaporates, grows, eats, orbits). Some use ofpassives to define and/or describe actions where

6 Glasswell, K., Parr, J., & Aikman, M.agent is obscured or unimportant in the explanationsequence (e.g., “Gradually, these rocks are erodedand sand is formed”).Conjunctions show linkages in time and placerelationships for sequencing (e.g., first, then,following, finally). Conjunctions of consequence(cause and effect) link aspects and reasons throughcausal relationships (e.g., if …, then …, so, as aconsequence).Table 2Explanations for the criteria in the rubric “toargue or persuade”Purpose or function of writingTo argue or persuadeTo argue a position or persuade a reader to thewriter’s point of view. Arguments are expositorytexts centring on an assumption that the writer mustconvince the reader through the presentation ofrelevant points with supporting evidence.TypesMany types of persuasive texts with variations infocus. E.g., arguments: analytic exposition(describing component elements) and hortatoryexposition (tending to exhort readers to a course ofaction or beliefs). Persuasive texts:advertisements. Main focus for this rubric isargument.Generic Features (Content)Staging:Thesis or position statement (provides the readerwith the context).Body: Main points with elaboration or supportingevidence take the reader through a structured andlogical presentation of information (evidence and/orillustration) to support the writer’s position (thesis).Conclusion: Re-states the writer’s position (and/orrecommendation for action – what ought or oughtnot to be done – hortatory exposition).Text Organisation/ Structure – sequencing andlinkingFocus on objects and ideas (rather than events,happenings, or processes).Logical Organisation: Information and ideasgrouped logically and linked thematically.Organising devices such as paragraphing andconjunctions are used to show relations amongcontent items or ideas.Language Resources for TextThe arguments name and describe generalisedparticipants or abstract concepts (e.g., “parents”,”blue whales”, or “the gun-control lobby”).Arguments employ declarative mood choices tomake statements of fact and offer personal opinionson the topic.Language resources: Precise, descriptive, factuallanguage employed to give detail and credibility tothe argument. Persuasive/emotive languagecommonly used to add to reader impact and makeargument seem powerful. Use of idiomaticlanguage to appeal to readers’ senses andemotions. Technical language related to the topic(where appropriate) adds authority to the text andwriter.Verbs: Many existing and relational verbs (beingand having verbs such as is, are, have, belongs to).Verbs are used to make clear the “state of play”.Choice and use of verb-vocabulary often reflectsthe desire to create particular information-ladenmeanings for the reader.Modals (must, might, can, ought to, should, etc) areoften used to give information about the degree ofobligation or certainty involved in the argument.Verbs are commonly in timeless present tense.This adds to the authority of the text as readers aregiven a version of the world “as it is”. Passivestructures are also employed to make the text seemmore objective and formal.Nouns: Arguments often make use ofnominalisation and abstract nouns to enhance (theappearance of) objectivity and formality. Nounpackingis a common device for developing conciseand precise descriptions. Adjectivals are often“stacked” to produce densely packed noun-groups.Conjunctions: Additive and causal relations arecommon in these texts which define and elaboratepositions and their underlying reasons (e.g., inaddition to, and, if …, then …).Table 3Explanations for the criteria in the rubric “toinstruct or lay out a procedure”Purpose or function of writingTo instruct or lay out a procedureTo give instructions or to lay out a procedure.Usually to describe how something may beaccomplished through a sequence of actions orsteps. (To tell someone how something is done.)TypesSeveral types: recipes, appliance manuals,assembly instructions, games’ rules, etc.Generic Features (Content)Instructions and procedures contain informationstatements (often imperative and declarative) thattell another person how something may beachieved. Domain elements include:Goal statement/title: provides information for thereader about the nature of the procedure to beoutlined. Identifies the product to be made or theprocess to be done.Materials (not required for all procedural texts):Tells the reader what resources may be required tocomplete the procedure, usually ordered.Steps (1-n): The description of the sequence ofsteps required in order for the reader to achieve thegoal.

Technical Report 6: Writing Assessment Rubrics 7Advice/background information may be included atany time as a means of clarifying the procedure.Text Organisation/Structure – sequencing and linkingGenerally organised around a process. Focus is onactions (rather than events or things) and humanagency.Content is structured according to a prescribedsequence of events. Temporal sequence isemployed to tell the reader the order of the steps.Text organisers such as titles, headings, and subheadingsmay be used to orient readers.Language Resources for TextPrecise, descriptive language is employed to clarifyaspects of procedure (e.g., action verbs, adverbials,adjectivals add detail and clarity what is neededand what is to be done).Pronoun use or omission (where the reader isinferred) refers to reader in a generalised way (e.g.,“First you break the egg” or “Break the egg”).Verbs: Many action verbs are used, to describeprocesses to be done by the reader (e.g., whisk,cut, deal, transfer, twist).Choice and use of verb-vocabulary: Precise verbchoices reflect the desire to clarify meanings for thereader (e.g., trim rather than cut).Verbs are commonly in simple present tense.Mood choice is often imperative (command-likestatements), though declarative statements may beused to contextualise the action or give advice tothe reader.Conjunctions: Time/sequence relationships inprocedure generally indicated by the use oftemporal conjunctions (e.g., first, then, next, after)or numbering.Some cause-and-effect conjunctions may bepresent (e.g., if …, then …).Table 4Explanations for the criteria in the rubric “toclassify, organise, describe, or reportinformation”Purpose or function of writingTo classify, organise, describe, or reportinformationTo document, organise, and store factualinformation on a given topic. Usually to classify anddescribe whole classes of things (reports) orspecific things (descriptions) living and non-living(scooters, Pikachu, my teddy, blue whales).TypesMany types – This rubric deals specifically withinformation reports and descriptions.Generic Features (Content)Reports and descriptions contain informationstatements (often declarative). Domain elementsinclude:General classification statement – providesinformation for the reader about the nature of thesubject of the text (e.g., “Kiwis are flightless birds”;“My teddy is the most precious toy that I have”).Descriptions – Tells what the phenomenon/itemunder discussion “is like”. Information is managedin parts to provide details about (depending on topicof report or description) components and theirfunctions, properties, behaviours, uses, locations orhabitats, types, and relationship to writer).Conclusion (optional) – The writer may concludethe text in a simple manner. The text does nothave an ending as such, but writer may round offwith a general statement about the topic (e.g.,“Today the Kiwi is well known around the world as asymbol of New Zealand”. Or “I love my teddy morethan any other toy I have. I hope I never lose him”).Text Organisation/Structure – sequencing and linkingGenerally organised around things and theirdescription.Logical ordering of information (no temporalsequence) – content is grouped/structuredaccording to common themes evident in theinformation given. Sentences are linkedthematically to topic of paragraph or section.Text organisers such as titles, headings, and subheadingsare commonly used to orient readers.Language Resources for TextThe descriptions name and describe specificpeople or things (e.g., “my teddy”).Reports name and describe generalisedparticipants or whole classes of things (e.g., bluewhales or the kiwi as species, telephones).Reports and descriptions employ declarative moodchoices to make statements of fact.Language resources: Precise, descriptive, factuallanguage employed rather than “flowery” oraesthetically pleasing language. Technicallanguage related to the topic is common in reports.Language of comparison is common (comparatives,superlatives). Simile and metaphor may also beutilised as devices of comparison.Verbs: Many existing and relational verbs (beingand having verbs such as is, are, have, belongs to).Verbs are used to classify, to identify what thephenomenon is like, and what it comprises.Some action verbs are used to describe behaviours(if living) or uses (if non-living).Choice and use of verb-vocabulary often reflectsthe desire to create particular information-ladenmeanings for the reader (e.g., forage rather thansearch for food).Verbs are commonly in timeless present tense.This adds to the authority of the text as readers aregiven a version of the world “as it is”. Passivestructures are also employed to make the text seemmore objective and formal.Nouns: Noun-packing is a common device fordeveloping concise and precise descriptions.Adjectivals are often “stacked” to produce denselypacked noun-groups.

8 Glasswell, K., Parr, J., & Aikman, M.Conjunctions: Additive relations are common inthese texts which define and elaborate throughdescriptions (e.g., in addition to, and).Table 5Explanations for the criteria in the rubric “toinform or entertain through imaginativenarrative”Purpose or function of writingTo inform or entertain through imaginative narrativeTo inform or entertain a reader or listener byconstructing a view of the world that the reader canenter. Narratives centre on a problem that isusually resolved in the course of the telling.TypesMany types of narrative with variations in focus:folktales, fairytales, myths, legends, short stories –e.g., historical, romance, fantasy, crime, sciencefiction, adventure.Generic Features (Content)Narratives develop characters, setting(s), plot, andtheme. Point of view (perspective from which thestory is told). Use of dialogue.Staging – most writing for this purpose contains thefollowing elements (not always in this order):Orientation (provides the setting and usuallyintroduces the main character(s));Complication (a problem or crisis – somethingis/goes wrong) – this usually necessitates goingthrough a…Series of events (steps to resolve the problem) untilreaders are taken through to a…Resolution (the problem is solved – for better orworse);Coda (optional) (a reflective statement often relatedto the theme may occur at any time – in some typesof narrative it is common at the end).Text Organisation/Structure – sequencing and linkingGenerally organised around events or happenings.Temporal sequence: Organised through time(conjunctions and adverbials show linkages insetting events in time, ordering the events and thepassage of time).Language Resources for TextSpecific people, places, and events are named(e.g., “Winnie the Pooh and the Hundred AcreWood” rather than “bears and forests”.Language resources (figurative language devicessuch metaphor, idiom, and onomatopoeia, anddescriptors such as adverbials and adjectivals) arecommonly used to add interest, engage the reader,and give detail to characters, settings, and events.Dialogue (or direct speech) is often used to developcharacters and plot and to give the story a “realistic”feel.Verbs: Many action verbs that tell of happeningsand behaviours.Some sensing and thinking verbs used to describethe thoughts and feelings of characters.Some saying (verbal) verbs that tell of charactersspeaking.Some existing and relational verbs used to tell ofsettings establish and reflect on characters andproblems as they are.Choice and use of verb vocabulary often reflectsthe desire to create particular images or feelings forthe reader.Verbs are commonly in past tense, although tensecan vary (e.g., flash-back may use present tense torelate a past event “as it happens”).Conjunctions and adverbials show linkages insetting events in time, ordering the events and thepassage of time).Table 6Explanations for the criteria in the rubric “toinform or entertain through recount”Purpose or function of writingTo recountTo inform or entertain a reader or listener byreconstructing a view of the world that the readercan enter. Recounting centres on the sequencedretelling of experience (real or imagined).TypesThere are three common types of recounting –each with its own variations in focus.Personal recounting involves the reconstruction of apersonal experience. Often include reflections onthe writer’s feelings.Factual recounting involves the recounting ofevents from an informational perspective (e.g., avisit to McDonalds). Often include statements ofobservation as asides to the recounting of events(e.g., “The ice cream machine behind the counter isbig and shiny. I saw people polishing it. It takes alot of work to keep it that shiny”).Imaginative recounting may involve the writer inrecounting events from an imagined perspective(e.g., A day in the life of a Viking Raider) orrecounting imagined events from a personalperspective (e.g., A field trip to Mars); theserecounts may include both imagined observationand comment.Generic Features (Content)Staging:Orientation recounting uses a succinct orientatingdevice early in the piece to introduce characters,setting(s), and events to be recounted (who, what,why, where, when, and how).Point of view is often established here (perspectivefrom which the recount is told).Events related in time order.Comment, observation, and/or reflection is used toforeground events or details of significance to thewriter. May be interwoven with the retelling.

Technical Report 6: Writing Assessment Rubrics 9Re-orientation (optional) – an ending statementoften used to reflect on or comment on the eventsrecounted, or to predict (e.g., “I had a great time atCamp Hunua. I wonder what will happen to us nextyear!”).Text Organisation/Structure – sequencing and linkingRecounts are organised around a sequencedaccount of events or happenings.Temporal sequence: Organised through time –conjunctions and adverbials show linkages insetting events in time, ordering the events, and thepassage of time.Language Resources for TextSpecific people, places, and events are named(e.g., “On Saturday, our class had a sleepover atKelly Tarlton’s Underwater World in Auckland.” or“Today, we raided Lindisfarne Abbey to gathermore gold for our longboat”).Detailed recounting makes effective use ofdescriptive verbs, adverbs, adjectives, andidiomatic language to catch and maintain readerinterest. Frequent use of prepositional phrases,adverbials, and adjectivals to contextualise theevents that unfold.Dialogue (or direct speech) is often used to give therecount a “realistic” feel, to assist in thereconstruction of the events, or to provideopportunities to comment on the happenings.Verbs: Many action verbs tell of happenings andbehaviours of those involved. Some relationalverbs to tell how things are as the writer reflects,observes, or comments. Choice and use ofvocabulary often reflects the desire to createparticular images or feelings for the reader.Verbs are commonly in past tense, although tensecan vary in comments (e.g., “On Tuesday, Maryand I went to the shop. We are best friends”).Conjunctions and adverbials show linkages insetting events in time, ordering the events, and thepassage of time).The actual rubrics in the form they appearfor scoring each of these functions are inAppendix 1.Overview of Process of Development and TrialImplementationMethodThe development of the rubrics was aniterative process and involved both conceptualwork on the part of the researchers and trial anddevelopment work with teachers. From theconceptual framework outlined above, theresearchers drafted rubrics for each of the fivemajor functions to score tasks designed todemonstrate that function. Seventeen teacherswere recruited to work for four hours a sessionfor five sessions to score using the rubrics andprovide feedback on the process. This feedbackconcerned both the framework for the rubricsand the teachers’ ability to score using them.The teachers were given an explanation ofthe features of each rubric in turn before theybegan any scoring. They then individuallyscored a test set that consisted of 8 to 10 scriptsin randomised order with a core of four scriptsthe same for each teacher. The pool of 30scripts for each function of writing wasrandomly selected from three schools that hadparticipated in trials of the extended writingtasks. The schools were selected to representthree different points on the decile range(schools are classified from 1 to 10 on the basisof socio-economic features of their catchmentarea). Teacher feedback was formally sought atselected points and informal feedback waswelcomed throughout the sessions.Data SourcesThe National Curriculum document, andother official material (e.g., Dancing with thePen, Planning and Assessment in English)published for teachers to support the curriculumdocument, were consulted to ensure that thefunctions of writing that we finally agreed onwere congruent with the way that thecurriculum views the functions of writing. Thecurriculum also outlines general levels at whichwe would expect various functions to appearand in what broad form. Our scoring rubricshad to be consistent with this while taking intoaccount what other local professionaldevelopment literature reported that teachersactually did in their classrooms in this respect.The other major data source was feedbackfrom the teachers recruited to trial the draftrubrics. This was obtained in several ways.During the trial week, there was a cycle inwhich teachers were introduced to each of fiverubrics in turn, then they all marked a set ofscripts. After this, there was a discussion andfeedback session. We also asked for writtenfeedback at the end of the second day, guidedby some broad questions that drew from theobjectives we placed before them on Day One.The objectives were that we wanted to develop

10 Glasswell, K., Parr, J., & Aikman, M.rubrics that provided information useful toclassroom teachers, that linked to ENZC, andthat could be scored reasonably expeditiously(that is, the pay-off would be sufficient for theeffort in scoring). Then, there was a moreformal questionnaire on the final day (seeAppendix 2 attached). There were also datafrom the annotations the teachers made on thescoring sheets as they dealt with individualscripts.Clearly, another important set of data indeveloping these scoring rubrics was the extentto which teachers could reliably use the rubrics.During the development phase, the scoringsheets were eyeballed and reliability wasroughly calculated after the test set for eachrubric was scored. Changes were made to eachrubric in the light of discussion and the extent towhich teachers were scoring consistently.The TeachersThe teachers who took part were simplythose who responded to an advertisement underthe heading “professional development” in theEducation Gazette and agreed to participateduring their summer holidays. The reasoningbehind this method of selection was that therubrics should be able to be used by anyclassroom teacher. However, the sample ofteachers who presented may not have beenrepresentative of New Zealand primaryclassroom teachers. Of the 17 teachers whotook part, three had trained in South Africa; twoin India, and two in the United Kingdom,although all but two of the teachers had hadsome teaching experience in New Zealand.One was a secondary science teacher bytraining and another was primarily a tertiaryteacher. Feedback from non-primary, nonlanguageteachers – some trained in a systemwith a radically different view of languageteaching – is helpful to an extent to refine thedescriptions of the criteria in each rubric.However, if the sample had contained moreteachers like the two or three who were clearlyexpert in literacy teaching, more feedback mayhave been available at a more conceptual level.The teachers ranged in years of experiencefrom a teacher with one year’s experience toseveral teachers with over 30 years’ experience.For such an experienced group, the professionaldevelopment they could recall receiving in thewriting area was limited. Their responses to theprompt in the evaluation questionnaire toprovide details of the ways in which theyusually assess writing in their classroomsindicated that there is a move towards morecriteria-based evaluation of unassisted samplesand the use of exemplars and benchmarking.However, much of writing assessmentinformation came from the “log in the head”obtained through observation, conferencing, orsimple checklists.The aim was to design rubrics that not onlycould be scored reliably and reasonablyexpeditiously, but that would also providediagnostic information as described above.Presumably, such rubrics would also fulfil someprofessional development role in terms ofteachers’ knowledge of teaching writing.Consequently, the rubrics were designed toreflect and, hopefully, construct best practiceaspects of teaching writing, not for scoring bythe lowest common denominator in terms ofteacher skill. In this respect, we faced aconundrum. To oversimplify the scoringrubrics to accommodate the needs of a smallminority of the teachers in the group woulddefeat this purpose. In the future researchersshould take care to ensure that teachers selectedrepresent the target population of primaryteachers currently practising in New Zealand.Feedback from Teachers Concerning RubricsThe development exercise with the teachersyielded data in several forms. The tapeddiscussion sessions following the trial markingof a set of papers resulted in some revisions tothe draft rubrics, largely to clarify wording ofthe criteria or to make the criteria more explicit.The researchers were able to establish what wasan accessible way (“teacher-friendly”, theparticipants termed it) to talk about criteria likegrammatical features. Many practising teachershad, apparently, received no formal training ingrammar and admitted in the evaluation tobeing unfamiliar with the professional languageused to refer to tense, for example.At the outset, we explained that our goalswere to create rubrics that were able to be used

Technical Report 6: Writing Assessment Rubrics 11expeditiously, were relevant to the NewZealand context, were useful in the classroom,and were able to both reflect and construct goodpractice. It was in these terms that we soughtfeedback.Ease of UseIf teachers are to use the rubrics, then theyneed to be able to understand them and also feelthat the pay-off received in terms of informationgained is worth the effort of scoring using therubrics. With respect to understanding therubrics, the group had about an hour ofintroduction to the first rubric and, with eachsucceeding one, it was necessary only tohighlight the important features in a briefintroduction. As the teachers noted at the endof Day Two and the second rubric, they weregetting more skilled at using them. One teachercommented, “With more experience, I havebecome more comfortable.”This process was helped by the fact that eachrubric has a common framework and layout.This does not mean that the task of scoring waseasy. One teacher claimed to be putting ingreater effort than s/he would normally, andothers noted the concentration required, onesaying “they [the rubrics] demanded closeinspection”.There were aspects involved in the use of therubrics that were difficult and the teacher groupendorsed the need for some professionaldevelopment to accompany the asTTle package.The concept of “hot links” on the CD-ROM toprovide help was appealing. In the finalevaluation, nearly 90% strongly agreed that theconcept of hot links would help the assessmentprocess when using the rubrics. Our sense isthat such support would have enormousprofessional development benefits. This wasconfirmed by teacher comments, including onethat stated, “We need a more comprehensiveresource than what is available. Whataccompanies these rubrics would be logical andappropriate”.The professional language used to describefeatures of text or linguistic terms like articlewere an issue for some. Several teachers wereunsure of their syntactical expertise, noting thatgrammar, for example, was never taught duringpre-service training and they did not feelconfident in this area. However, the use of therubrics had a professional development spin-offfor many of the teachers, who talked ofbecoming more familiar with the genres andabout the complexities of language andgrammar use associated with them. Those whowere already quite expert felt that they learnedmore and were also confirmed in many of theirunderstandings about teaching and assessingwriting.In the feedback sessions and in the finalevaluation questionnaire where teachers wereasked what was a difficult aspect of using therubrics, most discussion centred round thedescriptors selected to describe movementwithin a level (between basic, proficient, andadvanced). Early reliability indicators from thescoring of test sets suggested that, while therewas consistency in placing aspects of a piece ofwriting at a particular level, there was lesssuccess in agreeing where the weight ofevidence in terms of the criteria placed thepiece within a level. Experienced languageteachers expressed a preference for familiardescriptors like beginning, developing, andproficient to describe progression within a leveland they said that they thought teachers saw thecontinuum as beginning, middle, and end. Webegan the development exercise with thedescriptors beginning, developing, andproficient. However, given the initial trends inscoring consistency and given general researchconsensus that such terms are imprecise andsubject to interpretation, we switched to themore generally accepted terms – basic,proficient, and advanced. The initial trendsmay have been confounded with lack ofpractice/training, however.In the summative evaluation questionnaire,many of the teachers indicated they were stillunhappy with the nomenclature, blaming it forthe continuing struggle to place within a level.One of the expert language teachers was stillconcerned that there is too big a step from basicto proficient, as the term proficient suggestscapable. This teacher said, “Proficient is asuperlative, not a middle-ground descriptor”.The interval between proficient and advanceddid not seem to the teachers to be as great as

12 Glasswell, K., Parr, J., & Aikman, M.that between basic and proficient. Despite theconcern surrounding the levels or, rather, theintra level placement, there was also a positivespin-off, with one teacher noting that the rubricsgave her “a better sense of levels and what theymeant in reality”.In terms of whether the pay-off is worth theeffort, it seems that this may depend on theteacher’s view of the purpose of the assessment.It is well to note that even after carefulexplanation, it was clear that one teacher, at theend of Day Two, was still confused as to thepurpose of such assessment. This teacherclaimed that an interview was still the preferredway of reporting to parents and the rubricwould not be a suitable way of doing this. Thisteacher viewed the accountability aspect as themain reason for assessment. The most likelyexplanation for this is that the teacher trained ina system quite different to that in New Zealand.Other teachers saw what the value of the rubricscould be and their evaluation wasoverwhelmingly positive.The bottom line is whether the teacherswould use the rubrics. Asked to rate on a scaleof 1 to 5 (where 5 is definitely and 1 is definitelynot), how likely, as a classroom teacher, theywould be to use the rubrics, the mean responsewas 4.9. “I can’t wait to use them in myschool,” wrote one. “They are manageable andtherefore useable,” wrote another. The estimateof the time they would have to expend scoringeach sample varied from 4 to 8 minutes and ourobservation confirms that, once the teacherswere familiar with the rubrics, 4 to 5 minutesper sample was the norm.Relevance to NZ ContextAlthough teachers were not asked directlyabout the relevance to the New Zealand context,it was clear from several responses that theysaw a role for the rubrics. One claimed thatENZC should be more specific in its demandsfor the teaching of what s/he termed “genre”.For two teachers, the rubrics clarified thecurriculum or “what to teach”. Another wrote,“If only this sort of detail was in thecurriculum”. As noted above, for some it madetransparent what the curriculum levels actuallymeant in real terms.After scoring a number of scripts from thetrials, disquiet was expressed about whetherwriting was being taught as well as it should be.“We have to do better,” wrote one. Anothersaid that scoring the scripts “reinforced afeeling that our standard in writing has beenexacerbated by the curriculum document and alack of clear instruction, professionaldevelopment, and national norms”.Usefulness for Classroom TeachersAt the most general level, teachers talked ofthe rubrics moving them away from relying on“gut feeling” or “professional intuition” inassessing writing. They were pleased to see a“more formalised system”. “This [rubric] ismore specific and I appreciate that”. “It defineswhat we are marking.” “It goes a long waytowards removing subjectivity and will provideconsistency across schools”. One teachersummed it up saying, “I can see myself usingthis as a major assessment, diagnostic, andplanning tool”.Teacher evaluation of the utility of therubrics was positive, and teachers saw this interms of the rubrics’ usefulness for diagnosisand planning as well as assessment. One ofways teachers suggested they might use therubrics was to give a profile of “where the childwas at” and a profile of his/her “strengths andweaknesses”. The rubrics were seen as giving“excellent detail about a child’s current level ofexpertise”. The teachers appreciated that theywould get a profile – information that wouldshow that a child may not be on the same levelin each of the seven areas specified in the rubriccategories.The teachers were asked to rate thehelpfulness of the rubrics for diagnosing wherea writer is at in each of the different functionsor purposes of writing. Using a 5-point scalewhere 5 is very helpful and 1 is not at allhelpful, they rated the rubrics’ helpfulness inthis regard as 4.5. This profile may also bedifferent for the five different functions ofwriting and teachers found the rubrics helpfulfor providing information about this (mean =4.8, where 5 = very helpful).In addition, teachers noted that the use of therubrics had helped them to see the different

Technical Report 6: Writing Assessment Rubrics 13aspects that should be assessed: “It showed meareas to look for that I had not previouslyconsidered”. “It helped me focus on what it isthat we are assessing”. The rubrics would alsoserve the function one teacher noted of “helpingus to teach all [genres], not just those we arefamiliar with”.In terms of teaching writing, there werenumerous comments about the utility of therubrics. One aspect was that the rubrics wouldhelp a teacher to be more focused and clearer inplanning objectives – “Planning can becomemore directed”. “I will know where to placeemphasis”. Another teacher wrote, “I realisenow the importance of teaching the detail of thedifferent genres, and I see the key elements thatneed to be identified and explained to thestudents”. And another, in similar vein, wrote,“It [the set of rubrics] fills an important gap,namely, detail about where specifically we wantto take the child’s writing”.The detailed nature of the information wouldbe useful, several teachers thought, for groupingchildren for focused instruction. Anothercommented that s/he “could see now thedevelopmental process that a child would movethrough to gain control of a genre”. Asked howhelpful the rubrics were in terms of informationon which to base teaching the writer about eachof the different functions or purposes forwriting, the teachers gave a mean rating of 4.6(again on a 1 to 5 scale where 5 was veryhelpful).Similarly, for helpfulness in providinginformation on which to base teaching about thedifferent aspects within a particular purpose orfunction of writing, teachers gave a mean ratingof 4.5. Some also saw the scoring from therubrics as providing a means of reflecting onteaching. “It [the outcome of the scoring]would alert me to realise how I, as a teacher,had taught that unit, from the child’sresponses”.This trial was important as it demonstratedseveral points with respect to producingcarefully designed, but flexible, diagnosticassessment tools. The first is that, in designingsuch tools, researchers have to be aware of thecontext, particularly the curriculum context andthe skills and practices of the teachers who willuse them. Second was the, not unexpected,finding from this exercise that significantprofessional growth can result from the use oftools such as the scoring rubrics.Although the development and trial sessionswere not designed as a professionaldevelopment exercise, teachers gave unsolicitedfeedback that it had been a “rewardingexperience in terms of learning about teachingand assessing writing”. Some of theprofessional development they reported took theform of acquiring a shared vocabulary to talkabout what they did as practitioners or what wascharacteristic of the text types associated withcertain functions of writing. Another valuablepiece of knowledge for some came from aclarification of function or process as opposedto text form and of the relationship between thetwo. This gave them a new way of viewingteaching writing, congruent with the curriculumand with the beliefs they held about reading ashighly contextualised.Support for Teachers when Using RubricsWhile the team has never doubted thecommitment of New Zealand teachers toliteracy, we have been aware that their expertisein writing does not match their expertise inreading. The ENZC is a purposely nonprescriptive,open document. The teachers seemuch of the official supporting material asvague. Teachers lack a shared language fortalking about writing. Feedback, from those inour sample who were employed primaryteachers, confirms that they feel far lessconfident about their ability to teach writingthan about their ability to teach reading. Manyof the terms common in the professionalliterature were foreign to them, suggesting alack of pre- and inservice development in thisarea.We recommend that the rubric for eachfunction of writing be accompanied by adescription highlighting the major structural andlinguistic features of the process. In addition,while working with the teachers, we developeda “Tips for Scoring” sheet (see Appendix 3). Asimilar sheet should accompany the rubrics.In the course of developing the rubrics, wehave identified areas where it would be useful

14 Glasswell, K., Parr, J., & Aikman, M.on the CD-ROM to have hot links that a scorercan invoke to clarify a term or to see anexample. We think that the scoring rubricsshould be accompanied by detailed exemplars.We feel that a very useful addition to theCD-ROM would be an instructional video thatwould include the type of introduction that theteam, and Kath Glasswell, in particular, gavefor each rubric. A productive format might beto have regional inservice sessions for literacyleaders, then an instructional video to assist theresource people when they return to theirschools. Alternatively, the video could serve asa resource for teachers to teach themselvesmore so they can gain maximum benefit fromthe scoring of their class samples.The Writing Tasks“The rubrics make more sense than thetasks” was a common sentiment from the group.The teachers were not necessarily aware of theconstraints under which the initial trial taskswere written. In our view, for the rubrics towork well as assessment tools, there needs to beconsistency between the extended writing tasksand the rubrics. Several issues need to be bornein mind when constructing further tasks. Theseissues concern the context of the task, includingthe accompanying instructions, as well as thenecessity to provide a communicative context inwhich to embed the task and also the actualcontent of the tasks themselves.With respect to the context of instructions, itwas apparent from a close reading of numerousbatches of samples from schools thatadministration was not standardised. Someteachers (or adults) appear to have scribed forstudents unable to write, while others appear tohave developed a planning web with thechildren or have given additional instructionslike “write your views in a letter to theprincipal”. In order to achieve some degree ofstandardisation, instructions need to be madeexplicit and given verbatim.Perhaps the most striking omission in thedevelopment of the extended writingassessment tasks, for a teaching profession thatworks top-down and is concerned with context,is the failure of the tasks to specify the functionand audience for the piece of writing. Writingalways has a social function and New Zealandteachers have tried to move away from writingfor the teacher as audience, yet this is what thetasks do. Each task should say for whom it is tobe written or where it will be published. It isimportant to state who will see it and read it, asthe writing is crafted with this in mind. A set ofinstructions for a young child may look verydifferent from a set of instructions devised for amore expert person. The task that required anargument, the changing school hours, did notstipulate an audience, so some studentsaddressed the principal, others seemed to bewriting for their fellow students, and othersmoved between audiences, usuallyunsuccessfully.In addition to specifying the purpose andaudience, the issue of instructions thataccompany the task needs considerable andcareful thought. The invitation to illustrate inthe muffin task meant that some children didnot write much, if anything, or spent valuabletime drawing, leaving little text to score. If weintend to score literacy that represents a “turn tothe visual” (after Kress, 1999), then we need todevelop a specific task and scoring rubric.Also, there is a need to rethink the hints to makesure that they dovetail with the scoring rubric.Further, each task should call for only onefunction for writing. Although genre blurringmay happen in real life writing and may bedeliberately used as a device by authors, fordiagnostic assessment scoring purposes, thetask should include only one process.With regard to content, some of the topicsset may cause undue bias. Although there willalways be differential knowledge in terms ofcontent to write about (and content knowledgeis an important source of variance in writingscores), items should be constructed to avoidcultural bias. Migrant ESOL learners areunlikely to have the English vocabulary to talkabout making muffins. Instead of instructionsabout how to cook muffins (not an item in manymigrant cuisines), perhaps the task could be tocook a dish for a special occasion. The taskcould be to describe this preparation for a newfriend who has tasted it at your place and wantsto know how to make it. Similarly, the topicconcerned with the time machine caused

Technical Report 6: Writing Assessment Rubrics 15problems for those not exposed to Dr Who orBack to the Future! Some children seemed tohave no concept of what a time machine was.Others either did not know or could not workout how to read the word from the context andwrote about a typewriter machine!Tasks such as the recounting task need to bequite specific, because being open-ended, as inthis case, increases the difficulty level of thetask, which calls for selection of significantcontent. A writing task about a school camp orholiday needs to specifically refer to animportant or interesting event in order to avoida dawn to dusk recount of a week’s camp or afortnight’s holiday.When scoring, standard decisions need to bemade about what is an acceptable length of textin order to make a judgement. Rather thanplace a two-word piece at below level 2, shouldit be declared “unable to be scored”? Therealso arose the issue of what to do with creativeresponses, particularly to the narrative task.Some of these could be considered off task interms of the topic set, but they may have beenquite successful examples of writing to informor entertain through narrative (and may haveprovided the teacher with valuable diagnosticinformation). We decided to score these andincluded a new line under content inclusion thatsimply asked whether the writing was on topicor not (this was partly because of the nature ofsome of the tasks where it seemed unfair topenalise students for unclear or biased tasks).SummaryOverall, the teachers in the trial werepositive about the writing assessment rubrics.Once the teachers were more familiar withusing the rubrics, they seemed to find them easyto use, although some teachers expressed apreference for the proficiency level terms thatthey currently used over those in the rubrics. Inspite of these reservations, teachers indicatedthat they were highly likely to use the rubrics intheir classrooms when they became available.The teachers in the trial felt that they wouldbenefit from the specific nature of the rubrics;some felt that the detail given in the rubricsadded to their understanding of the writingcurriculum and what they should be teaching aswell as assessing in writing. Teachers indicatedthat they would use the rubrics for diagnosis ofindividual children’s strengths and weaknessesand for planning for the class, as well as forassessment. They generally saw the rubrics asbeing very helpful for diagnosis and felt thattheir planning would be more focused.The teachers in the study also felt that therubrics were very helpful in informing themabout the functions and purposes of writing,which they indicated they would feel moreconfident about teaching to their students.Although the study was not intended to be aprofessional development exercise, it did havethis effect for many of the teachers, in terms ofthe “shared vocabulary” they now had and whatthey had learnt about function or process asopposed to text form.More importantly, the study has provided aset of rubrics for the evaluation of writingconsistent with the New Zealand Englishcurriculum levels framework and internationalresearch on writing. The rubrics can be usedwith confidence as valid, manageable, andreliable means of establishing children’sprogress towards desired levels of writingachievement.ReferencesBerkenkotter, C. (1993). A rhetoric fornaturalistic enquiry and the question ofgenre. Research in the Teaching of English,27, 293–303.Britton, J., Burgess, A., Martin, N., McLeod,A., & Rosen, H. (1975). The development ofwriting abilities. London: MacMillanEducation.Bruner, J. (1983). Child’s talk. New York:Norton.Chapman, M. (1999). Situated, social, active:Rewriting genre in the elementaryclassroom. Written Communication, 16,469–490.Clay, M.M. (1998). By different paths tocommon outcomes. York, MN: Stenhouse.Derewianka, B. (1990). Exploring how textswork. Sydney, Australia: Primary TeachersAssociation.Freedman, A., & Medway, P. (1994). Locatinggenre studies: Antecedents and prospects. In

16 Glasswell, K., Parr, J., & Aikman, M.A. Freedman & P. Medway (Eds.), Genreand the new rhetoric (pp. 1–22). London:Taylor & Francis.Gerot, L., & Wignall, P. (1994). Making senseof functional grammar. Cammeray, NSW:Antipodean Educational Enterprises.Knapp, P., & Watkins, M. (1994). Context,text, grammar: Teaching the genres andgrammar of school writing in infant andprimary classrooms. Sydney, Australia:Text Productions.Glasswell, K. (1999). The patterning ofdifference: Teachers and childrenconstructing development in writing.Unpublished doctoral dissertation. TheUniversity of Auckland.Limbrick, E., Keenan, J., & Girven, A. (2000).Mapping the English Curriculum.Unpublished Technical Report. ProjectasTTle. University of Auckland.Martin, J. R., Christie, F., & Rothery, J. (1987).Social processes in education. In I. Reid(Ed.), The place of genre in learning:Current debates. Geelong, Australia:Deakin University Press.McNaughton, S. (1995). Patterns of emergentliteracy: Processes of development andtransition. Melbourne: Oxford UniversityPress.Ministry of Education (1994). English in theNew Zealand Curriculum. Wellington, NewZealand: Learning Media.Ministry of Education (1997). Planning andAssessment in English. Wellington, NewZealand: Learning Media.Nold, E. W. (1981). Revising. In C. H.Frederiksen & J. F. Dominic (Eds.), Writing:The nature, development and teaching ofwritten composition (pp. 67–79). Hillsdale,NJ: Lawrence Erlbaum Associates.Northwest Regional Educational Laboratory(1998). Six-Trait Analytic WritingAssessment Model. Portland OR: NorthwestRegional Educational Laboratory.Parr, J. M. (1989). Revision in writing:Cognitive and linguistic aspects.Unpublished doctoral dissertation. TheAustralian National UniversityWhite, E. (1994). Issues and problems inwriting assessment. Assessing Writing, 1,11–27.Wing Jan, L. (1991). Write ways: Modellingwriting forms. Melbourne, Australia: OxfordUniversity Press.Wray, D., & Lewis, M. (1999). From learningto teaching: Towards a model of teachingliteracy. Unpublished Report, England(ERIC Document Service No. ED 432 777).

Appendix Table 1asTTle Writing Scoring Rubric: to explainAudienceawarenessand purposeContentinclusionCoherence:sequencingideas andlinkingThe writer recognises that anexplanation is required and thathe/she is writing for an audienceother than the self. Assumedshared knowledge with the readermay interfere with meaning.Appendix 1 – Scoring RubricsRhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P ALanguage and writing style ofexplanation is appropriate to theaudience. May rely on context andrequire some reader inference tounderstand explanation.The writer makes an attempt toidentify the phenomenon orprocess and gives two or moresimple reasons for its occurrence.May include statements that aretangential to the topic and/or taskor include a personal perspectiveto the explanation (e.g., “I likerocks.” “I saw a tuatara at the zooin Auckland”).Language use and writing styleappropriate to purpose anddirected to the reader/audience(e.g., evidence that needs of thereader are being considered).Explanation is clear and can “standalone.Text: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AThe writer identifies thephenomenon or process clearly.Body of text contains furtherelaboration and gives associatedreasons for why or how it occurs.Limited tangential informationevident.Generally organised at sentencelevel. May attempt to show causeand-effectrelationships in theexplanation by using withinsentencelinks (e.g., because, so).Evidence of attempts at structuringcontent through the grouping ofideas within and across sentences.May be attempting to associateideas and reasons by usingdevices such as paragraphing andbetween-paragraph links.The writer identifies thephenomenon or process clearly inan introduction that may also givecontextualising information. Bodyof text contains a sequencedaccount of elaborated aspects orprocesses, and gives detailedassociated reasons for why or howit occurs. Includes only relevantcontent.Attempts at grouping orsequencing of explanation evident(e.g., descriptions and reasons aregrouped thematically). Across thetext there is a sense of an attemptto sequence content (usingconjunctions and expressingcausal relationships through links)to give a full and explicitexplanation sequence.(continues…)Technical Report 6: Writing Assessment Rubrics 17

Languageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P ASimple factual statementsEvidence of use of taskappropriateConsistent use of appropriate(declarative) evident. Topicrelatedlanguage (e.g., actionlanguage (for task and topic)vocabulary present but littleverbs mainly present tense to tellenhances the clarity anddetail conveyed through thehow it is or happens). Topicrelatedcoherence of the explanation (e.g.,language (e.g., nouns may lackvocabulary contributes totechnical language is includedadjectivals). Action verbs choicesunderstanding of parts or aspectswhere appropriate). Conjunctionsreflect topic and task. Topicrelatedof phenomenon to be explained.used to link ideas within andvocabulary used. ShowsSome use of adjectivals and/oracross sentences. Reference linkssome understanding of pronounadverbials to give detail andclear.use (related to cohesion) butprecision to explanation. Someunclear or overused pronoununclear or repetitious reference.reference may interfere withmeaning.Appendix Table 2asTTle Writing Scoring Rubric: to argue or persuadeAudienceawarenessand purposeContentinclusionRhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P AEvidence that writer recognisesLanguage use and writing styleLanguage use and writing stylethat his/her opinion is needed.generally appropriate to audience.appropriate and directed toUses language to state opinionsWriter states his/her position.audience (e.g., writing attempts tofrom a personal perspective.Some attempt to influence thepersuade reader). Clearly statedreader is evident.consistent position is evident.Text: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AMakes two or more argumentstatements related to the topic.Content can be tangentially relatedto the topic.Position statement present. Mostother argument elements (e.g.,main points, evidence, restatement)are included. Someelaboration. May includeinformation that does notcontribute to argument.Content relevant and adds weightto argument. Argument elements(i.e., position statement, mainpoints, evidence, re-statement) arecomprehensive. Containselaboration of main points.(continues…)18 Glasswell, K., Parr, J., & Aikman, M.

Coherence:sequencingideas andlinkingLanguageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AEvidence of attempts at structuringContent managed effectivelycontent through grouping ideasthrough grouping and/orwithin and across sentences – mayparagraphing main ideas anduse devices such as paragraphingvaried use of linking words andand simple linking words (e.g.,phrases (e.g., on the one hand,because, and).however, although).Organised at sentence level (e.g.,limited because of haphazard orstream of consciousness-typeorganisation).Topic-related vocabulary present.Often speech-like in structure anduses a personal voice (e.g., “Ireckon”).Appendix Table 3asTTle Writing Scoring Rubric: to instruct or lay out a procedureAudienceawarenessand purposeContentinclusionUses topic-appropriate vocabulary.Attempts to use language to makearguments seem more objective(e.g., passive structures) andpowerful (e.g., emotive language).May attempt to use persuasivelanguage (e.g., emotivevocabulary) to influence readers orincludes or refers to the reader(e.g., "you would"). Useslanguage to make argumentsseem more objective (e.g., passivestructures) and/or powerful (e.g.,certainly, must, absolutely).Rhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P AThe writer recognises the purposeLanguage use and writing style isInterprets needs of audience.for writing and that he/she isappropriate to audience. Relies onLanguage use and writing stylewriting for an audience other thancontext and requires some readerdirected to audience. Requiresthe self (e.g., uses you orinference to complete thelittle reader inference to completeimperative form). May assumeprocedure.procedure.shared knowledge with the reader.Text: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AAll basic procedure elements (e.g.,Procedure elements (i.e.,headings, actions, materials)headings, sub-headings, materials,included. Some elaboration ofactions) sufficiently elaborate,elements. Limited tangentialprecise, and comprehensive.information.Includes only appropriate content.(continues…)Some elements of procedure (e.g.,headings, actions, materials)included. Some topic-relatedinformation included. Evidence ofinstruction-like statements.Technical Report 6: Writing Assessment Rubrics 19

Coherence:sequencingideas andlinkingLanguageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P ASemblance of order to procedure.Some grouping or sequencing ofElements of procedure grouped orMay use a simple ordering deviceprocedure elements evident.sequenced appropriately.(e.g., numbers).Some use of ordering devices butEffective use of ordering devices.limited in number and/or scope(e.g., repetitive use of then).Simple, unelaborated statementsevident. Some command-likestatements present. Actionsrecounted from a personalperspective.Evidence of use of taskappropriatelanguage (e.g., use ofdescriptors – action verbs,adverbs, adjectives to describematerials and actions). Commandlikestatements predominate.Attempts to use generalised other(e.g., second or third person).Appendix Table 4asTTle Writing Scoring Rubric: to classify, organise, describe, and report informationAudienceawarenessand purposeContentinclusionEvidence that the writer recognisesthe purpose for writing. Givesinformation from a writer’sperspective (e.g., may requirereader to infer or select informationto make sense of complete text).Consistent use of task-appropriatelanguage (i.e., precise and varieduse of descriptors – action verbs,adverbs, adjectives) to clarifyprocedure. Consistently refers toreader in generalised way (usingone, or you). May adjust languageto both instruct and advise.Rhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P ALanguage use and writing stylegenerally appropriate to audienceand purpose. Informs but mayrequire some reader inference.Evidence of statements of fact.Writing includes some factsrelevant to the topic and task,covering, for example, some (2 ormore) task-appropriate domains:attributes, behaviours, properties,functions, location, etc. Caninclude many statementstangential to the topic and/or task.Interprets needs of audience.Language use and writing styledirected to audience andappropriate to purpose. Informs(i.e., is comprehensive and explicitenough to require little or noreader inference).Text: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AMost domain elements appropriateDomain elements areto the task present (e.g., the writercomprehensive and elaboratedclassifies and deals with attributes,(i.e., the writer classifies, dealsbehaviours, properties, functions,with attributes, behaviours,location, etc.). May include someproperties, functions, location,material tangential to the topic andetc.), given task. Almost allthe task.material related to topic and task.(continues…)20 Glasswell, K., Parr, J., & Aikman, M.

Coherence:sequencingideas andlinkingLanguageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AEvidence that the writer is using aLogical, effective, and obviousframework for ordering contentframework for ordering description(e.g., categorising or classifying).(e.g., categorisation orMay not be consistently orclassification). Elementsoptimally ordered, and elementsappropriately assigned. Thematicmay be inappropriately assigned tolinking of sentences to topic ofparts of framework.paragraph or section.Semblance of framework (e.g.,some grouping of information).For example, text is limitedbecause of presentation of factstatements as discrete elements ormixture of text types.Simple factual descriptionsevident. Topic-related vocabularypresent but little detail conveyedthrough language (e.g., nouns maylack adjectivals). Shows someunderstanding of the use ofpronouns but pronominal reference(the who or what) may be unclearor overused.Evidence of use of taskappropriatelanguage (e.g., relatingverbs – to be or to have – forclassifying; action verbs fordescribing behaviours or uses,most often present tense; someuse of adverbs and adjectives).Some pronominal referenceunclear; some repetitiousreference.Appendix Table 5asTTle Writing Scoring Rubric: to inform or entertain through imaginative narrativeAudienceawarenessand purposeContentinclusionConsistent use of appropriatelanguage for task and topic (e.g.,relating verbs – to be or to have –for classifying; action verbs fordescribing behaviours or uses,most often present tense; addinginformation to the noun-noun"packing"). Reference links clear.Language of comparison isevident.Rhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P AEvidence that the writer recognisesEvidence of attempts to captureEngages audience and sustainsthe purpose for writing (i.e., to tellthe reader’s interest. Languagereader attention. Language usea story) and that he/she is writinguse and writing style appropriate toand writing style enhance thefor an audience other than the self.telling a story. Attempts to adopt atelling. The writer’s “voice” mayperspective to tell the story.enter the text and invoke areaction.Text: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AWriting includes importantStory includes comprehensiveelements of story (i.e., haselements (i.e., orientation,essentials of characters, settings,complication, resolution, andand events. Evidence of inclusionsometimes coda). Clear focus onof problem or complication. Mayand development of specificattempt to conclude events.events, characters, and settings.(continues…)Some attempt at a story. Writing isa series of loosely relatedsentences or a series of sentencesthat all describe a single event.Technical Report 6: Writing Assessment Rubrics 21

Coherence:sequencingideas andlinkingLanguageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P ASemblance of order evident butSome sequencing of storyStory element sequencinglimited because of haphazard orelements evident. The story ismanaged well (e.g., effective plotstream of consciousness-typeorganised around happenings andor development of events).organisation.has a point. Ideas and events mayEffective linking is evident throughbe linked through the use ofthe use of some linking devicesdevices such as paragraphing or(e.g., conjunctions of time –linking words and/or phrases (e.g.,afterwards, next, meanwhile),“Later that evening…”, because).which make the story flow.Language is simple. Actionsrecounted with little elaboration,and, overall, style lacks variety ormay be limited for topic (e.g.,pedestrian use of descriptors –adverbials, adjectives – such asnice or nicely). May insert directspeech but context lacks clarity.Appendix Table 6asTTle Writing Scoring Rubric: to recountAudienceawarenessand purposeEvidence of attempts to addinterest and detail through the useof descriptors (e.g., adverbials,adjectives). May attempt to usedialogue to add to story.May use language devices (e.g.,figurative language) anddescriptors (e.g., adverbials,adjectives) to engage the audienceand give detail to and developcharacters, actions, and settings.Purposeful use of dialogue (whereincluded).Rhetorical: the writer and the contextLevel 2 B P A Level 3 B P A Level 4 B P AThe writer recounts to tell of a pastLanguage use and writing styleLanguage use and writing styleexperience or happening.appropriate to recounting a pastenhance the recounting. The textRecognises that he/she is writinghappening. Audience hasis complete for audiencefor an audience other than the self,sufficient information to makeunderstanding. Engages audiencebut may be limited by assumptionsense of the experienceand sustains attention. Theof shared knowledge.recounted. Recount showswriter’s “voice” may be evident.evidence of attempts to capturethe audience’s interest.(continues…)22 Glasswell, K., Parr, J., & Aikman, M.

ContentinclusionCoherence:sequencingideas andlinkingLanguageresources forachieving thepurposeText: content and structureLevel 2 B P A Level 3 B P A Level 4 B P AWriting includes, in addition toOrientation is comprehensive, yetwhere, when, who, what, and whysuccinct. Clear focus on andetc., evidence of foregrounding ofdevelopment of specific events ofsignificant content. Evidence ofinterest. Recount is enriched withattempts to add detail to, commentinterpretive comments, evaluation,on, or evaluate selected points ofand observation. Writing is movinginterest. There may be an attemptto a satisfactory conclusion.to conclude.Writer recounts events. Writingbegins with an orientation(background information) usingsome of the elements ofrecounting (i.e., when, where, who,what, and why). May be someevidence of selection of events forinclusion or of comment on events.May include content that is notrelevant.Events are largely sequenced intime order. Events are linked byusing words that indicate thepassage of time (e.g., then, later,next).Language is simple. Some actionverbs used but limited in scope(e.g., “I went”, “I got”). Usessimple past tense. Events andactions recounted with littleelaboration (may be limited byrepetitive use of sentence structureand/or language to indicatepassage of time). May insert directspeech.Events are in time order and seemto follow on. Events are linked in avariety of ways (e.g., by using awider variety in devices thatindicate passage of time).Evidence of attempts to add detailto content through using a varietyof verbs, adverbials, andadjectivals. Pronoun use isappropriate and consistent. Usesappropriate verb tense for contentinclusion. Variety in sentencestructure. May include dialogue toassist reconstruction of events.Events are in time order, andsequencing is managed well. Thedetailed sequence of events isinterwoven with evaluativecomment and/or observation.Language devices may be used(e.g. figurative language,metaphors) to amplify content.Varied use of verbs to describeactions and events and to capturethoughts and feelings. Appropriatetenses used throughout. Whereused, dialogue enhances thetelling.Technical Report 6: Writing Assessment Rubrics 23

Appendix Table 7asTTle Writing Scoring Rubric: Conventions: sentences and wordsGrammaticalconventionsSpellingPunctuationConventions: sentences and wordsLevel 2 B P A Level 3 B P A Level 4 B P AMany simple sentences correct butMost sentences correct. Control offew complex sentences evident.complex sentences evident (whereSome errors in sentences (e.g.,appropriate). Uses completesubject and finite verb not presentsentences with subject-verbor do not agree, splitagreement and appropriate wordinappropriately infinitive, wordorder. Maintains consistent tense.order). Tenses may varyunintentionally.Sentence-like structure evident,but errors interfere with meaning(e.g., uses participle phrase; mayhave errors in agreement, tense,and/or word order).Many HFW* (Lists 1–4) speltcorrectly. Evidence of someknowledge of common spellingpatterns and approximate phoneticspelling.Some of the basic punctuationused correctly (e.g., end points (! .?), capitals for sentencebeginnings and proper nouns,commas in lists, speech marks).May use apostrophe forcontraction.Most HFW* (Lists 1–6) andcommon spelling patterns correct.Spelling shows someunderstanding of more complexspelling patterns.Basic punctuation is mostlycorrect. May be attempting morecomplex punctuation (e.g.,apostrophe for ownership,commas for parentheses andclauses, semi-colons and colons).Note: *HFW (High Frequency Words) taken from NZCER Essential Word Lists 1–7.Few errors. Familiarity with HFW*(Lists 1–7) and common spellingpatterns evident. Evidence ofability to spell multi-syllabic,irregular, or technical words.Basic punctuation correct.Evidence of correct use of someexamples of complex punctuation.24 Glasswell, K., Parr, J., & Aikman, M.

Technical Report 6: Writing Assessment Rubrics 25Appendix 2 – Teacher Summative Evaluation QuestionnaireasTTle Writing Rubric Development QuestionnairePlease help us by answering the following questions. Be assured that your responses areconfidential.Professional background1. Qualifications (type, when gained and where from eg. Dip Tchg, AucklandCollege of Education, 1988)2. Years of teaching-related experience (please specify where and age rangestudents)3. Details about the ways in which you usually assess writing in your classroom.(eg. portfolio to showcase etc.)4. Although this week was not designed as a professional developmentexercise, we are interested to know what you have “taken” from theexperience.• About assessing writing?• About teaching writing?• More generally about writing?5. As a classroom teacher, how likely would you be to use the asTTle concept (atask bank to select from; tools to assess; individual and group norms available) inthe writing area?No, definitely not, not likely, probably, very likely, definitely (please circle)6. The rubrics have sufficient detail for scoringStrongly disagree ,mostly disagree, slightly agree, moderately agree, mostly agree, strongly agree7. The concept of “hot links” would help the assessment process when using therubricsStrongly disagree, mostly disagree, slightly agree, moderately agree, mostlyagree, strongly agree

26 Glasswell, K., Parr, J., & Aikman, M.8. How helpful are the rubrics for:• Diagnosing where a writer is at in each of the different functions or purposes ofwriting (eg. to argue, to instruct etc).Very helpful, helpful, of some help useful, not very helpful, not at all helpful• Information on which to base teaching the writer about each of the differentfunctions or purposes of writing.Very helpful, helpful, of some help useful, not very helpful, not at all helpful• Diagnosing where a writer is at within a “genre” (like to argue or persuade) interms of the different aspects (audience awareness, coherence, languageresources etc)Very helpful, helpful, of some help useful, not very helpful, not at all helpful9. Information on which to base teaching about the different aspects (audienceawareness etc) within a “genre”.Very helpful, helpful, of some help useful, not very helpful, not at all helpful10. A difficult thing about using the rubrics was:11. The good thing about using the rubric was:12. Comments on the tasks as they are currently framed.13. Comments on the concept of basic, proficient, advanced.

Technical Report 6: Writing Assessment Rubrics 27Appendix 3 – Tips on ScoringAsTTle Writing Rubrics Design WorkshopsTips on ScoringGet a handle on the WHOLE RUBRIC (see what is being scored where)Read the child’s whole text. Consider options for invented spellings-focus on context and meaning.Know what the child is saying before you begin to score.Think FUNCTION (What is the purpose here?)Think TASK (What has this child been asked to do?)Read CHARACTERISTICS carefully in the light of the child’s response.Select the characteristic at the LEVEL 2/3/4 that best suits this sample. This is your workinghypothesis! Check the sample carefully for evidence to support your judgement. For example, if thecharacteristic is about sentences, isolate sentences for examination. Check, which sentences aresimple and which are complex (appropriate to the task). Is there evidence to support your workinghypothesis? Look for disconfirming evidence. Re-check your Level 2/3/4 judgement.Selecting B P A within a levelThe inclusions on the scoring rubrics are common characteristics for children at the levels. Forexample a skilled argument writer will limit his/her statements to those that are relevant but aseven-year-old child who is learning to argue will often include tangential information. This doesnot mean that they “lack” ability to argue but that they are learning to control that function of writing.The criteria are relevant to the LEVEL of the curriculum. We would expect to see the embryonicform of argument in early curriculum levels and have written the criteria accordingly.Basic (criteria at this level/category) Showing signs of these elements. Elements are evident inembryonic form.Proficient (at this level/category) There is evidence that the child is controlling the criteria elements.Advanced (at this level/category) Child is consistently meeting the criteria at this level. Littledisconfirming evidence is found.Tick the box that BEST FITSSeductive details• In each question- Think task: A child may have written well and at length but did not answer thequestion and has not achieved the purpose as outlined in the task.• May have lots of content but it is incidental rather than central to the argument.• Interference from surface features (e.g. spelling, handwriting, punctuation).• Knowing the writer or making inferences about the writer from information given.• Distraction by interesting language use that does not contribute to the task .Goals for the Writing Rubric designRelevant in the NZ contextUseful in the classroomExpeditious (in terms of teacher time in training and use)Reflective of current research, theory and models of good practiceChasing precision in criteria

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!