Assessment: Setting and Marking Assessment Tasks

Does the task provide opportunities forstudents to develop self-evaluation skills? Does the task link in any way with workforcerequirements in the particular discipline? Is the task significantly different from theassessment in the previous year? Does the assessment require evidence ofindividual engagement with the task? Does the task encourage or require students todemonstrate process competencies as well asproduce a finished product? Have in-class opportunities been provided todiscuss and practise the assessmentrequirements? Does the assessment task provide for thelearning of material and /or the developmentof skills as well as measuring studentcompetency? Does the assessment link with the otherassessment tasks? Does the task enable students to make use ofearlier formative feedback? Are the teaching approaches in line with theassessment requirements? If this is a final summative assessment, havethe students had ample formative assessmentopportunities to develop the requiredcompetencies and understanding?• 3 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

Writing clear taskinstructions andestablishing dialoguearound the assessmentprocessOne of the most common concernsexpressed by students is that they do notunderstand what a particular assessment taskrequires. Correspondingly, students often begina task with a very crude sense of what is required.Sometimes, a vague and muddled piece of workis directly related to a lack of clarity in the taskinstructions. There used to be a view that wasvery prevalent among academics that a part of the“test” for students was to unravel the mystery ofan assessment task. This view is part of a broaderperception of assessment as a test, a notion whichalso incorporates the notion of the assessor seeingassessment as an instrument of power andcontrol. However, if we focus on the learningopportunities offered by assessment and seeassessment as part of the teaching and learningexperience, it is vital that students fullyunderstand requirements and that the assessmenttask is framed by as much conversation andpreparation as possible. Furthermore, if anothergoal of assessment is to prepare students forlifelong learning, then it is important to invitethem to actively participate in the assessmentprocess at all stages.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 4 •

► Where possible, set tasks that invite studentsto contribute examples from their owncultures.► Where possible design an assignment in anumber of smaller stages so that you canquickly gauge students’ understanding, pickup problems and provide instructionalformative feedback. In this way, students candevelop their learning, as opposed to findingout that they have completely missed the boatwhen a piece of assessment is returned.► Experiment with getting students to write foror present in a range of contexts and todifferent audiences (see handbook on types ofassessment tasks).► Provide classroom time forpre-assignment discussion.► Give students opportunitiesto practise using taskmarking criteria or invitestudents’ active participationin the development of taskcriteria.► Show students previousexamples of work on similartasks.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 6 •

Marking criteriaDialogue around assessment criteria and ensuringgeneral understanding of what they signify is oneof the most significant parts of a successfulassessment regime and one that promotes optimalstudent learning. Getting common understandingof the assessment criteria BEFORE studentsundertake the task has numerous benefits:▪ Students and lecturers are likely to have abetter shared understanding of the assessmenttask requirements.▪ Students have tools to self-evaluate as theyprepare their assessment and developingstudent self-assessment competencies is animportant learning goal.▪ Very well understood and articulated criteriaenable teachers to mark with greater precisionand consistency.▪ Mutually agreed on criteria provide anappropriate basis for the provision offormative feedback and post-completion taskdiscussion.▪ Mutually understood criteria provide a basisfor discussion should students want toquestion any aspect of the feedback or marksthat have been allocated to a task▪ Dialogue around criteria helps to make theassessment processes more transparent anddiminishes the power differential betweenlecturer and students. The approach is inkeeping with a constructivist approach toteaching and learning.• 7 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

Useful termsIt is important to remember that when weevaluate work in relation to criteria we areworking within a criteria-referenced system.However, many academics and mangers are stillinfluenced by the models and thinking associatedwith a norm-referenced system in which studentsare ranked in relation to the group of students as awhole. As a consequence, people sometimesmark in relation to criteria and are then worried ifthe sample does not conform to a bell curve.Sometimes managers expect this kind of spread,but it is entirely possible to get an unevendistribution of marks ifworking within a criteriareferencedsystem. Moon(2002) provides helpfuldefinitions for the criteriareferencedsystem:Criteria-referencedassessmentA student’s learningperformance is measured in relation to a set ofclearly defined criteria which have been designedin advance of the assessment and which areexplicitly linked to course learning outcomes.Threshold Assessment CriteriaThese criteria designate the minimum level of thelearning outcomes needed to ensure a pass.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 8 •

Grading Assessment CriteriaAssessment criteria must be directly linked to thelearning outcomes. Moon (2002) suggests that thelearning outcome indicates the base level of therequired performance, while grading criteriaindicate “how well students achieve above thethreshold”(p.90).…Grading criteria Grading criteria indicate the standard orindicate the level quality of the student’s performance of aof learning above particular outcome. A dilemma for teachersthe base level of is to find the balance between giving clearguidance as to what is expected and beingthe learningtoo prescriptive and thereby encouraging aoutcomes that a mechanistic approach. Criteria can bestudent has holistic or analytical. Holistic criteriaattained. indicate the standard of performance inrelation to key learning outcomes that areMoon (2002), p.90required in order to attain different grades. Itis usually easiest to begin with the optimumand lowest levels of performance when workingout criteria and then work out the grades inbetween.Analytical criteria breaks a task up into itscomponent parts or domains. A percentage of thetotal mark is allocated to each domain.While analytical criteria can be precise and clear,they can easily encourage a mechanistic approachfrom students and lecturers and may get in the• 9 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

way of an appreciation of the work as anintegrated whole.Examples of Assessment CriteriaHolistic CriteriaTask:In general terms the assignment requires studentsto investigate and report (in 2000 words), theextent of human rights protection for a particularinterest group and the desirability of a bill of rightsfrom the point of view of the interest group.CriteriaPass: The report demonstrates a soundunderstanding of human rights and of itsapplication in the context of a particular interestgroup. It reflects an awareness (where appropriate)of gender and cross-cultural issues as they affecthuman rights in law, and it shows evidence of acapacity to examine critically a range of legal andnon-legal issues in human rights. The argument inthe report is generally logical and objective, and thereport is clearly and concisely written.Credit: In addition to satisfying the requirements fora pass, the report reflects mastery of a wide rangeof literature concerning human rights in law. Theargument is consistently logical and objective, andthe report is very clearly and concisely written.Distinction/High Distinction: The report isexceptionally well researched and argued. Inaddition to satisfying the requirements for a credit,it demonstrates a high order of critical andargumentative skills appropriate to the discipline orfield, and a highly developed ability to write clearly,creatively and concisely.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 10 •

(Instead of pass, credit, distinction you couldsubstitute C, B and A grades)Analytical criteria for the same assessment may havebeen written like this:▪ Coverage of key relevant material and issues: 20▪ Logical, well-presented argument: 25▪ Written communication skills: 15▪ Familiarity with appropriate research: 25Dunn, Morgan, O’ Reilly, & Parry (2004), pp.27 & 28• 11 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

An approach to designingassessment criteria: outline of aprocess(Michael K. Potter and Erika Kustra, CTL, Universityof Windsor)STEP ONE: Choose your Domains▪ Domains are the criteria you’re using to assess astudent’s performance.▪ Choose domains by identifying knowledge or skillor dispositions that the activity is supposed togive students a chance to develop anddemonstrate.▪ For example, a rubric for assessing essay-writingmight use the following domains: comprehensionof subject, mechanics (grammar and spelling),organisation, and quality of argument (logic,rhetoric, use of evidence).***Your domains are the ROWS of your rubric ***STEP TWO: Choose your Levels of Achievement▪ Decide how many levels of achievement you willrecognize and what you will call them—thenames matter!▪ Most rubrics use four levels of achievement.▪ There is a lot of variation when it comes to thenames. Some variation of “exceptional, good,average, and poor” is more common, but somedon’t even use names at all.*** Your levels of achievement are the COLUMNS ofyour rubric***ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 12 •

STEP THREE: Work on your Descriptions▪ You’re likely to spend most of your time on thiscrucial step, which involves describing whatperformance at each level of achievement, in eachdomain, looks like.▪ Students need to know the difference between, say,“exceptional” organisation of a paper and “good”organisation of a paper. And so do you!▪ Your descriptions tell your students (and yourself)what counts as evidence of achievement at eachlevel of each domain.***Your descriptions fill the BOXES of your rubric ***STEP FOUR: Choose your Rating Scheme▪ How many marks will the entire assessment beworth — and how much will a score at each level ofeach domain be worth?Domains Levels of Achievement (Description) (Description) (Description) (Description)(Description) (Description) (Description) (Description)(Description) (Description) (Description) (Description)• 13 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

Further Examples of CriteriaExample 11 learning outcome: Master’slevel, reproductive healthAt the end of the module, learners will be expected tobe able to appraise the consequence of a range of keysocio-cultural influences on sexual and reproductivehealth (including sexually transmitted diseases,adolescent sexuality, female genital mutilation, theeffects of culture and media).Assessment method:Assessment criteria are derived from the learningoutcome, and no specific task is identified. The taskcould be an unseen question in an examination or anessay question.Grade assessment criteria:Fail: the work produced in response to a relevantquestion suggests that the learner is not able toperform the process of appraisal, e.g. cannotadequately show the consequences of the influencesmentioned in the learning outcome; does not identifysufficient or adequately, the nature of the influences.There is no evidence of knowledge even to the extentof that discussed in the lecture, or the question is notanswered.Third: the work identifies some of the socio-culturalinfluences, suggests some understanding of theireffect on sexual and reproductive health, but thetreatment of the topic is superficial and/or notdiscussed in sufficient breadth. There is no evidenceof knowledge further than that covered in the lecture.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 14 •

Second: the treatment of the question is adequate.The learner identifies and discusses at least the socioculturalinfluences mentioned in the learningoutcome. The work discussion demonstrates usefulunderstanding of the influences and their action onsexual and reproductive health, is off sufficient depthand breath and demonstrates some reading aroundthe topic in addition to the material recommended.First: as the description of the criterion for theSecond, but there is evidence of reading and thoughtaround the topic that goes well beyond that discussedin the lecture or in the recommended reading.Example 12 learning outcome: Master’slevel, learning log module in a leadershipprogrammeAt the end of the module, in an oral presentation,making reference to their learning journal entries,learners will be expected to evaluate the role ofreflection in their work situations, indicating its valuesand the role or potential role of negative influences.They will be able to indicate how they can improvetheir use of learning journals in future use.Assessment method: an oral presentation for 15minutes in front of peers and a tutor, all of whom willjudge the quality of the presentation against a seriesof questions on its quality. The individual questionswill be judged to be passed if they are ticked by atleast all but two of the peers. The tutor will count upresponses.• 15 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

(Comment: peer assessment is to be used here.Learners are to be assessed as ‘adequate’ or ‘not yetadequate’. Those who are judged to be ‘not yetadequate’ will be expected to repeat the presentationat a later stage in their programme.)Assessment criterion—the real criterion is that at leastfive out of seven more potential passes on questionwill indicate ‘adequate’, and two or more ‘not passed’will constitute ‘not yet adequate’. The means ofjudging whether the criterion has been reached posedas questions:In the presentation does the learner evaluate the roleof reflection in the work situation in a consideredmanner?Yes/NoDo the illustrations read from the learning journalrepresent good examples of reflection in the worksituation?Yes/NoDoes the evaluation indicate the values of reflection?Yes/NoDoes the evaluation appropriately discuss negativeinfluences?Yes/NoHas the presenter considered how the learningjournal can be used in a future situation? Yes/NoASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 16 •

Example of grade assessmentcriteriaExample 15 learning outcome: level 1introduction to acting/drama programmeAt the end of the module, the student will beexpected to be able to work with others in small taskorientatedgroups, participating and interacting in thegroup in a productive manner for him/herself and forthe group as a whole.This learning outcome, according to the definition oflearning outcome statements given in the lastchapter, is written at threshold. An example of adesirable learning outcome that could guide thewriting of grade assessment criteria is:The high-achieving learner will be able to work withand to lead others in small task-orientated groups,participating in and interacting with the group in aproductive manner for him/herself and for the groupas a whole. S/he will be aware of his/her role in thegroup, and be able to describe his/her strategies andactions.Grade assessment criteria will now be guided by boththe learning outcome that provides the pass-fail pointinformation and the desirable learning outcome thatindicates the qualities of better performance that willattain a higher grade. The assessment criteria mightbe:Fail: the learner cannot or does not participate ordoes not work towards helpful co-operation in agroup situation.• 17 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

Average pass: the learner works with others in a taskorientatedgroup, participates and interacts in aproductive manner for her/himself and the group.High average: the learner works well with others in atask-orientated group, participating and interacting avery helpful manner that suggests an increasingawareness of his/her role in the group and anincreasing orientation towards the taking of leadersroles when appropriate.Excellent: the learner is able to lead and to act as aparticipant in a task-orientated group, is aware of his/her role in the group and is able to describe strategiesand actions.Moon (2002), pp.92. 101-102ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 18 •

A Sample Feedback Sheet for EssaysKnowledgeTextDeep, thorough,detailed knowledgeSuperficialknowledgeAuthorWide knowledge usedin analysisKnowledge lacking ornot usedGenreWide knowledge usedin analysisKnowledge lacking ornot usedHistorical andsocial contextWide knowledge usedin analysisKnowledge lacking ornot usedEssayStructure Clear logical structure Confused listQuotationsCorrect, purposefuluse, properlyreferencedReferences lacking,or incorrectOther sourcesWide range, relevant,properly referencedNone or irrelevantGrammar andspellingCorrect Many errorsPersonalResponse to text Vivid, personal No responseViewpoint Clearly expressed Viewpoint lacking orunoriginalCreativityCritical TheoryImaginative,surprisingPredictableUnderstanding Clear grasp No graspUse of methodsWide range,appropriately usedRange limitedHaines (2004), p.88• 19 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

▪ Critical and creative approach: 15Tips► Develop criteria that match the particular taskand associated learning outcomes.Departmental or generic programme criteriaare generally bland and written in a way thatdoes not make much sense to students and arenot precisely associated with the particularlearning outcomes.► Make sure that students and teachers have ashared understanding of the criteria beforestudents undertake the task.► Where possible, involve the students innegotiating the criteria.► Allow students to practise on exercises thatuse the same criteria.► Provide samples of work for students to markusing the criteria - this promotes a betterunderstanding of the meaning of the criteria.► Be sure that course team members have ashared understanding of the task criteria. Ifpossible, course team members should mark asample of student work and compare anddiscuss the marks that they have allocated.► Use a system of cross marking.► Mark a sample of scripts and then revisit tocheck allocation of grades.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 20 •

► Use peer and self evaluation to complementteacher evaluation.Dunn et al (2004) provide a useful quick checklistfor teachers that teachers in relation to gradingcriteria: Have I determined in advance of the task whatthe standards of performance - and thereforeconceptions of quality - are in relation to thistask? How can I be sure that performance standardswill be the same no matter who marks theassessment or whether it is marked first orlast? How do the various markers of this task knowwhat the performance standards are? How dothe students know? How clear to students are the standards ofperformance expected of them? How do theyknow this? (p.28).Remember there is no recipe for absolutelyconsistent and reliable allocation of grades, butwe need to put as many safeguards as possible toensure a high level of consistency and accuracy.These include :▪ Well-defined criteria around which there hasbeen dialogue and which are well-understoodby all parties.▪ Sample marking and discussion by membersof a marking team.• 21 •ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU

▪ Providing samples of marking to new tutors orlecturers.ReferencesDunn, L., Morgan, C. & O’Reilly, M. (2004). TheStudent assessment handbook. London:Routledge Falmer.Moon, J. (2002). The Module and programmedevelopment handbook. London: KoganPage.ASSESSMENT: SETTING & MARKING ASSESSMENT TASKS • TDU• 22 •

Produced by:Teaching Development Unit | Wāhanga Whakapakari AkoPrivate Bag 3105Hamilton 3240 New ZealandPhone: +64 7 838 4839Fax: +64 7 838 4573tduadmin@waikato.ac.nzwww.waikato.ac.nz/tdu