PP654 UniSa Freney - Final Report Feb 2010.pdf - Office for ...

Evaluation of the systems indicates that staff felt that they were better able to providedetailed formative feedback comments, and more consistent summative assessment.Students agreed that the feedback they were receiving was helpful and detailed, andthey appreciated the convenience of receiving feedback via email.Due to resource limitations, some proposed functionality was not achievable. Theability to display an analysis (e.g. graphs) of the class’ performance for each individualassessment criterion and the ability to attach audio files (in lieu of typed feedbackcomments) were not achieved, however the CAFAS Excel version does provide audiorecording functionality. These objectives were pursued but ultimately it was necessaryto abandon them to ensure that higher priority functionality was operating correctlyprior to trials. This functionality has been listed as future improvements on theEnhancement Requests page of the project website.3.0 CAFAS ObjectivesBoth the CAFAS Online (web based) and CAFAS Excel (offline/desktop) systems guideacademics through the process of designing a feedback form, which can then beemailed to students prior to them undertaking the assessment task. This approachprovides the student with detailed information about the assessment scheme including:weighted assessment criteria; graduate attributes that will be developed by theassessment task; and grade descriptors of performance levels and indicators for eachparticular grade. This ensures that students are provided with clear communicationabout the criteria prior to them undertaking the assessment task.Having designed the feedback form, academics use the system to assess and providefeedback on the students’ assignment submissions. A copy of the feedback form isgenerated for each student so that formative and summative assessment and feedbackcan be entered by the academic(s) and then emailed to each student. Thus eachstudent receives a feedback report via email, which clearly indicates how theirassignment has been assessed, listing all assessment criteria with grades/marks andfeedback comments for each criterion. The form also contains a graph that displaysgrade distribution for the assignment and a graph for each assessment criterion thatcompares the student’s performance in a particular assessment criterion to the classaverage for that criterion. This helps students gauge their performance in relation totheir peers while preserving the confidentiality of each student’s individual result.All marks and grades entered into the system are tabled in a “Mark Book” clearlydisplaying results for all assessment tasks in sortable columns. At the conclusion of thecourse/unit/subject a final mark is calculated (based on the weightings of theassessment tasks) and displayed in the Table. The Table can then be exported fromthe system (in CSV format) and opened in spreadsheet software such as MicrosoftExcel. This enables transfer of the data into other systems used for reporting finalresults within the institution.While both CAFAS Online and Excel versions provide the same feedback andassessment functionality described above, the online version provides a more intuitiveuser interface and improved collaborative tools (due to the online nature of thesystem). Despite these advantages, the online system requires that institutionscooperate with the lead university in establishing the correct protocols forauthentication. The limitations of current authentication system protocols createdsignificant difficulties during trials with external institutions. Eventually, the issues wereable to be overcome with two institutions, and further improvements of the ShibbolethComputer Aided Feedback & Assessment System (CAFAS) 7