10. Briefing Paper Template - Higher Education Academy

Briefing Paper 

Closing the Loop: Conceptualising Feedback as Dialogue to Share 

Evidence-based Practice in Formative Assessment 

Briefing Paper by: Dr Stylianos Hatzipanagos & Dr Emma Medland, Kings College London (July 2011) 

Overview 

This seminar draws from our research that investigated the subjective processes 

involved in assessing and providing feedback; and the significance of formative 

assessment in higher education. 

1. Abstract: please provide a brief abstract of the seminar delivered (maximum 200 words). 

This seminar drew from our research on the subjective processes involved in assessing and 

providing feedback; and the significance of formative assessment in higher education by 

establishing and comparing attitudes to assessment amongst tutors and students. The 

seminar: 

put forward a conceptual model of formative assessment based on the 

literature and explored whether this model has any implications for learning 

and teaching in face to face or blended learning environments; 

identified examples of good practice in formative assessment and examined 

how formative assessment can be made to work purposefully in providing 

opportunities for dialogue between academic staff and students; 

examined critically the processes involved in producing feedback; 

explored current feedback practices and examined whether and how these 

practices support formative assessment. 

We combined discussion of our research findings with opportunities for the participants to 

explore and compare their own conceptualisations of formative assessment and feedback. 

Participants were also encouraged to critically reflect upon their own practice in light of 

the research findings by evaluating a piece of their own feedback in order to enhance its 

effectiveness in supporting students to close the loop. The seminar was therefore centred 

around audience participation and aimed to support evidence-based practice. 

EvidenceNet is a Higher Education Academy resource. 

www.heacademy.ac.uk/evidencenet 

Page 1 of 11


2. Rationale: please provide the background context, such as the research/evidenceinformed 

practice context, which provided the impetus for the seminar. 

Feedback is a significant attribute of formative assessment and yet academic staff and 

students’ conceptualisations of it are under-researched (McDowell, 2010). Feedback is 

believed to be based on a marker’s ‘intuition’ (Ecclestone, 2001) or ‘tacit knowledge’ 

(Eraut, 1995) of assessment standards, which is largely subjective (Bloxham 2009). Whilst 

disagreement between markers is often viewed as somehow compromising the integrity of 

the grade (Sadler, 2009), Medland (2010) posits that the opportunity for dialogue 

concerning the implicit values and beliefs informing personal judgement could instead be 

viewed as a tool for clarifying why mismatches occur. 

The view of feedback as an active and participative, or ‘sustainable’ (Hounsell, 2007; 

Carless et al., 2011) process, contrasts with the notion of feedback as a transmissive 

process that involves ‘telling’ or passing on information (Nicol & Macfarlane-Dick, 2006). 

This understanding of feedback as dialogue is fundamental to the process of ‘closing the 

loop’ (Sadler, 1989). Communication forms part of the mechanism by which the learner 

monitors, identifies and then is able to ‘bridge’ the gap in the learning process 

(Hatzipanagos, & Warburton, 2009). The outcome of this dialogue can be disconcerting for 

the students as there is no ‘pre-determined’ handed-down set of judgements but a 

mutually constructed set of targets that they need to act upon. In other words, 

communication becomes a vital part of the feedback cycle that enables students and 

academic staff to actively construct their own understanding of what can be, complex and 

difficult messages to both create and decipher (Higgins et al. 2002). 

Increasing levels of self-awareness and reflection in both academic staff and students, and 

their willingness to engage in dialogue concerning how feedback can be used as a tool of 

formative assessment to support the active construction of understanding is, therefore, 

key to gaining insight into how it is conceptualised by both academic staff and students. 

The seminar drew from our research, namely two projects that (1) investigated the 

subjective processes involved in assessing and providing feedback; and (2) explored the 

significance of formative assessment in higher education by establishing and comparing 

attitudes to assessment amongst tutors and students. 

3. Generation of evidence: please describe how the reported research/evaluation findings 

were generated e.g. methods used 

Evidence was generated for each of the research projects as follows [1) relates to Dr Emma 

Medland’s research and; 2) relates to Dr Stylianos Hatzipanagos’ research]: 

(1) Semi-structured interviews regarding how each tutor assesses and produces feedback 

for the same piece of coursework were undertaken with academic developers from the 

same team. The intention was to illuminate some of the values and beliefs that shape the 

professional judgement informing the way assessment and feedback are undertaken as a 

means of identifying some of the reasons why mismatches between markers occurs. The 



Page 2 of 11


research participants formed part of a team of academic staff working within an academic 

development unit in a large research-intensive university in the UK, who were chosen in 

view of the call by their external examiners for greater consistency between first and 

second markers. The sample size of six research participants was decided prior to the start 

of the research as a number that would provide variation and richness of data but would 

be a manageable number of transcripts to analyse. Furthermore, Interpretative 

Phenomenological Analysis, or IPA, emphasises individual experiences over broader 

generalisations and is idiographic in nature and therefore it is recommended that sample 

size does not exceed ten participants (Reid et al., 2005). 

(2) Semi-structured interviews with tutors focusing on their perceptions of assessment and 

evidence of dialogue and interaction around formative assessment and feedback led to 

identifying individual cases of practitioners employing formative assessment. It explored 

assessment practices in distance education with a focus on tutors’ orientations. Twenty 

tutors and seventeen students from three Open and Distance Learning environments 

participated in open-ended interviews. The aim was to include tutors and students from 

the same disciplines. Where that was not possible we looked at courses which had a similar 

disciplinary context, so that context would not be dissimilar. An online questionnaire based 

on the interview outcomes was administered to a large cohort of students and within the 

three environments; there were 1,032 returns. 

4. Existing evidence: please provide details of research/evaluation evidence drawn on and 

reported in the seminar 

The seminar drew upon two pieces of research relating to 1) Dr Emma Medland’s research 

and; 2) Dr Stylianos Hatzipanagos’ research as follows: 

1) Medland reported on one project that illuminated some of the values and beliefs that 

shape the tacit knowledge (Eraut, 1995) or intuition (Ecclestone, 2001) informing the way 

marking is undertaken by academic developers as a means of exploring why mismatches 

between markers in the same team occur. The conclusion drawn aimed to contribute to 

the ‘discourse on marking in higher education’ that Bloxham (2009) believes is currently 

lacking. Each participant was invited to mark the same piece of coursework, a 3000 word 

assignment. In addition, the research participants took part in an open-ended semistructured 

interview (30-50 minutes) that focused on the processes involved in marking the 

assignment and assessment processes more generally. The example assignment was 

produced by a new lecturer who had successfully completed a qualification offered by the 

academic development unit and was chosen in view of the disagreement amongst the 

original markers with regard to the mark it should receive. The findings reported indicate 

the need to reconceptualise subjectivity in assessment. 

2) Hatzipanagos reported on two projects that explored tutors’ and students' perceptions 

of assessment. Data from three Open and Distance Learning environments - King’s College 

London, the external programmes of the University of London and the Open University 

(OU) – were collected. A key focus of the exploration was to establish whether assessment 

activities in such environments are used to enhance dialogue, interaction and collaborative 



Page 3 of 11


work and consequently improve learning, and how these activities are perceived by tutors 

and students. This case study put forward a conceptual model of formative assessment and 

discusses how this can be made to work purposefully to support students in higher 

education. 

5. Research findings/new evidence: please describe any new findings or evidence reported in 

the seminar. 

The seminar drew upon two sets of findings relating to 1) Dr Emma Medland’s research 

and; 2) Dr Stylianos Hatzipanagos’ research as follows: 

1) The findings indicated that differences in perspective or subjectivity amongst a team of 

academic developers was believed to have a positive impact, which could encourage the 

articulation of personal values and beliefs and lead to the development of increasing selfawareness 

and a greater level of shared understanding amongst a team of markers. It is 

acknowledged that a large proportion of disagreement between markers can be 

problematic and result, as the external examiners commented, in a greater amount of time 

being invested in marking. However, this study also indicated that the outcome of this 

disagreement can be highly useful in initiating discussions surrounding the personal values 

and beliefs that shape the manner in which one marks. 

2) We considered feedback intertwined with the notion of dialogue, as a two-way 

communication between the student and the tutor and also among the students 

themselves and teams of markers. Our evidence suggests that assessment and e- 

assessment practices which involved the provision of formative feedback seemed to 

encourage student self-assessment and self-regulation. The research contributed to the 

development of a framework for rationalising formative assessment practices. Within this 

framework, assessment methods are most effective if they move practitioners and students 

towards the use of formative assessment. 

6. Outcomes of research /evaluation evidence and the implications for policy and practice: 

please identify any application or outcomes of research/evaluation evidence and detail the 

implications for policy and practice for different stakeholder groups such as: academics, 

learning technology practitioners, professional developers, senior managers, policy makers, 

students, sector organisations, employers and professional bodies. 

The seminar drew upon two sets of research outcomes relating to 1) Dr Emma Medland’s 

research and; 2) Dr Stylianos Hatzipanagos’ research, as follows: 

1) From a poststructuralist perspective of assessment (Orr, 2007), mismatches between 

markers can, therefore, be rewarded with an insight into the subjectivity that implicitly 

pervades the discourse of assessment (Bloxham, 2009). In other words, rather than the 

traditionally positivist perspective of the role of subjectivity in the assessment system as 

somehow compromising the integrity of the mark (Sadler, 2009), it could instead be viewed 

as a tool for clarifying why mismatches between markers occurs. Thus alleviating some of 

the anxiety that inhibits discussions surrounding the often implicit factors that Price (2005) 

believes ‘are at the foundation of our awards’. In essence, one’s understanding of what 



Page 4 of 11


constitutes a ‘good’ piece of work ‘...is created through a social process involving dialogue 

and experience’ (Bloxham 2009, 218). When marking is viewed through this lens, 

subjectivity may then be reconceptualised as a potentially useful tool in developing greater 

self-awareness and levels of coherence between teams of markers. 

2) The implication is that a categorisation of assessment practices in four stages could 

benefit tutors working in different disciplines by encouraging them to move from stage 

Zero (limited evidence of engagement with formative assessment practices) to Stage Three 

(making formative assessment central in teaching practices). 

In stage Zero assessment is implemented mainly through exams and end of assessment 

term projects. In this setup, feedback on performance is either limited or non-existent. 

There are no opportunities for peer/self assessment and there is no use of learning 

technologies to support assessment practices. 

Exams are a common assessment method in higher education. However, in ‘end-loaded’ 

assessments such as end-of-year exams, students do not benefit from feedback. 

Implementation of formative assessment practices and the provision of feedback can be 

problematic in courses where the emphasis is on end-of-year assessments, as the ‘closing 

the loop’ component of the assessment process very rarely takes place. 

Stage One is characterised by the provision of generalised feedback on student work. This 

is (by default) of limited customisation to the needs of the individual learner. There is 

limited use of learning technologies to support assessment practices. 

A possible drawback to providing generalised feedback is that it is not tailored to the needs 

of individual students, unless it is customised by the tutor. A more adaptive approach 

tailored to learner needs would benefit the students. This can be logistically difficult with 

big cohorts of students. However, a solution could be a concise template including 

feedback on performance and developmental issues which the students would need to 

consider. This template (very close to a generic feedback sheet) could be 

adapted/personalised quite easily by the tutor for every student. 

Examiner reports with model answers are also used to monitor and evaluate assessment 

practice. Examiners’ reports for the use of tutors and students are useful because they are 

not model answers to exam questions but a concise and sometimes reflective account of 

the issues related to a correct approach to answering assessment questions. 

Enhancing Learning through Assessment in Business and Management, Hospitality, Sport, 

Leisure and Tourism 8 Tutor Engagement in Formative Assessment for an Impact on the 

Learner Experience 

Periodic/continuous assessment is central in Stage Two, helping learners to rehearse 

arguments that they will use in end of the assessment period assignments. The notion of 

continuous assessment that some distance learning institutions have endeavoured to put 

in place tends to be more of a periodic rather than a continuous nature. Some form of 

periodic assessment is necessary to ensure that monitoring progress and study support 

measures are in place for the students, before they reach the final assessment. However, 



Page 5 of 11


even periodic assessment may play a summative role, if there is no opportunity for the 

students to revisit and use the feedback subsequently. 

Feedback is monitored, using computer mediated communication, to ensure that students 

will act upon negotiated targets and the feedback loop is closed. In this way, learner 

responses to feedback become an essential part of the assessment cycle. 

Peer- and self-assessment play an important role in Stage Three, as do activities designed 

to help students to acquire ownership of the assessment process. This may also take the 

form of student involvement in setting marking criteria. 

Use of learning technologies to peer-review and to construct knowledge can facilitate 

assessment activities. Further emphasis on formative assessment can be facilitated by the 

use of computer communication tools that encourage dialogue about feedback and 

assessment (such as blogs and wikis, synchronous and asynchronous discussion forums and 

social networking tools). 

(Hatzipanagos, 2010) 

7. Emerging themes: please detail the discussion topics or themes that were raised by 

delegates during the course of the seminar - suggesting areas that would merit further 

investigation. 

The seminar culminated in an activity designed to encourage the peer evaluation of 

personal examples of feedback as a means of identifying aspects of effective feedback and 

areas for development. The aim was to produce a list of traits that participant groups felt 

could help or hinder the student to ‘close the loop’ (i.e. reflect and act upon feedback 

comments received). Participants were divided into seven groups and their conclusions 

are summarised below: 

Group A: 

Sufficiency – got to be enough to help students learn but not overwhelm 

Personalised – we prefer 1 st person to third person 

Forward looking – good feedback tells students where they can go next 

Legible 

Balance of positive comments/praise and constructive critique 

Clear links between comments and grade – some of the ones we reviewed had 

stark disjunctions between comments and grade. 

Grades/Comments should clearly link to criteria. 

All KCL comments followed a template: we are not convinced it is necessarily 

applicable to all assignments. 

We liked the way some of the examples clearly linked to the literature of the field. 

Good feedback provides detailed, granular information about how to improve. 

The examples of feedback that used full sentences contained less ambiguity than 

the ones that used note form and come across as more respectful 

All four examples came across as judgemental rather than developmental in tone. 

There were few opportunities evident for dialogue. 



Page 6 of 11


*NB We each considered how well our own practice matches up to our own principles 

and didn’t feel particularly smug. 

Group B: 

Need to ensure positive feedback as well as negative (feedback sandwich) – makes 

action points feel more manageable. 

If we are encouraging criticality, we have to accept that at some point learners will 

critique the questions we ask and we should support that. 

Split general feedback (e.g. structure, focus etc.) from specific feedback (e.g. 

relating to particular sentence or paragraph). 

Try to find 3-5 key action points that will maximise development in a manageable 

way and put clearly at end. 

In text specific comments use different colours or word comment functions. 

Directly address the learner via ‘you’, your or by name (for draft/formative only). 

Use of questions to learner to help development. 

Invite discussion via reflexive journals, PDPs, tutorials or class discussion. 

Group C: 

Respect for students’ work 

Feedback should equate with grade 

Balance of positive and negative 

Talk to student: ‘conversation’ to encourage sense of dialogue or start detached 

Does it relate to criteria? 

Mirroring not useful to student 

A grade/moderation justification 

Language: context needed/giving examples of what you mean, not assuming 

knowledge/interpretation; leading into feedforward 

Group D: 

Clearly written with explanations and signposting of specific examples in students’ 

text. 

Although there are specific ideas and examples on how this assignment could be 

improved, less on how future/different assignments could be approved 

Feedback framework both helpful and potentially straitjacketing 

Not much use of questions 

Group E: 

Specific (e.g. point out exact areas for improvement) 

Opportunities for dialogue 

Disciplined – based feedback is important to be aware of 

o Medical/Science approach to teaching generates students with a thirst for 

facts! 

Opportunities for improvement 

Group F: 



Page 7 of 11


Fragmented, modularised programmes - a problem for feedforward? (e.g. if 

different forms of assignment) 

How do we give “feedback” to the markers/teaching team? 

Group G: 

Portfolio of Evidence feedback 

- Very extensive – perhaps too extensive 

- Is it digestible? 

- Each rubric starts on a positive note 

Principles of Good Chair Side Teaching 

- Doesn’t provide food/constructive examples 

- Too broad brush 

- Doesn’t focus on relevant areas 

These results were disseminated to all participants who attended the seminar, as well as 

those who were interested in the seminar but unable to attend. These summaries will also 

be fed-forward into the ‘Benchmarking Feedback’ initiative that the HEA has recently 

initiated, for which Dr Medland and Dr Hatzipanagos are both members. 

8. Any other comments: please use this box to include any additional details. 

N/A 

9. Bibliography/references (preferably annotated): please list any references mentioned in 

or associated with the seminar topic. Where possible, please annotate the list to enable 

readers to identify the most relevant materials. 

Bailey, R., & Garner, M. (2010) Is the Feedback in Higher Education Assessment Worth the 

Paper it is Written on? Teachers’ reflection on their practices. Teaching in Higher Education, 

15(2), 187-198. 

Beaumont, C., O’Doherty, M., & Shannon, L. (2011) Reconceptualising Assessment 

Feedback: a key to improving student learning? Studies in Higher Education. 

Black, P., & Wiliam, D. (1998) Assessment and Classroom Learning. Assessment in 

Education, 5(1), 7-74. 

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2003) Assessment for Learning. 

Buckingham: Open University Press. 

Bloxham, S. (2009) Marking and Moderation in the UK: false assumptions and wasted 

resources. Assessment & Evaluation in Higher Education, 34(2), 209-220. 

Bloxham, S., & Campbell, L. (2010) Generating Dialogue in Assessment Feedback: exploring 

the use of interactive cover sheets. Assessment and Evaluation in Higher Education, 35(3), 

291-300. 



Page 8 of 11


Bloxham, S., & Boyd, S. (2007) Developing Effective Assessment in Higher Education. Open 

University Press and McGraw-Hill Education: Berkshire. 

Brown, S. (1999) Assessment Matters in Higher Education: choosing and using diverse 

approaches. SRHE and Open University Press: Buckingham. 

Carless, D., Salter, D., Yang, M., & Lam, J. (2011) Developign Sustainable Feedback Practices. 

Studies in Higher Education, 36(5) [online]. Available: 

http://dx.doi.org/10.1080/03075071003642449 

Carless, D., Salter, D., Yang, M., & Lam, J. (2010) Developing Sustainable Feedback Practices. 

Studies in Higher Education. 

Carless, D. (2006) Differing Perceptions in the Feedback Process. Studies in Higher 


Chanock, K. (2000) Comments on Essays: do students understand what tutors write? 

Teaching in Higher Education, 5(1), 95-105. 

Ecclestone, K. (2001) ‘I know a 2:1 when I see it’: Understanding criteria for degree 

classifications in franchised university programmes. Journal of Further and Higher 


Eraut, M. (1995) Developing Professional Knowledge and Competence. London: Falmer 

Press. 

Hattie, J., & Timperley, H. (2007) The Power of Feedback. Review of Educational Research, 

77(1), 81-112. 

Hatzipanagos, S. (2010) Closing the loop: Tutor engagement in formative assessment for an 

impact on the learner experience in Kemp, P. and Atfield, R. (Eds) Enhancing learning 

through Assessment. Threshold Press, ISBN 978-1-61520-937-8, 32-41. 

Hatzipanagos, S & Rochon, R. (eds) (2010). Special Issue on Approaches to Assessment that 

Enhance Learning in Journal of Assessment and Evaluation in Higher Education. 35 (5), 491- 

646. 

Hatzipanagos, S. & Warburton, S. (2009) Feedback as Dialogue: Exploring the Links between 

Formative Assessment and Social Software in Distance Learning. Learning, Media and 

Technology. 34(1), 45–59. 

Higgins, R., Hartley, P., & Skelton, A. (2002) The Conscientious Consumer: reconsidering the 

role of assessment feedback in student learning. Studies in Higher Education, 27(1), 53-64. 

Hounsell, D. (2007) Towards more Sustainable Feedback to Students, in D. Boud & N. 



Page 9 of 11


Falchikov (eds.) Rethinking Assessment in Higher Education (Routledge: Oxon) pp. 101-113. 

Juwah, C. Macfarlane-Dick, D., Matthew, B., Nicol, D., Ross, D., Smith, B. Enhancing student 

learning through effective formative feedback. At 

http://www.heacademy.ac.uk/assets/York/documents/resources/resourcedatabase/id353 

_senlef_guide.pdf 

Knight, P.T., & Yorke, M. (2003) Assessment, Learning and Employability. Buckingham: 

Society for Research in Higher Education and Open University Press. 

McDowell, L. (2010) Assessment for Learners: when practice and theory meet. Keynote at 

EARLI/Northumbria Assessment Conference, 1-3 September, Slaley Hall Hotel, 

Northumerland, UK [online]. Available: 

http://www.northumbria.ac.uk/sd/central/ar/academy/cetl_afl/earli2010/themes/lmcdow 

ell/?view=Standard 

Medland, E. (2010) Subjectivity as a Tool for Clarifying Mismatches between Markers. The 

International Journal of Learning, 17(7), 399-412 

Nicol, D. (2010) From Monologue to Dialogue: improving written feedback processes in 

mass higher education. Assessment and Evaluation in Higher Education, 35(5), 501-517. 

Nicol, D.J., & Macfarlane-Dick, D. (2006) Formative Assessment and Self-Regulated 

Learning: a model and seven principles of good feedback practice. Studies in Higher 


Orr, S. (2007) Assessment moderation: constructing the marks and constructing the 

students. Assessment & Evaluation in Higher Education, 32(6), 645-656. 

Orrell, J. (2006) Feedback on Learning Achievement: rhetoric and reality. Teaching in Higher 


Orsmond, P., & Merry, S. (2011) Feedback Alignment: effective and ineffective links 

between tutors’ and students’ understanding of coursework feedback. Assessment and 

Evaluation in Higher Education, 36(2), 125-136. 

Poulous, A., & Mahony, M.J. (2008) Effectiveness of Feedback: the students’ perspective. 

Assessment and Evaluation in Higher Education, 33(2), 143-154. 

Price, M., Handley, K., Millar, J., & O’Donovan, B. (2011) Feedback: all that effort, but what 

is the effect? Assessment and Evaluation in Higher Education, 35(3), 277-289. 

Reid, K., Flowers, P., & Larkin, M. (2005) Exploring Lived Experiences. The Psychologist, 

18(1), 20-23. 

Sadler, D.R. (2010) Beyond Feedback: developing student capability in complex appraisal. 

Assessment and Evaluation in Higher Education, 35(5), 535-550. 



Page 10 of 11


Sadler, D.R. (2009) Grade Integrity and the Representation of Academic Achievement. 

Studies in Higher Education, 34(7), 807-826. 

Sadler, D.R. (1989) Formative Assessment and the Design of Instructional Systems. 

Instructional Science, 18, 119-144. 

ONLINE RESOURCES: 

Assessment Futures: www.assessmentfutures.com 

Assessment Standards Knowledge exchange (ASKe): http://www.brookes.ac.uk/aske/ 

Centre for Excellence in Teaching and Learning in Assessment for Learning: 

http://www.northumbria.ac.uk/sd/central/ar/academy/cetl_afl/ 

Enhancing Feedback: http://www.tla.ed.ac.uk/feedback.htm 

HEA (2006) Assessment and Feedback Video Clip: student perspective [online]. Available: 

http://www.heacademy.ac.uk/ourwork/learning/assessment/senlef 

Improving the Effectiveness of Formative Assessment in Science Teaching (FAST Project): 

http://www.open.ac.uk/science/fdtl/ 

NUS (2010) Charter on Feedback and Assessment [online]: Available: 

http://www.nusconnect.org.uk/asset/news/6010/FeedbackCharter-toview.pdf 

Transforming the Experience of Students through Assessment (TESTA): 

http://www.testa.ac.uk/ 



Page 11 of 11

10. Briefing Paper Template - Higher Education Academy

Create successful ePaper yourself

Delete template?

Save as template?