Do Compare Yourself To Others! Using Comparative Judgement to Help Students Engage with Feedback

This article was originally published in NATE’s (National Association for the Teaching of English) publication Teaching English: Issue 37.

Don’t Do Compare Yourself To Others!

Using Comparative Judgement to Help Students Engage with Feedback

Marking assessments is time-consuming and often fails to deliver meaningful improvements. Comparative judgement can transform feedback, helping students take ownership of their learning and fostering a growth-focused mindset.

It is a truth universally acknowledged that a mound of freshly completed assessments waiting expectantly on an English teacher’s desk is not an entirely welcome sight. This is not because we do not want to give our students meaningful feedback. In fact, many teachers would say that they are eager to see their students’ successes and learn more about how future teaching could be adapted to meet their emerging needs.

What inspires such feelings of ambivalence is the foreknowledge of how much time will be consumed by the marking, and what will need to be sidelined – well-planned and resourced lessons, opportunities for professional development and, let’s admit it, life outside of school – in order to plough through it in a timely way.

Nevertheless, having laboured tirelessly to offer each student detailed commentary and personalised action targets for improvement, it can then be even more frustrating to be met with a class of befuddled expressions, or even apathy when that feedback is shared. How can they cry ‘I don’t get it’ or ‘I don’t know what I’m supposed to do’ when we’ve invested so much in peppering their responses with regular suggestions and recommendations, before providing them with a lengthy, bespoke summary of their work at the end? How can our efforts not immediately affect the change necessary in our students’ work?

Unfortunately, as many teachers recognise, in-depth feedback does not necessarily result in quality improvement because this process can often, despite our best intentions, leave students feeling overwhelmed and uncertain of what is expected next. The reality is that informing students of their gaps and instructing them on ways they may address them does not mean that they know how to enact this process.

While this article cannot profess to offer solutions that will eliminate the time spent on marking, it will hopefully provide some simple strategies which ensure that teacher feedback can empower students to better understand their work and know how to improve it.

Understandable, Helpful, Actionable Feedback

As teachers, we know feedback that places marks or grades at the forefront can distract even the most diligent learners from engaging with their precise successes, as well as the steps they need to take to progress further. Moreover, the recording of a mark or a grade on a piece of work, which is intended as formative, can transform it into something summative and finite. What would be the point, some students might ask, in improving a piece of work which they perceive as complete since it has already been graded?

How can we then move students away from this preoccupation with marks and grades, and towards an understanding of feedback as a helpful and actionable process that facilitates their ongoing progress? Kate Jones (2021), drawing on the working of Dylan Wiliam and Siobhan Leahy, suggests three concise principles, which offer a useful starting point for making feedback purposeful.

Asking ourselves how understandable, helpful, and actionable our feedback is inevitably shifts the focus from ‘What did you get?’ to ‘What should you do now?’ and ‘How can you do it?’ The focus then also shifts from improving the work to improving the learner. But these latter questions should not just be ones posed by the teacher to their students.

Instead, to make feedback truly purposeful, we should create opportunities for students to ask these questions of themselves and others, so that self-reflection and self-regulation becomes an internalised process.

The Role of Comparative Judgement

Introducing the Strategy

An effective method for cultivating this way of thinking is comparative judgement.

Imagine this: the brief specifies that an artist needs to create a convincing image of a house, which includes the following:

• At least two windows
• A door
• A chimney
• A garden with trees

How successful has the artist been in fulfilling the brief? How confident would you feel grading it out of 10 based on this criteria?

Then, look at these two further images. Which one has fulfilled the task most successfully? How would you rank these two examples with the first? How does viewing further examples help you to measure how effectively each artist has completed the task? Why might it now be easier, and more accurate, to grade each piece of work?

This strategy, often referred to as comparative judgement, is outlined by Daisy Christodoulou in her book Making Good Progress? (2017). She describes it as a process that enables teachers to assess work more accurately by scaling several examples based on quality. ‘Human judgement,’ she argues, ‘is essentially comparative, not absolute,’ making it difficult to identify strengths and areas for development when looking at a single piece of work in isolation. However, when placed against another example (or several), those strengths and areas for development become more perceptible. This is because the work is being seen in the context of many other responses which have completed the task to varying degrees of efficacy, some more successfully and some not so.

Benefits for Teachers and Students

Using comparative judgement will be familiar to English departments that make regular and ongoing use of standardisation to gauge what aspects of the learning students are grasping well, and what additional input or re-teaching is required. Knowing how valuable it is to scale multiple pieces of work alongside each other, we would suggest that it would be equally informative for students to do the same.

We would recommend that teachers return the work without marks or grades recorded on them. Yes, some students may ask the immediate question, ‘What was my grade?’ or helpfully point out ‘You forgot to put my grade on my work’, but once they are well rehearsed in this mode of feedback, the individual mark will become less of a preoccupation.

After students have re-read their work and re-familiarised themselves with the objectives of the task, teachers should guide them through the following steps:

1. Organise students into small groups and provide them with at least three new responses to the same task, which have been completed to varying levels of success. These need not be full responses. For an analytical response, it may be the introduction and first paragraph, while for a writing task, it could be the two opening paragraphs.
2. Display the objectives or success criteria for the task again on the board. Ask students to actively read the responses and discuss where the success criteria have or have not been demonstrated.
3. Ask students to rank the responses from most to least successful. Explain that they need to justify their decision by referring to the success criteria. One approach may be for students to record on sticky notes what, for instance, Example B should do to achieve the same level as Example C, or how Example B has demonstrated a particular skill more effectively than Example A.
4. Guide students to verbalise these thoughts during whole-class feedback. Use this process to address any misconceptions that some students may have about the success criteria.
5. Prompt students to return to their own work and reflect on where they would place it within the rank order and why. As with the modelled examples, guide them to carefully scrutinise how effectively they have met the success criteria and the incremental changes that they would need to make to improve (we use the word ‘incremental’ because students should only identify 1-2 changes that will result in the greatest improvement). Allow students the appropriate amount of time to fully action these changes.

At this stage, teachers can share their feedback with students, including the action targets that they have identified as a priority. Students should be encouraged to consider to what extent their priorities for improvement match those recommended by their teacher, especially because as students become more confident and skilled with this process, they will often identify the same targets. In this way, the use of comparative judgement is a tool for helping learners become more self-motivated and independent, because it makes them feel knowledgeable about their own progress. This is empowering because it creates a sense of efficacy and develops a growth mindset by demonstrating that improvement is possible and ongoing.

Detective work

We have found that sharing feedback in this way has been particularly fruitful in our classrooms because it adheres to the three principles outlined by Kate Jones (2021). Firstly, it is understandable because it gives students tangible examples of what success does and does not look like through the modelled responses. When students can see where their work fits into a spectrum of possible outcomes, they develop a clearer grasp of how far they have met the objectives or success criteria.

It is helpful because it presents students with authentic examples of the very work they have completed. A target at the end of a piece of work, even one that is SMART, can feel vague or inaccessible if students cannot see what it would look like in practice. The comparative judgement responses contextualise students’ targets and enable them to see how they could be successfully demonstrated within a piece of work.

Finally, it is actionable because it enables students to make precise improvements to the current piece of work while also serving as a diagnostic for the next one – students understand the next incremental steps they need to actively take to progress.

Using comparative judgement shifts students’ focus from marks to quality. It is improvement-driven because it cultivates a classroom culture of what Dylan Wiliam describes as ‘detective work’ (2017) by making students delve deeply into their work and the process they undertook to complete it, in order to understand why the outcome is what it is. Ranking three examples that are not their own against a success criteria essentially acts as a scaffold: students find the process of conducting ‘detective work’ far easier with someone else’s response than they do with their own since it is low-stakes and unbiased. This then builds them up to investigating their own responses more thoroughly.

We are always astounded by the number of students who do not ask for their marks after completing a comparative judgement reflection task, as they are already thinking ahead to what they will do differently (and better) next time.

Many students even express some frustration with their work – but not because they’re unhappy with their grade, or don’t know how to improve. Instead their exasperated comments tend to be something akin to ‘I can’t believe I didn’t do that!’ or ‘I definitely could have done that!’ Teachers need to galvanise this feeling by reminding students that learning is a continuous process of making mistakes and therefore there is always scope for improvement.

Planning, Embedding, Refining

Comparative judgement is an effective strategy because it can be adapted to the emerging needs of the class and to the point that students have reached within a scheme of work. When embedded successfully, it need not be a laborious process for teachers. Below are some of the approaches that we have identified for refining the use of comparative judgement within our classrooms:

• Use student responses to expose the class to authentic examples of work produced by their peers. These could be taken from another class or a previous cohort. Seeing the real outcomes of other learners of the same age can be very motivating in terms of helping students visualise what they could achieve.
• Create teacher-generated responses that mirror the recurring errors that you may have identified in their work. Use these to help students pinpoint the mistakes that they may not immediately recognise in their own responses, in a low-stakes, unbiased way, so that they know what to look out for in their own work.
• Deliberately challenge students by providing them with modelled examples which are designed to tackle common misconceptions about what success looks like. By doing so, you can ensure that they have a secure grasp of the success criteria, and do not fall into the trap of thinking that wordiness equals sophistication, or that writing more must make it better work, or that using a PEAL writing frame will result in quality analysis.
• Always have a response higher than the highest in the class so that there is no ceiling of achievement for any learner. Since all students have access to this top example, regardless of their predicted/target grade, teachers can foster a culture of constant high standards for all and help build self-efficacy.
• Similarly, always have a response lower than the lowest in the class. All students should be given scope to identify successes in their work, when compared to others, regardless of the level that they are working at. This is particularly pertinent in a mixed ability setting so that students don’t feel pigeon-holed.
• Keep modelled examples short to reduce the cognitive load of the task.
• Use comparative judgement before an assessment, midway through a unit of work, as a diagnostic task to help students identify and prioritise their key action targets. In doing so, you will be embedding knowledge and understanding of the success criteria.
• Train students to use the strategy effectively by making it an integral part of all schemes of work, using consistent language and questions. We have found comparative judgement to be insightful for students from Key Stage 3 to Key Stage 5, across reading, writing and speaking and listening tasks.

Equipping students with knowledge about the quality of their work and their progress is empowering. Done well, students will begin to ask reflective and metacognitive questions – ‘What should I do now?’ and ‘How can I do it?’ of themselves and in discussion with their peers. Fostering such independence is essential to developing confident learners who move away from the belief that only teachers hold the knowledge about how they might improve, and instead, recognise that they too can form accurate conclusions to move themselves forward. Ultimately, the residual effect of this is a safe classroom where students don’t fear comparison, but rather see it as insightful.

As English teachers, we have found comparative judgement to be a welcome addition to our toolkit, but this strategy is not exclusive to the teaching of English. If you trial it in your classroom and find it to be useful, please do share it with other subject areas – it can only serve to help students become better reflective learners when adopted across a school.

Bibliography

• Christodoulou, D. (2017) Making Good Progress? The Future of Assessment for Learning, Oxford: Oxford University Press.
• Jones, K. (2021) Wiliam and Leahy’s Five Formative Assessment Strategies in Action, Woodbridge: John Catt.
• Wiliam, D. (2017) ‘Assessment, Marking and Feedback’ in What Does This Look Like in the Classroom? Bridging the Gap Between Research and Practice, edited by C. Hendrick and R. McPherson, Woodbridge: John Catt.