On Wednesday 24 January, we were visited by Dr Ender Özcan and Dr Carmen Tomas from the University of Nottingham for a research seminar about moderation. Ender is an assistant professor of Operational Research and Computer Science with the Automated Scheduling, Optimisation and Planning (ASAP) research group in the School of Computer Science at the University of Nottingham, and the level 3 and level 4 undergraduate project coordinator for the School. Carmen is the Assessment Adviser for the University of Nottingham and works on the Teaching Transformation Programme leading on the area of assessment.
Ender and Carmen explained about their implementation of a novel approach to moderation of dissertations and project reports. Carmen explained the background to the project, which aims to ensure consistency and confidence in determining final grades. Like all the best projects, the assessment advisor and the head of computer science found that they were thinking along the same lines and were able to combine forces to create a new approach. I may be over-simplifying here, but the process goes something like this: the supervisor first-marks the assignment and submits a grade. At the same time, three colleagues read the submission in less depth. Each allocates the assignment to a grade band and submits this. Ender then compares the median marks. If the grades are in the same band, as 79% are (16% were identical), then the supervisor’s mark stands. If there is a wide gap, then the project is systematically referred to a full second marking. If there is more than six marks of difference across the markers, then the panel meets to discuss the final grade.
Student submissions are 15,000 words each. Each panel member reviews around 28 submissions and reported that they took between 10 and 30 minutes to review each one, compared to 90 minutes for a full, detailed, grading with feedback production.
Following the seminar, we had a lively debate about the pros and cons of introducing such an approach at Manchester Met. We talked about how this approach would mitigate the risk of single/bilateral marking groups. We also talked about whether it would mask weaker supervision, because the extreme grades may get removed during the process. Ender and Carmen said that there are now more first class marks than there used to be, but that this may be because of the simultaneous introduction of an analytic rubric.
Our thanks to Carmen and Ender for coming over to present to us and for engaging in a stimulating and robust discussion.