On Thursday 24 November 2011, the Student Assessment and Classification Working Group (SACWG) hosted a one-day seminar, ‘The efficiency and effectiveness of assessment in challenging times’ at Woburn House, Tavistock Square, London.
To open the seminar, Dr Marie Stowell (University of Worcester) set out the context for the day in her presentation ‘Efficiency and effectiveness in assessment’. She identified that one of the aims of SACWG is to explore variations in practice across the sector and how they impact differently on students, retention, and learning success, and she observed the importance of placing students at the centre of the process given the fee structure proposed for 2012 entry coupled to the implications of assessment and feedback to student satisfaction. In light of the new funding model, one particularly pertinent observation she made concerned the cost of teaching in relation to the cost of assessment: the latter is resource-heavy, particularly once one factors in elements such as formative assessment (for which quality is less assured than its summative counterpart), moderation, external examining, reassessment of failed components, and the possibility that students may be over-assessed in the first instance. She also suggested that assessment criteria may not warrant the detailed attention they are typically accorded, as students tend to take the more direct approach towards assessment of endeavouring by less formal means to uncover exactly what it is that the lecturer is expecting them to produce. These arguments may indicate that both the efficiency and effectiveness of assessment could usefully be enhanced.
The next talk, by Professor Alison Halstead (Aston University), explored how institutions have responded to the challenges of recent years, specifically, the White Paper and its implications to students and to Higher Education. She noted that the potential increase in the students’ financial burden will inevitably lead to heightened expectations concerning teaching quality, learning, and employability, in which respect assessment and feedback are currently among the most important issues. She warned that student challenges to the regulatory framework for assessment may be on the rise in the future and identified that it was imperative, in these changing times, to nurture outstanding, innovative teachers and for staff to support student learning and e-learning (including assessment). Calling for the abandonment of the rigid distinction often drawn between ‘teachers’ and ‘researchers’, she suggested that promotions should award teaching excellence on a par with research. Later sections of her presentation outlined recent initiatives at Aston, for instance, standardizing the use of Virtual Learning Environment across the institution, and introducing learning technologies such as lecture capture and electronic voting systems. Her view was that teaching-enabled practice, while it took more time upfront to implement, was worth the investment in terms of teaching quality and learning success.
A structured group discussion and question-and-answer session with the morning’s speakers ensued. One point that emerged strongly was the importance of maintaining a variety of assessments, organized in a carefully considered schedule that takes a holistic overview at programme level. The latter becomes much more difficult in degree courses that incorporate elective modules, though there are both pedagogical and satisfaction-related reasons for offering choice to students and giving them ownership of their programme pathway. Another preoccupation amongst delegates was that assessments do not become too atomized, but relate to one another even beyond the confines of the module with which they are associated; one of the more innovative solutions proposed was the possibility of assessments straddling two or more modules. The need to develop sustainable structures was also discussed (for instance, moving towards group assessment to cope with rising student numbers), as was the importance of considering (as part of change management) what the benefits of effecting the change might be; if these cannot be persuasively articulated to staff and students, the change may not be worth implementing. A final warning concerned being too driven by regulations in designing efficient and effective curricula: it may be more useful in the long term to refer obstacles presented by the regulatory framework upwards so that they can be addressed.
The seminar resumed in the afternoon with a talk from Professor Chris Rust (Oxford Brookes University) on ‘Tensions in assessment practice’, which opened by reiterating the themes of the seminar in noting that current practices are neither efficient nor effective. He discussed that students have a tendency to focus on the mark they will obtain from the assessment rather than on the educational content of their studies, and that their approach often becomes increasingly surface-level as they progress through their programme. He defended modes such as formative, self-, and peer assessment as potentially yielding more ‘authentic’ assessment, arguing that graduates should be able to evaluate themselves and their peers as an outcome of their programme, and that making greater use of these options might also free up staff resources for summative assessment. Noting that students do not warm to the notion of being assessed, he suggested that perhaps the word ‘assessment’ should not be used for formative tasks. He further observed that feedback practices might be made more efficient by strengthening the relationship between modules, such that students are encouraged to learn from feedback received in one module and to carry what they have learnt over to others. Lessening the sense of compartmentalization of individual modules would, in his view, lead to more inclusive structures albeit less flexible ones, in that standardization (for instance, in terms of the same word limit for all assessments) does not always result in appropriate assessments.
Then followed a second group workshop session, on the theme of ‘What can institutions do to mitigate tensions?’. After a structured discussion of the issues, each group reported back to the seminar as to the problems that they had identified and the possibilities for efficient or effective solutions. It would be impossible to do justice here to the vast amount of ground covered between the several contributing groups. To cite just a few examples, key tensions that were raised included giving formative assessment a greater purpose (a proposed solution being to tie formative and summative assessments together in more meaningful ways), the problem of ensuring parity when using several examiners for the same assessment task (which may be solved by grading the assessment as pass/fail only), and the evergreen question of quality of feedback versus timeliness of feedback (for which there was some discussion about feedback becoming ‘quick and dirty’). On the question of standardization of process, I took the microphone to report back on the standardized feedback proforma that had been created in liaison with the students and implemented across one programme at City University London (see this post for details), and suggested, with much support from the floor, that students should be more involved in consultation regarding matters of assessment and feedback.
Prior to the close of the seminar a final speaker, Professor Paul Hyland (Bath Spa University), provided some reflections upon the day’s discussion. Noting that assessment was a large topic with which to deal, he categorized the day’s discussion as having crystallized around four main areas: external scrutiny (ranging from students’ parents to formal regulatory bodies); administration and management; the tutors’ perspective on assessment; and the students’ perspective. He argued that discussions of effectiveness and efficiency should always be mindful of the purpose of assessment. In his view, assessment should be concerned with measuring students’ performance and nurturing learning, whereas there exists a danger of (to put it crudely) simply setting assessments in order to get the students to do some work. In this context, a greater level of student involvement and engagement with assessment would therefore be beneficial. He also observed the need to use technology to improve existing practice, for instance, to supplement traditional modes of feedback with video and screen casts. Finally, he commented upon the importance of tutors having access to students’ feedback on previous assessments in order to understand where they are coming from and to be able to support them in their ongoing studies.
SACWG has kindly made available the presentation slideshows used by the speakers, and the comprehensive notes distilled from the two very productive group discussions (as reported back to the seminar by nominees from the groups), at the following link: http://web.anglia.ac.uk/anet/faculties/alss/sacwg.phtml.
