​​Session 1A : Generative AI and Assessment – Challenges and Opportunities​

Dr Simon Hayley (Senior Lecturer in Finance) Faculty of Finance, Bayes Business School

Dr Jane Secker (Senior Lecturer Educational Development) LEaD, City, University of London

Professor Susan Blake (Associate Dean (Education)) The City Law School

Professor Stian Reimers  (Director for Educational Enhancement and Digital Innovation) School of Health and Psychological Sciences, City, University of London

Dr Julie Voce (Senior lecturer and Head of Digital Education) LEaD, City, University of London

[Workshop]

Generative AI makes some traditional types of assessment impractical; It also opens up new types of formative assessment in which students learn by using AI; It is likely to change required educational outcomes, as ability to use AI with critical judgement become a key employment skill.

​​Generative artificial intelligence (AI) tools such as ChatGPT pose a dramatic challenge to traditional forms of assessment (Rudolph, Tan & Tan, 2023; Gamage, Dehideniya, Xu and Tang, 2023). Banning AI use is generally neither practical nor desirable, since using such AI tools effectively and with critical judgement is likely to become a key employment skill. This leaves three broad approaches to assessment design: 

1. ​Restricting access to AI (e.g. with some element of live performance or invigilation); 

1. ​Setting assessment tasks that AI is not yet good at (“outrunning AI”) 

1. ​Working with AI, e.g. allowing students to use AI tools, whilst taking a critical approach to AI-generated output. 

​These approaches form the basis for City’s guidance on assessment design (City, University of London, 2023). Compton (n.d) suggests that the disruption caused by generative AI provides us with an opportunity to rethink assessment, for example by considering process-focussed assessments and alternatives to written assessments. Meanwhile Bali (2023) urges us to think about how we develop critical AI literacies in our students.  

​This workshop will draw upon the experience of developing this guidance, and also on the wide range of innovative approaches that are being developed for different subjects in different parts of the university. This workshop aims to give participants the knowledge they require to adapt their own assessments in effective and innovative ways. ​ 

References

​​Bali, M (2023) What I Mean When I Say Critical AI Literacy. Available at: https://blog.mahabali.me/educational-technology-2/what-i-mean-when-i-say-critical-ai-literacy/  ​ 

​City, University of London (2023). Generative AI: Guidance for Module Design and Assessment. 

​City University, Generative Artificial Intelligence (AI) and Learning and Teaching, available at: https://cityuni.sharepoint.com/sites/cs_CityLearningTeachngPortal/SitePages/generativeai.aspx?CT=1681895842672&OR=OWA-NT&CID=cbfcdc6c-023d-0f22-4841-101e3afe5be4 

​Compton, M. (n.d.), AI = Assessment Innovation, https://growbeyondgrades.org/blog/ai-assessment-innovation 

​Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K. and McGrath, C., 2023. Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices. Assessment & Evaluation in Higher Education, pp.1-13. 

​Gamage, K. A.A. , Dehideniya, S. C.P., Xu, Z. and Tang, X. (2023) ChatGPT and higher education assessments: More opportunities than concerns? Journal of Applied Learning and Teaching, 6(2), pp. 358-369. https://doi.org/10.37074/jalt.2023.6.2.32 

​Mollick, E. and Mollick, L., 2023. Assigning AI: Seven approaches for students, with prompts. arXiv preprint arXiv:2306.10052. 

​Rudolph, J., Tan, S. and Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), pp.342-363. https://doi.org/10.37074/jalt.2023.6.1.9​