SEDA Spring Conference 2017 | Measuring Learning Gain

Measuring Learning Gain: Implications for development, enhancement and evaluation. Keynote speech from Camille Kandiko Howson of King’s College London

Camille opened by citing research carried out in the US by Richard Arum and Josipa Roksa and published in their book Academically Adrift, which found that many students don’t learn much in college, particularly in the way of skills like critical thinking and complex reasoning. She then asked the question: “Where is the evidence showing what students are gaining from Higher Education?” Her view is that the evidence is patchy and that we have to ask: is this partly down to the variability of teaching skills? In theory the Teaching Excellence Framework (TEF) sets out to measure this but Camille expressed the view that the TEF metrics are more of a measure of the student experience than of teaching quality.

Measuring teaching excellence

Measuring teaching excellence is a complex undertaking. In the UK the National Student Survey (NSS) is the closest proxy for learning, however this is a satisfaction survey and therefore it cannot provide a meaningful measure as there isn’t necessarily a correlation between satisfaction and learning.

Even if we don’t get learning gain completely right it will still be vastly superior to the poor proxies we use now @cbkandiko #sedaconf

— David Walker (@drdjwalker) May 11, 2017

Learning Gain – the attempt to measure the different ways in which students benefit from their learning experience – is now a core part of the Government’s plans for higher education. Learning gain measures have the potential to provide more robust evidence of teaching excellence, but there are many questions around what is being measured. Camille made the point that any measure depends on a value judgement when defining what learning is important, which is dependent on what you believe the purpose of education is. Is it to produce knowledge workers to benefit the economy? Is it because knowledge and learning are intrinsically valuable? Does education serve more of a civic and democratic function? Interestingly when students are asked this question the majority respond that their main purpose is to gain meaningful employment.

What does learning gain seek to measure?

#learninggain measures shaped by what ‘type’ of learning you are interested in. Lead to mismatched expectations of uni & employers #sedaconf

— Becky Turner (@DrBeckyTurner) May 11, 2017

When trying to pin down what learning gain seeks to measure, Camille set out three main areas:

1. Affective measures such as self-efficacy and resilience are valued by employers but it is arguable that these are personality characteristics which cannot be taught.
2. Behavioural measures such as student engagement with teaching, placements and learning platforms.
3. Cognitive measures such as general cognitive gain, disciplinary cognitive gain, critical reasoning skills, situational judgement and use of research methods.

Grades can be seen as a very narrow measure of education as they don’t necessarily mean that students have the skills and qualities that employers value. The concept of employability can also be critiqued as in many ways it is ill-defined. Does it mean being job ready? Having domain specific professional skills? Transferable skills? Self-efficacy and resilience?

How can we use learning gain measures to better support our students?

Camille concluded with some thoughts on how students might benefit from the use of learning gain measures. Utilising existing student data it is possible to use ‘nudge’ strategies. For example you could send communications to students which highlight successful student behaviours (if you spend x time or take y actions you are more likely to get z outcome). However, this approach is dependent on providing appropriate support services and interventions. In the context of our work in LEaD (our department is comprised of Learning Success, Student Counselling and Mental Health, Academic development and Educational Technology teams) this is possible but not all universities have this structure and even those that do are generally very resource pressured. She also highlighted the role of Technology Enhanced Learning (TEL) through the use of blended and flipped learning approaches and even personalised learning using adaptive learning. There is also a role for learning analytics although this may mean that academics, educational developers and learning technologists need to develop their quantitative and analytical skills.

Personalised learning, pedagogic and curriculum design, institutional enhancement, potential impacts of #learninggain #sedaconf

— Becky Turner (@DrBeckyTurner) May 11, 2017

Find out more

Measuring Learning Gain – Camille Kandiko Howson (SRHE presentation)
HEFCE’s Learning Gain programme