SEDA Spring Conference 2017 | Measuring Learning Gain

, , 4 Comments

Measuring Learning Gain: Implications for development, enhancement and evaluation. Keynote speech from Camille Kandiko Howson of King’s College London

Camille opened by citing research carried out in the US by Richard Arum and Josipa Roksa and published in their book Academically Adrift, which found that many students don’t learn much in college, particularly in the way of skills like critical thinking and complex reasoning. She then asked the question: “Where is the evidence showing what students are gaining from Higher Education?” Her view is that the evidence is patchy and that we have to ask: is this partly down to the variability of teaching skills? In theory the Teaching Excellence Framework (TEF) sets out to measure this but Camille expressed the view that the TEF metrics are more of a measure of the student experience than of teaching quality.

Measuring teaching excellence

Measuring teaching excellence is a complex undertaking. In the UK the National Student Survey (NSS) is the closest proxy for learning, however this is a satisfaction survey and therefore it cannot provide a meaningful measure as there isn’t necessarily a correlation between satisfaction and learning.


Learning Gain – the attempt to measure the different ways in which students benefit from their learning experience – is now a core part of the Government’s plans for higher education. Learning gain measures have the potential to provide more robust evidence of teaching excellence, but there are many questions around what is being measured. Camille made the point that any measure depends on a value judgement when defining what learning is important, which is dependent on what you believe the purpose of education is. Is it to produce knowledge workers to benefit the economy? Is it because knowledge and learning are intrinsically valuable? Does education serve more of a civic and democratic function? Interestingly when students are asked this question the majority respond that their main purpose is to gain meaningful employment.

What does learning gain seek to measure?

When trying to pin down what learning gain seeks to measure, Camille set out three main areas:

  1. Affective measures such as self-efficacy and resilience are valued by employers but it is arguable that these are personality characteristics which cannot be taught.
  2. Behavioural measures such as student engagement with teaching, placements and learning platforms.
  3. Cognitive measures such as general cognitive gain, disciplinary cognitive gain, critical reasoning skills, situational judgement and use of research methods.

Grades can be seen as a very narrow measure of education as they don’t necessarily mean that students have the skills and qualities that employers value. The concept of employability can also be critiqued as in many ways it is ill-defined. Does it mean being job ready? Having domain specific professional skills? Transferable skills? Self-efficacy and resilience?

How can we use learning gain measures to better support our students?

Camille concluded with some thoughts on how students might benefit from the use of learning gain measures. Utilising existing student data it is possible to use ‘nudge’ strategies. For example you could send communications to students which highlight successful student behaviours (if you spend x time or take y actions you are more likely to get z outcome). However, this approach is dependent on providing appropriate support services and interventions. In the context of our work in LEaD (our department is comprised of Learning Success, Student Counselling and Mental Health, Academic development and Educational Technology teams) this is possible but not all universities have this structure and even those that do are generally very resource pressured. She also highlighted the role of Technology Enhanced Learning (TEL) through the use of blended and flipped learning approaches and even personalised learning using adaptive learning. There is also a role for learning analytics although this may mean that academics, educational developers and learning technologists need to develop their quantitative and analytical skills.

Find out more

Measuring Learning Gain – Camille Kandiko Howson (SRHE presentation)
HEFCE’s Learning Gain programme

Print Friendly, PDF & Email
 

4 Responses

  1. Dominic Pates

    May 19, 2017 3:06 pm

    Great post – thanks for sharing this one, Thomas. It kind of gets at that evergreen challenge – that of measuring something that is inherently pretty untangible (‘learning’).

    I was interested to see that you’d brought up ‘nudge’ strategies. While there are evitable ethical issues around deploying a ‘nudge theory‘ approach to learning gain (given that it can be seen in some ways as social engineering), I’ve long wondered about the applicability to the classroom – and particularly HE (in this context).

    Do you have any thoughts on that?

    Reply
  2. Thomas Hanley

    May 19, 2017 6:15 pm

    Hi Dom, yes nudge theory is interesting as are the ethical considerations around predictive analytics. This Educause article looks at some of these issues. This is what they say on when a nudge becomes a shove:

    “The business and science of changing behaviors has mixed reviews. To be clear, we should note that nudging, by itself, is generally harmless and noncoercive. However, efforts to move students toward “rational” behaviors — efforts that are often seen as a nudge — can turn questionable when they become more coercive, restrictive, or punitive.”

    Reply

Leave a Reply to Thomas Hanley Cancel reply

Your email address will not be published. Required fields are marked *