If you are an academic based in a UK Higher Education Institution then you are probably bracing yourself for the next Research Excellence Framework (REF) submission in 2021. Details of the next REF were revealed by HEFCE at an OpenForum event in Manchester last week which have implications for all those working in development studies based at universities. The REF is the system for assessing the quality of research in UK universities and higher education colleges.
Being based at IDS, which is not part of the REF, you might wonder what I was doing there. As Director of the Impact Initiative which seeks to maximise the impact of almost two hundred UK Economic and Social Research Council (ESRC) and Department for International Development (DFID) funded projects I am deeply interested in the direction the REF goes in. Most of the Principal Investigators we support are based in UK Universities and will be part of the next REF. Will it support the impact agenda that the ESRC DFID partnership promote and provide further incentives to researchers to engage with non-academic audiences? The answer is probably yes but there was still some disappointing and, quite frankly, mystifying announcements to those of us operating at the interface between development research and policy and practice.
What development researchers might like about REF2021
A broader definition of impact
HEFCE staff kicked off the conference by talking about wanting to respond to some of the recommendations of the Stern report, the 2016 review of the REF, by broadening the definition of impact. This sounded encouraging – after all ESRC and DFID impact definitions encompass: (i) building individual and institutional capacity to close the gap between research and policy, (ii) influencing changes in attitudes, evidence use behaviours and ways of thinking about research issues and, of course, (iii) impacts on policy and practice. Meanwhile the REF2014 took a far narrower view dealing in mostly scholarly impacts. Could this give HE institutions with development research faculty the edge in the next REF?
Longer term impacts will be valued
Hooray acknowledgement that impact takes a long time! REF2014 case studies can be revisited if additional impacts have occurred. Though expect some complicated rules around ‘additionality’
Impact is a team sport
There was repeated attempts to reiterate that it is understood that it is not helpful to individualise impact. It is recognised that it is a team effort. They did not go quite as far as to start talking about co-production and networks of academics but the value of such approaches was implied.
The high value placed on inter-disciplinarity
The key example of this used was the potential of the Global Challenges Research Fund (GCRF) to promote these approaches. As I have blogged before, I hope GCRF can live up to these expectations.
A shift in focus from individuals to institutions
This sounded like a move that many in the development sector would recognise. It emphasises the value of long term research partnerships and enduring connections facilitated at an institutional level rather than a pure celebration of the super-star scientist.
What development researchers might not like about REF2021
A broader definition of impact
So it turns out they were teasing us. The broader definition is a much bigger focus on public engagement and impact on teaching in your own institutions. This, of course, is not a new agenda in the HE sector and it is quite understandable why it is deemed important. However, just how this is to be measured and how it links to what the literature and our experience tells us about research to action processes is unclear. Are researchers going to score more brownie points for showcasing research to their local communities and getting high profile media coverage than for involving those directly affected by the research or potential users of it? No one could answer this question. And still no mention of the value of building capacity to deliver engaged scholarship or use evidence in policy and practice.
No shift in focus from individuals to institutions until REF2028
Sorry, it turns out they decided to just pilot some institutional assessments in 2021. I felt sorry for the data analyst who turned up with exciting looking slides showing ways of tracking the impact of institutional level research agendas. Awkward.
A linear relationship between academic research outputs and impact
A member of a REF2014 sub panel informed us that it was now recognised that impact may happen during the research (ok good) and that in development research (we got a mention) impacts may precede the publication of research in journals (yes yes), but went on to apologetically explain that was just tough. If your research does not make the grade in journals (known as 2 star research) you won’t even make it to the next stage of assessment. This renders plenty of the ESRC DFID impact stories we have been working on in the Impact Initiative irrelevant despite evidence to show lives have been improved – even saved. I understand the REF is designed to assess research quality. Nonetheless, if quality partially depends on engagement, REF2021 is going to be problematic for many social scientists who believe in research as development not just for development.
Direct knowledge transfer is best
More of the same here – panels like a nice simple story of knowledge being transferred from academics to those in a position to use it for the benefit of the economy and society. This was helpfully illustrated with a diagram showing a nice big arrow from research to impact. It seems amazing that in 2021 this is the way research impact processes will be conceptualised. How does this relate to the experience of those of us in development (and elsewhere) grappling with the non-linearity and complexity, embracing it even, with some pretty robust methods of monitoring and evaluating impact in complex knowledge systems.
There is more to impact than the REF
Before getting too snooty about the REF it is worth reminding ourselves of the positive impact it has had on UK academia. I have written before about the misconception that UK academics are all stuck in ivory towers. As the Dr Catriona Firth, REF Deputy Manager at HEFCE put it: “REF14 has driven positive behaviours in the UK research sector and has helped demonstrate to government the value of research.” I agree and must admit that without the REF the chances of the Impact Initiative having such a committed bunch of academics to work with would be much slimmer. To be fair HEFCE and the universities presenting information on the next REF were very clear that they understood that there is more to impact than the REF. It was also said that many HE institutions have begun to really embed impact in their support for early career researchers and their systems for monitoring and evaluation. I met a wonderfully inspiring bunch of university impact officers and managers who valiantly attempt to support entire faculty single handed to demonstrate impact.
So by all means dread the paperwork and worry about exactly how REF2021 will be implemented. However, if you work in development studies rest assured that the REF is going to be an enabler of the impact agenda even if it does not turn out to be everything it could have been.
For further details, please see the source article here.