New Future Research Assessment Programme reports published – UKRI

REF costs

The first report, an in-depth analysis of the costs of the Research Excellence Framework (REF) 2021, was conducted by Technopolis.

The headline cost of REF 2021 was estimated as £471 million. This comprised:

• £17 million direct costs to the funding bodies
• £24 million for the cost of panel member effort
• £430 million in costs to higher education providers (HEPs) in preparing for their submissions

These costs amount to around 3 to 4% of the total funding distributed on the basis of the REF results, which is lower than the equivalent figure for project-based research funding of around 12%.

The cost to HEPs of REF 2021 increased by around 50% compared to REF 2014.

Reasons for the increase

Technopolis found that the increase was mostly because the expansion in the scope of the REF with a much larger number of staff submitted to this exercise. The number of staff submitting to the REF in 2021 was 76,000, as compared to 52,000 in 2014.

They also noted that a change in methodology making it difficult to draw direct comparisons with REF 2014.

The vast majority of community survey respondents believed that the overall benefits of REF 2021 outweigh the overall costs.

Minimising cost

The report makes three key recommendations for minimising the cost of any future exercise which are considered in the ‘Initial Decisions’ document:

• maintain continuity of rules
• issue guidance in a timely manner
• make REF requirements consistent with other UK research policies

We recognise the importance of ensuring that the administrative effort is proportionate and have introduced changes intended to reduce the cost, including:

• drawing data directly from the Higher Education Statistics Agency record
• simplifying or removing the processes for determining circumstances for reduction in output volume
• simplifying the output submission process
• reducing impact case study requirements

Collectively we estimate these measures could amount to nearly £100 million in savings.

Equality, diversity and inclusion (EDI) evidence

Also released today is a report investigating differential outcomes for staff with equality-related characteristics in REF 2021. This report examines outcomes related to the submission of staff, output selection and output scoring.

There is evidence that people with some characteristics were less likely to:

• be included in submissions
• have their outputs included in submissions
• receive higher scores for their output

The report also found variability between panels and that in some cases the small numbers of people with protected characteristics make drawing definitive conclusions impossible.

The findings from these analyses will inform the detailed content and planning for the codes of practice that institutions will be required to produce covering identification of significant responsibility for research and output selection. The findings will also inform panel assessment practices in due course

The engagement summary

The engagement summary report provides an outline of the key themes that have emerged from the range of activities. These include:

• the real-time REF review (understanding perceptions of the REF)
• early sector engagement activities (round table discussions)
• institutional and individual feedback on REF 2021
• consultation on the design of the UK’s future research assessment system
• meetings and other engagements with key stakeholders

Key emerging themes from these engagements align with the key decisions and issues for further consultation.

The reports will be published on Jisc.