How the REF’s regime of excellence is changing research for the worse

Ripubblichiamo questo articolo di Nick Butler e Sverre Spoelstra, originalmente apparso su The Conversation.

There can be little doubt that the research environment in universities is changing: it is now less a collegiate community of scholars than a competitive game of winners and losers. This transformation manifests in all sorts of ways, most notably in the rise to prominence of national research audits such as the recent Research Excellence Framework (REF) – the mechanism that provides the backbone for the way UK government spending on science and research in higher education is allocated.

A recent article by two professors at the University of Leicester drew attention to game-playing by business schools in the REF. They point out that submission rates for staff under the Business and Management Studies category differ dramatically between university departments, ranging from over 95% in some cases to less than 5% in others.

The upshot is that a business school that deems the majority of its staff “research inactive” can appear higher in the official rankings than a business school that submits, on principle, all of its academic personnel. Simon Lilley and Martin Parker underscored the absurdity of this situation:

Imagine if a school only submitted data about the pupils who got As, or a hospital could choose not to report death rates, or a local council could mention all the emails they’d sent while ignoring all the bins they didn’t collect. Then you get the basic idea.

Such tactics serve no other purpose than to inflate a department’s “grade point average” and accrue considerable financial and symbolic capital as a result. Similar game-playing around “impact” is now coming to light as the findings of the REF are scrutinised further.

This puts into question some of the assumptions made in the Department for Business, Innovation and Skills’ recent grant letter to the Higher Education Funding Council, which claims that the REF has “demonstrated substantial improvements in the UK’s research quality and delivered compelling evidence of the impact of research”. In fact, what the REF demonstrates is the ability of universities to massage the stats in their favour.

Scholarship in the shadow of excellence

Selectivity doesn’t just apply to the way departments engage with the REF in a competition for funding and resources. It also concerns the way individual academics relate to their own research to secure career advantages and peer esteem.

To a certain extent, research audits like the REF are imposed on scholars by governments and universities. But our recent research shows that senior academics are voluntarily modifying their behaviour in line with the instrumental logic of “research excellence”.

Increasingly, academics are more likely to work on “hot topics” that have a greater chance of being published. By the same token, academics may abandon fruitful research projects that are seen as unsuitable for top-rated outlets. This suggests that supposedly “neutral” and “objective” measures of research quality – such as journal rankings and other metrics – are playing an active role in shaping standards of scholarship, and not always for the better.

In a curious twist, scholars’ subjective view of research quality may differ greatly from official forms of evaluation. For example, one professor we interviewed described his own publication in a leading management journal as “awful” and “disgusting” as a piece of scholarship. Another admitted that a lot of papers in top-ranked management journals are simply not worth reading – his own included. Such behind-the-scenes confessions are a far cry from the bombastic rhetoric of the REF, which in 2014 classified three-quarters of research submitted to the exercise as “world leading” and “internationally excellent”.

How to become less excellent

The emphasis on rankings and league tables creates a climate in which the question “where do you publish?” has become at least as important as the question “what is your research about?” We feel it is time to reevaluate our orientation towards scholarship in a rapidly changing academic environment.

Of course, one solution would be to transform the structures that create incentives for academic behaviour. As a means to allocate university funding on an equitable basis, the REF is riddled with problems. It is little wonder that some have called for the REF to be completely overhauled to minimise institutional game-playing, or even scrapped entirely. But large bureaucracies, such as HEFCE, are notoriously difficult to change. This is why it is equally important to consider the role that individual academics can play in challenging the status quo.

An academic might begin by reflecting on how the managerialist regime of excellence has come to shape their own research interests and personal ambitions. They could then start raising uncomfortable questions about their motivations for working on certain topics, with certain co-authors and for certain journals. Becoming “less excellent” – and hopefully also a better scholar – involves recognising how one’s own scholarly ethos has, over time, become distorted by the external demands of research audits and journal rankings.

Social psychologist Michael Billig provides a striking example of such self-reflection in his 2013 book Learn to Write Badly when he takes himself to task for caring about his own citation rates. With remarkable candour, he admits being proud when his citations rise and insecure when overtaken by his peers: “Do I really care about the numbers? Yes, I must do. What a knob head.”

If more academics were to ruthlessly interrogate their own research values and vices in this way, then we may once again be able to pursue scholarship beyond the tyranny of excellence.

Nick Butler
Sverre Spoelstra

This article was originally published on The Conversation.
Read the original article.