Print This PostI’ve now been in my permanent academic position for three years. That means I’m in my third cycle of applying for research funding. Every year, I’ve applied for several small grants (<£50k) and one large grant (>£300k), with varying degrees of success. I’ve also reviewed grants for major research funders. It’s with this experience that I’ve concluded that a strong case can be made for reforming the research funding process. In this blog, I’ll explain why.
Before considering why the research funding process may need reforming, and what this might actually look like, it’s important to first understand how research funding is currently allocated. The process varies by funder and by scheme, but for your typical project grant most applications for research funding will initially go through a triage stage where a team within the funding organisation will review the applications to ensure they meet the eligibility criteria, and the remit of the funder or particular scheme. Sometimes full applications will be reviewed at this triage stage, but other times funders may just request an outline to save applicants spending significant time writing an application that is not eligible or within remit. Full applications are then invited from those who pass triage.
The next stage involves applications being sent for external peer review. Reviewers are asked to comment on the different sections of the application and give it an overall score on a graded scale. For example, the Biotechnology and Biological Sciences Research Council (BBSRC) use a scale of 1-6, where a score of ‘1’ would be ‘unfundable – work that is of no significant scientific merit, flawed, or duplicative of other research, or for which the applicants do not present evidence of a satisfactory track record, and which does not meet the majority of assessment criteria to an adequate level. Unlikely to advance the field’. A score of ‘6’ would be ‘exceptional – work that is at the leading edge internationally, addresses all of the assessment criteria, and meets the majority of them to an exceptional level. Likely to have a significant impact on the field’.
Following external review, funding may be distributed based on these scores, or the reviews may be sent to the applicants for rebuttal. A panel then provides further review, where an allocated panel member, informed by the reviewer’s comments, will lead a discussion about the application before further scoring takes place. This allows final funding allocation decisions to be made.
Peer review isn’t just used in the research funding process. You’re probably most familiar with it for selecting scientific papers to publish. It’s also a process used in decisions about academic promotions, conference presentations, and scientific awards. All these cases involve scientific experts judging and assessing the quality of the work of other scientists, and they receive no recognition or remuneration in return. As a result, the burden of peer review is becoming increasingly problematic, and the cost of this is arguably highest when the process is necessary for funding decisions to be made.
Aside from the challenges in recruiting enough peer reviewers, there are questions around whether reviewers really can be objective when assessing the scientific merit of their peers. The process of peer review is inherently subjective, which is at odds with the strive for objectivity in science. Research has shown that a number of factors leave the research funding process vulnerable to subjectivity and bias. Common biases include those against protected characteristics, such as age, sex, and ethnicity of the applicant, but also includes the research institution they’re affiliated to. Subjectivity and bias are seen most acutely in cases where funding decisions have felt largely dependent on which panel member was allocated to present the application to the rest of the panel. Research supports this, showing there’s significant variation in funding decisions, depending on which panel member is appointed to present a particular application. I’ve found myself saying “I hope we get someone good on the panel” simply because if one of my applications was to fail, I want it to fail because it wasn’t good enough, not because it wasn’t given a fair chance.
Research funders often need to justify their spending to Treasury officials with spreadsheets full of projected health savings. So part of getting dementia research funded is showing it’s cheaper to prevent memory loss than to pay for long-term care.
There’s also a negative bias where a single negative review carries more weight with a panel compared to multiple positive reviews, even when the sole negative review has been well-rebutted by the applicants. Evidence suggests reviewers and panel members are more likely to favour applications which support their own scientific views, known as cognitive particularism. In fact, reviewer and panellist bias is believed to be greatest towards the most innovative applications; those which challenge existing views in science. This anti-innovation bias stems from funding organisations becoming increasingly risk averse, as lower funding availability means they have to be more selective of what projects they decide to fund. As a consequence, the more innovative applications are more likely to receive mixed reviews and scores. In a highly competitive process with limited funding, innovative applications are therefore not likely to reach the threshold for funding.
Obtaining research funding is a determining factor in future funding success, and with grant income being commonly tied to academic promotion, researchers increasingly feel the pressure to submit multiple grant applications, choosing quantity over quality. Moreover, when assessing applicant track record, reviewers and panel members place significant emphasis on measures such as the number of citations, papers, and impact factor of journals where these papers are published. This implicitly rewards bad scientific practice which encourages researchers to publish as much as possible, with little regard for quality or risk of academic misconduct.
So if we’ve established that the research funding process is highly subjective, decisions are vulnerable to bias, and the process may be actively encouraging bad practice, what alternatives are available? Firstly, an easy solution to tackling potential conscious or unconscious bias against protected characteristics would be to blind the applications. Some funders have already taken steps to implement this.
An approach which has been gaining popularity, with some funders as early adopters, is to use partial randomisation. This is where the typical peer review process still occurs, but the applications which reach a certain threshold of quality are allocated funding at random. The rationale for this approach is that all applications are of a high enough quality to be funded if enough money was available. The subjective decision making by the panel on which projects are to be funded is therefore removed.
A more radical approach which has been proposed, would involve completely eliminating the peer review process, and instead research funding would be allocated equally amongst researchers. This approach would solve the problem of bias and the significant time spent on writing and assessing grant applications. However, this would create additional problems where public and charity funds could be wasted on low quality research, and there wouldn’t be enough funding for researchers who conduct expensive research, for example, if they require access to costly equipment.
It’s clear there are some fundamental problems with the research funding process, which actively disadvantages minoritized academics and early career researchers in particular. With every grant application I submit, I know it’s likely to fail.
Each time I become more demotivated and less ambitious. Scientific research shouldn’t be like that.
There’s no perfect alternative, but there’s scope to make small adjustments that collectively may help to minimise some of the issues I’ve discussed. However, one under-explored area which I think has the potential for positive impact, would be for current approaches in allocating research funding to incorporate ways of rewarding good research practice, such as data sharing and transparent reporting, for example. Research funding can be an effective incentive and tying it to measures of good research practice could help eliminate some of the issues with the process I’ve discussed, whilst also improving how science is conducted more generally.
Dr Kamar Ameen-Ali
Author
Dr Kamar Ameen-Ali is a Lecturer in Biomedical Science at Teesside University & Affiliate Researcher at Glasgow University. In addition to teaching, Kamar is exploring how neuroinflammation following traumatic brain injury contributes to the progression of neurodegenerative diseases that lead to dementia. Having first pursued a career as an NHS Psychologist, Kamar went back to University in Durham to look at rodent behavioural tasks to completed her PhD, and then worked as a regional Programme Manager for NC3Rs.
Dementia Researcher
Rebecca Williams
Dr Becky Carlyle
Ajantha Abey