As one of the largest funders of biomedical research, Wellcome has a big influence on the research system and its culture. This influence can be used positively to drive change, for example through our campaign to reimagine research.
However, our own systems can also have unintended consequences – such as creating a focus on outputs and increased productivity at the expense of how research is achieved. This is often underpinned by the funding decisions we make and how we make them.
The problems with peer review
The way funding decisions are currently made in the UK is based on the Haldane Principle, set out in the early 1900s. Under the principle, researchers are best-placed to make decisions on how research is funded (also known as peer review).
From an expertise and independence perspective, this appears to be a compelling argument and one that feels intuitively right. However, there is very little empirical evidence to support it.
Peer review in grant funding is effective at differentiating between good and bad research proposals. It isn’t as effective at teasing apart the really good proposals from the relatively good ones.
Who gets funded is dependent on who makes the decision at the time, meaning the results can sometimes seem to be random at best. And if we look at borderline cases – those who just obtained funding versus those who just missed out – there is no clear relationship between funding outcome and future success.
In some instances, there is no difference in scientific achievement (for example publications, independence and leadership) between researchers who received funding versus those who didn’t. In other cases, researchers who weren’t funded initially went on to do better than those who were.
This raises questions for funders, including:
- how do you effectively choose the ‘best’ research ideas?
- how do you pick the right reviewers?
- are reviewers being asked to make predictions about the future, and are they qualified to do so?
Then there are questions about what reviewers are judging applicants on. For example, reviewers are often asked to select researchers with the potential to make the next big breakthrough, which is extremely difficult to do. They may rely, perhaps unconsciously, on the applicant's previous publication record, rather than future potential.
This type of assessment puts women and early career researchers at a disadvantage – two groups we want to encourage and support in research. In contrast, looking at the project ideas can level the playing field.
So, should Wellcome fund people or ideas? How do we mitigate against (unconscious) bias if we don’t even know what biases we are contending with?
We’re reviewing how we make funding decisions
To be able to answer these and other questions, we need to understand how funding decisions are currently made at Wellcome. This includes who makes them and what drives those decisions. We also need to consider both the explicit decisions and the implicit or ‘unwritten rules’.
We have commissioned three research teams to look at:
- how we make decisions on both response-mode and challenge-led funding
- our end-to-end funding processes
- specific areas such as translation funding and peer review.
We will be interviewing people at all levels of the organisation, as well as engaging with expert advisers.
The findings will help to inform Wellcome’s funding decision-making strategy, and we will share what we've learned in 2020.