Identifying outstanding research proposals: an examination of reviewers’ judgement processes

Research proposals are usually peer reviewed to help funders decide which projects to support. Recent research has revealed inconsistencies and lack of agreement in reviewers’ evaluations, which may bring into question the usefulness of peer reviewing grant proposals. To date, there has been little empirical research on the cognitive processes which underpin scholars’ judgements of grant proposals and the reasons why inconsistencies arise remain unclear.

The aim of this project is to understand how peer reviewers use the information in grant proposals to make recommendations. Our interdisciplinary research team will conduct a mixed-method study of grant reviewers’ judgement processes. A proof-of-principle study in the field of humanities and social sciences, informed by the social judgement theory framework, will be used to determine whether these judgement processes can be statistically modelled. 

If the judgement process can be made explicit, these empirical models could be used to develop evidence-based training to improve reviewers’ judgements and optimise future funding policy and practice.