This guidance provides information for Wellcome-funded organisations on how to implement responsible and fair approaches for research assessment that meet the expectations set out in Wellcome’s open access policy.
We published a draft version of this guidance for consultation in early 2020 and received more than 50 responses from research organisations and the broader research community. This final version has been revised to take account of the hugely valuable feedback we received.
Our open access policy requires Wellcome-funded organisations to publicly commit to:
We believe that research assessment processes used by research organisations and funders in making recruitment, promotion and funding decisions should embody two core principles (‘the principles’) as set out in the San Francisco Declaration on Research Assessment (DORA)(opens in a new tab):
We recognise that many organisations will implement these core principles as part of a broader approach to promoting the responsible and fair use of metrics in line with other equivalent declarations, such as the Leiden Manifesto(opens in a new tab) and the Hong Kong Principles(opens in a new tab).
Becoming a signatory to DORA, and/or endorsing equivalent declarations is one way that organisations can demonstrate their commitment to implementing the principles, both to their staff and communities.
However, there is no requirement for Wellcome-funded organisations to publicly endorse any declaration. We want organisations to implement responsible and fair approaches for research assessment in line with the two principles set out above, whether that is driven by commitments to one or more of these declarations or otherwise.
Wellcome-funded research organisations should have a:
As part of our regular audits, we may ask organisations to demonstrate that they have met these three high-level requirements.
The rest of this guidance is not intended to be prescriptive or exhaustive. We recognise that organisations will take different approaches to put these principles into practice, that reflect their own values, cultures and ways of working. We encourage organisations to consider other activities (that are not detailed below) where they feel they could add value in their own context, and to trial new ideas and approaches generated by their staff on an ongoing basis.
We also recognise that many organisations have already publicly committed to DORA, or equivalent statements, and have established processes for their implementation. We do not expect organisations to establish new public statements, implementation plans or parallel activities where these are already in place.
Work to implement the principles should be joined-up with the organisation’s broader efforts to enhance research culture and promote research integrity.
We strongly encourage organisations to be transparent and proactive in sharing their approaches and learnings, and to work together to drive change. Wellcome will work in partnership with organisations to develop and share good practice.
If you have developed approaches to promote fair and responsible assessment that could provide valuable learning and inspiration for other research organisations, please let us know and share with other organisations – such as DORA(opens in a new tab) – who are collating good practice examples and case studies.
Organisations must have a clear and easily accessible statement on their externally-facing website, highlighting their commitment to implementing the principles as part of a fair and responsible approach for research assessment.
The statement does not necessarily need to be published on a stand-alone webpage. For example, it could form part of an existing webpage which addresses the organisation’s broader approach to research evaluation, open research or enhancing research culture.
Alongside this statement, organisations are encouraged to:
Organisations must develop a plan for implementing the principles (where such a plan is not already in place) or have a clear process in place for developing a plan with a specified delivery date. This should incorporate elements from each of the subsections below.
The development and delivery of the implementation plan should be resourced sufficiently and have clear ownership and buy-in across the organisation.
Organisations should consider:
Organisations should develop their recruitment, promotion and career-advancement policies and practices in ways that reflect their commitment to the principles.
This might include:
Organisations should make sure that their staff are aware of their organisation’s policy and expectations in relation to the principles, and have the guidance and support needed to implement them.
This might include:
Organisations are required to have a process in place to monitor their progress in implementing the principles and ensure this progress is reviewed over time.
Wellcome will only require organisations to provide assurance that they have such a process in place. We will not require detailed reporting against specific metrics and we recognise that the ability to collect particular types of information will vary between organisations. As noted above, we will also explore how we can bring organisations together to help them develop effective monitoring frameworks and share good practice.
Organisations should consider:
Since we introduced our open access policy in 2005, Wellcome has adopted a clear position that research publications should be assessed on their intrinsic merit, and not on the journal in which they are published.
We were one of the first funders to sign the San Francisco Declaration on Research Assessment (DORA)(opens in a new tab) when it was launched in 2013.
There is a clear and growing consensus over the need to reform research assessment practices to help improve research culture and integrity, which is reflected in the Hong Kong Principles(opens in a new tab), and the need to ensure the metrics that underpin research assessment practices are used in a responsible and appropriate way, as considered in the Metric Tide report(opens in a new tab).
Many funders, universities and other research organisations have either signed DORA, indicated their support for related declarations such as the Leiden Manifesto(opens in a new tab), or otherwise started work to put the principles into practice.
In 2019, the cOAlition S(opens in a new tab) group, of which Wellcome is a member confirmed its commitment to assess research outputs based on their intrinsic merit by making it a core principle of Plan S.
To be successful in driving cultural change, funders and research organisations must work together to reform the assessment processes that underpin decisions relating to research funding, recruitment and career progression. Credible and trusted processes need to be put in place, through which researchers are genuinely judged on the value of their work and not on where they have published. Researchers must also be empowered to challenge situations where practices fail to reflect these standards. Many organisations have already made significant steps towards achieving these goals.
Our open access policy requires the organisations we fund to publicly commit to the two core DORA principles, and that this may be assessed as part of our regular audits from this time.
The cultural change required to embed the principles will take time, and we are committed to working with our funded organisations to drive this change.
If you have any questions about this work, contact Dave Carr.