Commitment to responsible research assessment: Wellcome and funded organisations

Wellcome believes that research assessment should recognise diverse outputs, practices, and activities to enhance inclusion and maximise the quality and impact of research. Organisations conducting research assessments should also be explicit about their criteria for evaluating research and researcher productivity.

To demonstrate our ongoing commitment to research assessment reform and the sharing of best practices, Wellcome signed the Declaration on Research Assessment (DORA) in 2013 and the Agreement for Reforming Research Assessment in 2023. 

The declaration sets a shared direction for changes in assessment practices for research, researchers, and research-performing organisations. Meanwhile, the agreement includes the principles, commitments, and timeframe for reforms and lays out principles for a coalition of organisations to work together on implementing change. 

We encourage Wellcome-funded organisations to commit, as a minimum, to publicly:

• assessing research outputs and other research contributions based on their intrinsic merit 
• discouraging the inappropriate use of proxies or metrics – such as the title or impact factor of the journal in which the work was published

We recognise that many organisations will implement these core principles as part of a broader approach to promoting the responsible and fair use of metrics. This is in line with DORA or other equivalent declarations, such as the Leiden Manifesto, the Hong Kong Principles or the Agreement for Reforming Research Assessment (CoARA). 

Becoming a signatory to DORA and CoARA or endorsing equivalent declarations is one way organisations can demonstrate their commitment to implementing the principles with their staff and in communities.  

However, there is no requirement for Wellcome-funded organisations to endorse any declaration. We want organisations to publicly implement responsible and fair approaches for research assessment in line with the two principles set out above, whether that is driven by commitments to one or more of these declarations or otherwise.  

Core Expectations

Wellcome-funded research organisations should have a:

• statement of commitment to implementing the principles on their website – this should be clear and accessible
• plan for implementing the principles or a transparent process in place for developing a plan (with a specified delivery date) 
• process in place for monitoring and reporting on implementing the principles

The rest of this guidance is not intended to be prescriptive or exhaustive. We recognise that organisations will take different approaches to practice these principles, reflecting their values, cultures, and ways of working. We encourage organisations to consider other activities where they feel they could add value and to try new ideas and approaches generated by their staff.

We also recognise that many organisations have already publicly committed to DORA, or equivalent statements, and have established processes for their implementation. We do not expect organisations to establish new public statements, implementation plans or parallel activities where these are already in place.

Work to implement the principles should be joined-up with the organisation’s broader efforts to enhance research culture and promote research integrity.  

We strongly encourage organisations to be transparent and proactive in sharing their approaches and learning, and to work together to drive change. We also support partnership and the sharing of good practice.

If you have developed approaches to promote fair and responsible assessment that could provide valuable learning and inspiration for other research organisations, please share with other organisations – such as DORA or CoARA – who are collating good practice examples and case studies.

Here is some suggested guidance for implementing DORA or other equivalent declarations. 

Statement of commitment

Organisations should consider having a clear and easily accessible statement on their website, highlighting their commitment to implementing the principles as part of a fair and responsible approach for research assessment.

The statement does not necessarily need to be published on a standalone webpage. For example, it could form part of an existing webpage which addresses the organisation’s broader approach to research evaluation, open research or enhancing research culture.  

Alongside this statement, organisations are encouraged to:

• specify how the principles are aligned to the organisation’s overarching vision, mission and values 
• be explicit on whether they:
  • have signed up to or endorsed DORA, and/or equivalent declarations
  • are actively considering signing up to one or more of these declarations
  • have decided not to sign up to a declaration but will adopt equivalent or enhanced principles and procedures
• explain the key steps they are taking to implement and embed the principles, linking through to relevant policies, guidance, and the implementation plan
• provide detailed materials, guidance and resources to staff through an intranet as appropriate (while considering making these publicly available wherever possible as exemplars of good practice)

Implementation plan

Organisations should develop a plan for implementing the principles (where a plan is not already in place) or have a clear process in place for developing a plan with a specified delivery date. This should incorporate elements from each of the subsections below.

Strategy and leadership

The development and delivery of the implementation plan should be resourced sufficiently and have clear ownership and acceptance across the organisation.  

Organisations should consider:

• making sure one or more senior leaders takes ownership for championing the principles and their implementation as part of broader efforts to enhance the research culture
• establishing a cross-organisation working group with a suitably diverse and representative membership – including across research disciplines and career levels – to develop the implementation plan and oversee its delivery
• seeking input and feedback from staff across the organisation during the development and application of the implementation plan, where appropriate
• designating one or more staff members to deliver the implementation plan as a key part of their role
• establishing a way for staff to report confidentially if the principles have been breached, and for appropriate remedial action to be taken by senior leaders (which may be through an existing whistleblowing process)

Hiring and promotion practices

Organisations should develop their recruitment, promotion and career-advancement policies and practices in ways that reflect their commitment to the principles.

This might include:

• making sure that the criteria used for recruitment promotion and other decisions on career advancement are clear and transparent, and specifically reference the principles
• developing clear guidance for staff involved in recruitment and promotion decisions which:
  • explicitly caution against the inappropriate use of publication metrics
  • encourage them to value a full and diverse range of research outputs and contributions
• encouraging candidates to highlight a broad range of research outputs and other contributions, in addition to publications
• asking candidates to highlight a limited number of key achievements (which may be based on one or several outputs), and to provide a narrative description of their significance and the role they played in the research
• prohibiting the use of language in job advertisements which refers directly or indirectly to journal title as a proxy for quality (for example ‘a track record of publication in leading journals’)
• discouraging the inappropriate use of lists of target journals for researchers that are based on perceived prestige associated with the journals concerned

Communication, advocacy and raising awareness

Organisations should make sure that their staff are aware of their organisation’s policy and expectations in relation to the principles, and have the guidance and support needed to implement them.

This might include:

• highlighting ongoing work through the organisation’s communication channels – such as websites, newsletters and social media
• running seminars or workshops to engage staff across the organisation about the principles, and responsible and fair research assessment more broadly (this could highlight the associated opportunities and challenges)
• incorporating information about the principles in staff inductions and in specific training for group leaders and others involved in recruiting and promotion
• identifying ‘champions’ across the organisation who can act as advocates of good practice and work with colleagues to identify and address any challenges and concerns

Approach for monitoring and sharing learning

Organisations should have a process in place to monitor their progress in implementing the principles and ensure this progress is reviewed over time.

Organisations should consider:

• gathering quantitative and qualitative information (for example, on hiring and promotion practices and on wider attitudes and practices among staff) to help monitor and report on progress against an initial baseline
• reviewing and adapting their implementation plan on a regular basis, in line with lessons learned and emerging good practice 
• sharing their experiences with other research organisations and the wider research community (for example through DORA and other bodies) including both successful and unsuccessful approaches

This guidance was first published as a draft for consultation in early 2020 and received more than 50 responses from research organisations and the broader research community. This final version has been revised to take account of the hugely valuable feedback we received and to include mention of CoARA. 

See our summary of the key issues raised in the consultation and how we have responded to these points in the final guidance.

Looking to the future

Areas of interest for Wellcome’s future alignment with the agreement commitments include, but are not limited to, the following: 

• evaluation of the narrative CV 
• enhancing the transparency of peer review 
• investing in bespoke teams for research culture, open research, bioethics, and engagement, as well as improving the grant funding data we collect 
• providing committee members with equity, diversity and inclusion training