Page History

This section summarises some of the most relevant international recommendations. Each recommendation contains a brief description of its background, the relevant parts of the recommendation and a link to the original recommendation, which can be consulted for further information.

San Francisco Declaration on Research Assessment (DORA)

The San Francisco Declaration on Research Assessment is a set of recommendations for improving the quality of evaluation in scholarly research activities. It was developed in December 2012 during the Annual Meeting of the American Society for Cell Biology (ASCB) in San Francisco, California. At the meeting, a group of editors and publishers of scholarly journals addressed the need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions and other parties. The group developed a set of recommendations, which was published in March 2013 and referred to as the DORA declaration (San Francisco Declaration on Research Assessment). 

A number of themes run through these recommendations:

• the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment and promotion considerations
• the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and
• the need to capitalise on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles and exploring new indicators of significance and impact).

The signatories of the San Francisco Declaration on Research Assessment promise to support the promotion and adoption of the practices in research evaluation. Several private individuals and organisations from different countries have signed the DORA declaration. DORA is the only international declaration on responsible metrics that requires commitment by signature.

The main DORA recommendations are:

• Methods: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions or in hiring, promotion or funding decisions.
• Criteria: Be explicit about the criteria used in evaluating the scientific productivity of grant applicants and clearly highlight, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the reputation of the journal in which it was published.
• Data: Be open and transparent by providing data and methods used to calculate all metrics. Provide the data under a licence that allows unrestricted reuse, and provide computational access to data, where possible.
• Diversity: For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to scholarly publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.

The Metric Tide

A review examining the coverage and reliability of a range of quantitative metrics for evaluating and managing the quality of research at both national and organisational levels, launched by David Willetts, the Minister of State for Universities, Science and Cities in the UK in 2014. The review process was also supported by the Higher Education Funding Council for England (HEFCE). The review was conducted by a multidisciplinary steering group of experts in research funding, research policy, publishing, scientometrics and university management and administration. The Metric Tide (PDF) review provides recommendations for various operators. See the original report for more detailed descriptions of the recommendations.

• Supporting the effective leadership, governance and management of research cultures. The recommendations pay attention to the indicators and their definition, the clarity and transparency of the evaluation criteria and the promotion of the principles of responsible publication metrics.
• Improving the data infrastructure that supports research information management. The recommendations emphasise the transparency and openness of the infrastructures, the use of persistent identifiers and the interoperability of the infrastructures.
• Increasing the usefulness of existing data and information sources.
• Using metrics in future research evaluations. On the basis of the recommendations, quantitative evaluation can be used to support quality evaluation.
• Coordinating metrics activity. A common discussion forum, increased funding for science policy and public debate on responsible metrics themes are recommended to promote coordination.

Leiden Manifesto

The Leiden Manifesto for Research Metrics, published in December 2016, presents ten principles that should be applied in research evaluation. The Manifesto arose from the need to address the abuse of research metrics, such as requesting an applicant’s h-index value during the recruitment process, basing promotion decisions on h-index values and on the number of articles published in high-impact journals, allocating research funding or bonuses based on numbers, etc.

The ten principles of the Manifesto were not new to experts in publication metrics, but this was the first time they were brought together. The aim was to draw up a summary of best practices so that researchers could have a practical look at the ways in which they are evaluated and evaluators could responsibly justify the metrics, or indicators, they use.

Ten principles

In this section, the principles are presented very briefly. Specifications for each principle can be found in the original article.

1. Quantitative evaluation should support qualitative expert assessment. 
2. Measure performance against the research missions of the institution, group or researcher.
3. Protect excellence in locally relevant research.
4. Keep data collection and analytical processes open, transparent and
   simple. 
5. Allow those evaluated to verify the data and analyses.
6. Account for variation by field in publication and citation practices.
7. Base assessment of individual researchers on a qualitative judgement of their portfolio.
8. Avoid misplaced concreteness and false precision.
9. Recognise the systemic effects of assessment and indicators.
10. Scrutinise indicators regularly and update them.

The Hong Kong Principles

The Hong Kong Principles for assessing researchers were formulated and endorsed at the 6th World Conference on Research Integrity, June 2019 in Hong Kong. These principles will help research institutions to minimise perverse incentives that invite to engage in questionable research practices.

The Hong Kong Principles complement the recommendations of the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto on responsible researcher and research evaluation. The principles have been developed with the idea in mind that their implementation could help to assess researchers for career advancement with a focus on behaviours that strengthen research integrity. For more detailed justification and examples can be found on the Hong Kong Principles. 

Five principles

1. Assess responsible research practices.
2. Value complete reporting.
3. Reward the practice of open science.
4. Acknowledge a broad range of research activities.
5. Recognise essential other tasks like peer review and mentoring.

Helsinki Initiative on Multilingualism in Scholarly Communication

The Helsinki Initiative on Multilingualism in Scholarly Communication was established in 2019. It was prepared by the Federation of Finnish Learned Societies (TSV), the Committee for Public Information (TJNK), the Finnish Association for Scholarly Publishing, the Universities Norway (UHR) and the COST action ‘European Network for Research Evaluation in the Social Sciences and the Humanities’ (ENRESSH). The Helsinki Initiative on Multilingualism in Scholarly Communication has several objectives. The aim is to protect multilingualism to keep locally relevant research alive. There is also a need to endorse impact, which is served by the dissemination of research results in your own language. Support is also needed in interacting with society and sharing knowledge beyond the scientific community. In addition to these, it is also encouraged to look after the fragile infrastructure of scholarly communication in national languages.

The signatories of the Helsinki Initiative on Multilingualism in Scholarly Communication support the following recommendations to be adopted by policy-makers, leaders, universities, research institutions, research funders, libraries and researchers:

1. Support dissemination of research results for the full benefit of the society.

• Make sure researchers are merited for disseminating research results beyond the research community and for interacting with heritage, culture and society.
• Make sure equal access to researched knowledge is provided in a variety of languages.

2.  Protect national infrastructures for publishing locally relevant research.

• Make sure not-for-profit journals and book publishers have both sufficient resources and the support needed to maintain high standards of quality control and research integrity.
• Make sure national journals and book publishers are safeguarded in their transition to open access.

3. Promote language diversity in research assessment, evaluation, and funding systems.

• Make sure that in the process of expert-based evaluation, high quality research is valued regardless of the publishing language or publication channel.
• Make sure that when metrics-based systems are utilised, journal and book publications in all languages are adequately taken into account.

Back to top