This section summarises some of the most relevant international recommendations. Each recommendation contains a brief description of its background, the relevant parts of the recommendation, especially from the point of view of publication metrics, and a link to the original recommendation, which can be consulted for further information.
Barcelona Declaration on Open Research Information
Barcelona Declaration was published in April 2024 and its agenda is to promote the openness of research information. By signing the Declaration, organizations commit to take a lead in transforming the way research information is used and produced. The declaration contains the following four commitments:
- We will make openness the default for the research information we use and produce.
- We will work with services and systems that support and enable open research information.
- We will support the sustainability of infrastructures for open research information.
- We will support collective action to accelerate the transition to openness of research information.
From the Finnish organizations, the declaration has been signed by University of Turku and the Federation of Finnish Learned Societies (TSV).
The OpenAlex database and the OpenCitations service are examples of open research information; the guide presents both sources in more detail.
Coalition for Advancing Research Assessment (CoARA)
In July 2022, Science Europe, an organisation representing the European research funders, published The Agreement on Reforming Research Assessment, which defines new research assessment practices. The agreement was created as a result of international cooperation initiated by the European Commission.
The Coalition for Advancing Research Assessment (CoARA), an alliance formed by the signatories, promotes the implementation of the agreement. Central to the agreement are practices that advance diversity, openness and equality in research and research assessment. Such practices are seen to improve the quality and impact of science and research.
Nearly 800 organizations has signed the agreement (situation in October 2024), among them many Finnish organizations, including almost all universities, most of the universities of applied sciences, as well as Finnish Union of Docents, Research Council of Finland and The Federation of Finnish Learned Societies (TSV). The entire agreement can be found on CoARA's website.
CoARA's agreement contains four core commitments:
- Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research
- Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators
- Abandon inappropriate uses in research assessment of journal- and publication- based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index
- Avoid the use of rankings of research organisations in research assessment
The core commitments have been supplemented with six supporting commitments:
Commit resources to reforming research assessment as is needed to achieve the organisational changes committed to
- Review and develop research assessment criteria, tools and processes
- Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use
- Exchange practices and experiences to enable mutual learning within and beyond the Coalition
- Communicate progress made on adherence to the Principles and implementation of the Commitments
- Evaluate practices, criteria and tools based on solid evidence and the state-of- the-art in research on research, and make data openly available for evidence gathering and research
CoARA Working Groups and National Chapters
In the spring of 2023, CoARA launched the call, where its member organizations could propose working groups and national chapters to be established to promote responsible evaluation. The working groups function as communities of practice that enable mutual learning and cooperation on selected themes of responsible assessment. Participating members exchange information, learn from each other's experiences, discuss and develop concrete outputs to promote research evaluation and support the implementation of their members' commitments. The duration of the working groups is two years, and all CoARA member organizations can participate in their activities. CoARA's steering group has appointed thirteen working groups in total, including the Working Group on Responsible metrics and indicators.
More information about CoARA Working Groups and instructions for joining the groups can be found on the CoARA webpage: https://coara.eu/coalition/working-groups/
The Finnish national CoARA chapter was established at the beginning of 2024. Almost all Finnish organizations that have signed the CoARA agreement, have joined the Finnish national chapter. The activities of the national chapter is coordinated by the Federation of Finnish Learned Societies. Additional information on the activities of the national chapter and instructions for joining the chapter, can be found on the Finnish National Chapter website at: https://coara.eu/coalition/national-chapters/coara-national-chapter-finland/
San Francisco Declaration on Research Assessment (DORA)
The San Francisco Declaration on Research Assessment is a set of recommendations for improving the quality of evaluation in scholarly research activities. It was developed in December 2012 during the Annual Meeting of the American Society for Cell Biology (ASCB) in San Francisco, California. At the meeting, a group of editors and publishers of scholarly journals addressed the need to improve the ways in which the output of scientific research is evaluated by funding agencies, academic institutions and other parties. The group developed a set of recommendations, which was published in March 2013 and referred to as the DORA declaration (San Francisco Declaration on Research Assessment).
A number of themes run through these recommendations:
- the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment and promotion considerations
- the need to assess research on its own merits rather than on the basis of the journal in which the research is published; and
- the need to capitalise on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles and exploring new indicators of significance and impact).
The signatories of the San Francisco Declaration on Research Assessment promise to support the promotion and adoption of the practices in research evaluation. Several private individuals and organisations from different countries have signed the DORA declaration. DORA is the only international declaration on responsible metrics that requires commitment by signature.
The main DORA recommendations are:
- Methods: Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions or in hiring, promotion or funding decisions.
- Criteria: Be explicit about the criteria used in evaluating the scientific productivity of grant applicants and clearly highlight, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the reputation of the journal in which it was published.
- Data: Be open and transparent by providing data and methods used to calculate all metrics. Provide the data under a licence that allows unrestricted reuse, and provide computational access to data, where possible.
- Diversity: For the purposes of research assessment, consider the value and impact of all research outputs (including datasets and software) in addition to scholarly publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.
The Metric Tide
A review examining the coverage and reliability of a range of quantitative metrics for evaluating and managing the quality of research at both national and organisational levels, launched by David Willetts, the Minister of State for Universities, Science and Cities in the UK in 2014. The review process was also supported by the Higher Education Funding Council for England (HEFCE). The review was conducted by a multidisciplinary steering group of experts in research funding, research policy, publishing, scientometrics and university management and administration. The Metric Tide (PDF) review provides recommendations for various operators. See the original report for more detailed descriptions of the recommendations.
- Supporting the effective leadership, governance and management of research cultures. The recommendations pay attention to the indicators and their definition, the clarity and transparency of the evaluation criteria and the promotion of the principles of responsible publication metrics.
- Improving the data infrastructure that supports research information management. The recommendations emphasise the transparency and openness of the infrastructures, the use of persistent identifiers and the interoperability of the infrastructures.
- Increasing the usefulness of existing data and information sources.
- Using metrics in future research evaluations. On the basis of the recommendations, quantitative evaluation can be used to support quality evaluation.
- Coordinating metrics activity. A common discussion forum, increased funding for science policy and public debate on responsible metrics themes are recommended to promote coordination.
Leiden Manifesto
The Leiden Manifesto for Research Metrics, published in December 2016, presents ten principles that should be applied in research evaluation. The Manifesto arose from the need to address the abuse of research metrics, such as requesting an applicant’s h-index value during the recruitment process, basing promotion decisions on h-index values and on the number of articles published in high-impact journals, allocating research funding or bonuses based on numbers, etc.
The ten principles of the Manifesto were not new to experts in publication metrics, but this was the first time they were brought together. The aim was to draw up a summary of best practices so that researchers could have a practical look at the ways in which they are evaluated and evaluators could responsibly justify the metrics, or indicators, they use.
Ten principles
In this section, the principles are presented very briefly. Specifications for each principle can be found in the original article.
- Quantitative evaluation should support qualitative expert assessment.
- Measure performance against the research missions of the institution, group or researcher.
- Protect excellence in locally relevant research.
- Keep data collection and analytical processes open, transparent and
simple. - Allow those evaluated to verify the data and analyses.
- Account for variation by field in publication and citation practices.
- Base assessment of individual researchers on a qualitative judgement of their portfolio.
- Avoid misplaced concreteness and false precision.
- Recognise the systemic effects of assessment and indicators.
- Scrutinise indicators regularly and update them.
The Hong Kong Principles
The Hong Kong Principles for assessing researchers were formulated and endorsed at the 6th World Conference on Research Integrity, June 2019 in Hong Kong. These principles will help research institutions to minimise perverse incentives that invite to engage in questionable research practices.
The Hong Kong Principles complement the recommendations of the San Francisco Declaration on Research Assessment (DORA) and the Leiden Manifesto on responsible researcher and research evaluation. The principles have been developed with the idea in mind that their implementation could help to assess researchers for career advancement with a focus on behaviours that strengthen research integrity. For more detailed justification and examples can be found on the Hong Kong Principles.
Five principles
- Assess responsible research practices.
- Value complete reporting.
- Reward the practice of open science.
- Acknowledge a broad range of research activities.
- Recognise essential other tasks like peer review and mentoring.
Helsinki Initiative on Multilingualism in Scholarly Communication
The Helsinki Initiative on Multilingualism in Scholarly Communication was established in 2019. It was prepared by the Federation of Finnish Learned Societies (TSV), the Committee for Public Information (TJNK), the Finnish Association for Scholarly Publishing, the Universities Norway (UHR) and the COST action ‘European Network for Research Evaluation in the Social Sciences and the Humanities’ (ENRESSH). The Helsinki Initiative on Multilingualism in Scholarly Communication has several objectives. The aim is to protect multilingualism to keep locally relevant research alive. There is also a need to endorse impact, which is served by the dissemination of research results in your own language. Support is also needed in interacting with society and sharing knowledge beyond the scientific community. In addition to these, it is also encouraged to look after the fragile infrastructure of scholarly communication in national languages.
The signatories of the Helsinki Initiative on Multilingualism in Scholarly Communication support the following recommendations to be adopted by policy-makers, leaders, universities, research institutions, research funders, libraries and researchers:
1. Support dissemination of research results for the full benefit of the society.
- Make sure researchers are merited for disseminating research results beyond the research community and for interacting with heritage, culture and society.
- Make sure equal access to researched knowledge is provided in a variety of languages.
2. Protect national infrastructures for publishing locally relevant research.
- Make sure not-for-profit journals and book publishers have both sufficient resources and the support needed to maintain high standards of quality control and research integrity.
- Make sure national journals and book publishers are safeguarded in their transition to open access.
3. Promote language diversity in research assessment, evaluation, and funding systems.
- Make sure that in the process of expert-based evaluation, high quality research is valued regardless of the publishing language or publication channel.
- Make sure that when metrics-based systems are utilised, journal and book publications in all languages are adequately taken into account.