The evaluation of universities, units or individual researchers should be based primarily upon qualitative expert review, but research metrics can be used to support the evaluation. The national recommendations are a continuation of the international debate on researcher evaluation and the responsible use of research metrics, which emphasises that the responsible use of publication metrics is only a part of the overall evaluation process in which different methods complement each other.
Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland.
The Federation of Finnish Learned Societies (TSV) published a recommendation in 2020: Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland. The recommendation is drafted from the perspective of an individual researcher evaluation. The same principles should be applied when evaluating research organisations, research units and research in a wider context.
General principles of researcher evaluation:
- Transparency: the evaluation is carried out as transparently as possible and it is thoroughly documented.
- Integrity: the evaluation must be conducted in accordance with practices recognised by the research community.
- Fairness: all those subject to evaluation must be treated equally and impartially.
- Competence: the evaluators must have the substantive competence and knowledge required for the evaluation process.
- Diversity: the evaluation must take into account the diversity of research and outputs.
Areas of evaluation and good practices:
A. Building the evaluation process: 1) The objectives and criteria of the evaluation must be essential and transparent to all parties; 2) The evidence used in the evaluation must be comprehensive and fair; 3) The selection of evaluators and evaluation guidelines must support the principles of responsible evaluation; 4) Ensuring equality protects against discrimination.
B. Evaluation of research: 5) The evaluation of scientific quality is primarily based on the scientific output of the research and the metrics relevant to the field of study; 6) Activities to promote open access must be considered as part of the evaluation; 7) The evaluation must take into account compliance with the ethical principles of research at all stages of research.
C. Diversity of activities: 8) Teaching and supervisory activities must be seen as an integral part of a researcher’s work; 9) When evaluating societal impact and interaction, it is necessary to define what is meant by them; 10) Researchers’ activities in research communities and other communities must be considered in the evaluation; 11) In relation to the goals of the evaluation, researchers must be evaluated as representatives of their field of research.
D. Researcher’s role in the evaluation process: 12) The researcher must be given an opportunity to express their own understanding of the objectives, significance and effectiveness of their work; 13) The evaluation must be designed to also benefit the researcher and to help them improve their work.
National recommendations on the responsible use of publication metrics
The Recommendations on the responsible use of publication metrics are included in the National recommendation for the responsible evaluation of a researcher in Finland. The Recommendations on the responsible use of publication metrics were drawn up by the National Responsible Metrics Working Group within the TUHA network in 2020. The recommendations are based on international recommendations and adapted to the national context. The recommendations are listed in full below:
Recommendations on the responsible use of publication metrics
- Quantitative indicators can be used to support qualitative peer review of scientific activity. Peer review should be the primary approach for evaluating individual researchers.
- Publication metrics should be based on data that is relevant for the unit of assessment. The known limitations of the data should always be disclosed.
- Being as open and transparent as possible in data collection, analytical processes and results is necessary. Those being evaluated should, as far as possible, be able to check both the data used and the results of the analysis.
- Disciplinary differences and interdisciplinarity should be taken into account in the application of publication metrics.
- The indicators used in assessment should be chosen to support the aims of the evaluation.
- Results should be reported with an accuracy relevant for the unit of assessment, methods and the data. Inapplicable indicators should not be reported.
- Specific expertise is needed in the production and interpretation of publication metrics.
- Organisations committed to this recommendation should provide sufficient resources and expertise needed for producing and interpreting publication metrics. Organisations should offer training for responsible use of publication metrics for their faculty and staff.
- Organisations committed to this recommendation should name the responsible party in their organisation who can be contacted in cases of irresponsible use of publication metrics.
The national recommendation for responsible use of publication metrics (only in Finnish) (pdf)
This recommendation in English is included in the Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland.
Responsible use of the Publication Forum classification
The Publication Forum (JUFO) is a publication channel rating and classification system to support research quality assessment, developed by the Finnish scientific community. User guide for the Publication Forum classification (2019) include the recommendations made by the steering group appointed by the board of the Federation of Finnish Learned Societies for the responsible use of Publication Forum classification as a tool for research assessment.
The Publication Forum classification is suitable for examining the average quality of large volumes of publications at a university level. As the classification is not intended for evaluating the quality of smaller publication volumes or individual publications, nor for evaluating or comparing individual researchers, it has been necessary to describe the rationale behind the classification and to provide guidance on how to use the classification in accordance with the principles of responsible metrics. If the Publication Forum classification is used as a tool for evaluating smaller units or individual researchers, the following things should be considered in the evaluation:
- The evaluation should take into account the limitations on the use of the Publication Forum classification, including level quotas, the range of quality and impact within levels and differences between fields of science.
- In addition to the Publication Forum classification, other research metrics specific to a publication channel or publication should be used in the evaluation as widely as possible, taking into account the differences and characteristics of different fields of science.
- Publication metrics experts should be involved in producing and interpreting the research metrics based on the Publication Forum classification.
- The subject of the evaluation must be told in a transparent manner in what context and how the Publication Forum classification is used.
- Researchers should be consulted on the applicability of the Publication Forum classification for the various evaluation purposes in their field of science or research.
User guide for the Publication Forum classification