| Info | ||
|---|---|---|
| ||
Click on the |
| Table of Contents |
|---|
1 Introduction
Publication Forum manual for panels contains information on the Publication Forum, scientific publications and publication channels, classification criteria, principles of the evaluation, and panels' work.
...
The Publication Forum established in 2010 under the auspices of the Federation of Finnish Learned Societies (TSV) is a system to support the quality assessment of scientific publication activities in all disciplines, based on the classification of domestic and foreign publication series and book publishers. The rating decisions are taken by the over 200 expert members of the 23 Evaluation Panels covering various disciplines, universities and research institutes. The Panel members are nominated on basis of proposals made by the universities, government research institutes, learned societies and science academies.
...
...
The publication forum rating serves as an indicator of the quality of university publication production in the ministry of education and culture’s funding model from 2015 onwards.Combined with publication data collected by MinEdu, the classification provides a comprehensive and comparable information on the volume and development of the publishing activities of organizations and disciplines. The classification disseminates information on the quality, impact and prestige of scientific journals, series and book publishers in the scientific community, and thus encourages Finnish researchers to publish their most important research in the leading national and international scientific publications channels in their field.
...
...
...
...
...
A ‘scientific publication channel’ refers to printed and digital publication series as well as to book publishers specialised in the publication of scientific research outcomes: they have an editorial board constituted by experts, and the publication calls for quality evaluation recognised by the scientific community, above all referee/peer evaluation. Scientific publication channels are evaluated primarily in terms of their scientific impact, not their societal impact. Scientific publication channels may also produce not-peer-reviewed publications, such as comments, reviews and handbooks. In Publication Forum publication channels are divided in three types that are journals/series, conferences and book publishers.
...
...
...
Book publishers are listed under the name in the ISBN registry or under other established name. In the Publication Forum and the data collection the “book publisher” means a publisher who is responsible for the publisher's ISBN (not other kind of publisher or printing house). Publishers who use self-publishing ISBNs are not primarily treated as book publishers. The imprint of the parent company is classified as an independent publication channel. Publisher’s activities in different countries are considered to belong to the same publication channel. If the faculties or institutes have their own ISBNs, they are considered the same publication channel as the university.
...
...
...
...
Publication Forum does not classify channels targeted at professional or general audiences. If the boundary is not clear, channels for the professional and the general public can be evaluated by the panels. If the panels find that such channels are not scientific publication series or book publishers, they will be transferred to a separate list of professional and general publication channels.
...
In computer science publishing channels have been set up for publishing open source software and descriptive articles. Journals specializing in articles describing software are normally processed for panel evaluation.
...
...
The journals/series and conferences are divided between the panels according to the discipline, so that the responsibility for evaluating each publication channel is limited to one panel. The exception to this rule is a small number of multidisciplinary journals (e.g. Nature, Science and PNAS). These multidisciplinary publications series, as well as most of the book publishers, have been placed on a panel of general publication channels (panel 24), which is common to all panels. All panels can participate in evaluation of the channels placed in Panel 24.
...
The Publication Forum includes active publication channels that specialize in publishing of research results and are relevant from the point of view of Finnish research. The classification thus covers the leading and internationally renowned journals/series and book publishers of the various disciplines as widely as possible, but national and local publication channels in different countries can be limited to those used by or relevant to Finnish researchers.
...
...
• The channel is specialised in the publication of scientific research outcomes;
• There is an editorial board constituted by experts;
• The scientific publications are subject to a peer evaluation focusing on the entire manuscript.
In principle, a publication channel meeting the criteria of a scientific publication channel must not be included in level 1 if
• over half (1/2) of the reviewers or authors come from the same research organisation that is also the publisher;
• the scientific level or relevance raises questions.
...
...
Level 1 channels must have an editorial board including specialists in the field working in universities or research institutes.
Peer Review refers to a procedure whereby a journal, conference or book publisher invites experts in the field to make an evaluation of the scientific merits and fitness for publication of manuscripts offered for publication. The peer review should focus on the entire manuscript to be published, not just of the abstract.
...
...
...
...
...
...
The publishing model based on author fees has increased the number of predatory journals and book publishers. Characteristic features of the predator include low-quality webpages, fast processing time of manuscripts, vague topic, aggressive email marketing and lack of contact information. Author fees alone do not make the journal suspicious, but for all kinds of fraud attested in the amount of fees, in the composition of the editorial board, database indexing and impact factor information or the peer review implementation is a good reason to leave the journal in level 0.
...
...
...
Level 2 can be awarded to leading scientific publication channels of the various disciplines that meet the following criteria:
...
• researchers from different countries seek to publish their best results
• editors, authors and readers represent various nationalities.
All publication channels meeting these criteria cannot be rated as level 2 but the Panels must choose, within their own level 2 quota (see below), the publication channels attracting and selecting the highest quality publications as a consequence of extensive competition and demanding peer reviews. Finnish channels aimed at international audiences are evaluated in relation to other international publication channels.
...
In humanities and social sciences (Panels 14, 16-23), level 2 can also include leading Finnish or Swedish-language publication channels having the widest coverage of the research on Finnish society, culture or history in their field. Due to the lack of citation data, the scientific impact of the domestic publication channels cannot be measured but the level 2 publication channels need to meet the following criteria:
• The quality assessment of the scientific writings must be in line with the best practices;
• The publication series must be among the ones that cover the research in the respective discipline – and the book publishers in their main discipline – most comprehensively and be used by the entire national scientific community in that particular discipline;
• The context of the research problems is strongly focused on the Finnish society or the Finnish or Swedish-speaking culture.
• Publishing in these channels is regarded as high merit as publishing in foreign level 2 channels.
...
...
...
...
Level 3 is the top category of the Publication Forum classification. Level 3 is a sub-category of the level 2, so level 3 channels need also to meet the level 2 criteria. Level 3 can include the topmost publication series of various disciplines meeting also the following criteria:
• The research published in them represents the highest level in the discipline and has very high impact (e.g., as measured through citation indicators);
• The series cover the discipline comprehensively, not limiting to the discussion of narrow special themes;
• Both the authors and readers are international and the editorial boards are constituted by the leading researcher in the field;
• Publication in these journals and series is highly appreciated among the international research community of the field.
The Panels will prepare a proposal of publication series to be rated as level 3, and the Panel Chairs take consensus decisions to include publication series in the level 3 category.
...
...
...
4.5.1 Counting the publication volume
...
The panels need to have a discussion about the importance of impact factors, as well as the Norwegian and Danish ratings, in the evaluation of publication channels in their field. There also needs to be a discussion about how to deal with review journals in the panel and how open access is taken into account in the assessment.
...
...
...
Various impact factors are available for the journals indexed in the Web of Science or Scopus databases, most importantly Journal Impact Factor (JIF), CiteScore, Source Normalized Impact per Paper (SNIP), and SCImago Journal Rank (SJR). Common to these indicators is that they try to measure the impact of publication channels based on the number of references they have received in their publications. The basic assumption is that the larger the number of citations to publications in a journal is, the greater its impact on the international scientific community. In the use of the impact factors, it should be noted that they are not comparable between different disciplines.
...
...
...
...
...
...
...
...
Open access journals are accepted to the rating and are evaluated on the same criteria as traditional publication series. For the levels 2 and 3, the aim is to promote the open science in the following way: Green OA channel can be favored if the embargo is at most 6 months in natural sciences and engineering, and 12 months in SSH. Also open data can be taken into account.”
• if candidates to the level 2 or 3 in the same field have equal impact or prestige, open access journal or the one permitting self-archiving of the peer-reviewed version of the manuscript is chosen to the higher level over the channel that does not support open access.
• the channel allowing for self-archiving can be favoured in comparison if the embargo period is in science and technology for six months and in human sciences for 12 months.
...
...
...
Channels permitting self-archiving (green open access) are traditional subscription-paid journals that allow articles to be stored, for example, in the University's open publication archive. The SHERPA/RoMEO service, which is integrated into the Publish Forum database, can be used to find a self-archiving policy. In general, the color code alone does not adequately describe the policy, but the journal specific information must be checked from SHERPA / RoMEO.
...
...
...
The panels are wide-ranging, so they have to be responsible for fair representation of channels from different fields in level 2 and 3. The panels need to look at the journals/series and conferences of each of the research disciplines as a whole in the disciplines of the Web of Science, Scopus, Norway, Denmark and ERIH Plus, or other groupings. Considerations should also be given to those journals belonging to the same category of these disciplines that are placed in the lists of other panels. The level 2 and 3 shares of publication volumes are monitored within disciplines, so that special attention can be paid to areas with significant under- or over-representation on the levels 2 and .
...
...
All channels regarded as leading by researchers do not have a level 2 or 3 rating, or the rating of the journals does not fully correspond to the impact factor ranking order of a particular discipline. Often, this is due to the structure of the rating, for example, the level quotas or the classification criteria. However, the evaluation work of the panel of experts in the publication forum should be justified and consistent, and take into account ethical considerations, thereby increasing the transparency and reliability of the rating.
...
...
The panels are required to produce more precise justification of what level 1 criteria the channels rated at level 0 do not meet (see 3.2). For Levels 2 and 3, panels need to produce on a general level the principles and criteria on basis of which the classification is made. In addition, the aim is to record channel-specific justifications for decisions that significantly deviate from the impact factors or the Norway and Denmark's ratings of expert panels:
...
• If the channel receives a higher level than another channel representing the same field of research, the impact of which in the international scientific community is significantly higher on the basis of the indicators.
The members of the panels are representatives of the national research community in their field of expertise. This means that panelists are not only guardians of interests of their own university / unit or a particular publishing channel (publisher or journal).
...
...
Panelists must announce their engagements each time a panel discusses evaluation of channels linked to them. This applies to publishing channels where the panelists have published more than once in the past 5 years or have been journalists or members of the Editorial Board. In these cases, the panelist will not be excluded from the decision-making, but the panel can draw attention to the engagements in the assessment situation. The Secretariat will compile information on the engagements.
...
...
...
The progress of the review of ratings requires panel meetings, in which the evaluation criteria and changes to the level 2 and 3 classification can be discussed. The reviewed rating is validated through the JUFO portal.
...
In the JUFO portal, panelists can view information about the publishing channels that are being evaluated by the panel, make proposals about the level of publication channels, and make notes about publishing channels. Panelists can also download their channel list as an Excel file.
...
In those years when an review or ratings is not carried out, the panels meet once a year to address feedback and suggested corrections to the ratings. Individual changes to Levels 2 and 3 can be made according to the current volume situation. That is, if a free level 2 and 3 quota has been created dut to addition of level 1 journals, it may be used for upgrades.
...
If there is a need to change or complement the panel composition during the term of office, the Secretariat and the Chair of the Steering Group will prepare a presentation to the Steering Group on possible panelists. Depending on the timetable, the Steering Group appoints panel extension members either at their meeting or at a separate e-mail meeting.
7 APPENDIX 1: Evaluation timeline for the term 2018-2021
...
Complementary evaluation
...
January-
February
...
1st panel meeting
- organization of the panels
- introduction to panel work
- Publication Forum and evaluation criteria
- panel fields and representativeness
- level 2 and 3 quotas
- JUFO-portal
- timetable
...
Steering-group meeting
- changes to panel compositions
- Top-10 indicator
- other issues
...
2nd panel meeting
- preliminary proposal for the level 2 and 3 ratings
Meeting of SSH panel chairs
- preliminary proposal for level 2 national language channels
...
Steering-group meeting
- examination of the preliminary propsal and feedback
- feedback to panels
...
September-
October
...
3rd panel meeting
- examination of the feedback
- final proposal for the level 2 and 3 ratings
...
Meeting of SSH panel chairs
- final proposal for level 2 national language channels
Meeting of all panel chairs
- final proposal for level 3 journals, and level 2 and 3 for the panel 24 channels (journals/series and book publishers)
...
Steering-group meeting
- confirming the classification
...
2019
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...
2020
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...
2021
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...