| Info | ||
|---|---|---|
| ||
Click on the |
| Table of Contents |
|---|
1 Introduction
Publication Forum manual for panels contains information on the Publication Forum, scientific publications and publication channels, classification criteria, principles of the evaluation, and panels' work.
...
...
...
The Panels are appointed and the evaluation work supervised by the Publication Forum Steering Group, nominated by the TSV Board and constituted by representatives of the Ministry of Education and Culture (MinEdu), the Finnish Council of University Rectors (UNIFI), the Academy of Finland, the Council for Finnish University Libraries (SYN), the National Library of Finland, the IT Centre for Science (CSC) as well as various major disciplines. The work of the Steering Group and the Evaluation Panels is prepared and supported by the Publication Forum Secretariat employed by TSV which also provides the Panels with meeting facilities at the House of Science and Letters in Helsinki. The IT services required by the Publication Forum are run in collaboration with CSC.
...
...
...
...
...
...
...
...
A publication series can be a scientific journal, book series or a series of proceedings of a regularly organised conference. Journals, book series and proceedings with ISSN are classified using the title recorded in the international ISSN centre. The main rule is that the book series of Finnish book publishers are classified separately in the Publication Forum, while the book series of foreign book publishers are not classified.
...
Book publishers are listed under the name in the ISBN registry or under other established name. In the Publication Forum and the data collection the “book publisher” means a publisher who is responsible for the publisher's ISBN (not other kind of publisher or printing house). Publishers who use self-publishing ISBNs are not primarily treated as book publishers. The imprint of the parent company is classified as an independent publication channel. Publisher’s activities in different countries are considered to belong to the same publication channel. If the faculties or institutes have their own ISBNs, they are considered the same publication channel as the university.
As a rule, publication channels will not be evaluated by the panels if publication activities are less than one year old, or if there are not enough issues or publications in the channel that allow content to be evaluated.
...
Publication Forum does not classify channels targeted at professional or general audiences. If the boundary is not clear, channels for the professional and the general public can be evaluated by the panels. If the panels find that such channels are not scientific publication series or book publishers, they will be transferred to a separate list of professional and general publication channels.
Nature and Elsevier have started publishing channels to support open access and reuse of research data. In the articles, the researchers describe the published data, including the methods and technical analyzes used for generating/collecting the data. Journals publishing only data descriptions are placed directly in level 0 without evaluation, but the development of the status of these channels is monitored. Journals publishing only data are normally processed for panel evaluation. However, the panels must assess carefully that journals fulfil the level 1 criteria (specialization on publishing research results and peer-review).
...
International publishers, for example, Palgrave Macillan and Routledge, have launched a new peer-reviewed publication format, the publication type of which corresponds to a monograph but the length of manuscripts is between an article and a traditional monograph. In the MinEdu data collection short monographs are instructed to be classified as book articles. If the publication channel for short monographs can be identified with an ISSN, it can be evaluated in the Publish Forum.
...
The journals/series and conferences are divided between the panels according to the discipline, so that the responsibility for evaluating each publication channel is limited to one panel. The exception to this rule is a small number of multidisciplinary journals (e.g. Nature, Science and PNAS). These multidisciplinary publications series, as well as most of the book publishers, have been placed on a panel of general publication channels (panel 24), which is common to all panels. All panels can participate in evaluation of the channels placed in Panel 24.
...
...
...
...
• The channel is specialised in the publication of scientific research outcomes;
• There is an editorial board constituted by experts;
• The scientific publications are subject to a peer evaluation focusing on the entire manuscript.
...
...
• the scientific level or relevance raises questions.
...
To level 1 can be accepted channels that publish peer-reviewed publications on a regular basis. Channels that occasionally publish individual scientific publications will not be accepted for level 1. Scientific publishing channels may also include not-peer-reviewed publications for both the professionals and the general public. However, it is recommended that the classification channels that are acceptable to the level 1 indicate as clearly as possible which publications are peerreviewed.
Level 1 channels must have an editorial board including specialists in the field working in universities or research institutes.
...
...
Only such channels can be approved for Level 1, in which researchers from different research organizations publish their research results. The local publishing channel refers in particular to the publications and dissertation series of universities and research institutes.
The panels do not have to accept channels formally satisfying the level 1 criteria if the scientific quality or relevance is questionable from the point of view of Finnish research. For this reason, e.g. questionable (“predatory”) open access journals or other channels that publish scientific papers for payment without proper quality evaluation can be placed on level 0.
...
...
Level 2 is the leading category of the Publication Forum classification, which contains only a very limited group of peer-reviewed publication channels. Level 2 can be awarded mainly to the leading international publication channels, but in the social sciences and humanities also leading national language channels.
Level 2 can be awarded to leading scientific publication channels of the various disciplines that meet the following criteria:
• has a wide reach and respect among the international experts in the field
• researchers from different countries seek to publish their best results
• editors, authors and readers represent various nationalities.
All publication channels meeting these criteria cannot be rated as level 2 but the Panels must choose, within their own level 2 quota (see below), the publication channels attracting and selecting the highest quality publications as a consequence of extensive competition and demanding peer reviews. Finnish channels aimed at international audiences are evaluated in relation to other international publication channels.
In humanities and social sciences (Panels 14, 16-23), level 2 can also include leading Finnish or Swedish-language publication channels having the widest coverage of the research on Finnish society, culture or history in their field. Due to the lack of citation data, the scientific impact of the domestic publication channels cannot be measured but the level 2 publication channels need to meet the following criteria:
• The quality assessment of the scientific writings must be in line with the best practices;
• The publication series must be among the ones that cover the research in the respective discipline – and the book publishers in their main discipline – most comprehensively and be used by the entire national scientific community in that particular discipline;
• The context of the research problems is strongly focused on the Finnish society or the Finnish or Swedish-speaking culture.
• Publishing in these channels is regarded as high merit as publishing in foreign level 2 channels.
All publication channels meeting these criteria are no rated as level 2 but it will only include a selection of the highest quality and most comprehensive Finnish and Swedish-language publication channels covering the disciplines where it is justified to produce and publish new scientific information in the national languages. The Panel Chairs acting in corpore are to make a consensus decision on the classification of Finnish and Swedish-language publication channels in level 2.
Level 3 is the top category of the Publication Forum classification. Level 3 is a sub-category of the level 2, so level 3 channels need also to meet the level 2 criteria. Level 3 can include the topmost publication series of various disciplines meeting also the following criteria:
• The research published in them represents the highest level in the discipline and has very high impact (e.g., as measured through citation indicators);
• The series cover the discipline comprehensively, not limiting to the discussion of narrow special themes;
• Both the authors and readers are international and the editorial boards are constituted by the leading researcher in the field;
• Publication in these journals and series is highly appreciated among the international research community of the field.
The Panels will prepare a proposal of publication series to be rated as level 3, and the Panel Chairs take consensus decisions to include publication series in the level 3 category.
...
...
...
4.5.1 Counting the publication volume
...
...
...
...
Depending on the discipline, the impact and prestige of the publication channels in the international scientific community can be estimated using the impact factors and level rating indicators that the secretariat provides for the use of panels. The citation databases cover better the natural and medical sciences than the journals/series of engineering, social sciences and the humanities. In case of SSH, the focus of citation databases in English newspaper articles is also problematic. The Norwegian and Danish panel ratings create a more comprehensive basis for evaluating international prestige of journals and series. None of the indicators, however, covers all publishing channels to be evaluated in the Publish Forum.
Various impact factors are available for the journals indexed in the Web of Science or Scopus databases, most importantly Journal Impact Factor (JIF), CiteScore, Source Normalized Impact per Paper (SNIP), and SCImago Journal Rank (SJR). Common to these indicators is that they try to measure the impact of publication channels based on the number of references they have received in their publications. The basic assumption is that the larger the number of citations to publications in a journal is, the greater its impact on the international scientific community. In the use of the impact factors, it should be noted that they are not comparable between different disciplines.
In addition, the panels have the ratings given for journals, series and book publishers in the Norwegian and Danish panel evaluation that is comparable to the Publication Forum. Common to these classifications is that they are also based on an expert assessment, although impact factors and may be used to inform the evaluation depending on the discipline. Norway and Denmark have a 2-tier rating, where class 1 is basic and class 2 is the leading level. In Denmark, there are about 3000 journals/series in Level 2, in Norway about 2000.
Members of the academic community have the opportunity to give panels feedback on ratings by proposing changes to the ratings on the Publications Forum website and on the JUFO portal. In addition, members of the Finnish research community can each determine the 10 most important journals/series and the book publisher for their own research on the JUFO portal. There is also expertise concerning the evaluation of channels belonging to each panel in other panels, and panelists can also make rating suggestions across panel boundaries. Panels can use all of this information to support their assessment. Panel members represent the national research community of their own discipline or research area, and the assignment also includes the need to consult with their background communities.
In natural and medical sciences review articles typically gain the most citations because of their nature, so the impact factors for journals that only publish reviews are higher than those publishing original research papers. The best review journals can be classified as levels 2 and 3, as long as they do not fill the entire quota.
...
...
• if candidates to the level 2 or 3 in the same field have equal impact or prestige, open access journal or the one permitting self-archiving of the peer-reviewed version of the manuscript is chosen to the higher level over the channel that does not support open access.
• the channel allowing for self-archiving can be favoured in comparison if the embargo period is in science and technology for six months and in human sciences for 12 months.
• openness of data can also be considered an advantage. It is checked in the case of comparison whether open access provides grounds to decide selection.
...
...
...
Channels permitting self-archiving (green open access) are traditional subscription-paid journals that allow articles to be stored, for example, in the University's open publication archive. The SHERPA/RoMEO service, which is integrated into the Publish Forum database, can be used to find a self-archiving policy. In general, the color code alone does not adequately describe the policy, but the journal specific information must be checked from SHERPA / RoMEO.
...
...
...
Closed society journals are publication channels that publish only from invited authors, or in which society membership is required or advantaged. In the review of ratings, the panels may take into account if membership restricts authorship too much and exlpain this in the grounds for rating.
All channels regarded as leading by researchers do not have a level 2 or 3 rating, or the rating of the journals does not fully correspond to the impact factor ranking order of a particular discipline. Often, this is due to the structure of the rating, for example, the level quotas or the classification criteria. However, the evaluation work of the panel of experts in the publication forum should be justified and consistent, and take into account ethical considerations, thereby increasing the transparency and reliability of the rating.
...
...
The panels are required to produce more precise justification of what level 1 criteria the channels rated at level 0 do not meet (see 3.2). For Levels 2 and 3, panels need to produce on a general level the principles and criteria on basis of which the classification is made. In addition, the aim is to record channel-specific justifications for decisions that significantly deviate from the impact factors or the Norway and Denmark's ratings of expert panels:
• If the channel gets level 2 or 3, even if the channel is classified in Norway and Denmark for level 1, or the channel gets level 1 even though it is classified in level 2 in Norway and Denmark.
...
The members of the panels are representatives of the national research community in their field of expertise. This means that panelists are not only guardians of interests of their own university / unit or a particular publishing channel (publisher or journal).
The evaluation work must be based on good scientific practice (honesty, general care and accuracy), including the announcement of engagements. Panel members are responsible for complying with the National Board of Research Ethics Guidelines for Responsible conduct of research, the principles of which are universally accepted by universities and research institutes.
Panelists must announce their engagements each time a panel discusses evaluation of channels linked to them. This applies to publishing channels where the panelists have published more than once in the past 5 years or have been journalists or members of the Editorial Board. In these cases, the panelist will not be excluded from the decision-making, but the panel can draw attention to the engagements in the assessment situation. The Secretariat will compile information on the engagements.
...
A continually ongoing complementary evaluation, in which panel has 2 months’ time-window to make a decision to place a channel on the level 0 or 1, is implemented entirely through the JUFO portal without panel meetings. The complementary evaluation includes publishing channels that have been proposed to be added to the classification through the Publications Forum web pages or identified from the publication data reported by the research organizations to the ministry of education and culture.
...
...
...
In the JUFO portal, panelists can view information about the publishing channels that are being evaluated by the panel, make proposals about the level of publication channels, and make notes about publishing channels. Panelists can also download their channel list as an Excel file.
The panels meet at the House of Science and Letters three times during the first year of their term of office, when the task is to carry out a review of ratings. For the last three years of its term of office, when panels perform exclusively a complementary evaluation, the panels meet once a year (September-October).
In those years when an review or ratings is not carried out, the panels meet once a year to address feedback and suggested corrections to the ratings. Individual changes to Levels 2 and 3 can be made according to the current volume situation. That is, if a free level 2 and 3 quota has been created dut to addition of level 1 journals, it may be used for upgrades.
...
...
7 APPENDIX 1: Evaluation timeline for the term 2018-2021
...
Complementary evaluation
...
January-
February
...
1st panel meeting
- organization of the panels
- introduction to panel work
- Publication Forum and evaluation criteria
- panel fields and representativeness
- level 2 and 3 quotas
- JUFO-portal
- timetable
...
Steering-group meeting
- changes to panel compositions
- Top-10 indicator
- other issues
...
2nd panel meeting
- preliminary proposal for the level 2 and 3 ratings
Meeting of SSH panel chairs
- preliminary proposal for level 2 national language channels
...
Steering-group meeting
- examination of the preliminary propsal and feedback
- feedback to panels
...
September-
October
...
3rd panel meeting
- examination of the feedback
- final proposal for the level 2 and 3 ratings
...
Meeting of SSH panel chairs
- final proposal for level 2 national language channels
Meeting of all panel chairs
- final proposal for level 3 journals, and level 2 and 3 for the panel 24 channels (journals/series and book publishers)
...
Steering-group meeting
- confirming the classification
...
2019
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...
2020
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...
2021
September-
October
...
1st panel meeting
- feedback and individual corrections to levels 2 and 3
- other issues
...
...