Kirjaudu Wikiin oikeasta yläkulmasta, jos haluat kommentoida opasta.

|

Log in from the top right corner if you want to comment on the guide.

The methods and forms of publishing vary in different fields of science. Evaluation based on citation data is most appropriate for the fields of science that publish primarily international journal articles. In fields that focus on books and conferences, other evaluation methods should also be used. Scientific research is most typically published in peer-reviewed publications.

A peer-reviewed article can be a research article (original research article), a review article (literature review, systematic review) or a conference article, i.e. a study presented at a scientific conference. In addition to these, data articles are also published.

Not all material published in peer-reviewed publication channels has undergone peer review: this material includes brief case reports, brief review articles and editorials.

When evaluating articles, it should be noted that the type of article itself can affect the attention it receives. For example, conference articles may only be available to a small audience, on a website restricted to those attending the conference.

Article types also have an impact on the number of citations: review articles tend to get more citations than research articles or conference articles.


Learn more about articles in the Publication section of this guide


Evaluation of articles

Number of citations

The most common way to evaluate articles is to examine the citations they have received. The number of citations received by articles is usually examined in the Web of Science, Scopus and Dimensions databases or via Google Scholar. The content of the database affects the number of citations: databases do not cover all fields of science in the same depth. On the other hand, the advantage of the above databases is that you can get detailed information about their content. Google Scholar is more extensive than the databases, but there is no information on its coverage available. 

The rate at which articles receive citations varies considerably from one field of science to another: in some fields, citations start to accumulate almost immediately, in others there is a long delay. The subject of the article may also have an impact on the accumulation of citations: an article on a subject that is new and potentially challenges the prevailing research paradigm often gets more citations after a longer delay. When examining the number of citations, it is worth considering the time frame: How old should the articles be for them to be relevant in terms of the number of citations?

The number of citations alone does not tell anything about the impact of an article either but should be compared to the average number of citations received by articles in the same field of science.  Field-normalised citation indicators reflect the impact of the publication set in its field of science. These indicators describe the ratio of the number of citations of a publication or set of publications to the average number of citations of publications of the same publication type in the same field of research in the same year. Read more in the chapter Field-normalised citation indicators.

When assessing citation counts, it is worth bearing in mind that citations are made for many reasons, including in a critical and contradictory sense, as a demonstration of academic scholarship and as a courtesy to the authorities or referees in the field. Recently, there has been a growing interest in this context of citations.

Open access

According to the Policy for Open Access to Scholarly Publications (PDF in Finnish), all new scientific articles and conference proceedings should be published for immediate open access. In the evaluation of publications, the open access to articles can therefore be evaluated separately. Open-access publishing also tends to have an impact on the number of citations.

When evaluating the openness of articles, it should be taken into account that high APCs and/or the publishing policies of journals may prevent an article from being published openly, even if authors wish to do so.

International or national research collaboration

Articles can also be evaluated from the perspective of collaboration. Most research organisations aim to support the internationalisation of research – the realisation of this can be monitored by examining the organisation and country data of the authors of the articles.

It is possible to examine the organisation and country data of the authors of the articles from their own research information systems and international citation databases such as Web of Science, Scopus and Dimensions. However, it should be noted that international citation databases contain limited material in languages other than English, so measuring the internationality of research on the basis of the material indexed in the citation databases may be limited to English-language research only.

Collaboration at national level can also be evaluated in the same way. In addition to this, author data can be used to obtain information on  with the possible corporate collaboration. However, it should be noted that it is difficult to establish natural international collaboration in all fields of science: in legal sciences, for example, much of the research is national.
 

Altmetrics

Altmetrics can be used to quickly identify the attention publications receive. If a publication has a persistent identifier (DOI, ISBN, URN), the mentions it receives on social media platforms (Twitter, Facebook) and news sites, for example, can be tracked to get an idea of the social visibility of the article.


Responsible analysis of articles

The recommendations for responsible metrics emphasise the priority of qualitative evaluation in the evaluation of articles and that quantitative indicators should not be given too much weighting in the evaluation of articles.

The recommendations also stress that individual articles should not be evaluated by the indicators used in the evaluation of publication channels.

Moreover, articles should not be evaluated by a single indicator but by a number of different indicators.

  • The DORA declaration encourages a transition from metrics based on the publication channel to an evaluation based on the scientific content of an article. In particular, the declaration discourages using journal-based metrics, such as Journal Impact Factors, as a substitute for measuring the quality of articles.
  • The user guide for the Publication Forum classification emphasises that the Publication Forum classification is intended for evaluating the average quality of large volumes of publications, not for evaluating individual researchers or articles.
  • No labels