This chapter deals with the assessment of the societal impact of science from the perspective of publication metrics. It is impossible to measure the societal impact of science by means of publication metrics, because impact is a broad and multifaceted phenomenon. However, publication metrics make it possible to verify impact-promoting interactions, which can be seen as popular and professional publications made by researchers, as well as the attention received by scientific publications in policy documents, media and patents.
Publications written by researchers for the professional community and the public
Publications directed by researchers to the professional community and the public (so-called YVV publications, YVV = social interaction and impact) can be considered one of the central means of social impact of science. In the publication classification of the Ministry of Education and Culture, such publications include articles in professional journals, books and book chapters for professionals, as well as popular journal articles, book chapters, books and blog posts.
Finnish universities, universities of applied sciences and research institutes collect information from these publications in their own research data collection systems. Nationally, information on publications is available on the Science and Research website and the Statistics Service Vipunen. The coverage of publication data in the above-mentioned sources is likely to be weaker for universities than for other types of publication because scientific publications play a central role in universities and professional and popular publications do not have a significant impact on university funding. In polytechnics, professional and popular publications play a more significant role, and these types of publications have a major impact on funding.
Responsible use
- Information on the number of publications can be used as support and evidence for qualitative description.
- When using databases as a source of information, possible gaps in the coverage of publications must be considered.
- In different fields of science and organisations, publications for professional and general audiences have different meanings, so their number may vary by field of science and organisation type.
- Differences between fields of science in the number of social interaction and impact publications have not been examined, so it is not possible to use standardization of the field of science.
- In reporting and description, it is a good idea to specify the different types of publications, because professional publications and publications intended for the public can contain a wide variety of publications.
References to scientific publications in policy documents
References in policy documents, recommendations and reports provided by publications indicate the social exploitation of research. Policy documents, recommendations and reports are produced by international organisations (e.g. WHO, IPCC, EU), state institutions (e.g. ministries), national organisations (e.g. Medical Society Duodecim), and research institutes (e.g. VTT, THL, SYKE, Finnish Meteorological Institute) and think tanks (e.g. SITRA, Demos Helsinki).
Information on the citations received by publications in policy documents can be followed in services such as Overton, SciVal Impact and Altmetric Explorer. SciVal Impact is based on Overton's data. It is more concise, containing only information on the references to publications indexed to Scopus. Altmetric Explorer has the lowest coverage for policy documents.
From policy documents, Overton best covers policy documents and recommendations of major international organisations (WHO, IPCC, EU). There may be variations in the coverage of publications by national actors such as state institutions. For example, the number of Finnish policy documents in the Overton database is smaller than in other Nordic countries. There may also be differences in coverage between different fields of science.
Responsible use
- Information on the number of publications that have received citations and citations can be used as support and evidence for qualitative description.
- When using databases as a source of information, possible gaps in the coverage of publications must be considered.
- Consideration of differences between fields of science. The relationship between fields of science and policy documents and recommendations varies, so publications in different fields of science have a different likelihood of being referred to in documents and recommendations.
- Differences between fields of science in policy documents and recommendations have not been considered, so it is not possible to use field of science normalisation.
Scientific publications and online media attention
The media attention that scientific publications receive is one form of interaction that promotes impact. The active party is not the authors of the research, but other people (individuals, other researchers, journalists, politicians) who can comment on or highlight the research publication in some media. Here we will deal only with the attention received by publications online.
1.1 Visibility of publications in online news services
The visibility of research in the news can be monitored using media monitoring services, but these tools are not commonly used to monitor the visibility of publications. The visibility of publications in the news can be monitored using altmetrics services. They can be used to monitor the visibility of publications on the websites of media companies producing online news services (e.g. YLE, Helsingin Sanomat, BBC).
Responsible use
- Information about the attention the publications receive in the news can be used as support and evidence for qualitative description.
- When using the data, it is necessary to consider the shortcomings in the coverage of the monitored publications, but also in the coverage of the monitored news services.
- Differences between fields of science in the amount of news attention have not been considered, so it is not possible to use standardization of the field of science.
1.2. The publicity of publications on social media
The visibility of publications on social media services (e.g. X, Facebook, Wikipedia, Sina Weibo) can be monitored using altmetric services.
Responsible use
- Information about the publicity that publications receive in social media services can be used as support and evidence for qualitative description.
- The attention received by the study can be of many kinds, so it is good to look at the content of the posts before drawing conclusions.
- When using the data, it is necessary to consider the shortcomings in the coverage of the monitored publications, but also in the coverage of the monitored services.
- Differences between fields of science in the amount of social attention have not been considered, so it is not possible to use the standardization of the field of science.
Sources
Muhonen R. (2021) Tutkimuksen yhteiskunnallisen vaikuttavuuden arvioinnin haasteet. Available: https://vastuullinentiede.fi/fi/jatkokaytto/tutkimuksen-yhteiskunnallisen-vaikuttavuuden-arvioinnin-haasteet