Reproducibility is an important part of providing evidence of the correctness of research results. Other researchers should be able to inspect the workflow and evaluate all of the steps that have been taken during the analysis and repeat them.
Reproducibility is defined as the possibility to obtain consistent results using the same data and code as the original study (computational reproducibility).
Replicability means obtaining consistent results across studies aimed at answering the same scientific question using new data or other new computational methods. In this case new data is collected or created.
Documenting and sharing research software and workflows are crucial elements of reproducibility. Research infrastructures and services should not only enable but also support reproducibility.
Please, feel free to add links to this list.
Methods and formats
Research compendium
https://research-compendium.science/
https://github.com/ropensci/rrrpkg
Research object crate
https://www.researchobject.org/ro-crate/background.html
https://github.com/ResearchObject/ro-crate
CWL
https://www.commonwl.org/user_guide/
https://github.com/common-workflow-language
Reports
FsF Report https://doi.org/10.5281/zenodo.4095092
FAIR data & software (Executable papers and software, p. 25-):
https://ec.europa.eu/info/sites/info/files/research_and_innovation/ki0120580enn.pdf
Knowledge Exchange report 2021 https://www.knowledge-exchange.info/event/publishing-reproducible-research-output
Articles
Open Science Software engineering
https://link.springer.com/chapter/10.1007%2F978-3-030-32489-6_17
Creating an executable paper is a journey through Open Science
https://www.nature.com/articles/s42005-020-00403-4
Using a PID graph for reproducible research
https://zenodo.org/record/4275872#.X7ukSBMzYXo
Implementing FAIR Data Infrastructures
https://doi.org/10.4230/DagMan.8.1.1
From FAIR research data toward FAIR and open research software
https://doi.org/10.1515/itit-2019-0040
Open Source Research Software
https://doi.org/10.1109/MC.2020.2998235
Taking a fresh look at FAIR for Research Software
https://doi.org/10.1016/j.patter.2021.100222
Articles about executable articles (different fields)
Executable Papers - improving the article format in computer science
eLife launches Executable Research Articles for publishing computationally reproducible results
Toward Executable Scientific Publications
https://www.sciencedirect.com/science/article/pii/S1877050911001323
RDA Health Data IG - Reproducible Workflows in Healthcare Guide
Demo
https://sorse.github.io/programme/software-demos/event-019/
Initiatives
RDA
Reproducibility IG
https://www.rd-alliance.org/groups/reproducibility-ig.html
Reproducible Health Data services WG
https://www.rd-alliance.org/groups/reproducible-health-data-services-wg
FAIR 4 research software
https://www.rd-alliance.org/groups/fair-4-research-software-fair4rs-wg
The Netherlands
Local Reseaarch software Directory https://www.research-software.nl/
Guidelines for FAIR software https://fair-software.nl/about
( Don't publish: Tom Bakker as contact point for further inquiries Tom Bakker t.bakker@esciencecenter.nl )
France
- Software Heritage INRIA https://www.softwareheritage.org/save-and-reference-research-software/
- Certification Agency for Scientific Code and Data (cascad): https://www.cascad.tech/
(With the YouTube boasting: https://www.youtube.com/watch?v=i17UI2bKh0E&feature=youtu.be)
- RunMyCode (an online repository allowing people to share code and data associated with scientific publications): http://www.runmycode.org/about.html
Misc
RDA Sweden/EOSC-Nordic webinar: "Placing research software into Open Science":