What services are offered?
Results reproduction before submission
You created a computer-based workflow as part of your latest paper submission? Great! Now it is time to make sure that your computational steps can be understood and recreated by a third party before you submit the workflow for peer review or publication of a preprint. Such an independent confirmation increases trust in your work.
During results reproduction, an R2S2 team member will evaluate the data and code provided by you (file organisation, documentation, code understandability, etc.) and follow the provided instructions to execute the workflow. We will then report on our results, compared to the ones provided by you, and give general feedback on what you may improve to further increase accessibility, understandability, and reusability for third parties. For more on the general feedback, see "Research compendium creation" below.
- We will not replicate your study (collect new data, use same analysis/code), make it robust (same data, different code/analysis), or generalise your work (different data, different analysis) (see The Turing Way’s definitions [en]).
- The reproduction does not cover the content of your paper, such as checking your research methodology, questioning your conclusions/assumptions, or making direct changes to the work.
- To be able to reproduce your workflow, your workflow or scripts must be based on software/programming language that is available to us, ideally free and open source software. Exemptions may be made for software available to employees of Münster University or where you can provide access for the R2S2 team member./span>
- No High-Performance-Computing (HPC) or Big Data: the R2S2 team does not have sufficient resources to reproduce very complex and extensive computations; please do get in touch if you are unsure if your data is "too big" or if you want to make a case why we should consider your workflow.
- Duration of computations should be under 24 hours on a reasonably sized desktop computer – consider creating a synthetic dataset or subset if this is the case; please do get in touch if you are unsure if your computation takes too long or if you want to make a case why we should consider your workflow./span>
Project set-up consultation and computing environment management
You can make your life and the life of collaborators (e.g., future you or the next PhD student) much easier if you consider reproducibility from the start of a research project. If there are no experts on computational reproducibility or open science at your lab/institute/working group, we are happy to have a conversation with you about your ideas and plans. Let’s try to look ahead into the future and see how you can not only avoid to shoot yourself in the foot, but be very efficient in your day-to-day work habits and score extra points with reviewers and become a reproducible research leader in your community of practice.
You already write R/Python packages and know about notebooks, virtual environments, Binder, version pinning, containers, and virtual machines? Get back to us for a results reproduction!
Research compendium creation
A research compendium accompanies, enhances, or is itself a scientific publication providing data, code, and documentation of a scientific workflow (cf. research-compendium.science [en] for more literature). It provides all materials for others to reproduce, re-use and extend a particular dataset or method. The term has been used in various disciplines to describe the desirable "package" of bits and pieces that make up the real scholarship, for which the article or papers is the "mere advertising" [en].
If you want to practice reproducible research and open science based on computers, a research compendium is a great approach to package computer-based methods for yourself and for sharing them with others. As a result of your consultation, you create an archivable package with all information in one place which is ready to be published in a repository and receive a persistent identifier (e.g., a DOI).
The compendium is created based on your current material. During the consultation, R2S2 team members will provide suggested code edits to facilitate reproduction and to ensures the transparency and reproducibility of your research, a high ease of access to data and code for others, and independent understandability by others. A research compendium creation may include:
- research data publication
- research software publication
- computing environment definition and publication
- citable data
- citable software
If your research compendium includes data, software, a containerised computing environment, and the possibility for users to manipulate parts of your workflow, then we would like to explore with you the possibility to create an Executable Research Compendium (ERC). The ERC is o2r’s own concept of the "research article of the future". You can learn about it in this publication about ERC [en] and see it in action in this reader’s perspective video [en]).
Even if you decide not to publish/submit the research compendium with the article (if journal policies permit) nor after publication of the article, you have the ability to promptly and confidently provide reproduction materials if reviewers or future readers ask for them.
- We may take a look at the paper, but only in so far as how it connects to the other building blocks of your compendium – the same limitations of results reproductions (see above) apply. Of course, we will not share anything before your research is published or your explicit confirmation.
- It may be more suitable to deposit research data and research software independently, or additionally, in different repositories – we try to find the best solution for your case together with you.