Supporting reproducible science in Earth Science communities through the connection of EGI and RELIANCE services

RELIANCE services enhance the discovery of and access to research data, including EO datasets (Copernicus), extract relevant knowledge from scientific text, and manage the research lifecycle holistically. Researchers in RELIANCE come from three different Earth Science communities, including:

• Sea monitoring community (represented by CNR-ISMAR – Italy): Marine ecologists, biologists, geologists, geophysicists, geochemists and oceanographers studying past and present dynamics of the ocean and the ocean-atmosphere interface
• Climate change community (represented by University of Oslo – Norway): Earth scientists working on climate impact studies, climate mitigation and adaptation, geo-engineering, monitoring
• Geohazard community (represented by INGV – Italy): Earth scientists working on improving geohazard scientific research and assessment, with scientific knowledge of volcanic and seismic Supersites, in support of Disaster Risk Reduction

RELIANCE implemented a research lifecycle management service ecosystem that relies on RO-crate-based research objects (ROs) as the overarching mechanism to manage scientific research activities, leveraging the ROHub platform as the reference service. ROs rely on Data Cubes for efficient and scalable EO data access and discovery via the Advanced geospatial Data Management (ADAM) platform and on text mining services based on expert.ai’s AI-based NLP/ U platform to extract and enrich research objects with machine-readable metadata from scholarly communications and other scientific text.

RELIANCE services allow researchers to continue doing their science as usual, but now being able to access larger datasets, including EO data from heterogeneous sources, share their results more easily, having all resources accessible via a single entry point and doing Open Science by default.

One of the key challenges regarding the adoption and implementation of Open Science practices, though, is enabling the reproducibility of results to support trust in science. Besides being able to access scientific knowledge, publications and research data, the replicability and reproducibility of research results is one of the main goals of Open Science. In fact, reproducibility is one of the key ambitions of the EU’s open science policy. However, as is evidenced in different studies in the literature, reproducibility of research results, even those published in prominent journals, is in most cases not possible. This so-called reproducibility crisis is a serious issue affecting many scientific studies. A survey from 2016 by Nature on 1,576 researchers found that more than 70% of researchers from different disciplines have tried and failed to reproduce another scientist’s experiment results, including 64% of earth and environmental scientists. More than half of all researchers have failed to reproduce their experiments. Even more recent studies like a 2019 study in scientific data estimated with 95% confidence that of 1,989 articles on water resources and management published in 2017, only 0.6% to 6.8% of results might be reproduced, even if each of these articles provided sufficient information that allowed for replication.

To support the reproducibility of research results, particularly in communities like Earth Science, researchers and/or reviewers need to be able to access not only the data used to produce the results but also the methods and/or code used to process the data and generate the results, and any relevant documentation and explanation of the research (e.g., paper, presentations, etc.). Additionally, to facilitate the reproducibility of these results, researchers should be able to easily reproduce the computing environment where these methods/codes were executed. As stated in this lecture, in computational / data science, a particular analysis, calculation, or notebook may depend on hundreds of different software packages, each with many different versions. The reproducibility of produced results depends on having the correct version of these packages.