Options
2025
Meeting Abstract
Title
FAIRification of sensor-based time-series data - a demonstration of the Helmholtz DataHub digital ecosystem
Abstract
In environmental sciences, observational data remains indispensable for monitoring and understanding of natural processes, validating earth system models and remote sensing products, and training of data driven methods. However, unified standards and interfaces for ensuring that such data is consistently available, usable, and compliant with FAIR and Open Science principles are still lacking.
The so-called DataHub initiative of the Helmholtz Research Field Earth & Environment, involving seven large environmental research Centers across Germany, addresses this gap by collaboratively developing a cohesive and unified research data space, including consistent data formats, metadata standards, tools, interfaces and services.
Since the beginning of the DataHub, we have been particularly focusing on unifying time-series data from environmental sensor systems, which are operated across all participating Centers. In this context, we have developed a digital ecosystem, that enhances and links existing and established research data infrastructures with well-defined interfaces and metadata standards. This ecosystem now covers the full processing chain from the integration of new sensor systems and their metadata over automatic and manual quality checks and flagging schemes to the visualization via dashboards and data portals or the usage in data analysis frameworks. In particular, our framework consists of multiple independent tools and services like the Sensor Management System (Brinckmann et al., 2024) as dedicated system for managing sensor metadata, the System for Automated Quality Control (SaQC, Schäfer et al. 2024) as common framework for QA/QC, a tailored metadata profile which adapts the SensorThings API (STA) from the Open Geospatial Consortium to common requirements from environmental sciences (Lorenz et al. 2024), the Earth Data Portal (https://earth-data.de) as overarching data portal and visualization suite as well as tools and services that link all these different building blocks.
While the first concepts of this ecosystem were based on temporary tools and interfaces, we have now reached a level of maturity, that allows us to confidently scale our solutions to new communities and user groups. In this presentation, we will hence give a brief overview of our ecosystem as well as the integrated tools and services. The main focus will be on a hands-on demonstration of the full workflow from deploying a new sensor system, the integration into the contributing services, the (meta)data provision via STA as well as the integration in different downstream systems like the Earth Data Portal for data visualization.
By this, we want to promote the potential of a decentralized research data infrastructure, that has been developed and adopted across multiple research Centers and reach out for new communities and user groups for ultimately creating a FAIR and inter-institutional open data space for our environmental sciences.
The so-called DataHub initiative of the Helmholtz Research Field Earth & Environment, involving seven large environmental research Centers across Germany, addresses this gap by collaboratively developing a cohesive and unified research data space, including consistent data formats, metadata standards, tools, interfaces and services.
Since the beginning of the DataHub, we have been particularly focusing on unifying time-series data from environmental sensor systems, which are operated across all participating Centers. In this context, we have developed a digital ecosystem, that enhances and links existing and established research data infrastructures with well-defined interfaces and metadata standards. This ecosystem now covers the full processing chain from the integration of new sensor systems and their metadata over automatic and manual quality checks and flagging schemes to the visualization via dashboards and data portals or the usage in data analysis frameworks. In particular, our framework consists of multiple independent tools and services like the Sensor Management System (Brinckmann et al., 2024) as dedicated system for managing sensor metadata, the System for Automated Quality Control (SaQC, Schäfer et al. 2024) as common framework for QA/QC, a tailored metadata profile which adapts the SensorThings API (STA) from the Open Geospatial Consortium to common requirements from environmental sciences (Lorenz et al. 2024), the Earth Data Portal (https://earth-data.de) as overarching data portal and visualization suite as well as tools and services that link all these different building blocks.
While the first concepts of this ecosystem were based on temporary tools and interfaces, we have now reached a level of maturity, that allows us to confidently scale our solutions to new communities and user groups. In this presentation, we will hence give a brief overview of our ecosystem as well as the integrated tools and services. The main focus will be on a hands-on demonstration of the full workflow from deploying a new sensor system, the integration into the contributing services, the (meta)data provision via STA as well as the integration in different downstream systems like the Earth Data Portal for data visualization.
By this, we want to promote the potential of a decentralized research data infrastructure, that has been developed and adopted across multiple research Centers and reach out for new communities and user groups for ultimately creating a FAIR and inter-institutional open data space for our environmental sciences.
Author(s)