EGI Federation Home
Service

DataHub

Access key scientific datasets in a scalable way

DataHub allows you to enable simple and scalable access to distributed data for computation, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites.

Overview

DataHub

DataHub allows you to bring data close to computing to exploit it efficiently, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites.
DataHub is based on the Onedata technology.

Main features

With EGI DataHub, you’ll enjoy a range of unique benefits:

  • Discover data with ease: Our central portal makes it simple to see the datasets that are available for exploitation and request access to it, saving you time and effort.
  • Customisable access: Whether you require unauthenticated open access, user registration, or restricted access for members of a Virtual Organisation, we’ve got you covered.
  • Multiple access options: Access data via GUI, POSIX or REST API, depending on your needs.
  • Resilient replication: Our platform is built to ensure maximum uptime and availability by replicating data from providers on-demand or automatically.
  • Authentication and Authorisation Infrastructure Integration (AAI): The integration with EGI Checkin seamlessly links EGI DataHub with other EGI components and with user communities existing infrastructure to ensure secure and convenient access to data.
  • Metadata and Shares Management: EGI DataHub simplifies the process of managing metadata and shares, saving you time and effort.
  • Data Import and Caching: Our platform uses file popularity to cache data, ensuring quick and easy access for users.
  • Multiple Backend Support: With support for CEPH, S3, GlusterFS, POSIX, and more, EGI DataHub is truly versatile and convenient.

TRL 8: Actual system proven in operational environment.

See the DataHub privacy policy to learn how personal data are processed when using this service.

LAGO: Unleashing astroparticle research in Latin America with EGI

LAGO is a collaborative organisation of IberoAmerican institutions operating the Latin American Giant Observatory. The observatory is a network of astroparticle detectors used for high-energy astrophysics, and space weather phenomena research across 10 Latin American countries. Facing data management and resource sharing challenges, EGI stepped in with a combination of services, including the EGI DataHub that empowered LAGO researchers to store, publish, and access crucial data and metadata. The service enabled them to access a 15TB cloud-stored library containing the expected flux of signals produced by more than 1013 cosmic rays showers. This service has been crucial in ensuring the continuity of research programmes in Latin America.

Get started

Trial access

Try the service, prototype and validate applications and services.

Custom access

Long-term access, service level guarantees, dedicated support.

Consultancy & Support

Are you wondering if this service can help solving your challenges? Are you a user and seek support?

Solutions

With DataHub, you can solve these challenges:
Batch computing

Fully managed batch processing in the EGI Infrastructure. EGI Solution.

...
Interactive computing

Data analytics and visualisation with Jupyter Notebooks. EGI Solution.

...
Data Spaces

Learn about EGI’s role in enabling Data Spaces to support digital transformation. EGI Solution.

...
Data Federation

Unified data access across distributed data providers. EGI Solution.

...
Service hosting

Scale-up domain-specific web hosting solutions on the EGI Federated Cloud. EGI Solution.

...

Our providers

This service is offered by providers of the EGI Federated Cloud Platform and the EGI Federated HTC Platform.

Funding

This service is co-funded by the European Commission.

Service documentation

Instructions, manuals and community resources

Documentation for users.

Service providers documentation

Related news

Loading...