EGI Federation Home
Service

DataHub

Access key scientific datasets in a scalable way

DataHub allows you to enable simple and scalable access to distributed data for computation, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites.

Overview

DataHub

DataHub allows you to bring data close to  computing to exploit it efficiently, and to publish a dataset and make it available to a specific community, or worldwide, across federated sites. DataHub is based on the Onedata technology.

You can access the service for evaluation using the PLAYGROUND shared space, or request support to publish your data and have dedicated storage assigned.

Main features

  • Discovery of data via a central portal.
  • Access to data conforming to required policies which may be:
    • unauthenticated open access;
    • access after user registration or
    • access restricted to members of a Virtual Organization (VO).
  • Access to data via GUI, POSIX, CDMI
  • Replication of data from data providers for resiliency and availability purposes. Replication may take place either on­-demand or automatically.
  • Authentication and Authorization Infrastructure (AAI) integration between the EGI DataHub and with other EGI components and with user communities existing infrastructure.
  • Metadata and shares management
  • Data import and  data caching based on file popularity
  • Support for many backends (CEPH, S3, GlusterFS, POSIX, etc)

TRL 8: Actual system proven in operational environment.

See the DataHub privacy policy to learn how personal data are processed when using this service.

Get started

Trial access

Try the service, prototype and validate applications and services.

Custom access

Long-term access, service level guarantees, dedicated support.

Consultancy & Support

Are you wondering if this service can help solving your challenges? Are you a user and seek support?

Solutions

With DataHub, you can solve these challenges:
Batch computing

Fully managed batch processing in the EGI Infrastructure. EGI Solution.

...
Interactive computing

Data analytics and visualisation with Jupyter Notebooks. EGI Solution.

...
Data Spaces

Integrated data access services, computing and domain-specific processing tools for big data analysis. EGI Solution.

...
Data Federation

Unified data access across distributed data providers. EGI Solution.

...
Service hosting

Scale-up domain-specific web hosting solutions on the EGI Federated Cloud. EGI Solution.

...

Case studies

How our community uses DataHub

Our providers

This service is offered by providers of the EGI Federated Cloud Platform and the EGI Federated HTC Platform.

Funding

This service is co-funded by the European Commission.

CYFRONET logo
Service documentation

Instructions, manuals and community resources

1
User Manual

Documentation for users.

2
Service Providers

Service providers documentation

Related news