The European Open Science Cloud (EOSC) is an emerging virtual environment underpinning the handling of high complexity volumes of information generated by the big data revolution. The EOSC roadmap foresees a seamless federation of existing and future research data infrastructures, under the umbrella of a common policy to enable FAIR datasets to be utilized and shared throughout entire value chains. NEANIAS is an H2020 research and innovation action addressing prototyping of new innovative services for the underwater, atmospheric and space research sectors. The action embarks from TRL 6 software solutions, to develop, validate and deliver a portfolio of cutting-edge thematic as well as reusable core services maturing into TRL 8 for EOSC ecosystem onboarding. We discuss our experiences in constructing a novel visualization gateway developed within the NEANIAS core services aiming at providing a unified visualization platform in contrast to existing solutions that, although effective, they are often intrinsically tightly coupled with highly specialised communities. Our core guiding principles are to prototype a general purpose cloud based visualization ecosystem to: (i) handle complex and heterogeneous datasets, (ii) support multiple visualization strategies and (iii) enable intuitive and user-friendly data exploration mechanisms. Our system deploys JupyterHub to provide a modular foundation for inserting bespoke high performance visualization services to allow users to design, construct, realise and fully validate bespoke workflows (e.g. by mixing and matching diverse visualization components to support different levels of composability and integration) to process and analyse complex datasets. Notebook servers are spawn on the cloud, streamlining end users deployments by avoiding manual installation of software tools locally and eliminating the necessity for taking care of complex software dependencies and housekeeping mechanisms. Our science gateway is integrated seamlessly with several cloud based services e.g. for authorisation and authentication, storage allocation and data sharing, logging, monitoring and accounting, as well as open publication of resulting data products, thus making it fully compliant and ready for onboarding the emerging EOSC ecosystem. Currently our gateway is populated with the VisIVO and Splotch services for data intensive visual discovery and is nearing its final release under the NEANIAS development infrastructure. The underlying philosophy focuses on usability and tools for seamless embedding within end-user workflows and interlinking with other NEANIAS core services, e.g. for visualization of AI-powered solutions, underpinning powerful virtual reality solutions or facilitating end-user data accessibility. The gateway was prototyped using KinD originally in a local setting and prior to releasing within the GARR Container Platform. The web interfaces are accessible via an authentication and authorization service based on Keycloak and offer notebook spawners that can be accessed on the notebook through interactive python demonstrations. Prior to spawning a notebook, the system offers the option to mount the spawned environments to a Nextcloud store via WebDAV. Integration with accounting and logging services is realized through Filebeat to pass on Kubernetes logs, integration with monitoring service is realized with Prometheus to collect monitoring metrics while linkages with Zenodo, towards Open Science integration, are underway. We outline lessons learned and discuss development guidelines, with pointers to future works.

Science Gateways in EOSC: The NEANIAS Visualisation Gateway

Raciti, Mario;
2023-01-01

Abstract

The European Open Science Cloud (EOSC) is an emerging virtual environment underpinning the handling of high complexity volumes of information generated by the big data revolution. The EOSC roadmap foresees a seamless federation of existing and future research data infrastructures, under the umbrella of a common policy to enable FAIR datasets to be utilized and shared throughout entire value chains. NEANIAS is an H2020 research and innovation action addressing prototyping of new innovative services for the underwater, atmospheric and space research sectors. The action embarks from TRL 6 software solutions, to develop, validate and deliver a portfolio of cutting-edge thematic as well as reusable core services maturing into TRL 8 for EOSC ecosystem onboarding. We discuss our experiences in constructing a novel visualization gateway developed within the NEANIAS core services aiming at providing a unified visualization platform in contrast to existing solutions that, although effective, they are often intrinsically tightly coupled with highly specialised communities. Our core guiding principles are to prototype a general purpose cloud based visualization ecosystem to: (i) handle complex and heterogeneous datasets, (ii) support multiple visualization strategies and (iii) enable intuitive and user-friendly data exploration mechanisms. Our system deploys JupyterHub to provide a modular foundation for inserting bespoke high performance visualization services to allow users to design, construct, realise and fully validate bespoke workflows (e.g. by mixing and matching diverse visualization components to support different levels of composability and integration) to process and analyse complex datasets. Notebook servers are spawn on the cloud, streamlining end users deployments by avoiding manual installation of software tools locally and eliminating the necessity for taking care of complex software dependencies and housekeeping mechanisms. Our science gateway is integrated seamlessly with several cloud based services e.g. for authorisation and authentication, storage allocation and data sharing, logging, monitoring and accounting, as well as open publication of resulting data products, thus making it fully compliant and ready for onboarding the emerging EOSC ecosystem. Currently our gateway is populated with the VisIVO and Splotch services for data intensive visual discovery and is nearing its final release under the NEANIAS development infrastructure. The underlying philosophy focuses on usability and tools for seamless embedding within end-user workflows and interlinking with other NEANIAS core services, e.g. for visualization of AI-powered solutions, underpinning powerful virtual reality solutions or facilitating end-user data accessibility. The gateway was prototyped using KinD originally in a local setting and prior to releasing within the GARR Container Platform. The web interfaces are accessible via an authentication and authorization service based on Keycloak and offer notebook spawners that can be accessed on the notebook through interactive python demonstrations. Prior to spawning a notebook, the system offers the option to mount the spawned environments to a Nextcloud store via WebDAV. Integration with accounting and logging services is realized through Filebeat to pass on Kubernetes logs, integration with monitoring service is realized with Prometheus to collect monitoring metrics while linkages with Zenodo, towards Open Science integration, are underway. We outline lessons learned and discuss development guidelines, with pointers to future works.
File in questo prodotto:
File Dimensione Formato  
IWSG2022_6.pdf

accesso aperto

Tipologia: Versione Editoriale (PDF)
Licenza: Creative commons
Dimensione 445.1 kB
Formato Adobe PDF
445.1 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11771/26020
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
social impact