Keeping your aggregative infrastructure under control

Abstract

“Aggregative Data Infrastructures” (ADIs) are systems devised to collect metadata descriptions (and files) from several data sources to construct uniform Information Spaces, hence providing cross-data source access via standard APIs or custom portals. ADIs typically deal with data collection workflows from arbitrary numbers of data sources, with heterogeneous access protocols, data exchange formats, and data models. Besides, they handle data processing work-flows for the harmonization and enrichment of aggregated metadata. Correct workflow management is crucial to ensure Information Space consistency, but is in general hard to sustain. This demo will present the solution offered in the context of the OpenAIRE infrastructure, which today collects metadata and files from around 450+ data sources (and growing) of several typologies. The D-NET Workflow Management Suite user interfaces support data curators at orchestrating overtime and in a sustainable way the configuration, execution, and monitoring of data collection and processing workflows for thousands of data sources.