Dev/mip 263/controller metadata registry
Created by: KFilippopolitis
Changelog: Jira-263
- Node Info no longer contains information about the data model and datasets.
- The information about the datasets per data model is now provided by the node task get_datasets_per_data_model.
- The period aggregation of information about the nodes and data models, in now handle by three different modules (Node Landscape Aggregator, Node Registry, Data Model Registry)
Node Registry:
- Its only responsibility is to contain and provide information about the nodes (NodeInfo).
Data Model Registry:
- Now contains two types of information data_models and datasets_location.
- data_models contains information about the data models and their corresponding cdes.
- datasets_location contains information about datasets and their locations(nodes).
Node Landscape Aggregator:
- Handles the aggregation of necessary information, to keep up-to-date and in sync the Node Registry and Data Model Registry
- Periodically sends requests (get_node_info, get_node_datasets_per_data_model, get_data_model_cdes) to the nodes to retrieve the current information that they contain.
- Validates and removes any data model that is incompatible across nodes from the aggregated informations.
- A data model is incompatible when the cdes across nodes are not identical, except one edge case. The edge case is that the cdes can only contain a difference in the field of 'enumerations' in cde with code 'dataset' and still be considered compatible.'
- Updates the 'enumerations' field in the cde with code 'dataset' of each data model to contain all the datasets across nodes.
- Provides the information to Node Registry and Data Model Registry, once all the information is aggregated and validated.