Skip to content

Dev/mip 263/controller metadata registry

Created by: KFilippopolitis

Changelog: Jira-263

  • Node Info no longer contains information about the data model and datasets.
  • The information about the datasets per data model is now provided by the node task get_datasets_per_data_model.
  • The period aggregation of information about the nodes and data models, in now handle by three different modules (Node Landscape Aggregator, Node Registry, Data Model Registry)

Node Registry:

  1. Its only responsibility is to contain and provide information about the nodes (NodeInfo).

Data Model Registry:

  1. Now contains two types of information data_models and datasets_location.
  2. data_models contains information about the data models and their corresponding cdes.
  3. datasets_location contains information about datasets and their locations(nodes).

Node Landscape Aggregator:

  1. Handles the aggregation of necessary information, to keep up-to-date and in sync the Node Registry and Data Model Registry
  2. Periodically sends requests (get_node_info, get_node_datasets_per_data_model, get_data_model_cdes) to the nodes to retrieve the current information that they contain.
  3. Validates and removes any data model that is incompatible across nodes from the aggregated informations.
  4. A data model is incompatible when the cdes across nodes are not identical, except one edge case. The edge case is that the cdes can only contain a difference in the field of 'enumerations' in cde with code 'dataset' and still be considered compatible.'
  5. Updates the 'enumerations' field in the cde with code 'dataset' of each data model to contain all the datasets across nodes.
  6. Provides the information to Node Registry and Data Model Registry, once all the information is aggregated and validated.

Merge request reports