Skip to content

Fix/mip 732/monet db memory error handling

Kostas FILIPPOPOLITIS requested to merge fix/MIP-732/monet-db-memory-error-handling into master

Created by: KFilippopolitis

Changelog: https://team-1617704806227.atlassian.net/browse/MIP-732. I implemented a first attempt to fix most of the problem with the monetdb. https://team-1617704806227.atlassian.net/browse/MIP-731.

Refactored monet_db_facade.py.

  1. Removed the many functionality from pymonetdb, because the is not compatible with the idempotent queries.
  2. Added the case of 'insert into' to the idempotent queries.
  3. Removed the _MonetDBConnectionPool.
  4. Now for each query execution we create and close a connection.
  5. Now monetdb facede will retry(x amount of times) to execute a query in case the error "table not found" is thrown. This fixes the MIP-731.

Added Monitoring tool 'Monit' to the monetdb configuration.

  1. Now monit will monitor the monetdb resource allocation.
  2. Added two actions on the monit 'hard memory limit' and 'soft memory limit'
  3. Soft/Hard memory limit will trigger once the memory of the system is above 70%/85%.
  4. Once soft limit is trigger, the monit will check if in the following 10 the memory usage drops below 70%, in case it does not the monit will lock, restart and then release again the database.
  5. Once hard limit is trigger monit will instantly lock, restart and then release again the database.

Merge request reports