Skip to content

MIP-703 UDF generator refactor

Kostas FILIPPOPOLITIS requested to merge dev/703/udfgen-refactor into master

Created by: jassak

This refactoring focuses primarily on two goals:

  • Break the udfgenerator module into multiple smaller modules.
  • Make some progress towards decoupling UDF generation logic from SMPC logic.

The first goal is fairly straightforward but requires the introduction of many auxiliary commits, in order to maintain git history. These commits are all named create X from udfgenerator.py, where udfgenerator.py is renamed to X, and split X from udfgenerator.py where all irrelevant lines are removed from X. This allows git to trace the history of every new module back to udfgenerator.py, thus maintaining git history.

The second goal is achieved by removing as much SMPC logic as possible currently from udfgenerator.py and moving it to a new smpc.py module. The actual SMPC logic is still coupled with the UDF generation, so the separation cannot be complete without changing multiple things in the system, which is out of scope for the current PR.

The PR also contains multiple smaller fixes:

  • Remove or deprecate obsolete features (traceback flag, scalar types, tensor operations)
  • request_id is not passed to generator. Instead it is exported as a placeholder for the node to substitute.
  • All algorithms import from udfgen package. Not from specific modules with the package.
  • Smaller refactors/renames.

Merge request reports