Topic: Modular Build — ESD build paths
The envisioned usage of the ESD includes:
- EBRAINS (jupyter) lab
- laptop use (i.e. run somewhere, use-as-downloaded or user-modified)
- HPC use (flat deployment or containerized, site-admin-managed or user-handled)
- service deployment (e.g., some k8s thing, or some classical
systemdservice) - GDPR-sensitive environments (e.g., "minimum-set" deployments)
→ i.e. to avoid duplication in the build flow, we need a modular build path.
Several aspects are important:
- caching (sources, build results,
spack buildcache, images) - handling of filesystem locations
- restricted build environments (i.e. non-privileged to varying degree)
Spack defines a typical workflow that involves:
- concretization → finding out the packages that should be "there", in which version/variant, etc.)
- fetching of sources → downloading source packages that are to be built
- fetching and installing from buildcache → downloading and installing from a buildcache
- building and installing of packages from source
- (and environment creation/handling things (first step: base compiler, but also env definition/creation/spezialiation, etc.)