Added/Updated spack packages for several BioBBs and their dependenciesMaster

Hi everyone,

All BioBB packages have been tested to work locally. Their dependencies (ambertools and acpype packages) were tested as well.

In the case of NGL (nglview and simpletraj packages) they were tested to "run flawlessly". However, I could not see them "working" in a strict sense and I would like to explain why.

NGL packages are graphical utilities meant for jupyter notebooks. They are molecular viewers. I installed the spack upstream package py-notebook and tried to run a jupyter notebook but I was not able to make it work. I can provide more context of the error I was stucked in if someone is interested but since here in ebrains you already have your jupyter lab working I will skip this part. Once I surrendered, I installed the notebook package using mamba (i.e. conda) while the rest of the modules were still in spack. I could run my whole workflow but the NGL windows were not opening, although they didn't throw errors either. Given the precarious nature of my jupyter notebook it is not strange that NGL was not working properly, but I think it may have worked in a more canonical scenario. That's why I am merging these packages also. I truly think they may work as they are.

Best regards

Hi @dbeltran,

Thank you for taking the time to package and test all those tools!

Apart from adding the Spack packages, you also need to edit the spack.yaml file and add one entry for each package that you want to be included to the environment (and the version you want installed). You should only do that for the top-level packages: no need to add those that are just dependencies (like ambertools and actype from what I understand).

added 1 commit

• d6aff117 - Update spack.yaml

Compare with previous version

Hi @elmath,

I just updated the spack.yaml file. I also added py-plotly since I'll need it in our workflows.

By the way, I just remembered what you told me about setting tests in the package.py files. I apologize since I totally forgot it. However, I was going to try to implement it and I just realized there is no '--test' flag in the spack install command so I can not check if my implementation is working. My spack version is 0.19.2, which is the same version than in the development lab. Any comment on this? Which version of spack should I use?

Best regards

added 1 commit

• 4a818098 - Update package.py

Compare with previous version

Hi @dbeltran! no worries, I think we should get the test build pipeline here to pass first and then work on the tests :)

(are you using just spack install --test <pkgname>? in Spack v0.19.2 it should be spack install --test root <pkgname>)

For now, 3 things should be fixed for the pipeline to pass:

• Spack has some trouble installing py-poetry, that is a dependency of acpype. I will look into why this happens, but I am 99% sure that acpype should only depend on py-poetry-core, and not py-poetry. So if you replace it here we can skip this problem completely :)
• Any external packages, such as py-plotly, should not be included in spack.yaml, that only contains top-level EBRAINS tools. Instead, you need to create a "meta-package" for your workflow (such as this one) and add them as dependencies there. However, py-plotly is coincidentally already available in the Lab as a dependency of another EBRAINS tool - so you can skip this completely, and just remove py-plotly from spack.yaml.
• simpletraj should depend on py-numpy

resolved all threads

added 1 commit

• cbb84083 - Update spack.yaml

Compare with previous version

added 1 commit

• 7e892724 - Update package.py

Compare with previous version

added 1 commit

• e5e88659 - Update package.py

Compare with previous version

changed target branch from master to update-biobb-packages

mentioned in commit 9029fa2a

merged

@dbeltran thanks for the changes! :)

I think everything is OK now, but because of some issues with the (shared) gitlab runners we use for testing (and the fact that python needs to be re-installed like you said) the job runs out of space.

I merged and opened !346 (merged) to test on our own, not shared runners, where we will not have this problem. The tests will probably take long because of the python thing.. I'll let you know when they pass.

(and sorry for the delay, usually this is a lot simpler )

Hi @elmath, I don't know why the pipeline did not fail yet but we are having 'Error: [Errno 28] No space left on device'. We may have run out of disk :/

@elmath, sorry for my silence, I just realized I was making comments which you could not see because apparently they were hidden until I 'finished my review' (facepalm). This GitLab interface is not very intuitive to me.

@dbeltran I finally managed to merge !346 (merged)

All packages are now available in the lab-int, and you can use them by selecting the EBRAINS-test kernel. Could you please have a look and let me know if everything is OK?

Hi @elmath,

Everything is working so far Even NGL, the viewer I was concern about, is working.

Thank you Eleni!

By the way, next week my boss has to give a training and it would be very convenient if the new packages could be reached from the lab in the wiki, which can be accessed trough free registration. I suppose this lab is indeed the production lab and updating it in a rush may be not possible. I totally understand if it can't be done, but I had to ask.

Hi @dbeltran, that's great!

Actually you didn't even have to ask :) the EBRAINS-experimental kernel at the production Lab is updated weekly with any new packages. This happens on Friday evening, so tonight, but the deployment job can take quite long sometimes. In any case, the new packages will definitely be available in the production Lab by Monday, I'll let you know as soon as it's done.

Your tools will also be part of our next (stable) official EBRAINS release at the end of the month, so if you could add some tests to the packages by then it would be great.

Good luck with the training, and if you need anything else added (e.g. any external package for your demos) let me know.

resolved all threads

Hi @dbeltran, your tools are now available in the prod Lab, in the experimental kernel (both CSCS and JSC)

Hi @elmath, yesterday I run our workflows and everything was working Thank you!