Change NEST tests from build to install phase

NEST conventionally focussed on make installcheck tests, so the build-phase tests do not provide a meaningful result at the moment. Spack however conventionally only calls make test before the install for CMakePackages. These changes override that behavior and add install-phase tests as suggested by @elmath .

fixes: #21 (closed)

added test-error label

requested review from @elmath

assigned to @terhorst

changed target branch from master to test-spacktests-workaround

Hi @terhorst,

Thanks for looking into it! For any changes related to the tests, Merge Requests should target the test-spacktests-workaround branch, that's where I'm trying out the spack tests.

Could you close the MR and re-open it with the correct target? I tried changing it but this way the MR pipeline is not triggered. Sorry for the inconvenience!

added 3 commits

• e4d09120 - 1 commit from branch test-spacktests-workaround
• 3b9b80cb - Update .gitlab/issue_templates/test-error.md
• 078f234e - Merge branch 'master' into nest-tests-to-post-install-phase

Compare with previous version

The branch had fallen behind master, so I merged and this re-triggered the pipeline.

added 12 commits

• 078f234e...68b83843 - 11 commits from branch test-spacktests-workaround
• 37689a4e - Merge remote-tracking branch 'upstream/test-spacktests-workaround' into...

Compare with previous version

added 1 commit

• 76ec171e - Update packages/nest/package.py

Compare with previous version

added 1 commit

• 9795c8ee - Update packages/nest/package.py

Compare with previous version

added 2 commits

• 6ee5e4d2 - add test dependencies
• 7817882e - Merge branch 'nest-tests-to-post-install-phase' of...

Compare with previous version

added 1 commit

• d8d8ede0 - add missing py-junitparser package dependency

Compare with previous version

added 1 commit

• edb7c445 - add missing py-terminaltables package dependency

Compare with previous version

added 1 commit

• 8281409b - fix typo

Compare with previous version

added 1 commit

• 6cd7e0d4 - add py-clang-format

Compare with previous version

(just in case you missed it) terminaltables is already in the Spack upstream, if you don't mind the older version, one less package to maintain

added 1 commit

• 527f52df - add clang-format@9

Compare with previous version

==> Installing py-junitparser-2.8.0-kncmua45fb7bghzhm4lzaor724je2yhm
==> No binary for py-junitparser-2.8.0-kncmua45fb7bghzhm4lzaor724je2yhm found: installing from source
==> Fetching https://github.com/weiwei/junitparser/archive/refs/tags/2.8.0.tar.gz
==> Error: ChecksumError: sha256 checksum failed for /builds/technical-coordination/project-internal/devops/platform/ebrains-spack-builds/.spack-tmp/root/spack-stage/spack-stage-py-junitparser-2.8.0-kncmua45fb7bghzhm4lzaor724je2yhm/2.8.0.tar.gz
    Expected a8d4290ab9fb93f2015e0dbef61ef93b7ea110e99099087a453f55c8e1481adf but got 94f7e0e45a1e6d729e35f29f28d06d5d314f17bdf68c9a4b1abc20eff0a6cca5

/builds/technical-coordination/project-internal/devops/platform/ebrains-spack-builds/spack/lib/spack/spack/package.py:1541, in do_fetch:
       1538        self.stage.fetch(mirror_only, err_msg=err_msg)
       1539        self._fetch_time = time.time() - start_time
       1540
  >>   1541        if checksum and self.version in self.versions:
       1542            self.stage.check()
       1543
       1544        self.stage.cache_local()


==> Warning: Skipping build of nest-3.3-sm4h4yo66keo6r4t4coqgreryvfg4zm6 since py-junitparser-2.8.0-kncmua45fb7bghzhm4lzaor724je2yhm failed

added 1 commit

• eec1b14f - fix sha from wheel to tgz sum

Compare with previous version

added 1 commit

• 05cb03ff - remove terminaltables package (already in upstream)

Compare with previous version

added 1 commit

• 3d38a0ad - add missing dependency to py-junitparser

Compare with previous version

ok, seems to go a bit further now. I have to look into

 /builds/technical-coordination/project-internal/devops/platform/eb
             rains-spack-builds/spack/opt/spack/linux-ubuntu20.04-x86_64/gcc-10
             .3.0/nest-3.3-sm4h4yo66keo6r4t4coqgreryvfg4zm6/share/nest/testsuit
             e/run_test.sh: line 175: ask_results: command not found

Which might need some discussion with other developers. I'm not sure what this is about, at the moment. Could be still a missing dependency, or a broken shell compatibility thing… I'll look into it.

(from quick google search - https://www.nest-simulator.org/mailinglist/hyperkitty/list/users@nest-simulator.org/message/BCHGWGOVXOD7523GY2ATE7RX5R6QUSJZ/ )

Yes, seems a known thing.

added 1 commit

• a568b865 - add readline dependency `type`

Compare with previous version

added 1 commit

• 0a77d2a3 - factor out test suite runs to a separate variant

Compare with previous version

added 1 commit

• e0a01386 - added possible wheel URL to py-junitparser (commented out)

Compare with previous version

added 65 commits

• e0a01386...0b44513a - 63 commits from branch test-spacktests-workaround
• 16a15825 - Merge branch 'test-spacktests-workaround' into nest-tests-to-post-install-phase
• a5e5019f - add debug output

Compare with previous version

Hi @terhorst,

You may have already found it but this seems to be the problem. It's fixed in the next version of Spack by adding a when= argument to run_after, but until we update you can use (in the function):

if self.spec.variants['testsuite'].value:

My apologies for the delay, busy times. Thanks a lot for basically solving the problem already! I just added your suggestion and removed the @when.

also in case you are wondering about the 3-hour pipeline, it's because py-nose depended on py-setuptools@:57, so now without py-nose Spack was able to get a more recent version and had to re-build every dependent of py-setuptools

added 1 commit

• 89d01a98 - fix condition on 'testsuite' variant

Compare with previous version

added 8 commits

• 89d01a98...a27ab5af - 7 commits from branch test-spacktests-workaround
• 071703b5 - Merge remote-tracking branch 'upstream/test-spacktests-workaround' into...

Compare with previous version

added 2 commits

• 3abde451 - 1 commit from branch test-spacktests-workaround
• 3e85fe9b - Merge branch 'test-spacktests-workaround' into 'nest-tests-to-post-install-phase'

Compare with previous version

OK, seems to be fine now:

==> Installing nest-3.4_rc1-trwon7p34md7mbabs7iyk5j37mym4k5p
==> No binary for nest-3.4_rc1-trwon7p34md7mbabs7iyk5j37mym4k5p found: installing from source
==> Fetching https://github.com/nest/nest-simulator/archive/refs/tags/v3.4_rc1.tar.gz
==> No patches needed for nest
==> nest: Executing phase: 'cmake'
==> nest: Executing phase: 'build'
==> nest: Executing phase: 'install'
==> nest: Successfully installed nest-3.4_rc1-trwon7p34md7mbabs7iyk5j37mym4k5p
  Fetch: 3.51s.  Build: 13m 27.37s.  Total: 13m 30.88s.

You can see the installation test results at the end of the build logs here.

I'll merge this and whenever you have the time you can open a new MR to fix the testsuite variant, is that OK?

Hey! That looks wonderful!

tail -n256 spack-build-03-install-out.txt shows the successful start of NEST:

-- Installing: /builds/technical-coordination/…/share/doc/nest/LICENSE
-- Installing: /builds/technical-coordination/…/doc/nest/README.md
==> [2023-02-01-15:34:30.927632] '/mnt/spack_v0.18.1_t/opt/spack/linux-ubuntu20.04-x86_64/gcc-10.3.0/python-3.8.11-5g66oktvlxp4dqzf7hcifbxqbi7thfpf/bin/python3.8' '-c' 'from pprint import pprint; import nest; pprint(nest.get())'

              -- N E S T --
  Copyright (C) 2004 The NEST Initiative

 Version: 3.4
 Built: Feb  1 2023 15:21:56

 This program is provided AS IS and comes with
 NO WARRANTY. See the file LICENSE for details.

 Problems or suggestions?
   Visit https://www.nest-simulator.org

 Type 'nest.help()' to find out more about NEST.

{'T_max': 1152921504606846.8,
 'T_min': -1152921504606846.8,
 'adaptive_spike_buffers': True,
 'adaptive_target_buffers': True,
 'biological_time': 0.0,
…
…

That's perfect and of course it's fine to merge, yes!

Thanks a lot for all the support and help, Eleni!

(The +testsuite thing I still need to investigate. Those tests however runs on every commit in the nest-simulator repository, so it should be fine.)

marked this merge request as ready

mentioned in commit 0f071298

merged

mentioned in issue #21 (closed)