- unit tests and some examples build their own mechanism catalogue
- repeated logic for detecting GPUs, passing compile time flags
- leverage the cmake variables and targets which are already there since these catalogues are internal to arbor
- build a static library with mechanisms, which links to arbor-private-deps

Fix a typo preventing the generation of GPU mechs.

This is prep for meson in addition to simplifying (ie reducing) the amount of scripting
we have. In the long run, I'd like to remove `BuildModules.cmake` too, but this might
be superseded by meson and this PR is self-contained.

# Changes
- `modcc` accepts now a list of NMODL files
- likewise, `modcc` is now able to spit out a catalogue.cpp file
- remove the `generate-catalogue` script, simplify a-b-c
- remove some options from `modcc` we never use
- No longer allow external modcc

# Changes
- Bump ABI to v0.1.1
- Add `arb_mechanism` to bundle `type` and `interface`s
- Adjust `generate_catalogue` and `build_catalogue`
- Return `nullptr` for GPU interfaces if not enabled

# Open Questions
- How to tell CMake not to link against `libarbor.a` and only use headers?
- ~~How to get around `#ifdef ARB_GPU_ENABLED`~~ Update: just embrace it.

# Linked Issues
Fixes #1834 

- build-catalogue
  - now installed by pip
  - enable GPU-backends
  - isolate from being installed in a non-default dir
  - rename to arbor-build-catalogue
  - allow hand-written C++ mechanisms to be linked in the same catalogue.
- CMake
  - enable PIC on arbor always to allow linking to shared objects
  - use relative paths when configuring a-b-c, to make relocation less of a problem
- Python
    - ensure a-b-c is installed by pip, along with headers and libarbor.a
    - throw errors if tools (cmake scripts, arbor package, modcc) missing
    - transition setup.py setuptools -> skbuild.
- Wheels
    - Add NML2 support to wheels
    - scripts/build-wheels.sh builds wheels, in principle valid for submission to PyPI, on your own hardware. This should be kept in sync with .github/workflows/ciwheel.yml
    - scripts/patchwheel.py corrects the rpath in the libraries in the wheels, working around a bad interplay between auditwheel and skbuild, see pypa/auditwh...

Implements #1376.

* Provide a common C linkage ABI for externally compiled mechanisms, for both CPU and GPU.
* Remove mechanism type hierarchy (`concrete_mechanism` etc.), and move corresponding functionality to the back-end shared state objects. Mechanism catalogue is no longer indexed by type id.
* Distinguish between SIMD optimal alignment and SIMD width with new `min_align` attribute. Mechanisms provide both pieces of information via ABI.

- Add an optional CMake option `ARB_CXX_FLAGS_TARGET` for setting target-specific flags to use when compiling for the target architecture (not applied to compilation of modcc).
- If `ARB_ARCH=="none"` CMake will not add architecture-specific `mtune/march/mcpu` flags
- Remove `ARB_CXXOPT_ARCH` from the installed `arbor-config.cmake`, and replace with more general `ARB_CXX_FLAGS_TARGET`.
- Update spack `package.py` to use this feature to pass custom flags.

Fixes #1519
Fixes #1522 

Replaces PR #1518 

* Use CMake check_cxx_source_compiles and check_cxx_source_runs to detect how std::filesystem
is enabled. The probes are executed first, if none succeeds, we use link-only to cover the possibility that we
are cross-compiling.

First phase of dynamically loaded mechanism catalogues.

- Add functionality to load and use catalogues from DSOs in Python and C++.
- Add documentation and support scripts to build catalogues DSOs.
- Re-factor CMakeLists.txt for mechanisms
- Limitations
  - Needs (compatible, preferably identical) arbor sources when building catalogues
  - Can only work on MacOS and Linux (uses dlfnc)

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Global temperature for mechanisms.

* Make 'celsius' magic in modcc: now an indexed variable.
* Add a new temperature data source for indexed variables.
* Add support to printers for indexed variables that reference a scalar.
* Check that indexed variables aren't used in PROCEDURE blocks (this is a problem not just for 'celsius').
* Modify built-in mod files to pass celsius as a parameter to rates() procedures.
* Add global temperature to shared_state classes, and initialize through backend mechanism superclasses.
* Add some infrastructure for unit-test only mechanisms.
* Set modcc flags globally in top level CMakeLists.txt.
* Add test mechanism/module for checking celsius setting.
* Add unit test for multicore and gpu mechanism celsius setting.
* Make common mechanism private field data access helper for unit tests.
* Use helper in temperature, synapses tests.
* Fix warning in `distribued_context.hpp` about errant semicolon.
* Fix global scalar ref for SIMD printing.
* Use correct ARB_CXXOPT_ARCH instead of incorrect CXXOPT_ARCH in various CMakeLists.txt files.
* Add special case for no-non scalar indexed variables in API loop in SIMD printing.

Fixes #386

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake and build refactoring

*   Use CUDA as first-class language (leading to CMake 3.9 minimum version requirement).

*   Use 'modern CMake' interface libraries for compiler options, include file and library dependency tracking. Interface library targets:
    * `arbor-deps`: compiler options and library requirements for the `libarbor.a` static library, as governed by configure-time options and environment.
    * `arbor-private-headers`: include path for non-installed headers, as required by unit tests and arbor itself.
    * `arbor-aux`: helper classes and utilities used across tests and examples.
    * `ext-json`, `ext-tclap`, `ext-tbb`, `ext-benchmark`, `ext-sphinx_rtd_theme`: externally maintained software that we include (directly or via submodule) in the `ext/` subdirectory.
 
*   Single static library `libarbor.a` includes all built-in modules and CUDA objects.

*   Simply configuration options:
    *  `ARB_WITH_TRACE`, `ARB_AUTORUN_MODCC_ON_CHANGES` `ARB_SYSTEM_TYPE` removed.
    * External `modcc` is provided by `ARB_MODCC` configuration option; if provided `modcc` is still buildable, but is not included in the default target.
    * `ARB_PRIVATE_TBBLIB`, defaulting to `OFF`, instructs the build to make TBB from the included submodule.

*   Extend `ErrorTarget` functionality to provide a dummy target or an error target based on a condition.
*   Generate header version defines and library version variables based on git status and project version, via new script `include/git-source-id`.
*   All generated binaries now placed in `bin/` subdirectory at build.
*   Install targets installs: public headers (incomplete); static library; `modcc` tool; `lmorpho` executable; `html` documentation (examples, tests and validation data are currently not installed).
*   Executable targets have had the `.exe` suffix removed; unit tests are labelled `unit` (arbor unit tests), `unit-modcc` (modcc unit tests), `unit-local` (distributed tests with local context), `unit-mpi` (distributed tests with MPI context).
*   More graceful handling of configure-time detection of `nrniv`, Julia and required Julia modules for validation data generation.
*   Add `cmake/FindJulia.cmake`, `cmake/FindTBB.cmake`  package finders, and adjust `cmake/FindUnwind.cmake` to use link library-style properties.
*  Adjust travis script to test `unit-local` and `unit-mpi` if appropriate.
*  Simply documentation `conf.py`.

Source relocation and reorganization

* All external project sources and files moved to `ext/`.
* Source code refactoring to decouple library-using code from the configure-time definitions that govern arbor behaviour: removes conditional code in public headers that depends upon `ARB_WITH_X`-type definitions at compile time. Affected code is is in the public interfaces for MPI, the threading implementation, and the profiler.
* Remove `util/debug.hpp`; split out functionality for pretty-printing from assertion handling.
* Make FVM cell non-physical voltage check a run-time cell-group parameter.
* Move spike double buffer implementation to `simulation.cpp`.
* Make timer utility wrap POSIX `clock_gettime` independent of threading configuration.
* Make `mpi_error` derive from `system_error` and follow C++11 `system_error` semantics.
* `EXPECTS` macro replaced by `arb_assert` macro.
* JSON dependency removed from `libarbor.a` and header files: moved to auxiliary library.
* Publicly visible macros garner an `ARB_` prefix as required.
* Move SWC test file to `test/unit` directory.
* Work-in-progress splitting of public from private includes: as a convention not entirely adhered to as yet, private headers within arbor source are included with `""`, public headers with `<>`.

Modcc interface changes

* Expose via `--namespace` option the functionality that sets the namespace in generated code.
* Use `--profile` option to add profiler hooks to generated code; uses public function interface directly rather than `PE/PL` macros in order to avoid public `PE` and `PL` defines.

Ben Cumming authored 7 years ago and Sam Yates committed 7 years ago

Separable compilation of the CUDA kernels generated by modcc from NMODL files.

CMake scripts:
* Update the `build_modules()` helper function to cleanly handle calls to modcc that generate multiple output files.
* Add a new library target `gpu_mechanisms` for the separately compiled CUDA kernels and the implementation of their C wrappers.
* Reduce verbosity of compilation messages.

* Simplify mechanism C++ namespace use: move everything in nest::mc::mechanisms::gpu::_mechanism-name_ into `nest::mc::gpu`, and similarly for multicore mechanism implementations, ions.
* Remove template parameters for `value_type` and `size_type` from all of the 
mechanism implementations, and use `fvm_value_type` and `fvm_size_type` everywhere instead.

modcc changes:
* Modify `CUDAPrinter` to keep track of 3 text buffers, one each for 
  "implementation", "interface" and "implementation interface":
* Write the CUDA implementation interface to `X_impl.hpp`, comprising the definition of the mechanism-specific 'X_ParamParck' struct used to pass function arguments to the CUDA kernels.
* Write the CUDA kernels and C wrappers to `X_impl.cu`.
* Write the public C++ mechanism interface (with calls to implementation wrappers) to `X.hpp`.
* Modify modcc driver to support multiple generated output files.

Vasileios Karakasis authored 8 years ago and Sam Yates committed 8 years ago

These tests are intended to test the sanity of the `modcc` generated code for the individual mechanisms. The don't have any physical background. Potentially optimized CPU-targeted mechanisms generated in the build are compared with unoptimized mechanisms generated from the reference modules.

* Add generic unit tests for individual mechanisms.
* Make unit tests exercise potential problems with aliased indexes (point processes).
* Ensure unit tests correspond to multiple low level vector operations.
* Ensure unit tests run with voltage, current and indices initialized with varying values.
* Refactor CMake code for module compilation to reduce cut-and-paste code and build complexity.