Implements #1376.

* Provide a common C linkage ABI for externally compiled mechanisms, for both CPU and GPU.
* Remove mechanism type hierarchy (`concrete_mechanism` etc.), and move corresponding functionality to the back-end shared state objects. Mechanism catalogue is no longer indexed by type id.
* Distinguish between SIMD optimal alignment and SIMD width with new `min_align` attribute. Mechanisms provide both pieces of information via ABI.

The visibility of symbols in the python shared library is set to 'hidden'. However, examining the dynamic symbol table of the generated `.so` file revealed that all the symbols of the static arbor libraries are still visible. This causes some issues on the Mac M1. To resolve, all source files are now compiled with `-fvisibility=hidden`.

- Add an optional CMake option `ARB_CXX_FLAGS_TARGET` for setting target-specific flags to use when compiling for the target architecture (not applied to compilation of modcc).
- If `ARB_ARCH=="none"` CMake will not add architecture-specific `mtune/march/mcpu` flags
- Remove `ARB_CXXOPT_ARCH` from the installed `arbor-config.cmake`, and replace with more general `ARB_CXX_FLAGS_TARGET`.
- Update spack `package.py` to use this feature to pass custom flags.

Fixes #1519
Fixes #1522 

Replaces PR #1518 

* Remove std::filesystem to workaround OSX features; use `std::string` in interfaces expecting a path.

* Auto-generate binary Python wheels, triggered by git tags 'v*'.
  * Wheels are generated for Python versions 3.6-3.9, for Linux using the [PyPA manylinux-2014 image](https://github.com/pypa/manylinux) and MacOS using macos-latest (10.15 at time of writing).
  * The Action generates `sdist` for any other platforms.
  * The cibuildwheel action automatically prepares the wheels using [auditwheel](https://cibuildwheel.readthedocs.io/en/stable/options/#repair-wheel-command), which means bundling of external deps (e.g. libxml2 for NeuroML).
  * The output is all that's needed for publication of a new version to (Test)PyPI.
* `setup.py` defaults `neuroml` to on.
* A small change to `CMakeLists.txt` was required to build the wheels: based on this [Pybind hint](https://pybind11.readthedocs.io/en/stable/compiling.html#findpython-mode) `Development.Module`is now required, and not `Development` when CMake searches for Python. I confirm this change is needed to make Arbor build in the manylinux docker images, but I do not understand what the change implies, other than that we now don't link to `libpython`.

Force nvcc to use the c++ compiler used to compile the c++ code.

* Move contents of `arbornml` to to `arborio` 
* `arborio` will be built with NeuroML2 morphology support if `ARB_CMAKE_NEUROML` is set. 
* Wrap `arborio::neuroml` and `arborio::morphology_data` for pyarb.
* Add documentation.
* add `neuroml` flag to `setup.py`

Fixes #1256
Fixes #1234

First phase of dynamically loaded mechanism catalogues.

- Add functionality to load and use catalogues from DSOs in Python and C++.
- Add documentation and support scripts to build catalogues DSOs.
- Re-factor CMakeLists.txt for mechanisms
- Limitations
  - Needs (compatible, preferably identical) arbor sources when building catalogues
  - Can only work on MacOS and Linux (uses dlfnc)

* Add the ability to use json and pybind11 libraries installed on the system instead of the
  vesrsions of those libraries bundled as submodules in the Arbor repository
  * Turned on by default
  * Toggled using the new `ARB_USE_BUNDLED_LIBS` CMake flag
* Update documentation for installation
* Update CI and pip workflows to opt in to the bundled libraries

Revert a change introduced in #1282 that required the python module for arbor to build the documentation.

This breaks the readthedocs build, where arbor is not available.

* Remove arbor dependence from `doc/conf.py`, reverting to the original "out of core" approach
* Fix other recently introduced issues that lead to CMake errors when configuring the documentation
  when the Python target is not configured.
  * The CMake variable `PYTHON_EXECUTABLE` wasn't set, but was required to make calls to the
     Python interpreter.
  * Instead of relying on pybind11 to find Python, explicitly detect it using `find_package(Python3 ...)`
  * Make Python3 a hard requirement when building the Python wrapper
  * And a soft requirement otherwise, that will fail only when the docs are built.

Fix CMAKE to use PYTHON_EXECUTABLE that corresponds to the version of Python used to build the python module.

* Docs: emphasised the option of having pip install a local copy of the Arbor source.
* CMake: default Python library install path to `${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))"`. Fixes #1100.
* CMake: remove ARB_PYTHON_PREFIX option, added ARB_PYTHON_LIB_PATH option. We cannot know in general where in the prefix the packages are expected by Python.

* Add CMake infrastructure for new `arbornml` library comprising Arbor's NeuroML2 (C++) support.
* Implement NeuroML2 parsing and interpretation, using libxml2 for XML parsing.
* Add associated documentation, unit tests.
* Replace `arb::util::optional` with `std::optional` in stitch morphology interface.
* Add optional prefix to `arb::label_dict` import.
* Update CI to test arbornml, with associated workarounds for OS X targets.
* Remove glob functionality from `sup`, as it is no longer needed (it was used for lmorpho) and it triggers yet another OS X issue.

Fixes #1088.

* Move label and s-expr parsing code and unit tests from Python to arbor C++ library.
* Update `locset` and `region` constructors that take string arguments to parse strings as s-expressions or "quoted" labels.
* Modify the input stream modifier used to parse asc files to use a lookup table for substitutions, renamed it transmogrifier.                                                             
* Replace `hopefully` type implemented in python headers with an `arb::util::expected`.
* Add `ARBDEV_COLOR` CMake option that forces gcc and clang to always output color output.
* Allow arbitrary strings in labels in region and locset expressions.
* Add `parse_region_expression` and `parse_locset_expression` functions alongside the existing `parse_label_expression` function for use when a region or locset is expected. These calls will promote a quoted string `"label"` to `(region "label")` or 
 `(locset "label")` respectively.
* Add user-defined string literals for labels so that the C++ interface can use `"soma"_lab` instead of awkward escaping `"\"soma\""`.
* Simplify Python wrapper code.

* Update CMake CXX version.
* Make CUDA 10 a minimum requirement and remove CUDA 9 support code.
* Set up g++ Travis tests to use g++ 8.
* Explicitly set C++14 version for nvcc.
* Properly split cuda compilation, including in unit tests.
* Remove unnecessary modcc `SOLVE` warning.
* Update pybind11 module to tag v2.5.0
* Replace `util::size` and `util::data` with `std::` equivalents.

* Set the default ARB_ARCH target to native.
* Tweak travis config to avoid gcc 6 bug that incorrectly implies amd64 target can do avx512.

- Add option to compile Arbor for HIP/CUDA backend using Clang. 
- Add new CMake option `ARB_GPU_COMPILE_TYPE` to distinguish between three possible gpu platforms/builds: `cuda-nvcc`; `cuda-clang`; `hip-clang`
- Add gpu wrapper functions in `arbor/backends/gpu/gpu_api.hpp` and `arborenv/gpu_api.hpp` which call HIP/CUDA functions depending on the platform
- Rename functions and files: cuda -> gpu
- Add downgraded warp primitives for HIP
- Implement `uuid` workaround for HIP
- Set correct alignment and warp size for AMD gpus
- Update installation guide and docs.
- Update pip/setuptools to support new gpu targets

Fixes #833

* Remove interleaved matrix solver on GPU.
* Fix out of bound memory bug when initialising matrix for all back ends
* fix singed-vs-unsigned comparison warning 

Fixes #1014 
Fixes #1013

* Replace TCLAP with tinyopt, remove TCLAP from `ext/`.
* Replace `sup/tinyopt.hpp` with copy of external tinyopt, update use of `tinyopt` in unit tests and examples to suit.
* Use tinyopt's 'smallopt' interface in `modcc`, `brunel` example.
* Update tinyopt interface in `single` example.
 Fix our attributions and add HBP acknowledgement

Addresses #557 #695 #803 

Improve the Python wrapper generation and installation:
  - install a proper module that can be extended with Python code;
  - give the user more control over where to install the module (e.g. as a user package or in a virtualenv).

During building, the following sub-directory is built in the build director (`CMAKE_BINARY_DIR`)
```
└── python
    └── arbor
            ├── __init__.py
            ├── arbor.so
            └── VERSION
```
This path can then be copied VERBATIM to the target installation path. By default this will be in `CMAKE_INSTALL_PREFIX/lib/python%d.%d/site-packages`.
An additional CMake parameter `ARB_PYTHON_PREFIX` can be used to specify an alternative destination for installing the Python module.

The Python part of the wrapper, implemented in `__init__.py` is currently very limited, only providing `__version__` and `__config__` variables.

The installation guide was updated to cover the Python installation.

A big update that wraps over 90% of the public C++ API, with enough functionality to let Python users to perform useful modelling.

Key features
- wrapping of cable cell functionality
  - user-defined explicit compartmentalisation not supported
- `single_cell_model` abstraction unique to the Python wrapper that simplifies developing and testing single cell models
- one-dimensional cell builder for Python wrapper that simplfies building cells that 
  - in and of itself limited in scope, but a very useful example of mapping a richer morphology builder onto `sample_tree`s.
- parsing of region and location expressions from strings
  - implements a generic s-expression parser that we can use for other purposes later

Extends the available locset/region expressions.

* Add `distal_interval` and `proximal_interval` expressions that return a region defined by an initial locset and a distance to extend in the distal or proximal directions respectively.
* Add `radius_lt`, `radius_le`, `radius_gt`, `radius_ge` region expressions describing the subset of the morphology with radius less than (or less than or equal to etc.) a given value.
* Add `z_dist_from_soma_lt`, `z_dist_from_soma_le`, `z_dist_from_soma_gt`, `z_dist_from_soma_ge` region expressions describing the subset of the morphology which is less than (resp. less than or equal to, etc.) the given distance along the z-axis from the root sample.
* Add `uniform` locset expression giving a uniformly sampled set of locations from a given region based on a supplied PRNG seed value and a closed interval [left, right] of elements from the corresponding pseudo-random sequence.
* Add `Random123` as a library to `ext` and `util::uniform` ...

* Make access to `segment`s in a `cable_cell` read only
* Change cell constructor to require morphology and label dictionary; `make_cable_cell` no longer required.
* Consolidate paint/place functionality on `cable_cell`.
* Add simple cell builder wrapping new API for use in unit tests.
* Remove validation test code: appropriate validation tests will need to be implemented in nsuite.

Fixes #871 

* libunwind is optional, and off by default.

* fix description of CMake option

* Replace old `arb::morphology` class with new `arb::sample_tree` and `arb::morphology` types, where `arb::sample_tree` manages a set of morphology sample points and their topology, and `arb::morphology` presents a branch-based view over samples described by an `arb::sample_tree`. The new classes support morphologies with cable-like or spherical somata.
* Supply a new conversion routine for forming a `arb::cable_cell` from a morphology, with support for both classes of soma.

Fixes #835 

Eric Müller authored 5 years ago and Benjamin Cumming committed 5 years ago

* cmake/FindUnwind.cmake used the wrong variable (caused by commit ad1c78ab)
* CMakeLists.txt seems to use wrong syntax for providing a compiler define
  of `WITH_UNWIND` (used in `arbor/util/unwind.cpp`)

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Remove the miniapp example.

* Remove the miniapp source from example/miniapp.
* Remove references to miniapp from CMake files.
* Remove miniapp from the travis tests.
* Fix unused variable warning in brunel miniapp.

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

First step towards the Python front end.

This commit sets up the structure of the python implementation
* directory structure
* git submodule for pybind11
* best practices for making bindings with pybind11
* unit testing for the python front end

It implements the following features in the Python front end
* execution contexts
* gpu detection
* thread count detection
* MPI initialization helpers.

Fixes #667.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Bump minimum CMake version to 3.9.

Fixes #691.

Implement a basic glob routine, supporting a subset of POSIX behaviour (e.g. no character classes), as a fallback for platforms such as Android which do not include it in their libc.

* Add CMake configuration option `ARB_USE_POSIX_GLOB`, defaults to `ON`, that determines if the fallback implementation is used or not.
* Extend `sup::path` functionality to add directory iterators and a couple more path manipulation routines, again following the C++17 `std::filesystem` interface.
* Add an NFA pattern matcher `glob_basic_match` and file system `glob_basic` function; the latter is abstracted over a file system provider object, primarily for testing/mocking purposes.
* Add unit tests for new `sup::path` functionality and for `glob_basic`.

FIxes #181.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Make a new installed library `libarborenv.a` covering a subset of the `sup` library functionality, with corresponding installed CMake target `arbor::arborenv`.

* Move NVML or CUDA 10 API decision for GPU UUID discovery to top level CMake.
* Move affinity, concurrency, MPI init guard, and gpu detection and negotiation functionality out of `sup` and into new library `arborenv`.
* Move `include/arbor` in project tree to `arbor/include/arbor` (for consistency across `sup`, `arbor`, and `arborenv` subdirectories.)
* Wrangle more explicit library dependency adding CMake code into the installed `arbor-config.cmake`, to help mitigate [CMake issue #18614](https://gitlab.kitware.com/cmake/cmake/issues/18614).
* Have `arborenv` code throw `std::runtime_exception` instead of `arb::arbor_error`. (We are still using `arb::mpi_error` though for a failure in `with_mpi`.)
* Move `scope_exit` into the `arb::util` namespace.
* Merge `affinity.hpp` into `concurrency.hpp`.
* Rename `gpu.hpp` to `gpu_env.hpp` in `arborenv` includes.

Fixes #647.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Extend sup library to support assigning unique GPUs to MPI ranks.

Fixes #648.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake wants to run a device link pass with nvcc despite
there being no CUDA seperable compilation enabled anywhere,
and then passes on -pthread to that unnecessary nvcc
invocation when we use the Threads dependency. The latter,
at least, is fixed in CMake 3.13.

We used the prefer -pthread option for compatibility with
our earlier build configuration; turning it off will
hopefully have no consequence.

We also enable device linking on the arbor library. Which
is not needed, but if they are going to insist on doing it,
it should be on the library rather than the executable.

CMake then goes and does it on the executable anyway. Great.

Fixes #645.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Forward CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES to compilation of arbor library and unit tests.

Fixes #651

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

This reverts commit be2a8a9f.