Force nvcc to use the c++ compiler used to compile the c++ code.

* Move contents of `arbornml` to to `arborio` 
* `arborio` will be built with NeuroML2 morphology support if `ARB_CMAKE_NEUROML` is set. 
* Wrap `arborio::neuroml` and `arborio::morphology_data` for pyarb.
* Add documentation.
* add `neuroml` flag to `setup.py`

Fixes #1256
Fixes #1234

First phase of dynamically loaded mechanism catalogues.

- Add functionality to load and use catalogues from DSOs in Python and C++.
- Add documentation and support scripts to build catalogues DSOs.
- Re-factor CMakeLists.txt for mechanisms
- Limitations
  - Needs (compatible, preferably identical) arbor sources when building catalogues
  - Can only work on MacOS and Linux (uses dlfnc)

* Add the ability to use json and pybind11 libraries installed on the system instead of the
  vesrsions of those libraries bundled as submodules in the Arbor repository
  * Turned on by default
  * Toggled using the new `ARB_USE_BUNDLED_LIBS` CMake flag
* Update documentation for installation
* Update CI and pip workflows to opt in to the bundled libraries

Revert a change introduced in #1282 that required the python module for arbor to build the documentation.

This breaks the readthedocs build, where arbor is not available.

* Remove arbor dependence from `doc/conf.py`, reverting to the original "out of core" approach
* Fix other recently introduced issues that lead to CMake errors when configuring the documentation
  when the Python target is not configured.
  * The CMake variable `PYTHON_EXECUTABLE` wasn't set, but was required to make calls to the
     Python interpreter.
  * Instead of relying on pybind11 to find Python, explicitly detect it using `find_package(Python3 ...)`
  * Make Python3 a hard requirement when building the Python wrapper
  * And a soft requirement otherwise, that will fail only when the docs are built.

Fix CMAKE to use PYTHON_EXECUTABLE that corresponds to the version of Python used to build the python module.

* Docs: emphasised the option of having pip install a local copy of the Arbor source.
* CMake: default Python library install path to `${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))"`. Fixes #1100.
* CMake: remove ARB_PYTHON_PREFIX option, added ARB_PYTHON_LIB_PATH option. We cannot know in general where in the prefix the packages are expected by Python.

* Add CMake infrastructure for new `arbornml` library comprising Arbor's NeuroML2 (C++) support.
* Implement NeuroML2 parsing and interpretation, using libxml2 for XML parsing.
* Add associated documentation, unit tests.
* Replace `arb::util::optional` with `std::optional` in stitch morphology interface.
* Add optional prefix to `arb::label_dict` import.
* Update CI to test arbornml, with associated workarounds for OS X targets.
* Remove glob functionality from `sup`, as it is no longer needed (it was used for lmorpho) and it triggers yet another OS X issue.

Fixes #1088.

* Move label and s-expr parsing code and unit tests from Python to arbor C++ library.
* Update `locset` and `region` constructors that take string arguments to parse strings as s-expressions or "quoted" labels.
* Modify the input stream modifier used to parse asc files to use a lookup table for substitutions, renamed it transmogrifier.                                                             
* Replace `hopefully` type implemented in python headers with an `arb::util::expected`.
* Add `ARBDEV_COLOR` CMake option that forces gcc and clang to always output color output.
* Allow arbitrary strings in labels in region and locset expressions.
* Add `parse_region_expression` and `parse_locset_expression` functions alongside the existing `parse_label_expression` function for use when a region or locset is expected. These calls will promote a quoted string `"label"` to `(region "label")` or 
 `(locset "label")` respectively.
* Add user-defined string literals for labels so that the C++ interface can use `"soma"_lab` instead of awkward escaping `"\"soma\""`.
* Simplify Python wrapper code.

* Update CMake CXX version.
* Make CUDA 10 a minimum requirement and remove CUDA 9 support code.
* Set up g++ Travis tests to use g++ 8.
* Explicitly set C++14 version for nvcc.
* Properly split cuda compilation, including in unit tests.
* Remove unnecessary modcc `SOLVE` warning.
* Update pybind11 module to tag v2.5.0
* Replace `util::size` and `util::data` with `std::` equivalents.

* Set the default ARB_ARCH target to native.
* Tweak travis config to avoid gcc 6 bug that incorrectly implies amd64 target can do avx512.

- Add option to compile Arbor for HIP/CUDA backend using Clang. 
- Add new CMake option `ARB_GPU_COMPILE_TYPE` to distinguish between three possible gpu platforms/builds: `cuda-nvcc`; `cuda-clang`; `hip-clang`
- Add gpu wrapper functions in `arbor/backends/gpu/gpu_api.hpp` and `arborenv/gpu_api.hpp` which call HIP/CUDA functions depending on the platform
- Rename functions and files: cuda -> gpu
- Add downgraded warp primitives for HIP
- Implement `uuid` workaround for HIP
- Set correct alignment and warp size for AMD gpus
- Update installation guide and docs.
- Update pip/setuptools to support new gpu targets

Fixes #833

* Remove interleaved matrix solver on GPU.
* Fix out of bound memory bug when initialising matrix for all back ends
* fix singed-vs-unsigned comparison warning 

Fixes #1014 
Fixes #1013

* Replace TCLAP with tinyopt, remove TCLAP from `ext/`.
* Replace `sup/tinyopt.hpp` with copy of external tinyopt, update use of `tinyopt` in unit tests and examples to suit.
* Use tinyopt's 'smallopt' interface in `modcc`, `brunel` example.
* Update tinyopt interface in `single` example.
 Fix our attributions and add HBP acknowledgement

Addresses #557 #695 #803 

Improve the Python wrapper generation and installation:
  - install a proper module that can be extended with Python code;
  - give the user more control over where to install the module (e.g. as a user package or in a virtualenv).

During building, the following sub-directory is built in the build director (`CMAKE_BINARY_DIR`)
```
└── python
    └── arbor
            ├── __init__.py
            ├── arbor.so
            └── VERSION
```
This path can then be copied VERBATIM to the target installation path. By default this will be in `CMAKE_INSTALL_PREFIX/lib/python%d.%d/site-packages`.
An additional CMake parameter `ARB_PYTHON_PREFIX` can be used to specify an alternative destination for installing the Python module.

The Python part of the wrapper, implemented in `__init__.py` is currently very limited, only providing `__version__` and `__config__` variables.

The installation guide was updated to cover the Python installation.

A big update that wraps over 90% of the public C++ API, with enough functionality to let Python users to perform useful modelling.

Key features
- wrapping of cable cell functionality
  - user-defined explicit compartmentalisation not supported
- `single_cell_model` abstraction unique to the Python wrapper that simplifies developing and testing single cell models
- one-dimensional cell builder for Python wrapper that simplfies building cells that 
  - in and of itself limited in scope, but a very useful example of mapping a richer morphology builder onto `sample_tree`s.
- parsing of region and location expressions from strings
  - implements a generic s-expression parser that we can use for other purposes later

Extends the available locset/region expressions.

* Add `distal_interval` and `proximal_interval` expressions that return a region defined by an initial locset and a distance to extend in the distal or proximal directions respectively.
* Add `radius_lt`, `radius_le`, `radius_gt`, `radius_ge` region expressions describing the subset of the morphology with radius less than (or less than or equal to etc.) a given value.
* Add `z_dist_from_soma_lt`, `z_dist_from_soma_le`, `z_dist_from_soma_gt`, `z_dist_from_soma_ge` region expressions describing the subset of the morphology which is less than (resp. less than or equal to, etc.) the given distance along the z-axis from the root sample.
* Add `uniform` locset expression giving a uniformly sampled set of locations from a given region based on a supplied PRNG seed value and a closed interval [left, right] of elements from the corresponding pseudo-random sequence.
* Add `Random123` as a library to `ext` and `util::uniform` ...

* Make access to `segment`s in a `cable_cell` read only
* Change cell constructor to require morphology and label dictionary; `make_cable_cell` no longer required.
* Consolidate paint/place functionality on `cable_cell`.
* Add simple cell builder wrapping new API for use in unit tests.
* Remove validation test code: appropriate validation tests will need to be implemented in nsuite.

Fixes #871 

* libunwind is optional, and off by default.

* fix description of CMake option

* Replace old `arb::morphology` class with new `arb::sample_tree` and `arb::morphology` types, where `arb::sample_tree` manages a set of morphology sample points and their topology, and `arb::morphology` presents a branch-based view over samples described by an `arb::sample_tree`. The new classes support morphologies with cable-like or spherical somata.
* Supply a new conversion routine for forming a `arb::cable_cell` from a morphology, with support for both classes of soma.

Fixes #835 

Eric Müller authored 5 years ago and Benjamin Cumming committed 5 years ago

* cmake/FindUnwind.cmake used the wrong variable (caused by commit ad1c78ab)
* CMakeLists.txt seems to use wrong syntax for providing a compiler define
  of `WITH_UNWIND` (used in `arbor/util/unwind.cpp`)

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Remove the miniapp example.

* Remove the miniapp source from example/miniapp.
* Remove references to miniapp from CMake files.
* Remove miniapp from the travis tests.
* Fix unused variable warning in brunel miniapp.

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

First step towards the Python front end.

This commit sets up the structure of the python implementation
* directory structure
* git submodule for pybind11
* best practices for making bindings with pybind11
* unit testing for the python front end

It implements the following features in the Python front end
* execution contexts
* gpu detection
* thread count detection
* MPI initialization helpers.

Fixes #667.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Bump minimum CMake version to 3.9.

Fixes #691.

Implement a basic glob routine, supporting a subset of POSIX behaviour (e.g. no character classes), as a fallback for platforms such as Android which do not include it in their libc.

* Add CMake configuration option `ARB_USE_POSIX_GLOB`, defaults to `ON`, that determines if the fallback implementation is used or not.
* Extend `sup::path` functionality to add directory iterators and a couple more path manipulation routines, again following the C++17 `std::filesystem` interface.
* Add an NFA pattern matcher `glob_basic_match` and file system `glob_basic` function; the latter is abstracted over a file system provider object, primarily for testing/mocking purposes.
* Add unit tests for new `sup::path` functionality and for `glob_basic`.

FIxes #181.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Make a new installed library `libarborenv.a` covering a subset of the `sup` library functionality, with corresponding installed CMake target `arbor::arborenv`.

* Move NVML or CUDA 10 API decision for GPU UUID discovery to top level CMake.
* Move affinity, concurrency, MPI init guard, and gpu detection and negotiation functionality out of `sup` and into new library `arborenv`.
* Move `include/arbor` in project tree to `arbor/include/arbor` (for consistency across `sup`, `arbor`, and `arborenv` subdirectories.)
* Wrangle more explicit library dependency adding CMake code into the installed `arbor-config.cmake`, to help mitigate [CMake issue #18614](https://gitlab.kitware.com/cmake/cmake/issues/18614).
* Have `arborenv` code throw `std::runtime_exception` instead of `arb::arbor_error`. (We are still using `arb::mpi_error` though for a failure in `with_mpi`.)
* Move `scope_exit` into the `arb::util` namespace.
* Merge `affinity.hpp` into `concurrency.hpp`.
* Rename `gpu.hpp` to `gpu_env.hpp` in `arborenv` includes.

Fixes #647.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Extend sup library to support assigning unique GPUs to MPI ranks.

Fixes #648.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake wants to run a device link pass with nvcc despite
there being no CUDA seperable compilation enabled anywhere,
and then passes on -pthread to that unnecessary nvcc
invocation when we use the Threads dependency. The latter,
at least, is fixed in CMake 3.13.

We used the prefer -pthread option for compatibility with
our earlier build configuration; turning it off will
hopefully have no consequence.

We also enable device linking on the arbor library. Which
is not needed, but if they are going to insist on doing it,
it should be on the library rather than the executable.

CMake then goes and does it on the executable anyway. Great.

Fixes #645.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Forward CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES to compilation of arbor library and unit tests.

Fixes #651

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

This reverts commit be2a8a9f.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Add a new Hines matrix solver implementation for the GPU that can solve a single tree in parallel with multiple threads. It replaces the interleaved solver, which used a single thread to solve each matrix.
Branches with the same common root in the tree can be solved independently on each of the forward and backward solution passes. 

* Add a matrix storage type, `arb::gpu::matrix_state_fine` that stores the branches of multiple trees for efficient backward and forward substitution.
* Extend the `arb::tree` data structure to support operations for choosing a new root node and determining a root node which minimises the maximum distance between the root and any of the trees leaves. 
* Implement code for rebalancing a set of matrix trees, a.k.a. a "forest" of trees.
* Add CUDA kernels for efficiently performing matrix assembly and matrix solution steps.
* Add CMake option `ARB_WITH_GPU_FINE_MATRIX` for toggling the new solver (default `on`).

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #622.

cf. CMake issue 16716: https://gitlab.kitware.com/cmake/cmake/issues/16716

* Bump version post 0.1 for development.
* Read version string from file VERSION.
* Strip suffix to make a numerical, CMake-compatible PROJECT_VERSION.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #618 and fixes #617.

*  Add convenience targets: 'examples' for all examples; 'tests' for all tests.
* Add support for component-testing in installed CMake package.
* Allow test for MPI support via find_package via component.
* Remove REQUIRED specification from `find_dependency()` commands in generated config.
* Update `mech_vec.cpp` to match new `fvm_lowered_cell_impl` constructor.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #612.

* Fix issues with permissions on directories created at install time (at least for CMake 3.11+).
* Add CMake export guff to various targets and install an `arbor-config.cmake` for consumption by other CMake-based projects.