akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Python 3.6 is tested (serial and mpi), in particular: 
- 3.6.3 on linux with a virtualenv, 
- 3.6.5 as downgrade from 3.7 on osx as in https://stackoverflow.com/questions/52584907/how-to-downgrade-python-from-3-7-to-3-6?rq=1 (other solutions did not work)

Fixes #715

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Rename `arb:simd::simd_detail` namespace to `arb::simd::detail`.
* Update SIMD documentation to suit.

Fixes #468.

Fixes #717.

* Move time interval update step after threshold check.

Rename mc_cell to cable_cell, cable1d_neuron to cable, and lif_neuron to lif in public API.

Fixes #701 

noraabiakar authored 6 years ago and Benjamin Cumming committed 6 years ago

Add gap junctions and improve general model overview in docs.

* Replace Arbor Model::Common Types with Arbor Model::Concepts, that introduces the concept of cells, connections and gap junctions. 
* Add gap junction documentation to cpp:recipe
* Document the implicit rule forcing cells connected by gap junctions to be in the same cell group.
* Add documentation that discusses core concepts in Arbor models.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Remove data/ directory, contents as they are not used now in Arbor.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Remove the miniapp example.

* Remove the miniapp source from example/miniapp.
* Remove references to miniapp from CMake files.
* Remove miniapp from the travis tests.
* Fix unused variable warning in brunel miniapp.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Adds a small self contained example demonstrating the use of SWC and morphology specifications, to fill a void that will be left when the miniapp is removed.

* Adds `single-cell` example, together with short README and example SWC file.
* Removes redundant constructors for context objects.
* Corrects and simplifies some of the context comments.
* Add a method to `mc_segment` for calculating (approximately) a lower bound on the length constant of that segment for a given frequency, for use with a NEURON 'd-lambda' style discretization rule.

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Remove old code and comments from Python config code.

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #700

Usage in python as: 

```
>>>import arbor
>>>arbor.config()
{'mpi': True, 'mpi4py': True, 'gpu': False, 'version': '0.1.1-dev'}
>>>d = arbor.config()
>>>arbor.print_config(d)
Arbor's configuration:
     mpi    :       True
     mpi4py :       True
     gpu    :      False
     version:  0.1.1-dev
```

Update Hardware API documentation

* split the domain decomposition and hardware API docs into separate pages
* update hardware API to reflect new *libarbor* and *libarborenv*
* add basic documentation for `optional`, `any` and `unique_any` types.

Update the installation guide to reflect the latest supported tool,
compiler and library versions.

Remove the Python docs, because they documented features that have not
been implemented yet. The Python docs can be added incrementally as
features are implemented.

Start work on oupdating the documenetation for hardware interfaces and
domain decomposition.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Add SIMD neon implementation for aarch64.
* Update unit tests to suit.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

Allow distinct point processes with the same linear dynamics to be combined if they reside in the same CV after discretization.

* Add `linear` flag to `mechinfo` struct.
* Add a linearity test to `modcc` that determines if a point process is a suitable candidate for coalescing: state and current updates must be linear, homogeneous functions of the state variables; state change on an event arrival (`net_receive`) must be independent of current state values.
* Add `mechanism_state_table()` inquiry function to backend mechanism interface.
* Add a field `multiplicity` to `fvm_mechanism_config` that accounts for merged synapse instances, and a corresponding `multiplicity` field in `mechanism::layout`.
* Merge linear synapses of the same type  in `fvm_build_mechanism_data` if they have the same parameters.
* Add global mc_cell property `coalesce_synapses` which enables or disables the merging of linear synapses at run time.
* Rename virtual `nrn_init()` interface funct...

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Update documentation for Python.

    splits the conceptual model ideas from the C++ docs into their own section
    has C++ and Python docs for recipes, domain decomposition, etc.

fixes #667

Added the following documentation (structure):

GETTING STARTED:

    Installing Arbor/Requirements/Optional Requirements/Python
    Installing Arbor/Building and Installing Arbor/Python Front End

MODEL BASICS:

    Overview
    Common Types
    Recipes
    Domain Decomposition
    Simulations

PYTHON:

    Overview
    Common Types
    Recipes
    Domain Decomposition
    Simulations

DEVELOPERS:

    Python Profiler
    Python Unit Testing

GETTING STARTED has two added sections of optional requirements using python and how to build the python front end.

MODEL BASICS describes Arbor's concepts in general (independent of programming language), thus general information on concepts in C++ API was moved here/ added.

PYTHON describes Arbor's pyth...

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

First step towards the Python front end.

This commit sets up the structure of the python implementation
* directory structure
* git submodule for pybind11
* best practices for making bindings with pybind11
* unit testing for the python front end

It implements the following features in the Python front end
* execution contexts
* gpu detection
* thread count detection
* MPI initialization helpers.

Fixes #667.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Bump minimum CMake version to 3.9.

Fixes #691.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

Fixes #683.

* Distinguish between per-cell matrix solver and per-'integration domain' state in the FVM back-end, adding new index for FVM back-ends mapping CVs to integration domain index.
* Move to one event queue per integration domain, avoiding extra logic for synchronisation of times across gap-junction connected cells. This requires a merging of event lanes in `mc_cell_group` when constructing the staged events in the multi-event queue.
* Add synapses to gap junction example.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

Add support for gap junctions in mc_cells, modelled as a conductance between 2 cell CVs. Gap junctions act as additional current sources on the CVs, as opposed to participating in the implicit voltage integration step.

Cells connected via gap junctions must be in the same cell group as determined by the provided domain decomposition.

* Extend `mc_cell` to hold a list of gap junction locations.
* Add `num_gap_junction_sites()` and `gap_junctions_on()` methods to the `recipe` interface.
* Add `gather_gids()` collective operation to distributed context interface and implementations.
* Extend `partition_load_balance()` functionality to ensure that cells connected by gap junctions are put in the same groups.
* Permute cells within `mc_cell_group` so that cells connected by gap junctions are contiguous.
* Add gap junction information to `multicore::shared_state` and `gpu::shared_state`, together with `add_gj_current()` method that computes GJ current contributions.
* Ad...

* Add unit test for event delivery pathing.
* Temporarily disable GJ-specific test pending GJ PR.

Somewhere along the line, some extensions were made to `cell_member_type` to facilitate some arithmetic on the `gid` field. These operations tend to blur the distinction between a gid and a `cell_member_type` value, and are besides used only in a couple of places.

* Remove `cell_member_type` arithmetic with `gid` methods/functions.
* Replace usage of these short cuts with explicit use of `gid` field in `dry_run_context_impl` and `symmetric_recipe`.

Implement a basic glob routine, supporting a subset of POSIX behaviour (e.g. no character classes), as a fallback for platforms such as Android which do not include it in their libc.

* Add CMake configuration option `ARB_USE_POSIX_GLOB`, defaults to `ON`, that determines if the fallback implementation is used or not.
* Extend `sup::path` functionality to add directory iterators and a couple more path manipulation routines, again following the C++17 `std::filesystem` interface.
* Add an NFA pattern matcher `glob_basic_match` and file system `glob_basic` function; the latter is abstracted over a file system provider object, primarily for testing/mocking purposes.
* Add unit tests for new `sup::path` functionality and for `glob_basic`.

FIxes #181.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Remove `sup::spike_emitter` and replace usages in `miniapp` and `brunel-miniapp`.

* Replaces instances of `spike_emitter` in `miniapp` and `brunel-miniapp` with a callback that saves spikes to a vector and which is written to a file after the simulation ends.
* Remove unused fields from `cl_options` in `brunel-miniapp`, including spike output file components.
* Hard code spike output path to `spikes.gdf` in `brunel-miniapp` and write saved spikes after simulation end.
* Correctly set 'rank' in `miniapp` (used in spike output paths).
* Read spike output options from JSON config in `miniapp` unconditionally.
* Make line comments in `miniapp` and `brunel-miniapp` main functions a bit more consistent in
formatting.

Fixes #677.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Make a new installed library `libarborenv.a` covering a subset of the `sup` library functionality, with corresponding installed CMake target `arbor::arborenv`.

* Move NVML or CUDA 10 API decision for GPU UUID discovery to top level CMake.
* Move affinity, concurrency, MPI init guard, and gpu detection and negotiation functionality out of `sup` and into new library `arborenv`.
* Move `include/arbor` in project tree to `arbor/include/arbor` (for consistency across `sup`, `arbor`, and `arborenv` subdirectories.)
* Wrangle more explicit library dependency adding CMake code into the installed `arbor-config.cmake`, to help mitigate [CMake issue #18614](https://gitlab.kitware.com/cmake/cmake/issues/18614).
* Have `arborenv` code throw `std::runtime_exception` instead of `arb::arbor_error`. (We are still using `arb::mpi_error` though for a failure in `with_mpi`.)
* Move `scope_exit` into the `arb::util` namespace.
* Merge `affini...

* Print a `meter-total` value of the accumulated meter values across all checkpoints to the meter `ostream` output.
* Add default behavior that prints the mean of a meter across all ranks if the meter type is not one of the known special cases (i.e. not one of time, memory or energy).

Fixes #671

* Fix bug in optimized (scalar) unconstrained indirect addition.
* Fix bug in indirect arithmetic and scatter tests that tested only a subset of the test data.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Optimize "none" index_constraint specialization of compound_indexed_add, so that it only reads/writes each distinct memory index once per vector.

Related to issue #637.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

* Fix incorrect specialization of AVX512 gather in SIMD library.

Related to #637.
Improves avx512 performance on Intel Xeon Gold 6130.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Extend sup library to support assigning unique GPUs to MPI ranks.

Fixes #648.

* Provide a helper wrapper for use behind the scenes in the
implementation of `sup::on_scope_exit` so that we can work around
`std::function` not being nothrow move constructible (and maintaining
the nothrow move on the `sup::scope_exit` structure).

Fixes #664.

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

Events arrive already sorted first by index then by time. 
* Remove sort by event index.
* Replace assertion that events are sorted by time with assertion that they are sorted by index. Assertion that the subrange of events with the same index is sorted by time already exists.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Refactoring that moves the logic for determining available concurrency and available GPUs from the core Arbor library to the sup library. This also constitutes work towards providing functionality for allocating GPUs to particular ranks when multiple GPUs are visible per rank.

* Move core/thread estimation code to sup library.
* Change default resource behaviour to use one thread and no GPU.
* Provide an interface in the sup library for: acquiring a default GPU; for coordinating an allocation of GPUs across multiple MPI ranks.

Ensure that all threads use the same GPU, which wasn't the case before.

* add `gpu_context::set_gpu()` method that will set all subsequent GPU calls from the calling thread run on the GPU of `gpu_context`.
* `fvm_lowered_cell_impl` now calls the `set_gpu` method on construction and `advance`.
* Also changed GPU memory allocation errors in `arb::memory` to throw `arb_exception` instead of calling `std::terminate` on error. Now errors due to poor GPU configuration can be caught by the calling application, and unit tests fail gracefully and allow other tests to run.

Fixes #655 

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake wants to run a device link pass with nvcc despite
there being no CUDA seperable compilation enabled anywhere,
and then passes on -pthread to that unnecessary nvcc
invocation when we use the Threads dependency. The latter,
at least, is fixed in CMake 3.13.

We used the prefer -pthread option for compatibility with
our earlier build configuration; turning it off will
hopefully have no consequence.

We also enable device linking on the arbor library. Which
is not needed, but if they are going to insist on doing it,
it should be on the library rather than the executable.

CMake then goes and does it on the executable anyway. Great.

Fixes #645.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Forward CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES to compilation of arbor library and unit tests.

Fixes #651

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

This reverts commit be2a8a9f.