Some small changes to facilitate smooth PyPi integration:
* update the Python installation guide
* update the license file
* add a `setup.cfg`
* add a stub for the Python readme

Add a `setup.py` for installing Arbor directly with pip/setuptools.

Implement a setuptools extension for CMake in `setup.py`. To be honest, I don't understand entirely how it works, but setuptools is obtuse enough that I don't feel at all guilty about this.

Define additional flags for optionally enabling GPUs, MPI, Vectorization and micro-architecture targets, for more adventurous users.

The documentation is updated with a "howto pip" for more casual users who don't want anything to do with CMake.

Fixes #958 .

Improve the Python wrapper generation and installation:
  - install a proper module that can be extended with Python code;
  - give the user more control over where to install the module (e.g. as a user package or in a virtualenv).

During building, the following sub-directory is built in the build director (`CMAKE_BINARY_DIR`)
```
└── python
    └── arbor
            ├── __init__.py
            ├── arbor.so
            └── VERSION
```
This path can then be copied VERBATIM to the target installation path. By default this will be in `CMAKE_INSTALL_PREFIX/lib/python%d.%d/site-packages`.
An additional CMake parameter `ARB_PYTHON_PREFIX` can be used to specify an alternative destination for installing the Python module.

The Python part of the wrapper, implemented in `__init__.py` is currently very limited, only providing `__version__` and `__config__` variables.

The installation guide was updated to cover the Python installation.

A big update that wraps over 90% of the public C++ API, with enough functionality to let Python users to perform useful modelling.

Key features
- wrapping of cable cell functionality
  - user-defined explicit compartmentalisation not supported
- `single_cell_model` abstraction unique to the Python wrapper that simplifies developing and testing single cell models
- one-dimensional cell builder for Python wrapper that simplfies building cells that 
  - in and of itself limited in scope, but a very useful example of mapping a richer morphology builder onto `sample_tree`s.
- parsing of region and location expressions from strings
  - implements a generic s-expression parser that we can use for other purposes later

* Add logo to read the docs as HTML header.
* Lighten nav bar background so as to not clash with logo.

Adds

* to `recipe.cpp`: 
`num_probes` and `get_probe` with helper function `cable_probe(kind, id, location)`
* `sample.cpp`: 
`sample` type with attributes `time` and `value`; a `sampler` with `samples()` and `clear()` and helper functions `attach_sampler(sim, dt)` for adding all probes as well as `attach_sampler(sim, dt, probe_id)` for adding one probe
* `morphology.cpp`: 
`location` as `arb::mlocation` with `branch` and `position` 
* corresponding docs and example
* update to new cable cell semantics in c++ validation test

fixes #762

* Add new "embedded morphology" `em_morphology`, constructed from a `morphology`, that: caches derived information such as terminal points, locations of sample points, branch lengths; provides services for canonicalizing locations and interpolation; and is used for the realisation of regions and locsets (see below).
* Add simple DSL for describing locsets and regions, which are abstract descriptions of sets of locations and regions on morphologies, together with a set of operations.
* Add a new class `label_dict` that associates names with regions and locsets.
* Extend `cable_cell` to hold a `label_dict` argument, which is used by a new method `paint` for associating mechanisms with regions.

Fixes #845.

Sam Yates authored 5 years ago and Benjamin Cumming committed 5 years ago

* Replace derivation of cylindrical cable equation formula with S–L approach.
* Relate analysis of cylindrical form to Rallpack 1 in a separate section.
* Add analysis of tapered cable case.
* Unify discussion of cable with constant electrical properties but varying geometry.
* Add section describing how one can well-estimate the inter-cell flux given point samples or CV means for the membrane voltage.

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

- wraps `partition_hint` struct for python
- `partition_hint`struct in `load_balance.hpp` is adjusted in a way that in case of zero unsigned int `cpu_group_size`/`gpu_group_size` it is set to the default value (via setter/getter)
- adds documentation for `partition_hint`  
- adds testing for `domain_decomposition` including `partition_hint` (in unit and unit_distributed)
- adds `partition_hint` in example `ring.py`
- corrects `config.cpp` (and doc) to test for `ARB_GPU_ENABLED` instead of `ARB_WITH_GPU`

Fixes #776 
Addresses #799, #769

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

* add interface for Python users to query schedule event times
* add schedule event tests and assertions
* add schedule events to doc

* Collect cable cell parameter setting structures into `include/cable_cell_param.hpp`.
* Restructure electrical specifications (axial resistance, membrane capacitance) and ionic concentrations and reversal potentials on cable cells, so that these can be specified with a global default, per-cell defaults, and per-segment values.
* Allow reversal potentials to be set by a mechanism of a new kind 'revpot', which are prohibited from maintaining state or writing to any shared state other than ionic reversal potentials.
* Specify reversal potential mechanisms as global defaults or per-cell. Reversal potential mechanisms may not be specified at the level of a segment in order to avoid non-linearities arising from the discretization.
* Supply default cable cell parameter data that matches NEURON values (this is _not_ used by default).
* Replace the d_lambda calculation with one that approximates more faithfully the effect of tapered segments, and which will use the electrical values inherited by cell or global defaults.
* Supply a bundled mechanism 'nernst' that replaces the previous hard-coded Nernst potential calculation, for use (together with ion rebinding) in the new ion description scheme.
* All ions used in a cable_cell model must be present in the top level global parameter table, together with their charge.
* Extend semantics of external variables in modcc, to permit direct assignment (as opposed to just += or -=.)
* Extend `util::binary_search_index` to allow for a projection functional analagous to other range utilites.
* Add documentation on the cable cell API, sketching over many of the details, but describing in particular the interface changes for default parameters and reversal potential behaviour.
* Adjust pyarb for new API
* Hard code global cable cell properties in the python recipe shim to useneuron default parameters.

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

* correct doc for meter_report and small word fixes
* shift make_report to next section

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

Wraps the `meter_manager` with
- the constructor
- `start(ctx)`
- `checkpoint(name, ctx)`
- `checkpoint_names`
- `times`

and the `meter_report` with
- `make_meter_report(meter_manager, ctx)`

Fixes #765

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

Add docs for Python wrapper to ReadTheDocs:

- Overview, Common Types, Hardware Management, Recipes, Domain Decomposition, Simulations, Metering
- Installing Arbor: Optional Requirements (Python), Buidling and Installing (Python Frontend), and Installation (Python Module)

Missing (, since not implemented yet): 

- probes
- arbor-sup 
- hint maps in domain_decomposition
- reset, events, empty schedule in event_generator
Also does not cover unit testing (since doc is user-centric).

Makes also defaults and units in wrapper consistent.

Fixes  #766

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

Domain decomposition in python wrapper
* functions: `partition_load_balance`
*  types: `group_description` and `domain_decomposition`

Also includes
* remove comment that `gid` in a `group_description` must be sorted.
* a host of small clean ups of python wrapper

Fixes #761

Add citation information to the readme.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

* Rename `arb:simd::simd_detail` namespace to `arb::simd::detail`.
* Update SIMD documentation to suit.

Fixes #468.

noraabiakar authored 6 years ago and Benjamin Cumming committed 6 years ago

Add gap junctions and improve general model overview in docs.

* Replace Arbor Model::Common Types with Arbor Model::Concepts, that introduces the concept of cells, connections and gap junctions. 
* Add gap junction documentation to cpp:recipe
* Document the implicit rule forcing cells connected by gap junctions to be in the same cell group.
* Add documentation that discusses core concepts in Arbor models.

Update Hardware API documentation

* split the domain decomposition and hardware API docs into separate pages
* update hardware API to reflect new *libarbor* and *libarborenv*
* add basic documentation for `optional`, `any` and `unique_any` types.

Update the installation guide to reflect the latest supported tool,
compiler and library versions.

Remove the Python docs, because they documented features that have not
been implemented yet. The Python docs can be added incrementally as
features are implemented.

Start work on oupdating the documenetation for hardware interfaces and
domain decomposition.

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Update documentation for Python.

    splits the conceptual model ideas from the C++ docs into their own section
    has C++ and Python docs for recipes, domain decomposition, etc.

fixes #667

Added the following documentation (structure):

GETTING STARTED:

    Installing Arbor/Requirements/Optional Requirements/Python
    Installing Arbor/Building and Installing Arbor/Python Front End

MODEL BASICS:

    Overview
    Common Types
    Recipes
    Domain Decomposition
    Simulations

PYTHON:

    Overview
    Common Types
    Recipes
    Domain Decomposition
    Simulations

DEVELOPERS:

    Python Profiler
    Python Unit Testing

GETTING STARTED has two added sections of optional requirements using python and how to build the python front end.

MODEL BASICS describes Arbor's concepts in general (independent of programming language), thus general information on concepts in C++ API was moved here/ added.

PYTHON describes Arbor's pyth...

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

First step towards the Python front end.

This commit sets up the structure of the python implementation
* directory structure
* git submodule for pybind11
* best practices for making bindings with pybind11
* unit testing for the python front end

It implements the following features in the Python front end
* execution contexts
* gpu detection
* thread count detection
* MPI initialization helpers.

Fixes #667.

noraabiakar authored 6 years ago and Benjamin Cumming committed 6 years ago

Updates the install docs. Fixes #604

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

changes: 
- .travis.yml:
  - added matrix for different osx's, since enumeration style only works for `env` and `compiler`

- scripts/travis/build.sh:
  - changed getting compiler version from ``${CXX} -dumpversion`` to ``${CXX} --version | grep -m1 ""`` 
  - added `--oversubscribe` flag to `mpiexec` on Mac to allow more processes on a node than processing elements
  - added `--mca btl tcp,self` flag for Open MPI to use the "tcp" and "self" BTLs for transporting MPI messages on Mac

noraabiakar authored 6 years ago and Benjamin Cumming committed 6 years ago

Dry-run mode: 
* An implementation of distributed_context that is used to mimic the performance of running an MPI distributed simulation with n ranks.
* Verifiable against an MPI run with the same parameters. 

Implementation: 
* Describe the model on a single domain (tile) and translate it to however many domains we want to mimic using arb::tile and arb::symmetric_recipe. This allows us to know the exact behavior of the entire system by only running the simulation on a single node.
* Mimic communication between domains using arb::dry_run_context

Example: 
* dryrun in example/ is a verifiable example of using dry-run mode with mc_cells

Other:
* Documentation of dry-run mode 
* unit test for dry_run_context 

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #587.

* Eliminate Clang warnings from GCC-tree-optimization bug work-around.
* Error with static-assert if simd type is used with a missing simd abi.
* Clarify install documentation regarding use of ARB_VECTORIZE with ARB_ARCH.

Make the execution context presented to users an opaque handle, moving all implementation of the gpu, thread and distributed contexts into the back end.

* move `execution_context` and `distributed_context` definitions to the back end
* create `execution_context` handle called `context` in the public API
* provide `make_context` helper functions that build different context configurations (default, user-specified local resources, with MPI)
* update documentation for all parts of the public API that touch contexts
* move `distributed_context` docs to the developer documentation (from the public API docs)

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Fixes #564

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

* Update the install docs for architecture build options
* Update to reflect new install target

Reduce redundant functionality across event_generator, time_seq
and schedule by providing a low-heap overhead interface to schedule
and using that for time sequences in event_generator and specialized
cell groups.

* Have schedule return pair of pointers as view to generated times.
* Fix missing DEBUG/TRACE functionality.
* Use rate instead of mean_dt for Poisson schedule.
* Move merge_events() functionality to simulator.cpp.
* Migrate event_generator to event span interface.
* Migrate tourney_tree to event span interface.
* Only invoke tourney_tree merge if generators have events in the epoch.
* Use schedule for times in event_generator implementations.
* Replace seq_generator with explicit_generator that keeps a copy of events.
* Replace vector_backed_generator and poisson_generator with schedule_generator.
* Replace time_seq uses with schedule.
* Add default empty schedule.
* Move rounding error test for regular time sequence into schedule test.
* Update sampling API documentation for schedule.

noraabiakar authored 6 years ago and Benjamin Cumming committed 6 years ago

Use correct profiling and mpi flags.
Fix profiler doc

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake and build refactoring

*   Use CUDA as first-class language (leading to CMake 3.9 minimum version requirement).

*   Use 'modern CMake' interface libraries for compiler options, include file and library dependency tracking. Interface library targets:
    * `arbor-deps`: compiler options and library requirements for the `libarbor.a` static library, as governed by configure-time options and environment.
    * `arbor-private-headers`: include path for non-installed headers, as required by unit tests and arbor itself.
    * `arbor-aux`: helper classes and utilities used across tests and examples.
    * `ext-json`, `ext-tclap`, `ext-tbb`, `ext-benchmark`, `ext-sphinx_rtd_theme`: externally maintained software that we include (directly or via submodule) in the `ext/` subdirectory.
 
*   Single static library `libarbor.a` includes all built-in modules and CUDA objects.

*   Simply configuration options:
    *  `ARB_WITH_TRACE`, `ARB_AUTORUN_MODCC_ON_CHANGES` `ARB_SYSTEM_TYPE` removed.
    * External `modcc` is provided by `ARB_MODCC` configuration option; if provided `modcc` is still buildable, but is not included in the default target.
    * `ARB_PRIVATE_TBBLIB`, defaulting to `OFF`, instructs the build to make TBB from the included submodule.

*   Extend `ErrorTarget` functionality to provide a dummy target or an error target based on a condition.
*   Generate header version defines and library version variables based on git status and project version, via new script `include/git-source-id`.
*   All generated binaries now placed in `bin/` subdirectory at build.
*   Install targets installs: public headers (incomplete); static library; `modcc` tool; `lmorpho` executable; `html` documentation (examples, tests and validation data are currently not installed).
*   Executable targets have had the `.exe` suffix removed; unit tests are labelled `unit` (arbor unit tests), `unit-modcc` (modcc unit tests), `unit-local` (distributed tests with local context), `unit-mpi` (distributed tests with MPI context).
*   More graceful handling of configure-time detection of `nrniv`, Julia and required Julia modules for validation data generation.
*   Add `cmake/FindJulia.cmake`, `cmake/FindTBB.cmake`  package finders, and adjust `cmake/FindUnwind.cmake` to use link library-style properties.
*  Adjust travis script to test `unit-local` and `unit-mpi` if appropriate.
*  Simply documentation `conf.py`.

Source relocation and reorganization

* All external project sources and files moved to `ext/`.
* Source code refactoring to decouple library-using code from the configure-time definitions that govern arbor behaviour: removes conditional code in public headers that depends upon `ARB_WITH_X`-type definitions at compile time. Affected code is is in the public interfaces for MPI, the threading implementation, and the profiler.
* Remove `util/debug.hpp`; split out functionality for pretty-printing from assertion handling.
* Make FVM cell non-physical voltage check a run-time cell-group parameter.
* Move spike double buffer implementation to `simulation.cpp`.
* Make timer utility wrap POSIX `clock_gettime` independent of threading configuration.
* Make `mpi_error` derive from `system_error` and follow C++11 `system_error` semantics.
* `EXPECTS` macro replaced by `arb_assert` macro.
* JSON dependency removed from `libarbor.a` and header files: moved to auxiliary library.
* Publicly visible macros garner an `ARB_` prefix as required.
* Move SWC test file to `test/unit` directory.
* Work-in-progress splitting of public from private includes: as a convention not entirely adhered to as yet, private headers within arbor source are included with `""`, public headers with `<>`.

Modcc interface changes

* Expose via `--namespace` option the functionality that sets the namespace in generated code.
* Use `--profile` option to add profiler hooks to generated code; uses public function interface directly rather than `PE/PL` macros in order to avoid public `PE` and `PL` defines.

Benjamin Cumming authored 6 years ago and Sam Yates committed 6 years ago

Move from choosing the distributed communication model from a compile time choice (the old `arb::communication::communication_policy` type) to a run time decision.

* Add `arb::distributed_context` class that provides the required interface for distributed communication implementations, using type-erasure to provide value semantics.
* Add two implementations for the distributed context: `arb::mpi_context` and `arb::local_context`.
* Allow distribution over a user-supplied MPI communicator by providing it as an argument to `arb::mpi_context`.
* Add `mpi_error` exception type to wrap MPI errors.
* Move contents of the `arb::communication` namespace to the `arb` namespace.
* Add preprocessor for-each utility `ARB_PP_FOREACH`.
* Rewrite all examples and tests to use the new distributed context interface.
* Add documentation for distributed context class and semantics, and update documentation for load balancer and simulation classes accordingly.

Fixes #472

* Update SIMD developer docs to reflect newly merged mechanism refactor work.

Ben Cumming authored 6 years ago and Sam Yates committed 6 years ago

Add two new documentation pages for the C++ API

* Add domain decomposition page that covers `domain_decomposition`, `node_info` and `partition_load_balance`.
* Add simulation page that describes `arb::simulation` API interface.
* Fix some small typos elsewhere in the docs.
* Use `std::move` when adding spike callbacks to `arb::simulation` (useful if callbacks are stateful).

Ben Cumming authored 6 years ago and Sam Yates committed 6 years ago

Fixes issue #467 

* Add GPU synchronization points where required for Kepler to coordinate CPU access of managed memory.
* Use hand-rolled double precision atomic addition for Kelper targets.
* Replace `ARB_WITH_CUDA` build option with `ARB_GPU_MODEL` option that takes one of 'none', 'K20', 'K80' or 'P100', and set up source-code defines accoringly.
* Clean up of redundant compiler flags and defines no longer required now that the project uses separate compilation for CUDA sources.

Fix some typos in the SIMD documentation.

Ben Cumming authored 6 years ago and Sam Yates committed 6 years ago

Add some C++ API documentation.

* Create C++ API section in docs.
* Document `arb::recipe`: both a class reference along with more explanatory text and best practices guide.
* Add some class documentation of basic types required to understand recipe definition.
* Some in-code comment clean up.
* Change `arb::cell_kind` from a vanilla enum to a scoped enum.

Ben Cumming authored 6 years ago and Sam Yates committed 6 years ago

The name `arb::model` did not clearly describe the role of the class, while `arb::simulation` better captures that this is an instantiation of a model for the purpose of running a simulation, as distinct from the description of a model represented by an `arb::recipe` instance.

* Rename sources `model.{hpp,cpp}` to `simulation.{hpp,cpp}`.
* Rename class `arb::model` to `arb::simulation`.
* Update docs and tests to suit.