* added notes re literal_include to python examples
* merged changes in #1504 
* deduped and includified (https://github.com/arbor-sim/arbor/pull/1558) tutorials.
* First divergence of tutorial and code spotted!
* Added some contrib documentation for examples and docs.

- Minor fixes to python examples.
- Fix `mpi.py` to use new label API.
- Rename `mpi.py`.

New structs and types:
* `cell_tag_type` (std::string): for labelling placeable items on a cell. The label refers to a number of items placed on a locset, equal to the number of locations in a locset. The number of locations in not always known to the user, so the previous way of using indices for items was no longer sufficient. 
* `lid_selection_policy`: for allowing a user to select a single item from a group of items sharing a label. Currently only `round_robin` and `assert_univalent` are supported. 
* `cell_local_label_type` and `cell_global_label_type`: for identifying the target and source of a connection or gap_junction connection. 
* `cell_label_ranges`, and `cell_labels_and_gids`: for propagating information about the labelled items on the cell from the cell groups back to the simulation and communicator. 
* `label_resolution_map` and `resolver`: for selecting an item (and retaining state) from a labelled group of items on a cell according to a user-selected policy.

Changes to the model-initialization: 
* The `communicator` now needs `label_resolution_maps` constructed from the cell group data in order to build the `connections` vectors. 
* The `simulation_state` object handles the transfer of the information when it is constructed. 
* Spike exchange at runtime remains unchanged, because `communicator::connections` remains unchanged. 

Changes to cells, cell_groups and recipe:
* `decor::place` expects a third label parameter, no longer returns an `lid_range`.
* `lif`, `source`, and `benchmark` cells need source/target labels in their constructors. 
* A `cell_group` needs to save data about the gid/labels/lid_ranges of each cell, to propagate back to the `communicator` constructor. 
* Connections/gap junction connections are formed between {label, policy} pairs on cells instead of indices. 
* `num_sources`, `num_targets`, `num_gap_junction_sites` deleted from `recipe`.

Additional changes:
* Add MPI wrapper for exchanging vectors of strings. 
* Corresponding updates to unit tests, Python wrapper, C++ and Python examples, documentation.

Fixes #1394

* Tutorial ring network added. A few typos fixed, crosslinks fixed.
* MPI tutorial added.
* Improved SVG generation, added separate label scaling, increased default label size.
* Bugfix: didnt generate cable region in gen-labels.py used in `concepts/morphology.rst`.
* Termified Interconnectivity page.
* Termified probe nomenclature page. Will be moved in [this PR](https://github.com/arbor-sim/arbor/issues/1353) from python/simulation to concepts/probes_samplers.

Extend Ptyhon recipe wrapper to support setting of global cell properties.

* Rename `pyrecipe::get_probes` to `py_recipe::probes`
* Remove `global_properties_shim` : It's easy to make mistakes with the class, as it holds a `mechanism_catalogue` and a `cable_cell_global_properties` which holds a pointer to the catalogue. This would likely have caused issues with the users.
* Expose `py_recipe::global_properties`

Make the `cable_cell` interface read only by passing the decorations
to its constructor in a new `decor` type.

C++ library
* Remove the paint/place/set_default interface from `cable_cell`
* Create a `decor` type that:
  * stores lists of paintings and placings
  * stores a set of cable cell parameters
  * uses the `paint`, `place`, `set_default` interface that was deprecated
    from `cable_cell`.
* Create `paintable`, `placeable` and `defaultable` variants that are sum types
  over the respective types that can be painted, placed and defaulted.
* Remove the overloaded `cable_cell::paint`, `cable_cell::place`,
  `cable_cell::set_default` methods to single methods that consume the sum types.

Unit Tests
* Many small changes because many many tests use cable cell API.
* There were no `cable_cell` unit tests! Not such a big deal, since cable_cell
  is tested implicitly in so many other tests
  * but I added cable_cell tests: not much at the moment but will quickly fill
    ...

Major changes in Python library and documentation:

* Add global state (immutable after initialization) for the Python module that
  manages the mapping between probe metadata types and the method by which the
  corresponding [sample data can be collected and presented as Python objects.
  This is necessary for decoupling the implementation of the simulation wrapper
  from that of the various cable cell probe types.

* Wrap each of the C++ cable cell probe types with a Python function that
  returns a `probe_info` object.

* Add code for converting and accumulating C++ probe metadata and sample
  data, registered in the module global state against the metadata type info.

* Make the `arbor.simulator` object responsible for recording all spike and
  sample data, with corresponding new methods.

* Use NumPy arrays and structured datatypes for returning spike and
  sample data.

* Place Python schedule proxies under an abstract interface so that consumers
  of the proxies can be made generic over schedule types.

* Add unit tests for the `arbor.simulator` class and its new functionality,
  including distributed spike collection.

* Rework all Python API documentation concerning probes, sampling, and spike
  recording; add information on newly exposed cable cell probe addresses.

* Add new python example `single_cell_recipe.py` which is a generic recipe
  version of `single_cell_model.py`.

* Adjust other code in the wrapper and examples to accommodate these changes.

Minor changes in Python library:

* Add equality tests for `arbor.location` and `arbor.cable` Python classes.

* Use the `py::` namespace shorthand more often in the Python wrapping sources.

* Add an implicit conversion for a pair of numbers to `arbor.cell_member`, so
  that e.g. probe ids can be specified simply as `(gid, index)`.

* Add an implicit conversion from a tuple to `arbor.mpoint` so that a
  morphological point can be specified simply as `(x, y, z, radius)`.

* Modify the interface to `arbor.regular_schedule` so that the overloads
  correspond more closely to the C++ API. In particular,
  `arbor.regular_schedule(dt)` now describes a schedule with times 0, dt, 2*dt,
  etc.

Minor changes in C++ library:

* Change test in FVM lowered cell implementation and exception message for
  `bad_source_description`: assert number of detectors is exactly the number
  of sources promised by the recipe.

* Add comment describing `probe_metadata` object in `sampling.hpp` which
  asserts that the type and value of probe-specific metadata completely
  determines the correct interpretation of sampled data provided to a callback.

Speed up plotting in seaborn.relplot by disabling calculation of confidence intervals

* Add some log messages in python examples
* Add python examples to CI

* Python examples: Updated to use region expressions (quotes)
* Python examples: Uniform plotting
* Python examples: Removed tutorial directory (example roughly matches python/example/single_cell_model.py)
* Python examples: Rewrote flat_cell_builder() to segment_tree()

* Update `dryrun` to generate some spikes again, add a voltage printer.
* Fix `ring.py` recipe.
* Fix labels in `single_cell_builder.py` and `single_cell_swc.py`.

* Remove spherical root/soma from morphology representations and logic.
* Replace `sample_tree` with `segment_tree`, which represents sample points in pairs and allows the representation of detached branches and cable segments.
* Remove sample point properties and predicates.
* Remove `cv_policy_flag::single_root_cv` as somata with a single attached dendrite will have a representation in the morphology of a single branch.
* Remove `arb::math::{volume/area}_sphere`, which are no longer being used.
* Remove spherical root special cases from `embed_pwlin` and `place_pwlin` objects.
* Add textual representation of `mnpos` in morphology exception text.
* Rename `sample_tree_from_swc` to `segement_tree_from_swc` and remove any special casing for one point somata.
* Remove `msample` and `mbranch` types.
* Update unit tests and python wrapper to reflect the changes.

Use locset names to set probe sites in single cell model.

1. Extend C++ cable_cell interface to provide a concrete list of locations associated with a locset on the cell.
2. Extend Python single_cell_model to allow placement of probes on locsets.
3. Remove "debug" output in Python ring example.

The single cell model maintains a trace callback for each probe location, and needs to know the explicit locations before starting the simulation. The first step is required to enumerate and record the locations associated with a locset when attaching probes.

Open issues: 

1. The extension to `cable_cell` won't sufficiently support locsets that rely on random number generators with global seeds (it would require passing additional meta data required to fully instantiate the locset)
2. Enhancing the probe recipe interface to be able to associate one `probe_id` with all locations in a locset (the number of which is not specified at the time the `probe_id` is created) would support the interface implemented for the `single_cell_model`.

Note: The richer interface on the C++ side will be implemented in time (see #957).

A big update that wraps over 90% of the public C++ API, with enough functionality to let Python users to perform useful modelling.

Key features
- wrapping of cable cell functionality
  - user-defined explicit compartmentalisation not supported
- `single_cell_model` abstraction unique to the Python wrapper that simplifies developing and testing single cell models
- one-dimensional cell builder for Python wrapper that simplfies building cells that 
  - in and of itself limited in scope, but a very useful example of mapping a richer morphology builder onto `sample_tree`s.
- parsing of region and location expressions from strings
  - implements a generic s-expression parser that we can use for other purposes later

Adds

* to `recipe.cpp`: 
`num_probes` and `get_probe` with helper function `cable_probe(kind, id, location)`
* `sample.cpp`: 
`sample` type with attributes `time` and `value`; a `sampler` with `samples()` and `clear()` and helper functions `attach_sampler(sim, dt)` for adding all probes as well as `attach_sampler(sim, dt, probe_id)` for adding one probe
* `morphology.cpp`: 
`location` as `arb::mlocation` with `branch` and `position` 
* corresponding docs and example
* update to new cable cell semantics in c++ validation test

fixes #762

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

- wraps `partition_hint` struct for python
- `partition_hint`struct in `load_balance.hpp` is adjusted in a way that in case of zero unsigned int `cpu_group_size`/`gpu_group_size` it is set to the default value (via setter/getter)
- adds documentation for `partition_hint`  
- adds testing for `domain_decomposition` including `partition_hint` (in unit and unit_distributed)
- adds `partition_hint` in example `ring.py`
- corrects `config.cpp` (and doc) to test for `ARB_GPU_ENABLED` instead of `ARB_WITH_GPU`

Fixes #776 
Addresses #799, #769

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Wraps the `meter_manager` with
- the constructor
- `start(ctx)`
- `checkpoint(name, ctx)`
- `checkpoint_names`
- `times`

and the `meter_report` with
- `make_meter_report(meter_manager, ctx)`

Fixes #765

akuesters authored 6 years ago and Benjamin Cumming committed 6 years ago

Add docs for Python wrapper to ReadTheDocs:

- Overview, Common Types, Hardware Management, Recipes, Domain Decomposition, Simulations, Metering
- Installing Arbor: Optional Requirements (Python), Buidling and Installing (Python Frontend), and Installation (Python Module)

Missing (, since not implemented yet): 

- probes
- arbor-sup 
- hint maps in domain_decomposition
- reset, events, empty schedule in event_generator
Also does not cover unit testing (since doc is user-centric).

Makes also defaults and units in wrapper consistent.

Fixes  #766

Support for recording spikes generated by a simulation in the Python wrapper
* Implement a `spike_recorder` that holds a shared pointer to a `std::vector` of spikes, and a callback for the `arb::simulation` spike recording API.
* Add `python/example/ring.py` that creates a ring network, then records and prints spikes.
* Some fixes to get the full `recipe` -> `domain_decomposition` -> `simulation` -> `spikes` workflow to work
  * always use default `global_parameters`: user customization of global parameters for cable cells can wait until the ion species interface is finished.
  * change the Python recipe interface for `recipe::connections_on` to use `pybind11::objects` because of shim.
* Some small improvements to error and help messages.

Fixes #764