Fix mistakes reported by @Helveg in the Python tutorial docs. 

* Fix the regular schedule generated in `single_cell_model::run`
* Adjust frequency of probes in python examples to the correct unit (KHz). 

Fixes #1457

Remove `gid` attribute of target of a connection, and local gap-junction site of a gap-junction connection (#1467)

* Change the type of the target of a `connection` from `cell_member_type` to `cell_lid_type`. The target cell's gid is always known in both the recipe and simulation.
* Change the type of the local site of a `gap_junction_connection` from `cell_member_type` to `cell_lid_type`, and make it such that the first argument of the connection is the peer site, and the second the local site. The local cell's gid is always known in both the recipe and simulation.
* Change the type of the target of an `event_generator` from `cell_member_type` to `cell_lid_type`. The target cell's gid is always known in both the recipe and simulation.
* New `cell_spike_events` and `cse_vector` for `simulation::inject_events`.
* Simplify recipe sanity checks and remove associated exception types. 
* Fix unit tests, examples, docs

* Add a method to `event_generator` that returns all of its targets. 
* When the simulation is constructed, check the targets of each event generator, and throw an exception if an the `gid` or `lid` is invalid.
* Add corresponding unit tests.

* Add Python version of brunel.cpp example.
* Enforce consistency in units: both C++ and Python APIs use unit `kHz` for all frequencies.
* Align Python version `poisson_schedule` signature with C++ equivalent.
* Update unit tests to suit API changes.
* Update documentation to suit, along with minor additions related to networks and interconnectivity.
* Add `brunel.py` to `basic.yml` tests

* Tutorial ring network added. A few typos fixed, crosslinks fixed.
* MPI tutorial added.
* Improved SVG generation, added separate label scaling, increased default label size.
* Bugfix: didnt generate cable region in gen-labels.py used in `concepts/morphology.rst`.
* Termified Interconnectivity page.
* Termified probe nomenclature page. Will be moved in [this PR](https://github.com/arbor-sim/arbor/issues/1353) from python/simulation to concepts/probes_samplers.

* add neuroML example
* no test, because GH actions arbor is built without neuroML support

Online STDP is implemented in the synapse and
and demonstrated by recording an STDP curve in
an example.

First phase of dynamically loaded mechanism catalogues.

- Add functionality to load and use catalogues from DSOs in Python and C++.
- Add documentation and support scripts to build catalogues DSOs.
- Re-factor CMakeLists.txt for mechanisms
- Limitations
  - Needs (compatible, preferably identical) arbor sources when building catalogues
  - Can only work on MacOS and Linux (uses dlfnc)

* Used Sphinx glossary directive to create terms that can be referred to in `concepts/morphology.rst` and `concepts/labels.rst`.
* Added note box on difference with Neuron, add source for NEURON cylinder discretisation
* Parameters added to svg generation script: turn off colors, root
* Fix cpp TOC duplication, match ordering in other sections.
* State default cv policy
* center/re -> midpoints
* Clarify and remove compartment term

* Add docs tutorial page for single_cell_recipe.py example
* Adapted single_cell_recipe.py slightly
* Improved labels and fixed some broken cross-linking
* Fixed a typo here and there

* Add 2 new examples to `python/examples` and their corresponding tutorials in the docs. 
    1. `single_cell_detailed.py` : a slightly more complex example using `single_cell_model`
    2. `single_cell_detailed_recipe.py`: equivalent to `single_cell_detailed.py` but using a recipe. 
* Change `load_swc`-> `load_swc_arbor` for consistency.
* Run Sphinx with specified PYTHON_EXECUTABLE.
* Set PYTHONPATH env var in doc/CMakeLists.txt, such that it propagates correctly, including locally.
* Build pyarb BEFORE html such that docs build can import arbor in Github Actions script.

Extend Ptyhon recipe wrapper to support setting of global cell properties.

* Rename `pyrecipe::get_probes` to `py_recipe::probes`
* Remove `global_properties_shim` : It's easy to make mistakes with the class, as it holds a `mechanism_catalogue` and a `cable_cell_global_properties` which holds a pointer to the catalogue. This would likely have caused issues with the users.
* Expose `py_recipe::global_properties`

Make the `cable_cell` interface read only by passing the decorations
to its constructor in a new `decor` type.

C++ library
* Remove the paint/place/set_default interface from `cable_cell`
* Create a `decor` type that:
  * stores lists of paintings and placings
  * stores a set of cable cell parameters
  * uses the `paint`, `place`, `set_default` interface that was deprecated
    from `cable_cell`.
* Create `paintable`, `placeable` and `defaultable` variants that are sum types
  over the respective types that can be painted, placed and defaulted.
* Remove the overloaded `cable_cell::paint`, `cable_cell::place`,
  `cable_cell::set_default` methods to single methods that consume the sum types.

Unit Tests
* Many small changes because many many tests use cable cell API.
* There were no `cable_cell` unit tests! Not such a big deal, since cable_cell
  is tested implicitly in so many other tests
  * but I added cable_cell tests: not much at the moment but will quickly fill
    up as we work on reading, writing, etc.                                                            

Python
* Wrap the new decor type.
* Update cable cell type to reflect read only API.
* Wrap the cv_policy types and implement the same operator overloading.

Major changes in Python library and documentation:

* Add global state (immutable after initialization) for the Python module that
  manages the mapping between probe metadata types and the method by which the
  corresponding [sample data can be collected and presented as Python objects.
  This is necessary for decoupling the implementation of the simulation wrapper
  from that of the various cable cell probe types.

* Wrap each of the C++ cable cell probe types with a Python function that
  returns a `probe_info` object.

* Add code for converting and accumulating C++ probe metadata and sample
  data, registered in the module global state against the metadata type info.

* Make the `arbor.simulator` object responsible for recording all spike and
  sample data, with corresponding new methods.

* Use NumPy arrays and structured datatypes for returning spike and
  sample data.

* Place Python schedule proxies under an abstract interface so that consumers
  of the proxies can be made generic over schedule types.

* Add unit tests for the `arbor.simulator` class and its new functionality,
  including distributed spike collection.

* Rework all Python API documentation concerning probes, sampling, and spike
  recording; add information on newly exposed cable cell probe addresses.

* Add new python example `single_cell_recipe.py` which is a generic recipe
  version of `single_cell_model.py`.

* Adjust other code in the wrapper and examples to accommodate these changes.

Minor changes in Python library:

* Add equality tests for `arbor.location` and `arbor.cable` Python classes.

* Use the `py::` namespace shorthand more often in the Python wrapping sources.

* Add an implicit conversion for a pair of numbers to `arbor.cell_member`, so
  that e.g. probe ids can be specified simply as `(gid, index)`.

* Add an implicit conversion from a tuple to `arbor.mpoint` so that a
  morphological point can be specified simply as `(x, y, z, radius)`.

* Modify the interface to `arbor.regular_schedule` so that the overloads
  correspond more closely to the C++ API. In particular,
  `arbor.regular_schedule(dt)` now describes a schedule with times 0, dt, 2*dt,
  etc.

Minor changes in C++ library:

* Change test in FVM lowered cell implementation and exception message for
  `bad_source_description`: assert number of detectors is exactly the number
  of sources promised by the recipe.

* Add comment describing `probe_metadata` object in `sampling.hpp` which
  asserts that the type and value of probe-specific metadata completely
  determines the correct interpretation of sampled data provided to a callback.

* Return `arb::morphology` from `load_swc_xxx` methods for consistency with the NeuroML interface. 
* Fix unit tests.
* Fix documentation

Speed up plotting in seaborn.relplot by disabling calculation of confidence intervals

* Add some log messages in python examples
* Add python examples to CI

* Python examples: Updated to use region expressions (quotes)
* Python examples: Uniform plotting
* Python examples: Removed tutorial directory (example roughly matches python/example/single_cell_model.py)
* Python examples: Rewrote flat_cell_builder() to segment_tree()

* Update `dryrun` to generate some spikes again, add a voltage printer.
* Fix `ring.py` recipe.
* Fix labels in `single_cell_builder.py` and `single_cell_swc.py`.

* Move label and s-expr parsing code and unit tests from Python to arbor C++ library.
* Update `locset` and `region` constructors that take string arguments to parse strings as s-expressions or "quoted" labels.
* Modify the input stream modifier used to parse asc files to use a lookup table for substitutions, renamed it transmogrifier.                                                             
* Replace `hopefully` type implemented in python headers with an `arb::util::expected`.
* Add `ARBDEV_COLOR` CMake option that forces gcc and clang to always output color output.
* Allow arbitrary strings in labels in region and locset expressions.
* Add `parse_region_expression` and `parse_locset_expression` functions alongside the existing `parse_label_expression` function for use when a region or locset is expected. These calls will promote a quoted string `"label"` to `(region "label")` or 
 `(locset "label")` respectively.
* Add user-defined string literals for labels so that the C++ interface can use `"soma"_lab` instead of awkward escaping `"\"soma\""`.
* Simplify Python wrapper code.

Fix a Python example that was out of date.

* Remove spherical root/soma from morphology representations and logic.
* Replace `sample_tree` with `segment_tree`, which represents sample points in pairs and allows the representation of detached branches and cable segments.
* Remove sample point properties and predicates.
* Remove `cv_policy_flag::single_root_cv` as somata with a single attached dendrite will have a representation in the morphology of a single branch.
* Remove `arb::math::{volume/area}_sphere`, which are no longer being used.
* Remove spherical root special cases from `embed_pwlin` and `place_pwlin` objects.
* Add textual representation of `mnpos` in morphology exception text.
* Rename `sample_tree_from_swc` to `segement_tree_from_swc` and remove any special casing for one point somata.
* Remove `msample` and `mbranch` types.
* Update unit tests and python wrapper to reflect the changes.

Add `cable_cell::concrete_region(region)` method that returns a the cable segments in a region.

This is required by the Arbor GUI #953 so that it can illustrate regions (and locsets) defined by users on a concrete cell morphology.

Note that the `concrete_region` and `concrete_locset` interfaces will need to be extended with random number seed information when we add full support for that, but shouldn't need to be replaced or changed otherwise.

Fixes #960.

Use locset names to set probe sites in single cell model.

1. Extend C++ cable_cell interface to provide a concrete list of locations associated with a locset on the cell.
2. Extend Python single_cell_model to allow placement of probes on locsets.
3. Remove "debug" output in Python ring example.

The single cell model maintains a trace callback for each probe location, and needs to know the explicit locations before starting the simulation. The first step is required to enumerate and record the locations associated with a locset when attaching probes.

Open issues: 

1. The extension to `cable_cell` won't sufficiently support locsets that rely on random number generators with global seeds (it would require passing additional meta data required to fully instantiate the locset)
2. Enhancing the probe recipe interface to be able to associate one `probe_id` with all locations in a locset (the number of which is not specified at the time the `probe_id` is created) would support the interface implemented for the `single_cell_model`.

Note: The richer interface on the C++ side will be implemented in time (see #957).

A big update that wraps over 90% of the public C++ API, with enough functionality to let Python users to perform useful modelling.

Key features
- wrapping of cable cell functionality
  - user-defined explicit compartmentalisation not supported
- `single_cell_model` abstraction unique to the Python wrapper that simplifies developing and testing single cell models
- one-dimensional cell builder for Python wrapper that simplfies building cells that 
  - in and of itself limited in scope, but a very useful example of mapping a richer morphology builder onto `sample_tree`s.
- parsing of region and location expressions from strings
  - implements a generic s-expression parser that we can use for other purposes later

Adds

* to `recipe.cpp`: 
`num_probes` and `get_probe` with helper function `cable_probe(kind, id, location)`
* `sample.cpp`: 
`sample` type with attributes `time` and `value`; a `sampler` with `samples()` and `clear()` and helper functions `attach_sampler(sim, dt)` for adding all probes as well as `attach_sampler(sim, dt, probe_id)` for adding one probe
* `morphology.cpp`: 
`location` as `arb::mlocation` with `branch` and `position` 
* corresponding docs and example
* update to new cable cell semantics in c++ validation test

fixes #762

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

- wraps `partition_hint` struct for python
- `partition_hint`struct in `load_balance.hpp` is adjusted in a way that in case of zero unsigned int `cpu_group_size`/`gpu_group_size` it is set to the default value (via setter/getter)
- adds documentation for `partition_hint`  
- adds testing for `domain_decomposition` including `partition_hint` (in unit and unit_distributed)
- adds `partition_hint` in example `ring.py`
- corrects `config.cpp` (and doc) to test for `ARB_GPU_ENABLED` instead of `ARB_WITH_GPU`

Fixes #776 
Addresses #799, #769

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

Wraps the `meter_manager` with
- the constructor
- `start(ctx)`
- `checkpoint(name, ctx)`
- `checkpoint_names`
- `times`

and the `meter_report` with
- `make_meter_report(meter_manager, ctx)`

Fixes #765

akuesters authored 5 years ago and Benjamin Cumming committed 5 years ago

Add docs for Python wrapper to ReadTheDocs:

- Overview, Common Types, Hardware Management, Recipes, Domain Decomposition, Simulations, Metering
- Installing Arbor: Optional Requirements (Python), Buidling and Installing (Python Frontend), and Installation (Python Module)

Missing (, since not implemented yet): 

- probes
- arbor-sup 
- hint maps in domain_decomposition
- reset, events, empty schedule in event_generator
Also does not cover unit testing (since doc is user-centric).

Makes also defaults and units in wrapper consistent.

Fixes  #766

Support for recording spikes generated by a simulation in the Python wrapper
* Implement a `spike_recorder` that holds a shared pointer to a `std::vector` of spikes, and a callback for the `arb::simulation` spike recording API.
* Add `python/example/ring.py` that creates a ring network, then records and prints spikes.
* Some fixes to get the full `recipe` -> `domain_decomposition` -> `simulation` -> `spikes` workflow to work
  * always use default `global_parameters`: user customization of global parameters for cable cells can wait until the ion species interface is finished.
  * change the Python recipe interface for `recipe::connections_on` to use `pybind11::objects` because of shim.
* Some small improvements to error and help messages.

Fixes #764