# Changes

- `cable_cell_global_properties` now holds a catalogue directy
- fixes for tests, examples, and python bindings accordingly

# Linked Issues
Fixes #1446 

New structs and types:
* `cell_tag_type` (std::string): for labelling placeable items on a cell. The label refers to a number of items placed on a locset, equal to the number of locations in a locset. The number of locations in not always known to the user, so the previous way of using indices for items was no longer sufficient. 
* `lid_selection_policy`: for allowing a user to select a single item from a group of items sharing a label. Currently only `round_robin` and `assert_univalent` are supported. 
* `cell_local_label_type` and `cell_global_label_type`: for identifying the target and source of a connection or gap_junction connection. 
* `cell_label_ranges`, and `cell_labels_and_gids`: for propagating information about the labelled items on the cell from the cell groups back to the simulation and communicator. 
* `label_resolution_map` and `resolver`: for selecting an item (and retaining state) from a labelled group of items on a cell according to a user-selected policy.

Changes to the model-initialization: 
* The `communicator` now needs `label_resolution_maps` constructed from the cell group data in order to build the `connections` vectors. 
* The `simulation_state` object handles the transfer of the information when it is constructed. 
* Spike exchange at runtime remains unchanged, because `communicator::connections` remains unchanged. 

Changes to cells, cell_groups and recipe:
* `decor::place` expects a third label parameter, no longer returns an `lid_range`.
* `lif`, `source`, and `benchmark` cells need source/target labels in their constructors. 
* A `cell_group` needs to save data about the gid/labels/lid_ranges of each cell, to propagate back to the `communicator` constructor. 
* Connections/gap junction connections are formed between {label, policy} pairs on cells instead of indices. 
* `num_sources`, `num_targets`, `num_gap_junction_sites` deleted from `recipe`.

Additional changes:
* Add MPI wrapper for exchanging vectors of strings. 
* Corresponding updates to unit tests, Python wrapper, C++ and Python examples, documentation.

Fixes #1394

* Raise an exception if:
1. `recipe.num_souces(gid)` != number of detectors placed on the cell.
2. `recipe.num_targets(gid)`  != number of synapses placed on the cell.
3. `recipe.connections_on(gid)` has connections with non-existent source or target gids or lids
* Raise better exception for gap junctions
* Fix unit test 

Addresses #681 

* Replace usages of `util::any` with `std::any`.
* Modify `util::any_cast` so that it forwards `std::any` arguments to `std::any_cast`; move into own header.
* Prevent `std::unique_any` from implicitly encapsulating `std::any` objects.
* Replace `test_any.cpp` unit tests with `test_any_cast.cpp` unit tests.
* Replace some usages of `std::is_same<X, Y>::value` with `std::is_same_v<X, Y>` as part of general C++17ification.
* Remove `util/variant.hpp`, `test_variant.cpp`.
* Replace occurrences of `util::variant` with `std::variant`.
* Replace `util::variant::get<N>` with `std::get<N>`.
* Replace `util::get_if<N>` with `std::get_if<N>`, passing a pointer to variant instead of a ref or value.
* Add some standard #includes where they were required in the source, but only included transitively.
* Some minor comment tidying and fix-ups.
* Update C++ API docs.

Fixes #1144.

* Make recipe return probes as a vector.
* Remove `probe_id` from `probe_info`.
* Rename `fvm_probe_info` to `fvm_probe_data` (everything was being called info, and it was getting confusing).
* Make `probe_association_map` specific to `mc_cell_group`/`fvm_lowered_cell`.
* Change probe_association_map representation: unordered map for probe_id -> tag; unordered multimap for probe_id -> fvm_probe_data. This allows multiple probes to be associated with the same probe id.
* Call sampler callback separately for each probe with the same probe_id.
* Replace location-based probes with locset equivalents.
* Add index for probes sharing a probe id.
* Bundle all probe metadata (id, tag, index, probe-specific meta) into `probe_metadata` struct for passing to sampler callbacks.
* Change simple_sampler to work on `trace_vector`, a vector of `trace_data`. The _i_th element is the data from probe with index i.
* Consolidate hash composition and `std::hash` specialization code in new header.
* Update python lib for new API.
* Update tests and examples for new recipe, internal probe, and simple_sampler APIs.
* Update docs to suit.

* Collect cable cell parameter setting structures into `include/cable_cell_param.hpp`.
* Restructure electrical specifications (axial resistance, membrane capacitance) and ionic concentrations and reversal potentials on cable cells, so that these can be specified with a global default, per-cell defaults, and per-segment values.
* Allow reversal potentials to be set by a mechanism of a new kind 'revpot', which are prohibited from maintaining state or writing to any shared state other than ionic reversal potentials.
* Specify reversal potential mechanisms as global defaults or per-cell. Reversal potential mechanisms may not be specified at the level of a segment in order to avoid non-linearities arising from the discretization.
* Supply default cable cell parameter data that matches NEURON values (this is _not_ used by default).
* Replace the d_lambda calculation with one that approximates more faithfully the effect of tapered segments, and which will use the electrical values inherited by cell or global defaults.
* Supply a bundled mechanism 'nernst' that replaces the previous hard-coded Nernst potential calculation, for use (together with ion rebinding) in the new ion description scheme.
* All ions used in a cable_cell model must be present in the top level global parameter table, together with their charge.
* Extend semantics of external variables in modcc, to permit direct assignment (as opposed to just += or -=.)
* Extend `util::binary_search_index` to allow for a projection functional analagous to other range utilites.
* Add documentation on the cable cell API, sketching over many of the details, but describing in particular the interface changes for default parameters and reversal potential behaviour.
* Adjust pyarb for new API
* Hard code global cable cell properties in the python recipe shim to useneuron default parameters.

Sam Yates authored 5 years ago and Benjamin Cumming committed 5 years ago

* Make global parameters and ion rebindings part of the instantiate interface, rather than insist that all concrete mechanisms implement these as methods.
* Mechanism catalogue instance() returns a pair, comprising the concrete mechanism for the requested backend, together with the override data.
* Extend catalgoue derive() method to take a list of old ion name -> new ion name remappings for a mechanism.
* Add exceptions for ion remapping errors, and check for these errors.
* Add convenience function for reparameterizing a mechanism with a single ion dependency over other ions. (This will be used for the future nernst pre-supplied mechanism.)
* Add unit tests for: chained renamings of ion names across multiple derivations; correct shared state ion assignment after renamings; ion remapping exceptions; parameterize_over_ion.

Addresses #757.

* Check if the multiplicity vector is empty before trying to use its contents. 
* Add unit test to catch the problem

Rename mc_cell to cable_cell, cable1d_neuron to cable, and lif_neuron to lif in public API.

Fixes #701 

noraabiakar authored 6 years ago and Sam Yates committed 6 years ago

Add support for gap junctions in mc_cells, modelled as a conductance between 2 cell CVs. Gap junctions act as additional current sources on the CVs, as opposed to participating in the implicit voltage integration step.

Cells connected via gap junctions must be in the same cell group as determined by the provided domain decomposition.

* Extend `mc_cell` to hold a list of gap junction locations.
* Add `num_gap_junction_sites()` and `gap_junctions_on()` methods to the `recipe` interface.
* Add `gather_gids()` collective operation to distributed context interface and implementations.
* Extend `partition_load_balance()` functionality to ensure that cells connected by gap junctions are put in the same groups.
* Permute cells within `mc_cell_group` so that cells connected by gap junctions are contiguous.
* Add gap junction information to `multicore::shared_state` and `gpu::shared_state`, together with `add_gj_current()` method that computes GJ current contributions.
* Add `time_dep` field to shared state structures that records which adjacent cells are GJ-connected for the purposes of determining integration time intervals.
* Add `sync_time_to()` method to shared state structures to determine the common integration time step across GJ-connected cells.
* Incorporate new shared state methods into the `fvm_lower_cell_impl::integrate()` integration loop.
* Add new arbor exception `arb::gj_kind_mismatch`.
* Add unit tests for new functionality.
* Add new example code `gap_junctions` for GJ demonstration.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

This time we're moving `recipe.hpp` and `simulation.hpp`, plus the requirements they bring.

Code changes:
* Pimplize `simulation`.
* Consolidate arbor exceptions: all non-cell kind specific exceptions that might be expected to reach user code now have consistent messages and fit in an exception hierarchy based at `arb::arbor_exception`. Internal errors throw an `arb::arbor_internal_error` exception.
* Renamed `postsynaptic_spike_event` to `spike_event`. (Note: `pse_vector` name is unchanged.)
* Repurposed `pprintf` and moved it into `strprintf.h` — further consolidation is a TODO.
* Made a generic `util::to_string` to avoid redundancy of `operator<<` overloads and other `to_string` definitions. Defaults to ADL `to_string`, `std::to_string`, and finally tries using `operator<<`.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Further work to public install target.

* Move SIMD classes, cell description classes, simple sampler to public include.
* Rename `cell` to `mc_cell`, `segment` to `mc_segment`, and remove `_description` from cell description class names and includes.
* Move `compartment_model` out of `mc_cell` interface and use only in `fvm_layout.cpp`.
* (Provisionally) remove area/volume methods on `mc_cell` and `mc_segment`.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake and build refactoring

*   Use CUDA as first-class language (leading to CMake 3.9 minimum version requirement).

*   Use 'modern CMake' interface libraries for compiler options, include file and library dependency tracking. Interface library targets:
    * `arbor-deps`: compiler options and library requirements for the `libarbor.a` static library, as governed by configure-time options and environment.
    * `arbor-private-headers`: include path for non-installed headers, as required by unit tests and arbor itself.
    * `arbor-aux`: helper classes and utilities used across tests and examples.
    * `ext-json`, `ext-tclap`, `ext-tbb`, `ext-benchmark`, `ext-sphinx_rtd_theme`: externally maintained software that we include (directly or via submodule) in the `ext/` subdirectory.
 
*   Single static library `libarbor.a` includes all built-in modules and CUDA objects.

*   Simply configuration options:
    *  `ARB_WITH_TRACE`, `ARB_AUTORUN_MODCC_ON_CHANGES` `ARB_SYSTEM_TYPE` removed.
    * External `modcc` is provided by `ARB_MODCC` configuration option; if provided `modcc` is still buildable, but is not included in the default target.
    * `ARB_PRIVATE_TBBLIB`, defaulting to `OFF`, instructs the build to make TBB from the included submodule.

*   Extend `ErrorTarget` functionality to provide a dummy target or an error target based on a condition.
*   Generate header version defines and library version variables based on git status and project version, via new script `include/git-source-id`.
*   All generated binaries now placed in `bin/` subdirectory at build.
*   Install targets installs: public headers (incomplete); static library; `modcc` tool; `lmorpho` executable; `html` documentation (examples, tests and validation data are currently not installed).
*   Executable targets have had the `.exe` suffix removed; unit tests are labelled `unit` (arbor unit tests), `unit-modcc` (modcc unit tests), `unit-local` (distributed tests with local context), `unit-mpi` (distributed tests with MPI context).
*   More graceful handling of configure-time detection of `nrniv`, Julia and required Julia modules for validation data generation.
*   Add `cmake/FindJulia.cmake`, `cmake/FindTBB.cmake`  package finders, and adjust `cmake/FindUnwind.cmake` to use link library-style properties.
*  Adjust travis script to test `unit-local` and `unit-mpi` if appropriate.
*  Simply documentation `conf.py`.

Source relocation and reorganization

* All external project sources and files moved to `ext/`.
* Source code refactoring to decouple library-using code from the configure-time definitions that govern arbor behaviour: removes conditional code in public headers that depends upon `ARB_WITH_X`-type definitions at compile time. Affected code is is in the public interfaces for MPI, the threading implementation, and the profiler.
* Remove `util/debug.hpp`; split out functionality for pretty-printing from assertion handling.
* Make FVM cell non-physical voltage check a run-time cell-group parameter.
* Move spike double buffer implementation to `simulation.cpp`.
* Make timer utility wrap POSIX `clock_gettime` independent of threading configuration.
* Make `mpi_error` derive from `system_error` and follow C++11 `system_error` semantics.
* `EXPECTS` macro replaced by `arb_assert` macro.
* JSON dependency removed from `libarbor.a` and header files: moved to auxiliary library.
* Publicly visible macros garner an `ARB_` prefix as required.
* Move SWC test file to `test/unit` directory.
* Work-in-progress splitting of public from private includes: as a convention not entirely adhered to as yet, private headers within arbor source are included with `""`, public headers with `<>`.

Modcc interface changes

* Expose via `--namespace` option the functionality that sets the namespace in generated code.
* Use `--profile` option to add profiler hooks to generated code; uses public function interface directly rather than `PE/PL` macros in order to avoid public `PE` and `PL` defines.

First commit of two for mechanism refactor work (refer to PR #484 and PR #483).

FVM/mechanism code:
* Refactor mechanism data structures to decouple backend-specific implementations and mechanism metadata.
* Add mechanism catalogue for managing mechanism metadata and concrete implementation prototypes.
* Add fingerprint-checking to mechanism metadata and implementations to confirm they come from the same NMODL source (fingerprint is not yet computed, but tests are in place).
* Split FVM discretization work out from FVM integrator code.
* Use abstract base class over backend-templated FVM integrator class `fvm_lowered_cell_impl` to allow separate compilation of `mc_cell_group` and to remove the dummy backend code.
* Add a new FVM-specific scalar type `fvm_index_type` that is an alias for `int` to replace
`fvm_size_type` in fvm layouts and mechanisms. This was chosen as an alternative
to making `unsigned` versions of all our SIMD implementation classes.
* Extend `cable1d_neuron` global data to encompass: mechanism catalogue; default ion concentrations and charges; global temperature (only for Nernst); initial membrane potential.

Modcc:
* Collect printer sources in modcc under `printer/`.
* Move common functionality across printers into `printer/printerutil.{hpp,cpp}`.
* Add string to file I/O implemented in routines read_all and write_all in `io/bulkio.hpp`.
* Implement indent-friendly source code generation via a `std::streambuf` filter `io::prefixbuf` defined in `io/prefixbuf.hpp`, together with manipulators and a corresponding std::ostream-derived wrapper.
* Rewrite printers to use new infrastructure: cpu target incorporates SIMD printing options; CUDA printer at this point produces only stubs for CUDA kernel wrappers.
* Modify SIMD printing command line options for modcc: `-s` enables explicit vectorization using the SIMD classes;  `-S <N>` allows a specific data width to be prescribed.
* Fix problem in `test_ca.mod` with uninitialized ion current.
* Add infrastructure support to allow future pre-computation of SIMD index conflict cases for (hopefully) faster scatters and updates.
* Simplify `IndexedVariable` expressions in the AST, making data source explicit via a `sourceKind` enum, and leaving the indexing method and index names up to the printers.
* Allow state variables in the AST to 'shadow' an ion concentration — these are assigned in the
generated `write_ions` method.

SIMD classes:
* Add `simd_cast` operation between SIMD value types of the same width, and with `std::array`. (Note: this was tested and used in an early development version of the code, but not in this version. It was still a lacuna in the original SIMD wrappers, so it has been left in.)
* Restructure SIMD gather/scatter API to use a `simd::indirect` expression,  which encapsulates a pointer and SIMD offset.
* Add `simd::index_constraint` scoped enum to describe knowledge of contention in indirect indices, so that we can branch on this to the appropriate implementation.
* Add SIMD concrete implementation routines `reduce_add` for horizontal reduction and `element0` for access to first lane scalar value.
* Add SIMD value method `sum()` that exposes implementation `reduce_add`.
* Add SIMD concrete implementation routine `compound_indexed_add` that provides the implementation for `indirect(p, simd_indices) += simd_value` construction.
* Fix SIMD `implbase` bug where some static methods were using the `implbase` fall-back functions instead of the derived class specialized implementations.
* Move SIMD mathematical functions into friend routines of `simd_impl` in order to resolve implicit conversions from scalars in mixed SIMD-scalar operations.
* Use a templated `tag` class to dispatch on SIMD concrete implementation types, to avoid problems with incomplete types in method signatures.
* Remove old SIMD intrinsics.

CMake infrastructure:
* Downcase some variables in `CMakeLists.txt` files to  distinguish them visually from CMake keywords and variables.
* Split arbor modcc vectorization option (now `ARB_VECTORIZE`) and target-architecture optimization (now `ARB_ARCH`).
* For `arbor` and `arbormech` targets, and in particular not the `modcc` target, use `ARB_ARCH` to generate corresponding target-appropriate binaries, including, for example, appropriate SIMD support.
* Extend `CompilerOptions.cmake` to map as best as able between the various target architecture names (we use the gcc names) and the correct option to pass to the compiler based on the compiler and platform.
* Add work-around for misidentification by CMake of XL C as Clang.
* As a temporary work-around, include `arbormech` library twice on link line to resolve circular arbor–arbormech dependencies.

Unit tests:
* Extend repertoire of generic sequence equality/near equality testing support  in `common.hpp`.
* Add warning suppression for icc for the malloc instrumentation code.
* SIMD unit tests for indirect expressions, compound indirect add, reduction.
* Make some exact tests into floating point 'near' tests when comparing computed areas and lengths in swc and fvm layout tests, to account for compiler (e.g. icc) performing semantically inequivalent floating point operation reordering or fusion at `-O3`.
* Split out some of the CUDA tests into separate .cpp/.cu files for  separate-compilation purposes.

Other:
* The `padded_allocator` has been modified to propagate alignment/padding on move and copy (these semantics make their use much easier and safer in the multicore mechanism instantiation code).
* Map/table searching utilities in `util/maputil.hpp`.
* Fixes for correct sequence type categorization and `begin/end` ADL.
* Fixes for type guards for range methods that take universal references.
* Removal of some redundant code in range utilities through the use of universal references.
* Add new range view `reverse_view` for ranges delineated by bidirectional iterators.
* Add single argument form of `make_span` to count up from zero, and associated helper `count_along` that gives a span that indexes a supplied container.
* Moved `prefixbuf` to `modcc` source.
* Make sequence positive and negative tests in algorithms generic.
* Add `private`-subverting helper code/macro to `tests/unit/common.hpp` to reduce the number of public testing-only interfaces in the library code.
* Add virtual destructors for virtual base classes.
* Add new arb::math:: functions: `next_pow2` for unsigned integral types, `round_up` to round a number away from zero to next largest magnitude multiple.
* New `index_into` implementation that supports bidirectional access (moved to `util::` namespace).
* Fix problem in `test_ca.mod` with uninitialized ion current.
* Rework dangerous `memory::array(Iter, Iter)` constructor to be less dangerous (and do the expected thing).
* Allow ranges to be constructed from other ranges if the iterators are compatible.

Ben Cumming authored 7 years ago and Sam Yates committed 7 years ago

Add some C++ API documentation.

* Create C++ API section in docs.
* Document `arb::recipe`: both a class reference along with more explanatory text and best practices guide.
* Add some class documentation of basic types required to understand recipe definition.
* Some in-code comment clean up.
* Change `arb::cell_kind` from a vanilla enum to a scoped enum.

Ben Cumming authored 7 years ago and Sam Yates committed 7 years ago

* Enable `cell` copy constructor, remove special tag type used to guard `cell` cloning.
* Provide sane defaults for `recipe` methods.

Implement generic `event_generator`, and some common generators that should be useful to users out of the box:
* poisson sequence                                                                                       
* uniform regular sequence
* from a user-prescribed sequence
* from a `std::vector` of post synaptic events.

Extend `recipe` API to provide an `recipe::event_generators(gid)` that returns a list of `event_generator`s attached to a cell with `gid`.

Update the `model` class and `merge_events` method to support event generators.

Fixes #401 

Fixes #350 

* Replace parameter_list with mechanism_spec.
* Add prototype for mechanism parameter schema checking.
* Allow mechanism weights to be set after construction.
* Combine range parameters on density mechanisms by linear contribution in CVs.
* Cable segment electrical parameters are now member variables.
* Publish mechanism parameter information through new method `mechanism::field_info`; note this will be replaced/improved in upcoming dynamic mechanism catalog work.
* Access mechanism parameter scalars and range data via `mechanism::field_view_ptr` and `mechanism::field_value_ptr` methods.
* Allow mechanism 'global' parameters to be set via a method of specializing mechanisms (and giving them corresponding aliases).
* Extend recipe interface to allow querying of per-cell-kind global information for use by cell group implementations.
* Add unit tests for above - note that linear density mechanism parameter test is tightly coupled with the FVM discretization scheme.

Change nest::mc namespaces to arb

Towards resolution of issue #283.

* Replace probe/sample infrastructure with new API as outlined in the `sampling_api.rst` documentation.
* Separate `cell` probe information from `cell` description object.
* Add `--list` option to `tsplot`, to summarize available time series for plotting together with their metadata.
* Add `--sample-dt` option to validation tests.
* Change validation time series comparison behaviour: linf distance metric now estimated by comparing reference data points against linearly interpreted simulation samples, rather than the other way around.
* Add utility class `any_ptr` which allows lightweight and type-checked access to a type-erased pointer.
* Replace `singleton_recipe` with two simple recipe base classes used for unit and validation tests, in `tests/simple_recipes.hpp`.
* Simplify RSS cell implementation.
* Add statistical functions for testing of probabilistic methods in `tests/unit/stats.hpp` (and .cpp):
   * simple summary stats class;
   * Kolmogorov–Smirnov one-sided statistic and CDF;
   * approximate Poisson CDF.
* Simplify and refactor miniapp `sample_trace` code.
* Add new utility class `handle_set` for managing a collection of integer handles. (Near-simplest implementation.)
* Relax const-ness constraints on `transform_iterator`.