Skip to content
Snippets Groups Projects
Commit 1f4eacd2 authored by akuesters's avatar akuesters Committed by Benjamin Cumming
Browse files

Python wrapper: documentation (#799)

Add docs for Python wrapper to ReadTheDocs:

- Overview, Common Types, Hardware Management, Recipes, Domain Decomposition, Simulations, Metering
- Installing Arbor: Optional Requirements (Python), Buidling and Installing (Python Frontend), and Installation (Python Module)

Missing (, since not implemented yet): 

- probes
- arbor-sup 
- hint maps in domain_decomposition
- reset, events, empty schedule in event_generator
Also does not cover unit testing (since doc is user-centric).

Makes also defaults and units in wrapper consistent.

Fixes  #766
parent 74df9f5a
No related branches found
No related tags found
No related merge requests found
Showing with 1271 additions and 31 deletions
......@@ -83,7 +83,7 @@ cells and members of cell-local collections.
Leaky-integrate and fire neuron.
.. cpp:enumerator:: spiking
.. cpp:enumerator:: spike_source
Proxy cell that generates spikes from a spike sequence provided by the user.
......@@ -148,6 +148,7 @@ Utility Wrappers and Containers
.. cpp:class:: unique_any
Equivalent to :cpp:class:`util::any`, except that:
* it can store any type that is move constructable;
* it is move only, that is it can't be copied.
......
.. _cppdomdec:
.. cpp:namespace:: arb
Domain Decomposition
====================
......@@ -8,7 +10,7 @@ The C++ API for partitioning a model over distributed and local hardware is desc
Load Balancers
--------------
Load balancing generates a :cpp:class:`domain_decomposition` given a :cpp:class:`recipe`
Load balancing generates a :cpp:class:`domain_decomposition` given an :cpp:class:`arb::recipe`
and a description of the hardware on which the model will run. Currently Arbor provides
one load balancer, :cpp:func:`partition_load_balance`, and more will be added over time.
......@@ -37,8 +39,6 @@ describes the cell groups on the local MPI rank.
Arbor provided load balancers such as :cpp:func:`partition_load_balance`
guarantee that this rule is obeyed.
.. cpp:namespace:: arb
.. cpp:function:: domain_decomposition partition_load_balance(const recipe& rec, const arb::context& ctx)
Construct a :cpp:class:`domain_decomposition` that distributes the cells
......
......@@ -236,7 +236,7 @@ The core Arbor library *libarbor* provides an API for:
.. cpp:function:: proc_allocation(unsigned threads, int gpu_id)
Constructor that sets the number of :cpp:var:`threads` and the id :cpp:var:`gpu_id` of
the
the available GPU.
.. cpp:member:: unsigned num_threads
......@@ -286,15 +286,15 @@ whether it has a GPU, how many threads are in its thread pool, using helper func
.. cpp:function:: bool has_gpu(const context&)
Query if the context has a GPU.
Query whether the context has a GPU.
.. cpp:function:: unsigned num_threads(const context&)
Query the number of threads in a context's thread pool
Query the number of threads in a context's thread pool.
.. cpp:function:: bool has_mpi(const context&)
Query if the context has an MPI communicator.
Query whether the context uses MPI for distributed communication.
.. cpp:function:: unsigned num_ranks(const context&)
......@@ -304,7 +304,7 @@ whether it has a GPU, how many threads are in its thread pool, using helper func
.. cpp:function:: unsigned rank(const context&)
Query the rank of the calling rand. If the context has an MPI
Query the rank of the calling rank. If the context has an MPI
communicator, return is equivalent to :cpp:any:`MPI_Comm_rank`.
If the communicator has no MPI, returns 0.
......@@ -317,18 +317,18 @@ Here are some simple examples of how to create a :cpp:class:`arb::context` using
#include <arbor/context.hpp>
// Construct a context that uses 1 thread and no GPU or MPI
// Construct a context that uses 1 thread and no GPU or MPI.
auto context = arb::make_context();
// Construct a context that:
// * uses 8 threads in its thread pool.
// * uses 8 threads in its thread pool;
// * does not use a GPU, regardless of whether one is available;
// * does not use MPI
// * does not use MPI.
arb::proc_allocation resources(8, -1);
auto context = arb::make_context(resources);
// Construct one that uses:
// * 4 threads and the first GPU.
// Construct one that uses:
// * 4 threads and the first GPU;
// * MPI_COMM_WORLD for distributed computation.
arb::proc_allocation resources(4, 0);
auto mpi_context = arb::make_context(resources, MPI_COMM_WORLD)
......
......@@ -130,7 +130,7 @@ Class Documentation
structures ahead of time and for putting in place any structures or
information in the concrete cell implementations to allow monitoring.
By default throws :cpp:type:`std::logic_error`. If ``arb::recipe::num_probes``
By default throws :cpp:type:`std::logic_error`. If :cpp:func:`num_probes`
returns a non-zero value, this must also be overridden.
.. cpp:function:: virtual util::any get_global_properties(cell_kind) const
......
......@@ -6,7 +6,7 @@ Simulations
From recipe to simulation
-------------------------
To build a simulation the following are needed:
To build a simulation the following concepts are needed:
* An :cpp:class:`arb::recipe` that describes the cells and connections
in the model.
......@@ -84,7 +84,7 @@ Class Documentation
.. cpp:function:: void inject_events(const pse_vector& events)
Add events directly to targets.
Must be called before calling :cpp:func:`simulation::run`, and must contain events that
Must be called before calling :cpp:func:`run`, and must contain events that
are to be delivered at or after the current simulation time.
**Updating Model State:**
......@@ -111,7 +111,7 @@ Class Documentation
sampling_policy policy = sampling_policy::lax)
Note: sampler functions may be invoked from a different thread than that
which called :cpp:func:`simulation::run`.
which called :cpp:func:`run`.
(see the :ref:`sampling_api` documentation.)
......@@ -128,7 +128,7 @@ Class Documentation
.. cpp:function:: std::size_t num_spikes() const
The total number of spikes generated since either construction or
the last call to :cpp:func:`simulation::reset`.
the last call to :cpp:func:`reset`.
.. cpp:function:: void set_global_spike_callback(spike_export_function export_callback)
......
......@@ -85,6 +85,15 @@ Alternative citation formats for the paper can be `downloaded here <https://ieee
.. toctree::
:caption: Python:
py_intro
py_common
py_recipe
py_cable_cell
py_hardware
py_domdec
py_simulation
py_profiler
.. toctree::
:caption: C++ API:
......
......@@ -117,10 +117,10 @@ More information on building with MPI is in the `HPC cluster section <cluster_>`
Python
~~~~~~
Arbor has a Python front end, for which Python 3.6 is required.
Arbor has a Python frontend, for which Python 3.6 is required.
In order to use MPI in combination with the python frontend the
`mpi4py <https://mpi4py.readthedocs.io/en/stable/install.html#>`_
Python package is also recommended.
Python package is recommended.
Documentation
~~~~~~~~~~~~~~
......@@ -354,10 +354,10 @@ example:
Arbor supports and has been tested on the Kepler (K20 & K80), Pascal (P100) and Volta (V100) GPUs
Python Front End
Python Frontend
----------------
Arbor can be used with a python front end which is enabled by toggling the
Arbor can be used with a python frontend which is enabled by toggling the
CMake ``ARB_WITH_PYTHON`` option:
.. code-block:: bash
......
......@@ -21,3 +21,4 @@ A *load balancer* generates the domain decomposition using the model recipe and
resources on which the model will run described by an execution context.
Currently Arbor provides one load balancer and more will be added over time.
Arbor's Python interface of domain decomposition and load balancers is documented in :ref:`pydomdec` and the C++ interface in :ref:`cppdomdec`.
......@@ -24,4 +24,4 @@ Execution Context
An *execution context* contains the local thread pool, and optionally the GPU state and MPI communicator, if available. Users of the library configure contexts, which are passed to Arbor methods and types.
See :ref:`cpphardware` for documentation of the C++ interface for managing hardware resources.
See :ref:`pyhardware` for documentation of the Python interface and :ref:`cpphardware` for the C++ interface for managing hardware resources.
......@@ -10,13 +10,13 @@ a *recipe* describes a model, and a *simulation* is an executable instantiation
To be able to simulate a model, three basic steps need to be considered:
* first, describe the model by defining a recipe;
* second, define the computational resources available to execute the model;
* finally, initiate and execute a simulation of the recipe on the chosen hardware resources.
1. Describe the model by defining a recipe;
2. Define the computational resources available to execute the model;
3. Initiate and execute a simulation of the recipe on the chosen hardware resources.
:ref:`modelrecipe` represent a set of neuron constructions and connections with *mechanisms* specifying ion channel and synapse dynamics in a cell-oriented manner. This has the advantage that cell data can be initiated in parallel.
:ref:`Recipes <modelrecipe>` represent a set of neuron constructions and connections with *mechanisms* specifying ion channel and synapse dynamics in a cell-oriented manner. This has the advantage that cell data can be initiated in parallel.
A cell represents the smallest unit of computation and forms the smallest unit of work distributed across processes. Arbor has built-in support for different :ref:`cell types <modelcells>` , which can be extended by adding new cell types to the C++ cell group interface.
A cell represents the smallest unit of computation and forms the smallest unit of work distributed across processes. Arbor has built-in support for different :ref:`cell types <modelcells>`, which can be extended by adding new cell types to the C++ cell group interface.
:ref:`modelsimulation` manage the instantiation of the model and the scheduling of spike exchange as well as the integration for each cell group. A cell group represents a collection of cells of the same type computed together on the GPU or CPU. The partitioning into cell groups is provided by :ref:`modeldomdec` which describes the distribution of the model over the locally available computational resources.
......
......@@ -96,4 +96,5 @@ subset of NEURON's mechanism specification language NMODL.
Examples
Common examples are the *passive/ leaky integrate-and-fire* model, the *Hodgkin-Huxley* mechanism, the *(double-) exponential synapse* model, or the *Natrium current* model for an axon.
Detailed documentation and best practices for C++ recipes can be found in :ref:`cpprecipe`.
Detailed documentation for Python recipes can be found in :ref:`pyrecipe`.
C++ :ref:`cpprecipe` are documented and best practices are shown as well.
......@@ -25,4 +25,5 @@ Simulations provide an interface for executing and interacting with the model:
* The model state can be *reset* to its initial state before the simulation was started.
* *Sampling* of the simulation state can be performed during execution with samplers and probes (e.g. compartment voltage and current) and spike output with the total number of spikes generated since either construction or reset.
See :ref:`cppsimulation` for documentation can be found in C++ simulation API.
The documentation for Arbor's Python simulation interface can be found in :ref:`pysimulation`.
See :ref:`cppsimulation` for documentation of the C++ simulation API.
.. _pycable_cell:
Python Cable Cells
====================
The interface for specifying cell morphologies with the distribution of ion channels
and syanpses is a key part of the user interface. Arbor will have an advanced and user-friendly
interface for this, which is currently under construction.
To allow users to experiment will multi-compartment cells, we provide some helpers
for generating cells with random morphologies, which are documented here.
.. Warning::
These features will be deprecated once the morphology interface has been implemented.
.. function:: make_cable_cell(seed, params)
Construct a branching :class:`cable_cell` with a random morphology (via parameter ``seed``) and
synapse end points locations described by parameter ``params``.
The soma has an area of 500 μm², a bulk resistivity of 100 Ω·cm,
and the ion channel and synapse dynamics are described by a Hodgkin-Huxley (HH) mechanism.
The default parameters of HH mechanisms are:
- Na-conductance 0.12 S⋅cm⁻²,
- K-conductance 0.036 S⋅cm⁻²,
- passive conductance 0.0003 S⋅cm⁻², and
- passive potential -54.3 mV
Each cable has a diameter of 1 μm, a bulk resistivity of 100 Ω·cm,
and the ion channel and synapse dynamics are described by a passive/ leaky integrate-and-fire model with parameters:
- passive conductance 0.001 S⋅cm⁻², and
- resting potential -65 mV
Further, a spike detector is added at the soma with threshold 10 mV,
and a synapse is added to the mid point of the first dendrite with an exponential synapse model:
- time decaying constant 2 ms
- resting potential 0 mV
Additional synapses are added based on the number of randomly generated :attr:`cell_parameters.synapses` on the cell.
:param seed: The seed is an integral value used to seed the random number generator, for which the :attr:`arbor.cell_member.gid` of the cell is a good default.
:param params: By default set to :class:`cell_parameters()`.
.. class:: cell_parameters
Parameters used to generate random cell morphologies.
Where parameters must be given as ranges, the first value is at the soma,
and the last value is used on the last level.
Values at levels in between are found by linear interpolation.
.. attribute:: depth
The maximum depth of the branch structure
(i.e., maximum number of levels in the cell (not including the soma)).
.. attribute:: lengths
The length of the branch [μm], given as a range ``[l1, l2]``.
.. attribute:: synapses
The number of randomly generated synapses on the cell.
.. attribute:: branch_probs
The probability of a branch occuring, given as a range ``[p1, p2]``.
.. attribute:: compartments
The compartment count on a branch, given as a range ``[n1, n2]``.
.. _pycommon:
Common Types
=====================
Cell Identifiers and Indexes
----------------------------
The types defined below are used as identifiers for cells and members of cell-local collections.
.. module:: arbor
.. class:: cell_member
.. function:: cell_member(gid, index)
Construct a cell member with parameters :attr:`gid` and :attr:`index` for global identification of a cell-local item.
Items of type :class:`cell_member` must:
* be associated with a unique cell, identified by the member :attr:`gid`;
* identify an item within a cell-local collection by the member :attr:`index`.
An example is uniquely identifying a synapse in the model.
Each synapse has a post-synaptic cell (with :attr:`gid`), and an :attr:`index` into the set of synapses on the post-synaptic cell.
Lexographically ordered by :attr:`gid`, then :attr:`index`.
.. attribute:: gid
The global identifier of the cell.
.. attribute:: index
The cell-local index of the item.
Local indices for items within a particular cell-local collection should be zero-based and numbered contiguously.
An example of a cell member construction reads as follows:
.. container:: example-code
.. code-block:: python
import arbor
# construct
cmem = arbor.cell_member(0, 0)
# set gid and index
cmem.gid = 1
cmem.index = 42
.. class:: cell_kind
Enumeration used to identify the cell kind, used by the model to group equal kinds in the same cell group.
.. attribute:: cable
A cell with morphology described by branching 1D cable segments.
.. attribute:: lif
A leaky-integrate and fire neuron.
.. attribute:: spike_source
A proxy cell that generates spikes from a spike sequence provided by the user.
.. attribute:: benchmark
A proxy cell used for benchmarking.
An example for setting the cell kind reads as follows:
.. container:: example-code
.. code-block:: python
import arbor
kind = arbor.cell_kind.cable
.. _pydomdec:
Domain Decomposition
====================
The Python API for partitioning a model over distributed and local hardware is described here.
Load Balancers
--------------
.. currentmodule:: arbor
Load balancing generates a :class:`domain_decomposition` given an :class:`arbor.recipe`
and a description of the hardware on which the model will run. Currently Arbor provides
one load balancer, :func:`partition_load_balance`, and more will be added over time.
If the model is distributed with MPI, the partitioning algorithm for cells is
distributed with MPI communication. The returned :class:`domain_decomposition`
describes the cell groups on the local MPI rank.
.. function:: partition_load_balance(recipe, context)
Construct a :class:`domain_decomposition` that distributes the cells
in the model described by an :class:`arbor.recipe` over the distributed and local hardware
resources described by an :class:`arbor.context`.
The algorithm counts the number of each cell type in the global model, then
partitions the cells of each type equally over the available nodes.
If a GPU is available, and if the cell type can be run on the GPU, the
cells on each node are put into one large group to maximise the amount of fine
grained parallelism in the cell group.
Otherwise, cells are grouped into small groups that fit in cache, and can be
distributed over the available cores.
.. Note::
The partitioning assumes that all cells of the same kind have equal
computational cost, hence it may not produce a balanced partition for
models with cells that have a large variance in computational costs.
Decomposition
-------------
As defined in :ref:`modeldomdec` a domain decomposition is a description of the distribution of the model over the available computational resources.
Therefore, the following data structures are used to describe domain decompositions.
.. class:: backend
Enumeration used to indicate which hardware backend to execute a cell group on.
.. attribute:: multicore
Use multicore backend.
.. attribute:: gpu
Use GPU backend.
.. Note::
Setting the GPU back end is only meaningful if the cell group type supports the GPU backend.
.. class:: domain_decomposition
Describes a domain decomposition and is soley responsible for describing the
distribution of cells across cell groups and domains.
It holds cell group descriptions (:attr:`groups`) for cells assigned to
the local domain, and a helper function (:func:`gid_domain`) used to
look up which domain a cell has been assigned to.
The :class:`domain_decomposition` object also has meta-data about the
number of cells in the global model, and the number of domains over which
the model is destributed.
.. Note::
The domain decomposition represents a division of **all** of the cells in
the model into non-overlapping sets, with one set of cells assigned to
each domain.
.. function:: gid_domain(gid)
A function for querying the domain id that a cell is assigned to (using global identifier :attr:`arbor.cell_member.gid`).
.. attribute:: num_domains
The number of domains that the model is distributed over.
.. attribute:: domain_id
The index of the local domain.
Always 0 for non-distributed models, and corresponds to the MPI rank
for distributed runs.
.. attribute:: num_local_cells
The total number of cells in the local domain.
.. attribute:: num_global_cells
The total number of cells in the global model
(sum of :attr:`num_local_cells` over all domains).
.. attribute:: groups
The descriptions of the cell groups on the local domain.
See :class:`group_description`.
.. class:: group_description
Return the indexes of a set of cells of the same kind that are grouped together in a cell group in an :class:`arbor.simulation`.
.. function:: group_description(kind, gids, backend)
Construct a group description with parameters :attr:`kind`, :attr:`gids` and :attr:`backend`.
.. attribute:: kind
The kind of cell in the group.
.. attribute:: gids
The list of gids of the cells in the cell group.
.. attribute:: backend
The hardware backend on which the cell group will run.
.. _pyhardware:
Hardware Management
===================
Arbor provides two ways for working with hardware resources:
* *Prescribe* the hardware resources and their contexts for use in Arbor simulations.
* *Query* available hardware resources (e.g. the number of available GPUs), and initializing MPI.
Available Resources
-------------------
Helper functions for checking cmake or environment variables, as well as configuring and checking MPI are the following:
.. currentmodule:: arbor
.. function:: config()
Returns a dictionary to check which options the Arbor library was configured with at compile time:
* ``ARB_MPI_ENABLED``
* ``ARB_WITH_MPI4PY``
* ``ARB_WITH_GPU``
* ``ARB_VERSION``
.. container:: example-code
.. code-block:: python
import arbor
arbor.config()
{'mpi': True, 'mpi4py': True, 'gpu': False, 'version': '0.2.1-dev'}
.. function:: mpi_init()
Initialize MPI with ``MPI_THREAD_SINGLE``, as required by Arbor.
.. function:: mpi_is_initialized()
Check if MPI is initialized.
.. class:: mpi_comm
.. function:: mpi_comm()
By default sets MPI_COMM_WORLD as communicator.
.. function:: mpi_comm(object)
Converts a Python object to an MPI Communicator.
.. function:: mpi_finalize()
Finalize MPI by calling ``MPI_Finalize``.
.. function:: mpi_is_finalized()
Check if MPI is finalized.
Prescribed Resources
---------------------
The Python wrapper provides an API for:
* prescribing which hardware resources are to be used by a
simulation using :class:`proc_allocation`.
* opaque handles to hardware resources used by simulations called
:class:`context`.
.. class:: proc_allocation
Enumerates the computational resources on a node to be used for a simulation,
specifically the number of threads and identifier of a GPU if available.
.. function:: proc_allocation()
By default selects one thread and no GPU.
.. function:: proc_allocation(threads, gpu_id)
Constructor that sets the number of :attr:`threads` and the id :attr:`gpu_id` of the available GPU.
.. attribute:: threads
The number of CPU threads available, 1 by default.
.. attribute:: gpu_id
The identifier of the GPU to use.
Must be ``None``, or a non-negative integer.
The :attr:`gpu_id` corresponds to the ``int device`` parameter used by CUDA API calls
to identify gpu devices.
Set to ``None`` to indicate that no GPU device is to be used.
See ``cudaSetDevice`` and ``cudaDeviceGetAttribute`` provided by the
`CUDA API <https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__DEVICE.html>`_.
.. cpp:function:: has_gpu()
Indicates whether a GPU is selected (i.e., whether :attr:`gpu_id` is ``None``).
Here are some examples of how to create a :class:`proc_allocation`.
.. container:: example-code
.. code-block:: python
import arbor
# default: one thread and no GPU selected
alloc1 = arbor.proc_allocation()
# 8 threads and no GPU
alloc2 = arbor.proc_allocation(8, None)
# reduce alloc2 to 4 threads and use the first available GPU
alloc2.threads = 4
alloc2.gpu_id = 0
.. class:: context
An opaque handle for the hardware resources used in a simulation.
A :class:`context` contains a thread pool, and optionally the GPU state
and MPI communicator. Users of the library do not directly use the functionality
provided by :class:`context`, instead they configure contexts, which are passed to
Arbor interfaces for domain decomposition and simulation.
.. function:: context()
Construct a local context with one thread, no GPU, no MPI.
.. function:: context(alloc)
Create a local context, with no distributed/MPI, that uses the local resources described by :class:`proc_allocation`.
.. attribute:: alloc
The computational resources, one thread and no GPU by default.
.. function:: context(alloc, mpi)
Create a distributed context, that uses the local resources described by :class:`proc_allocation`, and
uses the MPI communicator for distributed calculation.
.. attribute:: alloc
The computational resources, one thread and no GPU by default.
.. attribute:: mpi
The MPI communicator (see :class:`mpi_comm`).
mpi must be ``None``, or an MPI communicator.
.. function:: context(threads, gpu_id)
Create a context that uses a set number of :attr:`threads` and the GPU with id :attr:`gpu_id`.
.. attribute:: threads
The number of threads available locally for execution, 1 by default.
.. attribute:: gpu_id
The identifier of the GPU to use, ``None`` by default.
Must be ``None``, or a non-negative integer.
.. function:: context(threads, gpu_id, mpi)
Create a context that uses a set number of :attr:`threads` and gpu identifier :attr:`gpu_id` and MPI communicator :attr:`mpi` for distributed calculation.
.. attribute:: threads
The number of threads available locally for execution, 1 by default.
.. attribute:: gpu_id
The identifier of the GPU to use, ``None`` by default.
Must be ``None``, or a non-negative integer.
.. attribute:: mpi
The MPI communicator (see :class:`mpi_comm`).
mpi must be ``None``, or an MPI communicator.
Contexts can be queried for information about which features a context has enabled,
whether it has a GPU, how many threads are in its thread pool.
.. attribute:: has_gpu
Query whether the context has a GPU.
.. attribute:: has_mpi
Query whether the context uses MPI for distributed communication.
.. attribute:: threads
Query the number of threads in the context's thread pool.
.. attribute:: ranks
Query the number of distributed domains.
If the context has an MPI communicator, return is equivalent to ``MPI_Comm_size``.
If the communicator has no MPI, returns 1.
.. attribute:: rank
The numeric id of the local domain.
If the context has an MPI communicator, return is equivalent to ``MPI_Comm_rank``.
If the communicator has no MPI, returns 0.
Here are some simple examples of how to create a :class:`context`:
.. container:: example-code
.. code-block:: python
import arbor
import mpi4py.MPI as mpi
# Construct a context that uses 1 thread and no GPU or MPI.
context = arbor.context()
# Construct a context that:
# * uses 8 threads in its thread pool;
# * does not use a GPU, reguardless of whether one is available
# * does not use MPI.
alloc = arbor.proc_allocation(8, None)
context = arbor.context(alloc)
# Construct a context that uses:
# * 4 threads and the first GPU;
# * MPI_COMM_WORLD for distributed computation.
alloc = arbor.proc_allocation(4, 0)
comm = arbor.mpi_comm(mpi.COMM_WORLD)
context = arbor.context(alloc, comm)
.. _pyoverview:
Overview
=========
The Python frontend for Arbor is interface that the vast majority of users will use to interact with Arbor.
This section covers how to use the frontend with examples and detailed descriptions of features.
.. _prerequisites:
Prerequisites
~~~~~~~~~~~~~
Once Arbor has been built and installed (see the :ref:`installation guide <installarbor>`),
the location of the installed module needs to be set in the ``PYTHONPATH`` environment variable.
For example:
.. code-block:: bash
export PYTHONPATH="/usr/lib/python3.7/site-packages/:$PYTHONPATH"
With this setup, Arbor's python module :py:mod:`arbor` can be imported with python3 via
>>> import arbor
.. _simsteps:
Simulation steps
~~~~~~~~~~~~~~~~
The workflow for defining and running a model defined in :ref:`modelsimulation` can be performed
in Python as follows:
1. Describe the neuron model by defining an :class:`arbor.recipe`;
2. Describe the computational resources to use for simulation using :class:`arbor.proc_allocation` and :class:`arbor.context`;
3. Partition the model over the hardware resources using :class:`arbor.partition_load_balance`;
4. Run the model by initiating then running the :class:`arbor.simulation`.
These details are described and examples are given in the next sections :ref:`pycommon`, :ref:`pyrecipe`, :ref:`pydomdec`, :ref:`pysimulation`, and :ref:`pyprofiler`.
.. note::
Detailed information on Arbor's python features can also be obtained with Python's ``help`` function, e.g.
.. code-block:: python3
>>> help(arbor.proc_allocation)
Help on class proc_allocation in module arbor:
class proc_allocation(pybind11_builtins.pybind11_object)
| Enumerates the computational resources on a node to be used for simulation.
|...
.. _pyprofiler:
.. currentmodule:: arbor
Metering
========
Arbor's python module :py:mod:`arbor` has a :class:`meter_manager` for measuring time (and if applicable memory) consumptions of regions of interest in the python code.
Users manually instrument the regions to measure.
This allows the user to only measure the parts of the python code that are of interest.
Once a region of code is marked for the :class:`meter_manager`, the application will track the total time (and memory) spent in this region.
Marking Metering Regions
------------------------
First the :class:`meter_manager` needs to be initiated, then the metering started and checkpoints set,
wherever the :class:`meter_manager` should report the meters.
The measurement starts from the :func:`meter_manager.start` to the first :func:`meter_manager.checkpoint` and then in between checkpoints.
Checkpoints are defined by a string describing the process to be measured.
.. class:: meter_manager
.. function:: meter_manager()
Construct the meter manager.
.. function:: start(context)
Start the metering using the chosen execution :class:`arbor.context`.
Records a time stamp, that marks the start of the first checkpoint timing region.
.. function:: checkpoint(name, context)
Create a new checkpoint ``name`` using the the chosen execution :class:`arbor.context`.
Records the time since the last checkpoint (or the call to start if no previous checkpoints),
and restarts the timer for the next checkpoint.
.. function:: checkpoint_names
Returns a list of all metering checkpoint names.
.. function:: times
Returns a list of all metering times.
At any point a summary of the timing regions can be obtained by the :func:`make_meter_report`.
.. function:: make_meter_report(meter_manager, context)
Generate a meter report based on the :class:`meter_manager` and chosen execution :class:`arbor.context`.
For instance, the following python code will record and summarize the total time (and memory) spent:
.. container:: example-code
.. code-block:: python
import arbor
context = arbor.context(threads=8, gpu_id=None)
meter_manager = arbor.meter_manager()
meter_manager.start(context)
n_cells = 100
recipe = my_recipe(n_cells)
meter_manager.checkpoint('recipe create', context)
decomp = arbor.partition_load_balance(recipe, context)
meter_manager.checkpoint('load balance', context)
sim = arbor.simulation(recipe, decomp, context)
meter_manager.checkpoint('simulation init', context)
tSim = 2000
dt = 0.025
sim.run(tSim, dt)
meter_manager.checkpoint('simulation run', context)
Metering Output
------------------
Calling :func:`make_meter_report` will generate a measurement summary, which can be printed using ``print``.
Take the example output from above:
.. container:: example-code
.. code-block:: python
print(arbor.make_meter_report(meter_manager, context))
>>> <arbor.meter_report>:
>>> ---- meters -------------------------------------------------------------------------------
>>> meter time(s) memory(MB)
>>> -------------------------------------------------------------------------------------------
>>> recipe create 0.000 0.001
>>> load balance 0.000 0.009
>>> simulation init 0.026 3.604
>>> simulation run 4.171 0.021
>>> meter-total 4.198 3.634
.. _pyrecipe:
Recipes
=================
.. currentmodule:: arbor
The :class:`recipe` class documentation is below.
A recipe describes neuron models in a cell-oriented manner and supplies methods to provide cell information.
Details on why Arbor uses recipes and general best practices can be found in :ref:`modelrecipe`.
.. class:: recipe
Describe a model by describing the cells and network, without any information about how the model is to be represented or executed.
All recipes derive from this abstract base class.
Recipes provide a cell-centric interface for describing a model.
This means that model properties, such as connections, are queried using the global identifier (:attr:`arbor.cell_member.gid`) of a cell.
In the description below, the term ``gid`` is used as shorthand for the cell with global identifier.
**Required Member Functions**
The following member functions (besides a constructor) must be implemented by every recipe:
.. function:: num_cells()
The number of cells in the model.
.. function:: cell_kind(gid)
The cell kind of the cell with global identifier :attr:`arbor.cell_member.gid` (return type: :class:`arbor.cell_kind`).
.. function:: cell_description(gid)
A high level decription of the cell with global identifier :attr:`arbor.cell_member.gid`,
for example the morphology, synapses and ion channels required to build a multi-compartment neuron.
The type used to describe a cell depends on the kind of the cell.
The interface for querying the kind and description of a cell are seperate
to allow the cell type to be provided without building a full cell description,
which can be very expensive.
**Optional Member Functions**
.. function:: connections_on(gid)
Returns a list of all the **incoming** connections to :attr:`arbor.cell_member.gid`.
Each connection should have post-synaptic target ``connection.dest.gid``
that matches the argument :attr:`arbor.cell_member.gid`,
and a valid synapse id ``connection.dest.index`` on :attr:`arbor.cell_member.gid`.
See :class:`connection`.
By default returns an empty list.
.. function:: gap_junctions_on(gid)
Returns a list of all the gap junctions connected to :attr:`arbor.cell_member.gid`.
Each gap junction ``gj`` should have one of the two gap junction sites ``gj.local.gid``
or ``gj.peer.gid`` matching the argument :attr:`arbor.cell_member.gid`,
and the corresponding synapse id ``gj.local.index`` or ``gj.peer.index`` should be valid on :attr:`arbor.cell_member.gid`.
See :class:`gap_junction_connection`.
By default returns an empty list.
.. function:: event_generators(gid)
A list of all the :class:`event_generator` s that are attached to :attr:`arbor.cell_member.gid`.
By default returns an empty list.
.. function:: num_sources(gid)
The number of spike sources on :attr:`arbor.cell_member.gid`.
By default returns 0.
.. function:: num_targets(gid)
The number of post-synaptic sites on :attr:`arbor.cell_member.gid`, which corresponds to the number of synapses.
By default returns 0.
.. function:: num_gap_junction_sites(gid)
Returns the number of gap junction sites on :attr:`arbor.cell_member.gid`.
By default returns 0.
.. class:: connection
Describes a connection between two cells:
Defined by source and destination end points (that is pre-synaptic and post-synaptic respectively),
a connection weight and a delay time.
.. function:: connection(source, destination, weight, delay)
Construct a connection between the :attr:`source` and the :attr:`dest` with a :attr:`weight` and :attr:`delay`.
.. attribute:: source
The source end point of the connection (type: :class:`arbor.cell_member`).
.. attribute:: dest
The destination end point of the connection (type: :class:`arbor.cell_member`).
.. attribute:: weight
The weight delivered to the target synapse.
The weight is dimensionless, and its interpretation is specific to the type of the synapse target.
For example, the expsyn synapse interprets it as a conductance with units μS (micro-Siemens).
.. attribute:: delay
The delay time of the connection [ms]. Must be positive.
An example of a connection reads as follows:
.. container:: example-code
.. code-block:: python
import arbor
# construct a connection between cells (0,0) and (1,0) with weight 0.01 and delay of 10 ms.
src = arbor.cell_member(0,0)
dest = arbor.cell_member(1,0)
w = 0.01
d = 10
con = arbor.connection(src, dest, w, d)
.. class:: gap_junction_connection
Describes a gap junction between two gap junction sites.
Gap junction sites are represented by :class:`arbor.cell_member`.
.. function::gap_junction_connection(local, peer, ggap)
Construct a gap junction connection between :attr:`local` and :attr:`peer` with conductance :attr:`ggap`.
.. attribute:: local
The gap junction site: one half of the gap junction connection.
.. attribute:: peer
The gap junction site: other half of the gap junction connection.
.. attribute:: ggap
The gap junction conductance [μS].
Event Generator and Schedules
-----------------------------
.. class:: event_generator
.. function:: event_generator(target, weight, schedule)
Construct an event generator for a :attr:`target` synapse with :attr:`weight` of the events to deliver based on a schedule (i.e., :class:`arbor.regular_schedule`, :class:`arbor.explicit_schedule`, :class:`arbor.poisson_schedule`).
.. attribute:: target
The target synapse of type :class:`arbor.cell_member`.
.. attribute:: weight
The weight of events to deliver.
.. class:: regular_schedule
Describes a regular schedule with multiples of :attr:`dt` within the interval [:attr:`tstart`, :attr:`tstop`).
.. function:: regular_schedule(tstart, dt, tstop)
Construct a regular schedule as list of times from :attr:`tstart` to :attr:`tstop` in :attr:`dt` time steps.
By default returns a schedule with :attr:`tstart` = :attr:`tstop` = ``None`` and :attr:`dt` = 0 ms.
.. attribute:: tstart
The delivery time of the first event in the sequence [ms].
Must be non-negative or ``None``.
.. attribute:: dt
The interval between time points [ms].
Must be non-negative.
.. attribute:: tstop
No events delivered after this time [ms].
Must be non-negative or ``None``.
.. class:: explicit_schedule
Describes an explicit schedule at a predetermined (sorted) sequence of :attr:`times`.
.. function:: explicit_schedule(times)
Construct an explicit schedule.
By default returns a schedule with an empty list of times.
.. attribute:: times
The list of non-negative times [ms].
.. class:: poisson_schedule
Describes a schedule according to a Poisson process.
.. function:: poisson_schedule(tstart, freq, seed)
Construct a Poisson schedule.
By default returns a schedule with events starting from :attr:`tstart` = 0 ms,
with an expected frequency :attr:`freq` = 10 Hz and :attr:`seed` = 0.
.. attribute:: tstart
The delivery time of the first event in the sequence [ms].
.. attribute:: freq
The expected frequency [Hz].
.. attribute:: seed
The seed for the random number generator.
An example of an event generator reads as follows:
.. container:: example-code
.. code-block:: python
import arbor
# define a Poisson schedule with start time 1 ms, expected frequency of 5 Hz,
# and the target cell's gid as seed
target = arbor.cell_member(0,0)
seed = target.gid
tstart = 1
freq = 5
sched = arbor.poisson_schedule(tstart, freq, seed)
# construct an event generator with this schedule on target cell and weight 0.1
w = 0.1
gen = arbor.event_generator(target, w, sched)
Cells
------
.. class:: cable_cell
See :ref:`pycable_cell`.
.. class:: lif_cell
A benchmarking cell (leaky integrate-and-fire), used by Arbor developers to test communication performance,
with neuronal parameters:
.. attribute:: tau_m
Membrane potential decaying constant [ms].
.. attribute:: V_th
Firing threshold [mV].
.. attribute:: C_m
Membrane capacitance [pF].
.. attribute:: E_L
Resting potential [mV].
.. attribute:: V_m
Initial value of the Membrane potential [mV].
.. attribute:: t_ref
Refractory period [ms].
.. attribute:: V_reset
Reset potential [mV].
.. class:: spike_source_cell
A spike source cell, that generates a user-defined sequence of spikes
that act as inputs for other cells in the network.
.. function:: spike_source_cell(schedule)
Construct a spike source cell that generates spikes
- at regular intervals (using an :class:`arbor.regular_schedule`)
- at a sequence of user-defined times (using an :class:`arbor.explicit_schedule`)
- at times defined by a Poisson sequence (using an :class:`arbor.poisson_schedule`)
:param schedule: User-defined sequence of time points (choose from :class:`arbor.regular_schedule`, :class:`arbor.explicit_schedule`, or :class:`arbor.poisson_schedule`).
.. class:: benchmark_cell
A benchmarking cell, used by Arbor developers to test communication performance.
.. function:: benchmark_cell(schedule, realtime_ratio)
A benchmark cell generates spikes at a user-defined sequence of time points:
- at regular intervals (using an :class:`arbor.regular_schedule`)
- at a sequence of user-defined times (using an :class:`arbor.explicit_schedule`)
- at times defined by a Poisson sequence (using an :class:`arbor.poisson_schedule`)
and the time taken to integrate a cell can be tuned by setting the parameter ``realtime_ratio``.
:param schedule: User-defined sequence of time points (choose from :class:`arbor.regular_schedule`, :class:`arbor.explicit_schedule`, or :class:`arbor.poisson_schedule`).
:param realtime_ratio: Time taken to integrate a cell, for example if ``realtime_ratio`` = 2, a cell will take 2 seconds of CPU time to simulate 1 second.
Below is an example of a recipe construction of a ring network of multi-compartmental cells.
Because the interface for specifying cable morphology cells is under construction, the temporary
helpers in cell_parameters and make_cable_cell for building cells are used.
.. container:: example-code
.. code-block:: python
import sys
import arbor
class ring_recipe (arbor.recipe):
def __init__(self, n=4):
# The base C++ class constructor must be called first, to ensure that
# all memory in the C++ class is initialized correctly.
arbor.recipe.__init__(self)
self.ncells = n
self.params = arbor.cell_parameters()
# The num_cells method that returns the total number of cells in the model
# must be implemented.
def num_cells(self):
return self.ncells
# The cell_description method returns a cell
def cell_description(self, gid):
return arbor.make_cable_cell(gid, self.params)
def num_targets(self, gid):
return 1
def num_sources(self, gid):
return 1
# The kind method returns the type of cell with gid.
# Note: this must agree with the type returned by cell_description.
def cell_kind(self, gid):
return arbor.cell_kind.cable
# Make a ring network
def connections_on(self, gid):
src = (gid-1)%self.ncells
w = 0.01
d = 10
return [arbor.connection(arbor.cell_member(src,0), arbor.cell_member(gid,0), w, d)]
# Attach a generator to the first cell in the ring.
def event_generators(self, gid):
if gid==0:
sched = arbor.explicit_schedule([1])
return [arbor.event_generator(arbor.cell_member(0,0), 0.1, sched)]
return []
.. _pysimulation:
Simulations
===========
From recipe to simulation
-------------------------
To build a simulation the following concepts are needed:
* an :class:`arbor.recipe` that describes the cells and connections in the model;
* an :class:`arbor.context` used to execute the simulation.
The workflow to build a simulation is to first generate an
:class:`arbor.domain_decomposition` based on the :class:`arbor.recipe` and :class:`arbor.context` describing the distribution of the model
over the local and distributed hardware resources (see :ref:`pydomdec`). Then, the simulation is build using this :class:`arbor.domain_decomposition`.
.. container:: example-code
.. code-block:: python
import arbor
# Get a communication context (with 4 threads, no GPU)
context = arbor.context(threads=4, gpu_id=None)
# Initialise a recipe of user defined type my_recipe with 100 cells.
n_cells = 100
recipe = my_recipe(n_cells)
# Get a description of the partition of the model over the cores.
decomp = arbor.partition_load_balance(recipe, context)
# Instatitate the simulation.
sim = arbor.simulation(recipe, decomp, context)
# Run the simulation for 2000 ms with time stepping of 0.025 ms
tSim = 2000
dt = 0.025
sim.run(tSim, dt)
.. currentmodule:: arbor
.. class:: simulation
The executable form of a model.
A simulation is constructed from a recipe, and then used to update and monitor the model state.
Simulations take the following inputs:
The **constructor** takes
* an :class:`arbor.recipe` that describes the model;
* an :class:`arbor.domain_decomposition` that describes how the cells in the model are assigned to hardware resources;
* an :class:`arbor.context` which is used to execute the simulation.
Simulations provide an interface for executing and interacting with the model:
* **Advance the model state** from one time to another and reset the model state to its original state before simulation was started.
* Sample the simulation state during the execution (e.g. compartment voltage and current) and generate spike output by using an **I/O interface**.
**Constructor:**
.. function:: simulation(recipe, domain_decomposition, context)
Initialize the model described by an :class:`arbor.recipe`, with cells and network distributed according to :class:`arbor.domain_decomposition`, and computational resources described by :class:`arbor.context`.
**Updating Model State:**
.. function:: reset()
Reset the state of the simulation to its initial state.
.. function:: run(tfinal, dt)
Run the simulation from current simulation time to ``tfinal``,
with maximum time step size ``dt``.
:param tfinal: The final simulation time [ms].
:param dt: The time step size [ms].
.. function:: set_binning_policy(policy, bin_interval)
Set the binning ``policy`` for event delivery, and the binning time interval ``bin_interval`` if applicable [ms].
:param policy: The binning policy of type :class:`binning`.
:param bin_interval: The binning time interval [ms].
**Types:**
.. class:: binning
Enumeration for event time binning policy.
.. attribute:: none
No binning policy.
.. attribute:: regular
Round time down to multiple of binning interval.
.. attribute:: following
Round times down to previous event if within binning interval.
Recording spikes
----------------
In order to analyze the simulation output spikes can be recorded.
**Types**:
.. class:: spike
.. function:: spike()
Construct a spike.
.. attribute:: source
The spike source (type: :class:`arbor.cell_member`).
.. attribute:: time
The spike time [ms].
.. class:: spike_recorder
.. function:: spike_recorder()
Initialize the spike recorder.
.. attribute:: spikes
The recorded spikes (type: :class:`spike`).
**I/O interface**:
.. function:: attach_spike_recorder(sim)
Attach a spike recorder to an arbor :class:`simulation` ``sim``.
The recorder that is returned will record all spikes generated after it has been
attached (spikes generated before attaching are not recorded).
.. container:: example-code
.. code-block:: python
import arbor
# Instatitate the simulation.
sim = arbor.simulation(recipe, decomp, context)
# Build the spike recorder
recorder = arbor.attach_spike_recorder(sim)
# Run the simulation for 2000 ms with time stepping of 0.025 ms
tSim = 2000
dt = 0.025
sim.run(tSim, dt)
# Print the spikes and according spike time
for s in recorder.spikes:
print(s)
>>> <arbor.spike: source (0,0), time 2.15168 ms>
>>> <arbor.spike: source (1,0), time 14.5235 ms>
>>> <arbor.spike: source (2,0), time 26.9051 ms>
>>> <arbor.spike: source (3,0), time 39.4083 ms>
>>> <arbor.spike: source (4,0), time 51.9081 ms>
>>> <arbor.spike: source (5,0), time 64.2902 ms>
>>> <arbor.spike: source (6,0), time 76.7706 ms>
>>> <arbor.spike: source (7,0), time 89.1529 ms>
>>> <arbor.spike: source (8,0), time 101.641 ms>
>>> <arbor.spike: source (9,0), time 114.125 ms>
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment