Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

* Use `Unitful.uconvert` for scalar conversions (Float64 cast apparently does not work at the moment).
* Use .+ for scalar/array addition.
* Replace `immutable` with `struct`.
* Qualify included modules with `Main.` for using statements.
* Add informational note to FindJulia as component identification can take a long time as Julia may compile them from source.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake and build refactoring

*   Use CUDA as first-class language (leading to CMake 3.9 minimum version requirement).

*   Use 'modern CMake' interface libraries for compiler options, include file and library dependency tracking. Interface library targets:
    * `arbor-deps`: compiler options and library requirements for the `libarbor.a` static library, as governed by configure-time options and environment.
    * `arbor-private-headers`: include path for non-installed headers, as required by unit tests and arbor itself.
    * `arbor-aux`: helper classes and utilities used across tests and examples.
    * `ext-json`, `ext-tclap`, `ext-tbb`, `ext-benchmark`, `ext-sphinx_rtd_theme`: externally maintained software that we include (directly or via submodule) in the `ext/` subdirectory.
 
*   Single static library `libarbor.a` includes all built-in modules and CUDA objects.

*   Simply configuration options:
    *  `ARB_WITH_TRACE`, `ARB_AUTORUN_MODCC_ON_CHANGES` `ARB_SYSTEM_TYPE` removed.
    * External `modcc` is provided by `ARB_MODCC` configuration option; if provided `modcc` is still buildable, but is not included in the default target.
    * `ARB_PRIVATE_TBBLIB`, defaulting to `OFF`, instructs the build to make TBB from the included submodule.

*   Extend `ErrorTarget` functionality to provide a dummy target or an error target based on a condition.
*   Generate header version defines and library version variables based on git status and project version, via new script `include/git-source-id`.
*   All generated binaries now placed in `bin/` subdirectory at build.
*   Install targets installs: public headers (incomplete); static library; `modcc` tool; `lmorpho` executable; `html` documentation (examples, tests and validation data are currently not installed).
*   Executable targets have had the `.exe` suffix removed; unit tests are labelled `unit` (arbor unit tests), `unit-modcc` (modcc unit tests), `unit-local` (distributed tests with local context), `unit-mpi` (distributed tests with MPI context).
*   More graceful handling of configure-time detection of `nrniv`, Julia and required Julia modules for validation data generation.
*   Add `cmake/FindJulia.cmake`, `cmake/FindTBB.cmake`  package finders, and adjust `cmake/FindUnwind.cmake` to use link library-style properties.
*  Adjust travis script to test `unit-local` and `unit-mpi` if appropriate.
*  Simply documentation `conf.py`.

Source relocation and reorganization

* All external project sources and files moved to `ext/`.
* Source code refactoring to decouple library-using code from the configure-time definitions that govern arbor behaviour: removes conditional code in public headers that depends upon `ARB_WITH_X`-type definitions at compile time. Affected code is is in the public interfaces for MPI, the threading implementation, and the profiler.
* Remove `util/debug.hpp`; split out functionality for pretty-printing from assertion handling.
* Make FVM cell non-physical voltage check a run-time cell-group parameter.
* Move spike double buffer implementation to `simulation.cpp`.
* Make timer utility wrap POSIX `clock_gettime` independent of threading configuration.
* Make `mpi_error` derive from `system_error` and follow C++11 `system_error` semantics.
* `EXPECTS` macro replaced by `arb_assert` macro.
* JSON dependency removed from `libarbor.a` and header files: moved to auxiliary library.
* Publicly visible macros garner an `ARB_` prefix as required.
* Move SWC test file to `test/unit` directory.
* Work-in-progress splitting of public from private includes: as a convention not entirely adhered to as yet, private headers within arbor source are included with `""`, public headers with `<>`.

Modcc interface changes

* Expose via `--namespace` option the functionality that sets the namespace in generated code.
* Use `--profile` option to add profiler hooks to generated code; uses public function interface directly rather than `PE/PL` macros in order to avoid public `PE` and `PL` defines.

Ben Cumming authored 7 years ago and Sam Yates committed 7 years ago

Replace constant values for ion species reversal potentials taken from HH model with values that are updated on each time step via the Nernst equation.

* Implement Nernst equation in multicore and gpu back-ends.
* Extend interface of `ion` type to proved a method `update_reversal_potential` which calls the back-end nernst routine.
* Add `valency` and default concentration values to the ion species.
* Add interface for resetting ion species state (for restarting simulations).

Fixes #376 

* Use ARB_ and arb_ as variable prefixes in place of NMC_ and nmc_.
* Replace references to 'NestMC' and 'NEST MC' to refer instead to Arbor.
* Use 'arbor' as the sim name in generated validation data.
* Reflow long-line paragraphs in `tests/ubench/README.md`.
* Change names of CUDA mechanism and CUDA kernel libraries to include arbor name.

Incorporate symbolic GE code from prototype (with some simplifications) in msparse.hpp, symge.hpp and symge.cpp, together with unit tests.

Add two kinetic scheme test cases for validation: test_kin1 (simple exponential scheme) and test_kinlva (combination of exponential gate and a three-species kinetic scheme, modelling a low voltage-activated Calcium channel from Wang, X. J. et al., J. Neurophys. 1991).

Adapt numeric HH validation data generation to LVA Ca channel, with explicit stopping at stimulus discontinuities.

Add two new validation tests based on above: kinetic.kin1_numeric_ref and kinetic.kinlva_numeric_ref (multicore backend only).

Introduce a BlockRewriterBase visitor base class, as an aid for visitors that transform/rewrite procedure bodies; refactor KineticRewriter over this class.

Introduce common error_stack mixin class for common functionality across Module and the various procedure rewriters.

Implement visitors and public-facing convenience wrappers in symdiff.hpp and symdiff.cpp:

involves_identifer for testing if an expression contains given identifiers.
constant_simplify for constant folding with removal of trivial terms arising from a NumberExpression of zero or one.
expr_value to extract the numerical value of a NumberExpression, or NaN othereise.
is_zero to test if an expression is numerically zero.
symbolic_pdiff to perform symbolic partial differentiation; this adds a new (not parseable) expression subclass to represent opaque partial differential terms.
substitute to substitute identifiers for other expressions within an expression.
linear_test for linearity, diagonality and homogeneity testing (this is probably redundant, given ExpressionClassifier already exists).
Simplify unnecessary uses of make_unique with Vistor subclasses.

Make SOLVE statement rewriting more generic, through the use of solve-rewriter visitors CnexpSolverVisitor, SparseSolverVisitor, and DirectSolverVisitor; implementations in solvers.hpp and solvers.cpp. Supports multiple SOLVE statements for independent subsets of state variables with the BREAKPOINT block.

Add block rewriter for the removal of unused local variables, with convenience wrapper remove_unused_locals.

Generalize is_in utility in modccutil.hpp.

Simplify expression comparison in modcc unit tests with EXPECT_EXPR_EQ macro added to tests/modcc/test.hpp, that operates by comparing expression text representations.

Simplify and consolidate verbose printing in modcc unit tests with verbose_print function that tests the global verbose flag and handles expression_ptr and similar which have to_string methods.

* Fix CMakeLists to handle build as a subproject

When several CMake generated projects are build together, it is common
practice to have a 'superproject' CMakeLists that uses
  add_subdir(proj1)
  add_subdir(proj2)
  ...
where each subproject is a self contained CMake based project
(Example proj1=HPX, proj2=nestmc, proj3=another, ...)

CMAKE_SOURCE_DIR always points to the top level directory which
is the superproject dir in this case, whereas PROJECT_SOURCE_DIR
always points to the root of the current project() in the CMakeLists
so one shouod use PROJECT_SOURCE_DIR as this gets the relative paths
correct.

* Add option to turn off auto generation from *.mod files

* Fix #134 : Change CMake WITH_OPTION to NMC_WITH_OPTION, compiler #define to NMC_HAVE_OPTION

1) The user may select an option by saying NMC_WITH_XXX

2) This may trigger CMake to use find_package(...) or setup some
other variables. CMake can then set variable NMC_HAVE_XXX and add a
what has actually been used.

3) Code should use #ifdef NMC_HAVE_XXX to check for a feature

Old CMake/define      New CMake                 Compiler #define
----------------      ---------                 ----------------
THREADING_MODEL       NMC_THREAD_MODEL
    WITH_TBB          NMC_WITH_TBB              NMC_HAVE_TBB
    WITH_OMP          NMC_WITH_OMP              NMC_HAVE_OMP
    WITH_SERIAL       NMC_WITH_SERiAL           NMC_HAVE_SERIAL

WITH_MPI              NMC_WITH_MPI              NMC_HAVE_MPI
WITH_CUDA             NMC_WITH_CUDA             NMC_HAVE_CUDA
WITH_GPU                                        NMC_HAVE_GPU
WITH_ASSERTIONS       NMC_WITH_ASSERTIONS       NMC_HAVE_ASSERTIONS
WITH_TRACE            NMC_WITH_TRACE            NMC_HAVE_TRACE
WITH_PROFILING        NMC_WITH_PROFILING        NMC_HAVE_PROFILING

Other user visible CMake vars
-----------------------------
VECTORIZE_TARGET            -> NMC_VECTORIZE_TARGET
USE_OPTIIZED_KERNELS        -> NMC_USE_OPTIIZED_KERNELS
BUILD_VALIDATION_DATA       -> NMC_BUILD_VALIDATION_DATA
BUILD_JULIA_VALIDATION_DATA -> NMC_BUILD_JULIA_VALIDATION_DATA
BUILD_NRN_VALIDATION_DATA   -> NMC_BUILD_NRN_VALIDATION_DATA
VALIDATION_DATA_DIR         -> NMC_VALIDATION_DATA_DIR

Variables such as NMC_THREADING_MODEL and NMC_VECTORIZE_TARGET now use
enumerated cmake values so you can toggle between them in ccmake gui.
SYSTEM_TYPE_CRAY/BGQ        -> NMC_SYSTEM_TYPE (Generic/Cray/BGQ)

* Use generator expression for modcc path

Some IDE's (like Xcode for example), override the CMake binary paths
and add /Debug or /Release etc so rules that have hard coded paths
to binaries will fail.

* Use divided compartments to determine FVM coefficients.
* Pick correct control volume in FVM from sgement position (avoids
  off-by-half error.)
* Add colour override functionality to tsplot: `--colour` option.
* Add const accessor for cell soma.
* Source formatting, comments in `math.hpp`
* Fix `range_view`: was using incorrectly named type trait.
* Add unit test for `range_view`.
* Allow points of discontinuity to be omitted from L-infinity norm
  calculations.
* Add `-d, --min-dt` option to `validate.exe` to control time
  step in validation convergence tests.
* Add validation test: confirm divided compartment policy does
  not effect results on simple frustrum dendrites.
* Change default max compartments on validation tests to 100
  (ad hoc observed convergence limit at dt circa 0.001 ms;
  finder spatial division would required much finer dt.)
* Make NEURON validation data generation scripts use CVODE by
  default, and with `secondorder=2` when non-zero `dt` is given.

* Make `test_common_cells.hpp` and `ball_and_taper.py` agree.
* Add `ball_and_squiggle` model that has a tapering undulating
  profile.

Julia 0.5 deprecates use of `symbol` instead of
`Symbol`. This patch just substitutes the
correct call.

Julia 0.5 deprecates use of `symbol` instead of
`Symbol`. This patch just substitutes the
correct call.

* Implement Rallpack1 validation test (with a workaround
  for inability to set membrane conductance).
* Fix bug in L≠1 case in PassiveCable.jl (this may still be
  wrong).
* Fix bug in peak delta computation in trace analysis when
  both traces have no local maxima.
* Gentle failure on missing `numeric_soma.json`
* Allow multiple `-s` selection operations for `tsplot`,
  acting disjunctively.

* Amend data directory path in validation tests.
* Enmodulate `hh_soma.jl`
* Add HH channel reference data generations script.
* Switch `validate_soma.cpp` to numeric reference data.
* Consolidate common code in `validate_ball_and_stick.cpp`
* Add (nearly) Rallpack1 validation test (see below).
* Gentle failure on absence of reference data in
  `validate_ball_and_stick.cpp`

Can't yet override mechanism default parameter values,
so the cable cell model added to `test_common_cells.hpp`
lets the default stand; validation script will have
to use the default membrane conductance rather than that
given by Rallpack1.

* Use consistent scaling for y[1] scalar voltage in hh_soma.jl
* Also: add more reserved target names to CMakeLists.txt
  helper function.

* Move generation and data to top-level validation directory.
* Make BUILD_VALIDATION_DATA and VALIDATION_DATA_DIR cache vars.
* Add helper CMake functions for data generation.

Note `validation/ref/numeric/foo.sh` is just a placeholder.