required for making an emscripten build of the arbor python library

* make ARB_MODCC functional again

* move add_subdir

* no modcc/CMakeLists.txt is needed also when external modcc is used.

* review

Inline NMODL `PROCEDURE`s into their callers.

This is prep for meson in addition to simplifying (ie reducing) the amount of scripting
we have. In the long run, I'd like to remove `BuildModules.cmake` too, but this might
be superseded by meson and this PR is self-contained.

# Changes
- `modcc` accepts now a list of NMODL files
- likewise, `modcc` is now able to spit out a catalogue.cpp file
- remove the `generate-catalogue` script, simplify a-b-c
- remove some options from `modcc` we never use
- No longer allow external modcc

- Add stacktraces to
   1. arbor_exception (and all derived) as `where`
   2. arbor_internal_error as `where`
   3. arb_assert
- Propagate stacktraces to Python exceptions derived from the above.
- Expand dev/doc on debugging

Exports all symbols so arbor can be compiled as set of shared libraries. 

In order to export all symbols correctly, one macro per library and one global macro are added. The content of the macros is determined at configure time depending on build variant (static/shared), compiler, and platforms (linux, mac os) and goes into the library's include directory as `export.hpp` when installed (at build time it resides at cmake's temporary build directory). The per-library macro is named `ARB_LIBNAME_API` and goes in front of to-be-exported symbols. The global macro is `ARB_SYMBOL_VISIBLE`. 

This PR adds the annotation in all of the places where it is required. Most of them are in the public headers (and corresponding sources) but some are also added in internal headers, which were required for the unit tests to link properly.

Fixes #1752

- make fmt a dependency of libmodcc, not modcc
- set FMT_HEADER_ONLY as a compiler flag to avoid double definition

Implements #1376.

* Provide a common C linkage ABI for externally compiled mechanisms, for both CPU and GPU.
* Remove mechanism type hierarchy (`concrete_mechanism` etc.), and move corresponding functionality to the back-end shared state objects. Mechanism catalogue is no longer indexed by type id.
* Distinguish between SIMD optimal alignment and SIMD width with new `min_align` attribute. Mechanisms provide both pieces of information via ABI.

- Use parameter packs for multicore mechanisms.
- Adjust unit tests and modcc.
- CPU and GPU use the same mechanism_ppack_base.
- CPU code uses a split similar to GPU with `mech_XYZ_foo` and `kernel::mech_XYZ_foo.
- Make mechanism_ppack_base header public.
- Tracing macros for modcc code gen.

- Add option to compile Arbor for HIP/CUDA backend using Clang. 
- Add new CMake option `ARB_GPU_COMPILE_TYPE` to distinguish between three possible gpu platforms/builds: `cuda-nvcc`; `cuda-clang`; `hip-clang`
- Add gpu wrapper functions in `arbor/backends/gpu/gpu_api.hpp` and `arborenv/gpu_api.hpp` which call HIP/CUDA functions depending on the platform
- Rename functions and files: cuda -> gpu
- Add downgraded warp primitives for HIP
- Implement `uuid` workaround for HIP
- Set correct alignment and warp size for AMD gpus
- Update installation guide and docs.
- Update pip/setuptools to support new gpu targets

Fixes #833

* Replace TCLAP with tinyopt, remove TCLAP from `ext/`.
* Replace `sup/tinyopt.hpp` with copy of external tinyopt, update use of `tinyopt` in unit tests and examples to suit.
* Use tinyopt's 'smallopt' interface in `modcc`, `brunel` example.
* Update tinyopt interface in `single` example.
 Fix our attributions and add HBP acknowledgement

Addresses #557 #695 #803 

Nora Abi Akar authored 5 years ago and Benjamin Cumming committed 5 years ago

Add support for parsing and processing `LINEAR` blocks: 

Changes: 
* `SOLVE` expressions can be called from inside an `INITIAL` block, but only if they are solving a linear system
* Tilde expressions can now be either linear expressions or reaction expressions
* Linear expressions need to be rewritten before being sent to the solver, this is done using `LinearRewriter`
* The linear system is setup in `LinearSolverVisitor` fills the lhs and rhs of the symbolic matrix   
* The matrix is recued using `gj_reduce`, which now works on non-diagonal matrices. 

Fixes #839 

Fixes #139.

* Split colours and `pprintf(...)` into `io/pprintf.hpp` header.
* Remove generic `to_string()` function, replacing its very occasional usage with `pprintf`.
* Move block pretty printing into own .cpp file; this is the only place that the vector ostream printer was used.
* Remove `enum_hash`, as not needed with C++14.
* Move `is_in` utility function to `util.hpp`.
* Remove old SIMD printer backend code.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

Fixes #526.

Sam Yates authored 6 years ago and Benjamin Cumming committed 6 years ago

CMake and build refactoring

*   Use CUDA as first-class language (leading to CMake 3.9 minimum version requirement).

*   Use 'modern CMake' interface libraries for compiler options, include file and library dependency tracking. Interface library targets:
    * `arbor-deps`: compiler options and library requirements for the `libarbor.a` static library, as governed by configure-time options and environment.
    * `arbor-private-headers`: include path for non-installed headers, as required by unit tests and arbor itself.
    * `arbor-aux`: helper classes and utilities used across tests and examples.
    * `ext-json`, `ext-tclap`, `ext-tbb`, `ext-benchmark`, `ext-sphinx_rtd_theme`: externally maintained software that we include (directly or via submodule) in the `ext/` subdirectory.
 
*   Single static library `libarbor.a` includes all built-in modules and CUDA objects.

*   Simply configuration options:
    *  `ARB_WITH_TRACE`, `ARB_AUTORUN_MODCC_ON_CHANGES` `ARB_SYSTEM_TYPE` removed.
    * External `modcc` is provided by `ARB_MODCC` configuration option; if provided `modcc` is still buildable, but is not included in the default target.
    * `ARB_PRIVATE_TBBLIB`, defaulting to `OFF`, instructs the build to make TBB from the included submodule.

*   Extend `ErrorTarget` functionality to provide a dummy target or an error target based on a condition.
*   Generate header version defines and library version variables based on git status and project version, via new script `include/git-source-id`.
*   All generated binaries now placed in `bin/` subdirectory at build.
*   Install targets installs: public headers (incomplete); static library; `modcc` tool; `lmorpho` executable; `html` documentation (examples, tests and validation data are currently not installed).
*   Executable targets have had the `.exe` suffix removed; unit tests are labelled `unit` (arbor unit tests), `unit-modcc` (modcc unit tests), `unit-local` (distributed tests with local context), `unit-mpi` (distributed tests with MPI context).
*   More graceful handling of configure-time detection of `nrniv`, Julia and required Julia modules for validation data generation.
*   Add `cmake/FindJulia.cmake`, `cmake/FindTBB.cmake`  package finders, and adjust `cmake/FindUnwind.cmake` to use link library-style properties.
*  Adjust travis script to test `unit-local` and `unit-mpi` if appropriate.
*  Simply documentation `conf.py`.

Source relocation and reorganization

* All external project sources and files moved to `ext/`.
* Source code refactoring to decouple library-using code from the configure-time definitions that govern arbor behaviour: removes conditional code in public headers that depends upon `ARB_WITH_X`-type definitions at compile time. Affected code is is in the public interfaces for MPI, the threading implementation, and the profiler.
* Remove `util/debug.hpp`; split out functionality for pretty-printing from assertion handling.
* Make FVM cell non-physical voltage check a run-time cell-group parameter.
* Move spike double buffer implementation to `simulation.cpp`.
* Make timer utility wrap POSIX `clock_gettime` independent of threading configuration.
* Make `mpi_error` derive from `system_error` and follow C++11 `system_error` semantics.
* `EXPECTS` macro replaced by `arb_assert` macro.
* JSON dependency removed from `libarbor.a` and header files: moved to auxiliary library.
* Publicly visible macros garner an `ARB_` prefix as required.
* Move SWC test file to `test/unit` directory.
* Work-in-progress splitting of public from private includes: as a convention not entirely adhered to as yet, private headers within arbor source are included with `""`, public headers with `<>`.

Modcc interface changes

* Expose via `--namespace` option the functionality that sets the namespace in generated code.
* Use `--profile` option to add profiler hooks to generated code; uses public function interface directly rather than `PE/PL` macros in order to avoid public `PE` and `PL` defines.

First commit of two for mechanism refactor work (refer to PR #484 and PR #483).

FVM/mechanism code:
* Refactor mechanism data structures to decouple backend-specific implementations and mechanism metadata.
* Add mechanism catalogue for managing mechanism metadata and concrete implementation prototypes.
* Add fingerprint-checking to mechanism metadata and implementations to confirm they come from the same NMODL source (fingerprint is not yet computed, but tests are in place).
* Split FVM discretization work out from FVM integrator code.
* Use abstract base class over backend-templated FVM integrator class `fvm_lowered_cell_impl` to allow separate compilation of `mc_cell_group` and to remove the dummy backend code.
* Add a new FVM-specific scalar type `fvm_index_type` that is an alias for `int` to replace
`fvm_size_type` in fvm layouts and mechanisms. This was chosen as an alternative
to making `unsigned` versions of all our SIMD implementation classes.
* Exte...

* Remove optimization option (use SIMD options for vectorization).
* Remove arbor utility library dependencies from modcc (pending separation of utility lib from arbor lib source).
* Split target (cpu, gpu) specification from vectorization architecture (avx2, avx512).
* Remove `Options` singleton; replace with structure local to `modcc.cpp`.
* Tidy `modcc` option parsing and main function; allow a single invocation of `modcc` to generate code for multiple backends.
* Rename generated sources to include backend target in filename.
* Always run a constant simplification pass on generated procedures.
* Remove file i/o code from `Module` and `modcc` main function; move functionality to new functions in `io` namespace. (Note: in on-going mechanism revamp, other i/o utility code will reside in the `io` namespace and subdirectory.)
* Remove classes `ConstantFolderVisitor` and `ExpressionClassifierVisitor` that are no longer used.
* Modify CMakeLists.txt files, `backends/*/fvm.cpp` to reflect the new filenames of generated sources.
* Small formatting changes in `modcc` source to reflect coding guidelines (incomplete).

Fixes #399.

Update `test_printers.cpp`:
* Fix compilation and scope management issues, and re-include in CMakeLists.txt.
* Update to use `verbose_print` (run-time verbosity).
* Apply expected output tests to `CPrinter` and `CUDAPrinter` outputs.

Address binary operation rendering issue:
* Move C-style expression rendering common to `CPrinter` and `CUDAPrinter` to a specialized `Visitor`, `CExprRenderer`.
* Use operator associativity to determine need for parentheses around sub-expressions of a binary expression.

Basic features:

* Compile with -t avx512
* Automatically set up by CMake if USE_OPTIMIZED_KERNELS is on and VECTORIZE_TARGET is set to KNL 
* Generic SIMD printer that contacts a SIMD backend for emitting the actual SIMD intrinsics

Note: compilation for the avx512 target requires the Intel compiler. 

Incorporate symbolic GE code from prototype (with some simplifications) in msparse.hpp, symge.hpp and symge.cpp, together with unit tests.

Add two kinetic scheme test cases for validation: test_kin1 (simple exponential scheme) and test_kinlva (combination of exponential gate and a three-species kinetic scheme, modelling a low voltage-activated Calcium channel from Wang, X. J. et al., J. Neurophys. 1991).

Adapt numeric HH validation data generation to LVA Ca channel, with explicit stopping at stimulus discontinuities.

Add two new validation tests based on above: kinetic.kin1_numeric_ref and kinetic.kinlva_numeric_ref (multicore backend only).

Introduce a BlockRewriterBase visitor base class, as an aid for visitors that transform/rewrite procedure bodies; refactor KineticRewriter over this class.

Introduce common error_stack mixin class for common functionality across Module and the various procedure rewriters.

Implement visitors and public-facing convenience wrappers in symdiff.hpp and symdiff.cpp:

involves_identifer for testing if an expression contains given identifiers.
constant_simplify for constant folding with removal of trivial terms arising from a NumberExpression of zero or one.
expr_value to extract the numerical value of a NumberExpression, or NaN othereise.
is_zero to test if an expression is numerically zero.
symbolic_pdiff to perform symbolic partial differentiation; this adds a new (not parseable) expression subclass to represent opaque partial differential terms.
substitute to substitute identifiers for other expressions within an expression.
linear_test for linearity, diagonality and homogeneity testing (this is probably redundant, given ExpressionClassifier already exists).
Simplify unnecessary uses of make_unique with Vistor subclasses.

Make SOLVE statement rewriting more generic, through the use of solve-rewriter visitors CnexpSolverVisitor, SparseSolverVisitor, and DirectSolverVisitor; implementations in solvers.hpp and solvers.cpp. Supports multiple SOLVE statements for independent subsets of state variables with the BREAKPOINT block.

Add block rewriter for the removal of unused local variables, with convenience wrapper remove_unused_locals.

Generalize is_in utility in modccutil.hpp.

Simplify expression comparison in modcc unit tests with EXPECT_EXPR_EQ macro added to tests/modcc/test.hpp, that operates by comparing expression text representations.

Simplify and consolidate verbose printing in modcc unit tests with verbose_print function that tests the global verbose flag and handles expression_ptr and similar which have to_string methods.

Adds a new KineticRewriter visitor that transforms (after semantic analysis) a parsed KINETIC procedure into an equivalent DERIVATIVE procedure. The visitor takes a ProcedureExpression and composes the equivalent procedure, available via the as_procedure() method on the visitor object.

Move common functinality for 'local' variable insertion during transformation phase to new files astmanip.?pp.
Add Expression method for directly setting scope.
Use scope_ptr type alias widely.
Implement correct clone() behaviour for DerivativeExpression
Implement KineticRewriter transforming visitor class.
Add equivalence test for KineticRewriter: the test incorporates a simple ad-hoc algebraic expression simplifier.
Add unit test to Parser.parse_binop to exercise bug #94

* update the out of date version of modparser was added to the repository
* fix some warnings about unused static functions in a modcc test header
* move modcc/src path contents to modcc, because there was little point having
  the additional sub directory.

* only the source has been moved over and the cmake rules updated
* the tests still have to be moved