ANN activation functions for NMODL: `sigmoid(x)`, `relu(x)` and `tanh(x)` (#2052)
I have been playing around a bit with getting Neural ODE's working (and in general ANN's as black-box homeostatic mechanisms) and found arbor lacking in ANN activation functions. Now not everybody might agree that this is actually needed in Arbor, so I don't expect this PR to be merged directly. But if people like the idea, here it is :) The PR contains 3 common unary functions used in ANN's as nonlinear activations: `sigmoid(x)`, `relu(x)` and `tanh(x)` Here is a very minimal example of a 'neural ode': ``` NEURON { SUFFIX node } PARAMETER { A1 = -0.662 A2 = 0.282 A3 = 0.957 A4 = -0.189 A5 = -1.794 A6 = 1.094 A7 = -1.133 A8 = 0.625 A9 = 0.074 b1 = -1.434 b2 = -0.358 b3 = -0.093 } STATE { x y z } INITIAL { x = 0 y = 1 z = 1 } DERIVATIVE dstate { x' = tanh(A1 * x + A2 * y + A3 * z + b1) y' = tanh(A4 * x + A5 * y + A6 * z + b2) ...
Showing
- arbor/include/arbor/simd/implbase.hpp 20 additions, 0 deletionsarbor/include/arbor/simd/implbase.hpp
- arbor/include/arbor/simd/simd.hpp 2 additions, 2 deletionsarbor/include/arbor/simd/simd.hpp
- doc/dev/simd_api.rst 12 additions, 0 deletionsdoc/dev/simd_api.rst
- doc/fileformat/nmodl.rst 4 additions, 1 deletiondoc/fileformat/nmodl.rst
- modcc/expression.cpp 15 additions, 0 deletionsmodcc/expression.cpp
- modcc/expression.hpp 27 additions, 0 deletionsmodcc/expression.hpp
- modcc/parser.cpp 3 additions, 0 deletionsmodcc/parser.cpp
- modcc/perfvisitor.hpp 19 additions, 3 deletionsmodcc/perfvisitor.hpp
- modcc/printer/cexpr_emit.cpp 17 additions, 2 deletionsmodcc/printer/cexpr_emit.cpp
- modcc/printer/cprinter.cpp 1 addition, 0 deletionsmodcc/printer/cprinter.cpp
- modcc/symdiff.cpp 49 additions, 0 deletionsmodcc/symdiff.cpp
- modcc/token.cpp 6 additions, 0 deletionsmodcc/token.cpp
- modcc/token.hpp 5 additions, 0 deletionsmodcc/token.hpp
- modcc/visitor.hpp 3 additions, 0 deletionsmodcc/visitor.hpp
- test/unit-modcc/test_symdiff.cpp 5 additions, 1 deletiontest/unit-modcc/test_symdiff.cpp
- test/unit/test_simd.cpp 17 additions, 0 deletionstest/unit/test_simd.cpp
Please register or sign in to comment