Improved selection of default Python install path. (#1277)

* Docs: emphasised the option of having pip install a local copy of the Arbor source.
* CMake: default Python library install path to `${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))"`. Fixes #1100.
* CMake: remove ARB_PYTHON_PREFIX option, added ARB_PYTHON_LIB_PATH option. We cannot know in general where in the prefix the packages are expected by Python.

CMakeLists.txt

@@ -59,12 +59,14 @@ option(ARB_WITH_NEUROML "build NeuroML support library" OFF)
 
 option(ARB_WITH_PYTHON "enable Python front end" OFF)
 
-# Path in which to install the Python module.
-# For Python 3.8, the module would ibe installed in
-#       ${ARB_PYTHON_PREFIX}/python3.8/site-packages
-# To install Arbor eqivalently to `pip install --user` on a linux system:
-#       -DARB_PYTHON_PREFIX="${HOME}/.local/lib""
-set(ARB_PYTHON_PREFIX ${CMAKE_INSTALL_PREFIX} CACHE PATH "path for installing Python module for Arbor.")
+if (ARB_WITH_PYTHON)
+    # Find path in which to install the Python module.
+    find_package(PythonInterp REQUIRED)
+    # Ask the above found Python where it keeps its system (platform) packages.
+    execute_process (COMMAND ${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))" OUTPUT_VARIABLE ARB_PYTHON_LIB_PATH_DEFAULT OUTPUT_STRIP_TRAILING_WHITESPACE)
+    # Default to installing in that path
+    set(ARB_PYTHON_LIB_PATH ${ARB_PYTHON_LIB_PATH_DEFAULT} CACHE PATH "path for installing Python module for Arbor.")
+endif()
 
 #----------------------------------------------------------
 # Global CMake configuration

doc/install/build_install.rst

@@ -421,17 +421,19 @@ system with the executable in ``/usr/bin/python3.8``:
 
     cmake .. -DARB_WITH_PYTHON=ON -DPYTHON_EXECUTABLE=/usr/bin/python3.8
 
-By default the Python module will be installed in the standard ``CMAKE_INSTALL_PREFIX``
-location. To install the module in a different location, for example as a
-user module or in a virtual environment, set ``ARB_PYTHON_PREFIX``.
-For example, the CMake configuration for targetting Python 3.8 and install as a
+By default the Python module will be installed in the directory returned by
+``${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))"``.
+This returns the directory where the supplied or found PYTHON_EXECUTABLE looks for system packages.
+`See Python's sysconfig documentation <https://docs.python.org/3/library/sysconfig.html#installation-paths>`_.
+If CMake is run in a `venv` or Conda environment, this should pick up on the appropriate package directory. To install the module in a different location, set ``ARB_PYTHON_LIB_PATH`` to a custom path.
+For example, the CMake configuration for targeting Python 3.8 and install as a
 user site package might look like the following:
 
 .. code-block:: bash
 
-    cmake .. -DARB_WITH_PYTHON=ON                   \
-             -DARB_PYTHON_PREFIX=${HOME}/.local     \
-             -DPYTHON_EXECUTABLE=/user/bin/python3.8
+    cmake .. -DARB_WITH_PYTHON=ON                                              \
+             -DARB_PYTHON_LIB_PATH=${HOME}/.local/lib/python3.8/site-packages/ \
+             -DPYTHON_EXECUTABLE=/usr/bin/python3.8
 
 On the target LINUX system, the Arbor package was installed in
 ``/home/$USER/.local/lib/python3.8/site-packages``.
@@ -442,7 +444,7 @@ On the target LINUX system, the Arbor package was installed in
     ``/usr/local/include``, and the Python module will be installed in a path like
     ``/usr/local/lib/python3.8/site-packages``.
     Because ``/usr/local`` is a system path, the installation phase needs to be run as root,
-    i.e. ``sudo make install``, even if ``ARB_PYTHON_PREFIX`` is set to a user path
+    i.e. ``sudo make install``, even if ``ARB_PYTHON_LIB_PATH`` is set to a user path
     that does not require root to install.
 
 The Arbor Python wrapper has optional support for the mpi4py, though

doc/install/python.rst

@@ -3,30 +3,18 @@
 Python Installation
 ===================
 
-Arbor's Python API will be the most convenient interface for most users. Note that we only support Python 3.6 and later.
+Arbor's Python API will be the most convenient interface for most users. Note that we support Python 3.6 and later.
 Any instruction hereafter assumes you're using `python` and `pip` no older than that.
 
 Getting Arbor
 -------------
 
-The easiest way to get Arbor is with
+Every point release of Arbor is pushed to the Python Package Index. The easiest way to get Arbor is with
 `pip <https://packaging.python.org/tutorials/installing-packages>`_:
 
 .. code-block:: bash
 
-    pip3 install arbor --user
-
-Every point release is pushed to the Python Package Index. If you wish to install another version, it is possible to use
-Setuptools directly on a local copy of the source code, or instruct `pip` to install directly from our git repository:
-
-.. code-block:: bash
-
-    # use setuptools and python directly
-    git clone https://github.com/arbor-sim/arbor.git --recursive
-    python3 install ./arbor/setup.py
-
-    # tell pip to build and install from master
-    pip3 install git+https://github.com/arbor-sim/arbor.git --user
+    pip3 install arbor
 
 .. note::
     You will need to have some development packages installed in order to build Arbor this way.
@@ -36,6 +24,13 @@ Setuptools directly on a local copy of the source code, or instruct `pip` to ins
     * macOS: get `brew` `here <https://brew.sh>`_ and run `brew install cmake clang python3`
     * Windows: the simplest way is to use `WSL <https://docs.microsoft.com/en-us/windows/wsl/install-win10>`_ and then follow the instructions for Ubuntu.
 
+If you wish to get the latest Arbor straight from
+the master branch in our git repository, you can run:
+
+.. code-block:: bash
+
+    pip3 install git+https://github.com/arbor-sim/arbor.git
+
 To test that Arbor is available, try the following in a Python interpreter
 to see information about the version and enabled features:
 
@@ -46,14 +41,28 @@ to see information about the version and enabled features:
     >>> print(arbor.__config__)
 
 You are now ready to use Arbor! You can continue reading these documentation pages, have a look at the
-:ref:`Python API reference<pyoverview>` , or visit the :ref:`Quick Start page<gs_single_cell>`.
+:ref:`Python API reference<pyoverview>`, or visit the :ref:`gs_other_examples`.
 
 .. Note::
     To get help in case of problems installing with pip, run pip with the ``--verbose`` flag, and attach the output
     (along with the pip command itself) to a ticket on `Arbor's issues page <https://github.com/arbor-sim/arbor/issues>`_.
 
+Customising Arbor
+^^^^^^^^^^^^^^^^^
+
+If you want to work on Arbor's code, you can get a copy of our repo and point `pip` at the local directory:
+
+.. code-block:: bash
+
+    # get your copy of the Arbor source
+    git clone https://github.com/arbor-sim/arbor.git --recursive
+    # make your changes and then instruct pip to build and install the local source
+    pip3 install ./arbor/
+
+Every time you make changes to the code, you'll have to repeat the second step.
+
 Advanced options
-^^^^^^^^^^^^^^^^^^
+^^^^^^^^^^^^^^^^
 
 By default Arbor is installed with multi-threading enabled.
 To enable more advanced forms of parallelism, the following optional flags can
@@ -129,7 +138,7 @@ below demonstrate this for both pip and ``setup.py``.
     and installation.
 
 Dependencies
-^^^^^^^^^^^^^
+^^^^^^^^^^^^
 
 If a downstream dependency requires Arbor be built with
 a specific feature enabled, use ``requirements.txt`` to
@@ -142,8 +151,8 @@ with MPI support would add the following to its requirements:
     arbor >= 0.3 --install-option='--gpu=cuda' \
                  --install-option='--mpi'
 
-Performance
---------------
+Note on performance
+-------------------
 
 The Python interface can incur significant memory and runtime overheads relative to C++
 during the *model building* phase, however simulation performance is the same

python/CMakeLists.txt

@@ -73,11 +73,6 @@ set_target_properties(pyarb PROPERTIES LIBRARY_OUTPUT_DIRECTORY "${python_mod_pa
 file(COPY "${PROJECT_SOURCE_DIR}/python/__init__.py" DESTINATION "${python_mod_path}")
 file(COPY "${PROJECT_SOURCE_DIR}/VERSION" DESTINATION "${python_mod_path}")
 
-# Determine the installation path, according to the Python version.
-# The installation for Python 3.8 would be:
-#  ${ARB_PYTHON_PREFIX}/python3.8/site-packages
-# By default ARB_PYTHON_PREFIX is set to CMAKE_INSTALL_PREFIX, and can be optionally
-# used to install the Python module as a user module, or in a virtualenv.
-find_package(PythonInterp REQUIRED)
-set(arb_pyexecdir "${ARB_PYTHON_PREFIX}/lib/python${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR}/site-packages")
-install(DIRECTORY ${python_mod_path} DESTINATION ${arb_pyexecdir})
+# Set the installation path
+
+install(DIRECTORY ${python_mod_path} DESTINATION ${ARB_PYTHON_LIB_PATH})