Clarify Python installation from CMake (#1413)

Always install Python module inside CMAKE_INSTALL_PREFIX, unless the user explicitly sets the ARB_PYTHON_LIB_PATH CMake variable.

Update documentation on installing and building the Python packing using CMake to have a detailed example of how to install the Python module inside a virtual env.

Fixes #1403

doc/install/build_install.rst

@@ -444,25 +444,60 @@ system with the executable in ``/usr/bin/python3.8``:
 
     cmake .. -DARB_WITH_PYTHON=ON -DPYTHON_EXECUTABLE=/usr/bin/python3.8
 
-By default the Python module will be installed in the directory returned by
-``${PYTHON_EXECUTABLE} -c "import sysconfig; print(sysconfig.get_path('platlib'))"``.
-This returns the directory where the supplied or found ``PYTHON_EXECUTABLE`` looks for system packages.
-`See Python's sysconfig documentation <https://docs.python.org/3/library/sysconfig.html#installation-paths>`_.
-If CMake is run in a `venv` or Conda environment, this should pick up on the appropriate package directory.
-To install the module in a different location, set ``ARB_PYTHON_LIB_PATH`` to a custom path.
-For example, the CMake configuration for targeting Python 3.8 and install as a
-user site package might look like the following:
+
+By default the Python package will be installed in the appropriate sub-directory
+inside ``CMAKE_INSTALL_PREFIX``, determined by querying Python's sysconfig library.
+For example ``${CMAKE_INSTALL_PREFIX}/lib/python3.9/site-packages/``.
+
+To install the module in a different location, independent of ``CMAKE_INSTALL_PREFIX``,
+use ``ARB_PYTHON_LIB_PATH`` to specify the location where the Python module is to be installed.
 
 .. code-block:: bash
 
-    cmake .. -DARB_WITH_PYTHON=ON                                              \
-             -DARB_PYTHON_LIB_PATH=${HOME}/.local/lib/python3.8/site-packages/ \
-             -DPYTHON_EXECUTABLE=/usr/bin/python3.8
+    cmake .. -DARB_WITH_PYTHON=on -DARB_PYTHON_PATH_LIB=/custom/path
+
+If CMake is run in a `venv` or Conda environment, set ``CMAKE_INSTALL_PREFIX`` to the
+base path of the venv. The example below shows a workflow that creates a virtual
+environment, then installs Arbor inside the environment.
+
+.. code-block:: bash
+
+    # Set up your venv.
+    mkdir myenv
+    cd myenv/
+    python3 -m venv env
+    source env/bin/activate
+
+    # Install dependencies
+    pip install numpy
+
+    # Obtain arbor
+    git clone --recursive git@github.com:arbor-sim/arbor.git
+
+    # Determine the prefix path for your installation
+    # method 1: set it explicitly
+    export pyprefix=$(pwd)/env
+    # method 2: query python directly by instpecting the output of sysconfig.
+    # Note that the ending of the form lib/python3.x/site-packages must be dropped.
+    # E.g if the output was /home/xxx/myenv/env/lib/python3.9/site-packages, we
+    # want to set pyprefix=/home/xxx/myenv/env/
+    python -c "import sysconfig; print(sysconfig.get_path('platlib'))"
+
+    # Configure Arbor
+    mkdir build
+    cd build
+    cmake ../arbor -DARB_WITH_PYTHON=on       \       # enable python support.
+                   -DARB_USE_BUNDLED_LIBS=on  \       # use bundled versions of deps.
+                   -DCMAKE_INSTALL_PREFIX="$pyprefix" # set custom installation path.
+
+    # Build and install
+    make -j4
+    make install
 
-On the target LINUX system, the Arbor package was installed in
-``/home/$USER/.local/lib/python3.8/site-packages``.
+    # Test it out!
+    python -c "import arbor; print(arbor.__config__)"
 
-The Arbor Python wrapper has optional support for the mpi4py, though
+The Arbor Python wrapper has optional support for mpi4py, though
 it is not required to use Arbor with Python and MPI.
 CMake will attempt to automatically detect ``mpi4py`` if configured
 with both ``-DARB_WITH_PYTHON=ON`` and MPI ``-DARB_WITH_MPI=ON``.

doc/install/python.rst

@@ -136,8 +136,7 @@ below demonstrate this for both pip and ``setup.py``.
     :ref:`Python configuration <install-python>` section of the
     :ref:`installation guide <in_build_install>`.
     CMake is recommended for developers, integration with package managers such as
-    Spack and EasyBuild, and users who require fine grained control over compilation
-    and installation.
+    Spack and EasyBuild, and fine-grained control over compilation and installation.
 
 Dependencies
 ^^^^^^^^^^^^

python/CMakeLists.txt

@@ -82,12 +82,7 @@ file(COPY "${PROJECT_SOURCE_DIR}/VERSION" DESTINATION "${python_mod_path}")
 # Set the installation path
 
 # Ask Python where it keeps its system (platform) packages.
-
-if(CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
-    file(WRITE "${CMAKE_BINARY_DIR}/install-prefix" "")
-else()
-    file(WRITE "${CMAKE_BINARY_DIR}/install-prefix" "${CMAKE_INSTALL_PREFIX}")
-endif()
+file(WRITE "${CMAKE_BINARY_DIR}/install-prefix" "${CMAKE_INSTALL_PREFIX}")
 
 execute_process(
     COMMAND ${PYTHON_EXECUTABLE} -c