Update Python installation instructions (#1622)

Emphasize pip as the preferred method of installation.

doc/index.rst

@@ -13,7 +13,7 @@ You can find out how to :ref:`get Arbor<in_install>`; get started quickly with o
 What is Arbor?
 --------------
 
-`Arbor <https://arbor-sim.org>`_. is a high-performance library for computational neuroscience simulations with multi-compartment, morphologically-detailed cells,
+`Arbor <https://arbor-sim.org>`_ is a high-performance library for computational neuroscience simulations with multi-compartment, morphologically-detailed cells,
 from single cell models to very large networks. Arbor is written from the ground up with many-cpu and gpu architectures in mind, to
 help neuroscientists effectively use contemporary and future HPC systems to meet their simulation needs.
 

doc/install/build_install.rst

@@ -444,7 +444,6 @@ system with the executable in ``/usr/bin/python3.8``:
 
     cmake .. -DARB_WITH_PYTHON=ON -DPYTHON_EXECUTABLE=/usr/bin/python3.8
 
-
 By default the Python package will be installed in the appropriate sub-directory
 inside ``CMAKE_INSTALL_PREFIX``, determined by querying Python's sysconfig library.
 For example ``${CMAKE_INSTALL_PREFIX}/lib/python3.9/site-packages/``.
@@ -456,12 +455,22 @@ use ``ARB_PYTHON_LIB_PATH`` to specify the location where the Python module is t
 
     cmake .. -DARB_WITH_PYTHON=on -DARB_PYTHON_PATH_LIB=/custom/path
 
-If CMake is run in a `venv` or Conda environment, set ``CMAKE_INSTALL_PREFIX`` to the
-base path of the venv. The example below shows a workflow that creates a virtual
-environment, then installs Arbor inside the environment.
+.. note::
+    The location of libraries under a prefix in only guaranteed to be standard for Python's global library location.
+    Therefore, correct installation of the Python package to any other location using ``CMAKE_INSTALL_PREFIX``,
+    such as user directory (e.g. `~/.local`), a Python or Conda virtual environment, may result in installation to a wrong path.
+    
+    ``python3 -m site --user-site`` (for user installations) or a path from ``python3 -c 'import site; print(site.getsitepackages())'``
+    (for virtual environment installation) can be used in combination with ``ARB_PYTHON_LIB_PATH``.
+
+    In addition, installation via ``pip`` or ``python setup.py`` is guaranteed to find the right path. Please refer to the
+    :ref:`Python installation instruction <in_python_custom>`.
+
 
 .. code-block:: bash
 
+    # A demonstration using ARB_PYTHON_LIB_PATH
+
     # Set up your venv.
     mkdir myenv
     cd myenv/
@@ -469,26 +478,21 @@ environment, then installs Arbor inside the environment.
     source env/bin/activate
 
     # Install dependencies
-    pip install numpy
+    pip3 install numpy
 
     # Obtain arbor
     git clone --recursive git@github.com:arbor-sim/arbor.git
 
-    # Determine the prefix path for your installation
-    # method 1: set it explicitly
-    export pyprefix=$(pwd)/env
-    # method 2: query python directly by instpecting the output of sysconfig.
-    # Note that the ending of the form lib/python3.x/site-packages must be dropped.
-    # E.g if the output was /home/xxx/myenv/env/lib/python3.9/site-packages, we
-    # want to set pyprefix=/home/xxx/myenv/env/
-    python -c "import sysconfig; print(sysconfig.get_path('platlib'))"
+    # Manually set the prefix under which the python package will be installed.
+    # In this case, the first directory found by querying Python's list of site-package directories.
+    pyprefix=`python3 -c 'import site; print(site.getsitepackages()[0])'`
 
-    # Configure Arbor
+    # Setup CMake
     mkdir build
     cd build
     cmake ../arbor -DARB_WITH_PYTHON=on       \       # enable python support.
                    -DARB_USE_BUNDLED_LIBS=on  \       # use bundled versions of deps.
-                   -DCMAKE_INSTALL_PREFIX="$pyprefix" # set custom installation path.
+                   -DARB_PYTHON_LIB_PATH="$pyprefix"  # set Python installation path.
 
     # Build and install
     make -j4
@@ -497,6 +501,7 @@ environment, then installs Arbor inside the environment.
     # Test it out!
     python -c "import arbor; print(arbor.__config__)"
 
+
 The Arbor Python wrapper has optional support for mpi4py, though
 it is not required to use Arbor with Python and MPI.
 CMake will attempt to automatically detect ``mpi4py`` if configured

doc/install/python.rst

@@ -11,7 +11,9 @@ Arbor's Python API will be the most convenient interface for most users.
 Getting Arbor
 -------------
 
-Every point release of Arbor is pushed to the Python Package Index. The easiest way to get Arbor is with
+Every point release of Arbor is pushed to the Python Package Index.
+For x86-64 Linux and MacOS plaftorms, we provide binary wheels.
+The easiest way to get Arbor is with
 `pip <https://packaging.python.org/tutorials/installing-packages>`_:
 
 .. code-block:: bash
@@ -19,20 +21,14 @@ Every point release of Arbor is pushed to the Python Package Index. The easiest
     pip3 install arbor
 
 .. note::
+    For other platforms, `pip` will build Arbor from source. 
     You will need to have some development packages installed in order to build Arbor this way.
 
-    * Ubuntu/Debian: `sudo apt install git build-essential python3-dev python3-pip libxml2-dev`
-    * Fedora/CentOS/Red Hat: `sudo yum install git @development-tools python3-devel python3-pip libxml2-devel`
-    * macOS: get `brew` `here <https://brew.sh>`_ and run `brew install cmake clang python3 libxml2`
+    * Ubuntu/Debian: `git cmake gcc python3-dev python3-pip libxml2-dev`
+    * Fedora/CentOS/OpenSuse: `git cmake gcc-c++ python3-devel python3-pip libxml2-devel`
+    * MacOS: get `brew` `here <https://brew.sh>`_ and run `brew install cmake clang python3 libxml2`
     * Windows: the simplest way is to use `WSL <https://docs.microsoft.com/en-us/windows/wsl/install-win10>`_ and then follow the instructions for Ubuntu.
 
-If you wish to get the latest Arbor straight from
-the master branch in our git repository, you can run:
-
-.. code-block:: bash
-
-    pip3 install git+https://github.com/arbor-sim/arbor.git
-
 To test that Arbor is available, try the following in a Python interpreter
 to see information about the version and enabled features:
 
@@ -43,15 +39,24 @@ to see information about the version and enabled features:
     >>> print(arbor.__config__)
 
 You are now ready to use Arbor! You can continue reading these documentation pages, have a look at the
-:ref:`Python API reference<pyoverview>` , or visit the :ref:`Quick Start page<tutorialsinglecell>`.
+:ref:`Python API reference<pyoverview>`, or visit the :ref:`tutorial`.
 
 .. Note::
     To get help in case of problems installing with pip, run pip with the ``--verbose`` flag, and attach the output
     (along with the pip command itself) to a ticket on `Arbor's issues page <https://github.com/arbor-sim/arbor/issues>`_.
 
+.. _in_python_custom:
+
 Customising Arbor
 ^^^^^^^^^^^^^^^^^
 
+If you wish to get the latest Arbor straight from
+the master branch in our git repository, you can run:
+
+.. code-block:: bash
+
+    pip3 install git+https://github.com/arbor-sim/arbor.git
+
 If you want to work on Arbor's code, you can get a copy of our repo and point `pip` at the local directory:
 
 .. code-block:: bash
@@ -66,26 +71,25 @@ Every time you make changes to the code, you'll have to repeat the second step.
 Advanced options
 ^^^^^^^^^^^^^^^^
 
-By default Arbor is installed with multi-threading enabled.
-To enable more advanced forms of parallelism, the following optional flags can
-be used to configure the installation:
+By default Arbor is installed with multi-threading enabled. To enable more advanced forms of parallelism,
+Arbor comes with a few compilation options. These can be used on both local (``pip3 install ./arbor``) and
+remote (``pip3 install arbor``) copies of Arbor. Below we assume you are working off a local copy.
+
+The following optional flags can be used to configure the installation:
 
 * ``--mpi``: Enable MPI support (requires MPI library).
 * ``--gpu``: Enable GPU support for NVIDIA GPUs with nvcc using ``cuda``, or with clang using ``cuda-clang`` (both require cudaruntime).
   Enable GPU support for AMD GPUs with hipcc using ``hip``. By default set to ``none``, which disables gpu support.
-* ``--vec``: Enable vectorization. This might require choosing an appropriate architecture using ``--arch``. Note that on x86-64 platforms compilation will fail if you enable vectorization, but the CPU or ``--arch`` does not support any form of AVX.
-* ``--arch``: CPU micro-architecture to target. By default this is set to ``native``.
-
-If calling ``setup.py`` the flags must come after ``install`` on the command line,
-and if being passed to pip they must be passed via ``--install-option``. The examples
-below demonstrate this for both pip and ``setup.py``.
+* ``--vec``: Enable vectorization. The ``--arch`` argument, documented below, may also have to be set appropriately to generated vectorized code.
+  See :ref:`install-architecture` for details.
+* ``--arch``: CPU micro-architecture to target. The advised default is ``native``.
+  See `here <https://gcc.gnu.org/onlinedocs/gcc/x86-Options.html>`_ for a full list of options.
 
 **Vanilla install** with no additional features enabled:
 
 .. code-block:: bash
 
     pip3 install arbor
-    python3 ./arbor/setup.py install
 
 **With MPI support**. This might require loading an MPI module or setting the ``CC`` and ``CXX``
 :ref:`environment variables <install-mpi>`:
@@ -93,7 +97,6 @@ below demonstrate this for both pip and ``setup.py``.
 .. code-block:: bash
 
     pip3 install --install-option='--mpi' ./arbor
-    python3 ./arbor/setup.py install --mpi
 
 **Compile with** :ref:`vectorization <install-vectorize>` on a system with a SkyLake
 :ref:`architecture <install-architecture>`:
@@ -101,21 +104,18 @@ below demonstrate this for both pip and ``setup.py``.
 .. code-block:: bash
 
     pip3 install --install-option='--vec' --install-option='--arch=skylake' arbor
-    python3 ./arbor/setup.py install --vec --arch=skylake
 
 **Enable NVIDIA GPUs (compiled with nvcc)**. This requires the :ref:`CUDA toolkit <install-gpu>`:
 
 .. code-block:: bash
 
     pip3 install --install-option='--gpu=cuda' ./arbor
-    python3 ./arbor/setup.py install  --gpu=cuda
 
 **Enable NVIDIA GPUs (compiled with clang)**. This also requires the :ref:`CUDA toolkit <install-gpu>`:
 
 .. code-block:: bash
 
     pip3 install --install-option='--gpu=cuda-clang' ./arbor
-    python3 ./arbor/setup.py install --gpu=cuda-clang
 
 **Enable AMD GPUs (compiled with hipcc)**. This requires setting the ``CC`` and ``CXX``
 :ref:`environment variables <install-gpu>`
@@ -123,20 +123,21 @@ below demonstrate this for both pip and ``setup.py``.
 .. code-block:: bash
 
     pip3 install --install-option='--gpu=hip' ./arbor
-    python3 ./arbor/setup.py install --gpu=hip
 
 .. Note::
-    Setuptools compiles the Arbor C++ library and
-    wrapper, which can take a few minutes. Pass the ``--verbose`` flag to pip
+    Setuptools compiles the Arbor C++ library and wrapper, as well as dependencies you did not have installed
+    yet (e.g. `numpy`). It may take a few minutes. Pass the ``--verbose`` flag to pip
     to see the individual steps being performed if you are concerned that progress
     is halting.
 
+    If you had Arbor installed already, you may need to remove it first before you can (re)compile
+    it with the flags you need.
+
 .. Note::
-    Detailed instructions on how to install using CMake are in the
-    :ref:`Python configuration <install-python>` section of the
-    :ref:`installation guide <in_build_install>`.
-    CMake is recommended for developers, integration with package managers such as
-    Spack and EasyBuild, and fine-grained control over compilation and installation.
+    Detailed instructions on how to install using CMake are in the 
+    :ref:`Python configuration <install-python>` section of the :ref:`installation guide <in_build_install>`.
+    CMake is recommended if you need more control over compilation and installation, plan to use Arbor with C++,
+    or if you are integrating with package managers such as Spack and EasyBuild.
 
 Dependencies
 ^^^^^^^^^^^^