Skip to content
Snippets Groups Projects
Unverified Commit b0d2b90b authored by Han Lu's avatar Han Lu Committed by GitHub
Browse files

Update test.rst (#2313)

<!-- Please make sure your PR follows our [contribution
guidelines](https://github.com/arbor-sim/arbor/tree/master/doc/contrib)
and agree to the terms outlined in the [PR
procedure](https://github.com/arbor-sim/arbor/tree/master/doc/contrib/pr.rst

).
-->

---------

Co-authored-by: default avatarThorsten Hater <24411438+thorstenhater@users.noreply.github.com>
parent b795c104
No related branches found
No related tags found
No related merge requests found
......@@ -12,13 +12,13 @@ What to test?
Adding a feature should be accompanied by tests to ensure its functionality is
sound. That means
1. identify the core ideas of your feature
2. find the functions/classes that encapsulate these ideas
3. for each add a test case that covers it
1. identifying the core ideas of your feature;
2. finding the functions/classes that encapsulate these ideas;
3. adding a test case for each that covers it.
The core motivation is to capture the essence of your feature and to protect it
against accidental change. This is what enables us to freely add optimisations,
and perform refactoring.
and refactor code as needed.
Example
^^^^^^^
......@@ -27,9 +27,9 @@ This example might touch parts of Arbor that you are unfamiliar with. Don't
Panic! The details are less important than the general approach to adding tests.
Imagine adding a new feature that is intended to improve performance during the
communication step. Spikes should be transferred pre-sorted to avoid locally
sorting them after the exchange. Also assume, just for the sake of this example,
sorting them after the exchange. Also, assume, just for the sake of this example,
that you decided to add your own radix sort algorithm, since you expect it to be
faster for this particular usecase.
faster for this particular use case.
Thus, the tests added should be
......@@ -37,23 +37,23 @@ Thus, the tests added should be
- the application of `sort` sorts the given array. This seems trivial, but is
really the core of what you are doing!
- corner cases like: empty array, all elements equal, ... are treated greacefully
- if the sort is intended to be stable, check that equal elements do not switch order
- corner cases like: empty array, all elements equal, ... are treated gracefully.
- if the sort is intended to be stable, check that equal elements do not switch order.
2. local sorting
- spikes are -- after sorting -- ordered by their source id and in case of ties by time
- spikes are -- after sorting -- ordered by their source ids and in case of ties by time.
- corner cases: NaN, negative numbers, ...
3. global sorting
- after MPI exchange, each sub-array is still sorted
- after the MPI exchange, each sub-array is still sorted
- by the guarantees of ``MPI_Allgather``, the global array is sorted
Note that we added tests that are only applicable when eg MPI is enabled. Our test
Note that we added tests that are only applicable when, e.g., MPI is enabled. Our test
runners probe the different combinations automatically, see below.
Next, we would ask you to prove that this change does as promised, ie it
Next, we would ask you to prove that this change does as promised, ie, it
improves performance. When adding a new user-facing feature, also consider
adding an example showing off your cool new addition to Arbor.
......@@ -71,16 +71,16 @@ C++ tests
We are using the GTest library for writing tests. Each group of tests should be
contained in a ``.cpp`` file in ``test/unit`` (do not forget to add it to the
``CMakeLists.txt``!). To get access to the library and a battery of helpers
include ``common.hpp``. Test cases are defined vi the ``TEST`` macro which takes
including ``common.hpp``. Test cases are defined via the ``TEST`` macro, which takes
two arguments ``group`` and ``case``. Inside cases macros like ``ASSERT_TRUE``
can be used. Another helpful feature is that the test executable accepts
arguments on the commandline. Of these we would like to point out
arguments on the command line. Of these, we would like to point out:
- ``--gtest_catch_exceptions`` allows for disabling exception catching by the
framework. Handy when running the tests in a debugger.
- ``--gtest_throw_on_failure`` turns missed assert into exceptions, likewise
useful in a debugger
- ``--gtest_filter`` to filter the tests to run. Can cut down the runtrip time
- ``--gtest_filter`` to filter the tests to run. Can cut down the roundtrip time
when working on a specific feature.
For more information on GTest refer to the `documentation
......@@ -89,17 +89,17 @@ For more information on GTest refer to the `documentation
Python tests
------------
The Python tests uses the `unittest
The Python tests use the `unittest
<https://docs.python.org/3/library/unittest.html>`_ and its test discovery
mechanism. For tests to be discovered they must meet the following criteria:
mechanism. For tests to be discovered, they must meet the following criteria:
* Located in an importable code folder starting from the ``python/test`` root.
If you introduce subfolders they must all contain a ``__init__.py`` file.
If you introduce subfolders, they must all contain a ``__init__.py`` file.
* The filenames must start with ``test_``.
* The test case classes must begin with ``Test``.
* The test functions inside the cases must begin with ``test_``.
To run the tests locally use `python -m unittest` from the `python` directory.
To run the tests locally, use ``python -m unittest`` from the ``python`` directory.
Fixtures
^^^^^^^^
......@@ -108,9 +108,9 @@ Multiple tests may require the same reusable piece of test setup to run. You
can speed up the test writing process for everyone by writing these reusable
pieces as a `fixture <https://en.wikipedia.org/wiki/Test_fixture#Software>`_.
A fixture is a decorator that injects the reusable piece into the test
function. Fixtures, and helpers to write them, are available in
function. Fixtures and helpers to write them are available in
``python/test/fixtures.py``. The following example shows you how to create
a fixture that returns the arbor version, and optionally the path to it:
a fixture that returns the Arbor version and optionally the path to it:
.. code-block:: python
......@@ -124,8 +124,8 @@ a fixture that returns the arbor version, and optionally the path to it:
else:
return (arbor.__version__,)
Whenever you are writing a test you can now apply your fixture by calling it
with the required parameters, and adding a parameter to your function with the
Whenever you are writing a test, you can now apply your fixture by calling it
with the required parameters and adding a parameter to your function with the
same name as the fixture:
.. code-block:: python
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment