Writing and Running Barbican Tests

As a part of every code review that is submitted to the Barbican project there are a number of gating jobs which aid in the prevention of regression issues within Barbican. As a result, a Barbican developer should be familiar with running Barbican tests locally.

For your convenience we provide the ability to run all tests through the tox utility. If you are unfamiliar with tox please see refer to the tox documentation for assistance.

Unit Tests

Currently, we provide tox environments for Python 2.7 and 3.5. By default all available test environments within the tox configuration will execute when calling tox. If you want to run them independently, you can do so with the following command:

# Executes tests on Python 2.7
tox -e py27

Note

If you do not have the appropriate Python versions available, consider setting up PyEnv to install multiple versions of Python. See the documentation regarding Setting up a Barbican Development Environment for more information.

Note

Individual unit tests can also be run, using the following commands:

# runs a single test with the function named
# test_can_create_new_secret_one_step
tox -e py27 -- test_can_create_new_secret_one_step

# runs only tests in the WhenTestingSecretsResource class and
# the WhenTestingCAsResource class
tox -e py27 -- '(WhenTestingSecretsResource|WhenTestingCAsResource)'

The function name or class specified must be one located in the barbican/tests directory.

Groups of tests can also be run with a regex match after the --. For more information on what can be done with testr, please see: http://testrepository.readthedocs.org/en/latest/MANUAL.html

You can also setup breakpoints in the unit tests. This can be done by adding import pdb; pdb.set_trace() to the line of the unit test you want to examine, then running the following command:

# Executes tests on Python 2.7
tox -e debug

Note

For a list of pdb commands, please see: https://docs.python.org/2/library/pdb.html

Python 3.5

In order to run the unit tests within the Python 3.5 unit testing environment you need to make sure you have all necessary packages installed.

  • On Ubuntu/Debian:

    sudo apt-get install python3-dev

  • On Fedora 21/RHEL7/CensOS7:

    sudo yum install python3-devel

  • On Fedora 22 and higher:

    sudo dnf install python3-devel

You then specify to run the unit tests within the Python 3.5 environment when invoking tox

# Executes tests on Python 3.5
tox -e py35

Functional Tests

Unlike running unit tests, the functional tests require Barbican and Keystone services to be running in order to execute. For more information on setting up a Barbican development environment and using Keystone with Barbican, see our accompanying project documentation.

Once you have the appropriate services running and configured you can execute the functional tests through tox.

# Execute Barbican Functional Tests
tox -e functional

By default, the functional tox job will use testr to execute the functional tests as used in the gating job.

Note

In order to run an individual functional test function, you must use the following command:

# runs a single test with the function named
# test_secret_create_then_check_content_types
tox -e functional -- test_secret_create_then_check_content_types

# runs only tests in the SecretsTestCase class and
# the OrdersTestCase class
tox -e functional -- '(SecretsTestCase|OrdersTestCase)'

The function name or class specified must be one located in the functionaltests directory.

Groups of tests can also be run with a regex match after the --. For more information on what can be done with testr, please see: http://testrepository.readthedocs.org/en/latest/MANUAL.html

Remote Debugging

In order to be able to hit break-points on API calls, you must use remote debugging. This can be done by adding import rpdb; rpdb.set_trace() to the line of the API call you wish to test. For example, adding the breakpoint in def on_post in barbican.api.controllers.secrets.py will allow you to hit the breakpoint when a POST is done on the secrets URL.

Note

After performing the POST the application will freeze. In order to use rpdb, you must open up another terminal and run the following:

# enter rpdb using telnet
telnet localhost 4444

Once in rpdb, you can use the same commands as pdb, as seen here: https://docs.python.org/2/library/pdb.html