Testing

updated: None

Testing

The tests are run with tox and configured in tox.ini. The test results are tracked by testr and configured in .testr.conf.

Unit Tests

Run

In order to run the entire unit test suite, simply run the tox command inside of your source checkout. This will attempt to run every test command listed inside of tox.ini, which includes Python 2.7, 3.4, PyPy, and a PEP 8 check. You should run the full test suite on all versions before submitting changes for review in order to avoid unexpected failures in the continuous integration system.:

(sdk3)$ tox
...
py34: commands succeeded
py27: commands succeeded
pypy: commands succeeded
pep8: commands succeeded
congratulations :)

During development, it may be more convenient to run a subset of the tests to keep test time to a minimum. You can choose to run the tests only on one version. A step further is to run only the tests you are working on.:

(sdk3)$ tox -e py34                # Run run the tests on Python 3.4
(sdk3)$ tox -e py34 TestContainer  # Run only the TestContainer tests on 3.4

Functional Tests

The functional tests assume that you have a public or private OpenStack cloud that you can run the tests against. The tests must be able to be run against public clouds but first and foremost they must be run against OpenStack. In practice, this means that the tests should initially be run against a stable branch of DevStack.

DevStack

There are many ways to run and configure DevStack. The link above will show you how to run DevStack a number of ways. You’ll need to choose a method you’re familiar with and can run in your environment. Wherever DevStack is running, we need to make sure that python-openstacksdk contributors are using the same configuration.

This is the local.conf file we use to configure DevStack.

[[local|localrc]]
# Configure passwords and the Swift Hash
MYSQL_PASSWORD=DEVSTACK_PASSWORD
RABBIT_PASSWORD=DEVSTACK_PASSWORD
SERVICE_TOKEN=DEVSTACK_PASSWORD
ADMIN_PASSWORD=DEVSTACK_PASSWORD
SERVICE_PASSWORD=DEVSTACK_PASSWORD
SWIFT_HASH=DEVSTACK_PASSWORD

# Configure the stable OpenStack branches used by DevStack
# For stable branches see
# http://git.openstack.org/cgit/openstack-dev/devstack/refs/
CINDER_BRANCH=stable/OPENSTACK_VERSION
CEILOMETER_BRANCH=stable/OPENSTACK_VERSION
GLANCE_BRANCH=stable/OPENSTACK_VERSION
HEAT_BRANCH=stable/OPENSTACK_VERSION
HORIZON_BRANCH=stable/OPENSTACK_VERSION
KEYSTONE_BRANCH=stable/OPENSTACK_VERSION
NEUTRON_BRANCH=stable/OPENSTACK_VERSION
NOVA_BRANCH=stable/OPENSTACK_VERSION
SWIFT_BRANCH=stable/OPENSTACK_VERSION
ZAQAR_BRANCH=stable/OPENSTACK_VERSION

# Enable Swift
enable_service s-proxy
enable_service s-object
enable_service s-container
enable_service s-account

# Disable Nova Network and enable Neutron
disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service q-metering

# Enable Ceilometer
enable_service ceilometer-acompute
enable_service ceilometer-acentral
enable_service ceilometer-anotification
enable_service ceilometer-collector
enable_service ceilometer-alarm-evaluator
enable_service ceilometer-alarm-notifier
enable_service ceilometer-api

# Enable Zaqar
enable_plugin zaqar https://github.com/openstack/zaqar
enable_service zaqar-server

# Enable Heat
enable_service h-eng
enable_service h-api
enable_service h-api-cfn
enable_service h-api-cw

# Automatically download and register a VM image that Heat can launch
# For more information on Heat and DevStack see
# https://docs.openstack.org/heat/latest/getting_started/on_devstack.html
IMAGE_URL_SITE="http://download.fedoraproject.org"
IMAGE_URL_PATH="/pub/fedora/linux/releases/25/CloudImages/x86_64/images/"
IMAGE_URL_FILE="Fedora-Cloud-Base-25-1.3.x86_64.qcow2"
IMAGE_URLS+=","$IMAGE_URL_SITE$IMAGE_URL_PATH$IMAGE_URL_FILE

# Logging
LOGDAYS=1
LOGFILE=/opt/stack/logs/stack.sh.log
LOGDIR=/opt/stack/logs

Replace DEVSTACK_PASSWORD with a password of your choice.

Replace OPENSTACK_VERSION with a stable branch of OpenStack (without the stable/ prefix on the branch name).

os-client-config

To connect the functional tests to an OpenStack cloud we use os-client-config. To setup os-client-config create a clouds.yaml file in the root of your source checkout.

This is an example of a minimal configuration for a clouds.yaml that connects the functional tests to a DevStack instance. Note that one cloud under clouds must be named test_cloud.

clouds:
  test_cloud:
    region_name: RegionOne
    auth:
      auth_url: http://xxx.xxx.xxx.xxx:5000/v2.0/
      username: demo
      password: secrete
      project_name: demo
    example:
      image_name: fedora-20.x86_64
      flavor_name: m1.small
      network_name: private
  rackspace:
    cloud: rackspace
    auth:
      username: joe
      password: joes-password
      project_name: 123123
    region_name: IAD

Replace xxx.xxx.xxx.xxx with the IP address or FQDN of your DevStack instance.

You can also create a ~/.config/openstack/clouds.yaml file for your DevStack cloud environment using the following commands. Replace DEVSTACK_SOURCE with your DevStack source checkout.:

(sdk3)$ source DEVSTACK_SOURCE/accrc/admin/admin
(sdk3)$ ./create_yaml.sh

Run

Functional tests are run against both Python 2 and 3. In order to run the entire functional test suite, run the tox -e functional and tox -e functional3 command inside of your source checkout. This will attempt to run every test command under /openstack/tests/functional/ in the source tree. You should run the full functional test suite before submitting changes for review in order to avoid unexpected failures in the continuous integration system.:

(sdk3)$ tox -e functional
...
functional: commands succeeded
congratulations :)
(sdk3)$ tox -e functional3
...
functional3: commands succeeded
congratulations :)

Examples Tests

Similar to the functional tests, the examples tests assume that you have a public or private OpenStack cloud that you can run the tests against. In practice, this means that the tests should initially be run against a stable branch of DevStack. And like the functional tests, the examples tests connect to an OpenStack cloud using os-client-config. See the functional tests instructions for information on setting up DevStack and os-client-config.

Run

In order to run the entire examples test suite, simply run the tox -e examples command inside of your source checkout. This will attempt to run every test command under /openstack/tests/examples/ in the source tree.:

(sdk3)$ tox -e examples
...
examples: commands succeeded
congratulations :)
updated: None
Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.

found an error? report a bug questions?