5.23. OpenStack in a WAN Context Testing¶

status

draft

version

0.1

Abstract

This document describes a test plan for analysing how OpenStack behaves under Wide Area Network (WAN) conditions. In particular, measuring the effect of high latency, low bandwidth, packet loss and link failure.

Conventions

WAN A Wide Area Network (WAN) is a computer network that extends over a large geographical distance.
NB_COMPUTES The number of Computes considered by the tester.
NB_CORES/COMPUTE The number of cores on a Compute node.

5.23.1. Test Plan¶

New paradigms like Fog and Edge computing suggest pushing Computes close to the user. This kind of deployment puts Computes far from Control services that have to operate through WAN. The current test plan covers basic cloud operations (VM creations, Security Group management and IP associations) and communication testing (through several VMs) under such WAN conditions.

5.23.1.1. Test Environment¶

This section describes the setup for OpenStack testing under WAN conditions.

5.23.1.1.1. Preparation¶

The current test plan considers a fresh, bare-metal, simple deployment where Control, Network and Volume services are on the same nodes. Computes are on dedicated nodes. The tester should fix the number of Computes depending on the capacity of her testbed. In the rest of this document, the name NB_COMPUTES references the number of Computes.

Regarding network, the following section should describe in details network settings. In particular, links parameters (latency, bandwidth and reliability) between OpenStack services. It should also indicate the use of network emulation functionality (e.g.: netem). In case of network emulation with netem, netem constraints have to be set up right after the end of the OpenStack deployment to make tests reliable.

5.23.1.1.2. Environment description¶

The environment description includes hardware specification of servers, network parameters, operation system and OpenStack deployment characteristics.

5.23.1.1.2.1. Hardware¶

This section contains the list of all types of hardware nodes.

Parameter	Value	Comments
model		e.g. Supermicro X9SRD-F
CPU		e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz

5.23.1.1.2.2. Network¶

This section contains the list of interfaces and network parameters. In addition, it should include a topology diagram with latency, bandwidth and reliability of links between OpenStack services.

Parameter	Value	Comments
card model		e.g. Intel

card model | | e.g. Intel |

5.23.1.1.2.3. Software¶

This section describes installed software.

Parameter	Value	Comments
OS		e.g. Ubuntu 14.04.3
DB		e.g. MySQL 5.6
MQ broker		e.g. RabbitMQ v3.4.25
OpenStack release		e.g. Ocata
Network emulation		e.g. iproute2 4.9.0

5.23.1.1.2.4. Configuration¶

This section describes configuration of OpenStack and core services

Parameter	File
Keystone	./results/keystone.conf
Nova-api	./results/nova-api.conf
…

5.23.1.2. Test Case 1: OpenStack control services behaviour under WAN¶

5.23.1.2.1. Description¶

This test case aims to measure the correct behaviour of OpenStack under WAN at control plane level. It relies on Rally that runs loads on the current OpenStack. Then Rally reports can be used to get time of operations executions and percent of failure to evaluate OpenStack. Concretely, this test case considers followings Rally scenarios that are known to be sensitive to network performance:

Nova scenarios

NovaServers.boot_and_delete_multiple_servers
NovaServers.boot_and_associate_floating_ip
NovaServers.pause_and_unpause_server
NovaSecGroup.boot_server_and_add_secgroups

Neutron scenarios

NeutronNetworks.create_and_delete_networks
NeutronNetworks.create_and_delete_ports
NeutronNetworks.create_and_delete_routers
NeutronNetworks.create_and_delete_subnets
NeutronSecurityGroup.create_and_delete_security_groups

Glance scenario

GlanceImages.create_and_delete_image

Other scenarios such as live migration, VM snapshoting and telemetry may also be considered.

5.23.1.2.2. Parameters¶

Parameter name	Value
RTT latency (ms)	LAN, 20, 50, 100, 200
Traffic shaping (% loss)	0, 0.1, 1, 10, 25
Number of VMs (for `NovaServers. boot_and_delete_multiple _servers`)	1, NB_COMPUTES, NB_COMPUTES * NB_CORES/COMPUTE
Rally concurrency	1, NB_COMPUTES, NB_COMPUTES * NB_CORES/COMPUTE
VM image	CirrOS (~15MB), Alpine (~100MB), Ubuntu-16.04-server (~800MB)

The netem application can help to achieve WAN simulation by applying network constraints (RTT latency and packet loss) between the Control node and Computes. The tester may refer to the Test tool section for such purpose scripts.

Rally times parameter should be configured properly to produce accurate results. Furthermore, the tester may choose to execute scenarios in cold- or hot-cache.

5.23.1.2.3. Further parameters¶

WAN conditions often lead to link failures and so should be tested. A link failure should appear at a specific point in the workflow of a Rally scenario. For instance, just after the nova.compute.manager.ComputeManager.build_and_run_instance RPC call in the case of NovaServers.boot_and_delete_multiple_servers Rally scenario. The duration of link failure is another important parameter. However, to the best of our knowledge, there are no tools to automatically inject such failure at a specific point of an OpenStack workflow (something more convenient than modifying the OpenStack source code).

5.23.1.2.4. List of performance metrics¶

Priority	Value	Measurement Units	Description
1		sec	Time of operation execution
1		percentage	Operation execution failure

5.23.1.3. Test Case 2: OpenStack data plane behaviour under WAN¶

5.23.1.3.1. Description¶

This test case aims to measure the correct behaviour of OpenStack under WAN at data plane level. It relies on Shaker that starts VMs and runs network test on the current OpenStack. Then Shaker reports can be used to get throughput and round-trip time between VMs in order to evaluate OpenStack. Expectation here is that Neutron configuration has a strong impact on data plane, e.g.: going with DVR or Dragonflow. This test case considers followings Shaker scenarios:

openstack/full_l2
openstack/dense_l3_east_west
openstack/full_l3_east_west

Note that a Shaker test requires Heat service to be running.

5.23.1.3.2. Parameters¶

Parameter name	Value
RTT latency (ms)	LAN, 20, 50, 100, 200
Traffic shaping (% loss)	0, 0.1, 1, 10, 25
Shaker concurrency	1, NB_COMPUTES, NB_COMPUTES * NB_CORES/COMPUTE

5.23.1.3.3. List of performance metrics¶

Priority	Measurement Units	Description
1	ms	Ping
2	Mbits/s	TCP download
2	Mbits/s	TCP upload

5.23.1.4. Test tool¶

The tester can use EnOS to conduct the current test plan. EnOS is a tool that helps tester to (1) easily get testbed resources, (2) deploys and initialises OpenStack over these resources, (3) set network constraints, (4) invokes benchmarks and provides test reports. EnOS comes with a set of testing scenarios including some for the current test plan.

5.23. OpenStack in a WAN Context Testing