System(scenario) tests for Sahara project

System(scenario) tests for Sahara project

Authentication

You need to be authenticated to run these tests. To authenticate you should create openrc file (like in devstack) and source it.

#!/bin/sh
export OS_TENANT_NAME='admin'
export OS_PROJECT_NAME='admin'
export OS_USERNAME='admin'
export OS_PASSWORD='admin'
export OS_AUTH_URL='http://localhost:5000/v2.0'

Also you can specify the authentication details for Sahara tests using flags in run-command:

List of flags:
  --os-username
  --os-password
  --os-project-name
  --os-auth-url

Last way to set the authentication details for these tests is using a clouds.yaml file.

After creating the file, you can set OS_CLOUD variable or --os-cloud flag to the name of the cloud you have created and those values will be used.

We have an example of a clouds.yaml file, and you can find it in sahara-tests/unit/scenario/clouds.yaml.

Using this example, you can create your own file with clouds instead of setting the OS_CLOUD variable or the --os-cloud flag. Note that more than one cloud can be defined in the same file.

Here you can find more information about clouds

Template variables

You need to define these variables because they are used in mako template files and replace the values from scenario files. These names pass to the test runner through the -V parameter and a special config file.

The format of the config file is an INI-style file, as accepted by the Python ConfigParser module. The key/values must be specified in the DEFAULT section.

Variables and defaults templates

The following variables are currently used by defaults templates:

Variable Type Value
network_type string neutron or nova-network
network_private_name string private network name for OS_PROJECT_NAME
network_public_name string public network name
<plugin_name_version>_image string name of the image to be used for the specific plugin/version
{ci,medium,large}_flavor_id string IDs of flavor with different size

After finishing with authentication and configuration of file with template variables, you can run Sahara tests using Sahara Scenario Framework.

How to run

Scenario framework has default templates for testing Sahara. To use them, specify plugin and version (for transient check and fake plugin, version is not necessary):

$ tox -e venv -- sahara-scenario -p vanilla -v 2.7.1

Create the YAML and/or the YAML mako template files for scenario tests etc/scenario/simple-testcase.yaml. You can take a look at sample YAML files How to write scenario files.

The file templatevars.ini contains the values of the variables referenced by any testcase you are going to run.

If you want to run tests for the Vanilla plugin with the Hadoop version 2.7.1, you should create templatevars.ini with the appropriate values (see the section Variables and defaults templates) and use the following tox env:

$ tox -e venv -- sahara-scenario -V templatevars.ini sahara_tests/scenario/defaults/vanilla-2.7.1.yaml.mako

Credentials locate in sahara_tests/scenario/defaults/credentials.yaml.mako. This file replace the variables included into testcase YAML or YAML Mako files with the values defined into templatevars.ini.

$ tox -e venv -- sahara-scenario -V templatevars.ini sahara_tests/scenario/defaults/credentials.yaml.mako sahara_tests/scenario/defaults/vanilla-2.7.1.yaml.mako

The most useful and comfortable way to run sahara-scenario tests for Vanilla Plugin:

$ tox -e venv -- sahara-scenario -V templatevars.ini sahara_tests/scenario/defaults/credantials.yaml.mako -p vanilla -v 2.7.1

For more information about writing scenario YAML files, see the section section How to write scenario files.

Virtual environment flags

You can use the following flags to the Sahara scenario tests.

Optional arguments

Arguments Description
--help, -h show help message and exit
--variable_file, -V path to the file with template variables
--verbose increase output verbosity
--validate validate yaml-files, tests will not be run
--args ARGS pairs of argument key:value
--plugin, -p PLUGIN specify plugin name
--plugin_version, -v PLUGIN_VERSION specify plugin version
--release, -r RELEASE specify Sahara release
--report write results to file
--count COUNT specify count of runs
--os-cloud name name of cloud to connect
--os-auth-type, --os-auth-plugin name authentication type to use

Authentication options

Options specific to the password plugin.

Arguments Description
--os-auth-url OS_AUTH_URL authentication URL
--os-domain-id OS_DOMAIN_ID domain ID to scope to
--os-domain-name OS_DOMAIN_NAME domain name to scope to
--os-project-id --os-tenant-id OS_PROJECT_ID project ID to scope to
--os-project-name --os-tenant-name OS_PROJECT_NAME project name to scope to
--os-project-domain-id OS_PROJECT_DOMAIN_ID domain ID containing project
--os-project-domain-name OS_PROJECT_DOMAIN_NAME domain name containing project
--os-trust-id OS_TRUST_ID trust ID
--os-default-domain-id OS_DEFAULT_DOMAIN_ID optional domain ID to use with v3 and v2 parameters. It will be used for both the user and project domain in v3 and ignored in v2 authentication.
--os-default-domain-name OS_DEFAULT_DOMAIN_NAME optional domain name to use with v3 API and v2parameters. It will be used for both the user and project domain in v3 and ignored in v2 authentication.
--os-user-id OS_USER_ID user ID
--os-username, --os-user-name OS_USERNAME username
--os-user-domain-id OS_USER_DOMAIN_ID user's domain id
--os-user-domain-name OS_USER_DOMAIN_NAME user's domain name
--os-password OS_PASSWORD user's password

API Connection Options

Options controlling the HTTP API connections.

Arguments Description
--insecure explicitly allow client to perform "insecure" TLS (https) requests. The server's certificate will not be verified against any certificate authorities. This option should be used with caution.
--os-cacert <ca-certificate> specify a CA bundle file to use in verifying a TLS(https) server certificate. Defaults to env [OS_CACERT].
--os-cert <certificate> defaults to env[OS_CERT]
--os-key <key> defaults to env[OS_KEY]
--timeout <seconds> set request timeout (in seconds)

Service Options

Options control the specialization of the API connection from information found in the catalog.

Arguments Description
--os-service-type <name> service type to request from the catalog
--os-service-name <name> service name to request from the catalog
--os-interface <name> API Interface to use: [public, internal, admin]
--os-region-name <name> region of the cloud to use
--os-endpoint-override <name> endpoint to use instead of the endpoint in the catalog
--os-api-version <name> which version of the service API to use

How to write scenario files

The example of full scenario file with all these parameters you can find in etc/scenario/simple-testcase.yaml.

You can write all sections in one or several files, which can be simple YAML files or YAML-based Mako templates (.yaml.mako or yml.mako). Fox example, the most common sections you can keep in templatevars.ini and sahara_tests/scenario/defaults/credentials.yaml.mako.

Field "concurrency"

This field has integer value, and set concurrency for run tests

For example:
concurrency: 2

For parallel testing use flag --count in run command and setup cuncurrency value

Section "credentials"

This section is dictionary-type.

Fields Type Required Default Value
sahara_service_type string   data-processing service type for sahara
sahara_url string   None url of sahara
ssl_cert string   None ssl certificate for all clients
ssl_verify boolean   False enable verify ssl for sahara

Section "network"

This section is dictionary-type.

Fields Type Required Default Value
private_network string True private name or id of private network
public_network string True public name or id of private network
type string   neutron "neutron" or "nova-network"
auto_assignment_floating_ip boolean   False  

Section "clusters"

This sections is an array-type.

Fields Type Required Default Value
plugin_name string True   name of plugin
plugin_version string True   version of plugin
image string True   name or id of image
image_username string     username for registering image
existing_cluster string     cluster name or id for testing
key_name string     name of registered ssh key for testing cluster
node_group_templates object     see section "node_group_templates"
cluster_template object     see section "cluster_template"
cluster object     see section "cluster"
scaling object     see section "scaling"
timeout_check_transient integer   300 timeout for checking transient
timeout_poll_jobs_status integer   1800 timeout for polling jobs state
timeout_delete_resource integer   300 timeout for delete resource
timeout_poll_cluster_status integer   3600 timeout for polling cluster state
scenario array   ['run_jobs', 'scale', 'run_jobs'] array of checks
edp_jobs_flow string     name of edp job flow
hdfs_username string   hadoop username for hdfs
retain_resources boolean   False  

Section "node_group_templates"

This section is an array-type.

Fields Type Required Default Value
name string True   name for node group template
flavor string or object True   name or id of flavor, or see section "flavor"
node_processes string True   name of process
description string   Empty description for node group
volumes_per_node integer   0 minimum 0
volumes_size integer   0 minimum 0
auto_security_group boolean   True  
security_group array     security group
node_configs object     name_of_config_section: config: value
availability_zone string      
volumes_availability_zone string      
volume_type string      
is_proxy_gateway boolean   False use this node as proxy gateway
edp_batching integer   count jobs use for batching jobs

Section "flavor"

This section is an dictionary-type.

Fields Type Required Default Value
name string   auto-generate name for flavor
id string   auto-generate id for flavor
vcpus integer   1 number of VCPUs for the flavor
ram integer   1 memory in MB for the flavor
root_disk integer   0 size of local disk in GB
ephemeral_disk integer   0 ephemeral space in MB
swap_disk integer   0 swap space in MB

Section "cluster_template"

This section is dictionary-type.

Fields Type Required Default Value
name string     name for cluster template
description string   Empty description
cluster_configs object     name_of_config_section: config: value
node_group_templates object True   name_of_node_group: count
anti_affinity array   Empty array of roles

Section "cluster"

This section is dictionary-type.

Fields Type Required Default Value
name string   Empty name for cluster
description string   Empty description
is_transient boolean   False value

Section "scaling"

This section is an array-type.

Fields Type Required Default Value
operation string True   "add" or "resize"
node_group string True Empty name of node group
size integer True Empty count node group

Section "edp_jobs_flow"

This section has an object with a name from the section "clusters" field "edp_jobs_flows" Object has sections of array-type. Required: type

Fields Type Required Default Value
type string True   "Pig", "Java", "MapReduce", "MapReduce.Streaming", "Hive", "Spark", "Shell"
input_datasource object     see section "input_datasource"
output_datasource object     see section "output_datasource"
main_lib object     see section "main_lib"
additional_libs object     see section "additional_libs"
configs dict   Empty config: value
args array   Empty array of args

Section "input_datasource"

Required: type, source This section is dictionary-type.

Fields Type Required Default Value
type string True   "swift", "hdfs", "maprfs"
hdfs_username string     username for hdfs
source string True   uri of source

Section "output_datasource"

Required: type, destination This section is dictionary-type.

Fields Type Required Default Value
type string True   "swift", "hdfs", "maprfs"
destination string True   uri of source

Section "main_lib"

Required: type, source This section is dictionary-type.

Fields Type Required Default Value
type string True   "swift or "database"
source string True   uri of source

Section "additional_libs"

Required: type, source This section is an array-type.

Fields Type Required Default Value
type string True   "swift or "database"
source string True   uri of source
Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.