Testing

Zuul provides an extensive framework for performing functional testing on the system from end-to-end with major external components replaced by fakes for ease of use and speed.

Test classes that subclass ZuulTestCase have access to a number of attributes useful for manipulating or inspecting the environment being simulated in the test:

tests.base.simple_layout(path, driver='gerrit')

Specify a layout file for use by a test method.

Parameters:

  • path (str) – The path to the layout file.

  • driver (str) – The source driver to use, defaults to gerrit.

Some tests require only a very simple configuration. For those, establishing a complete config directory hierachy is too much work. In those cases, you can add a simple zuul.yaml file to the test fixtures directory (in fixtures/layouts/foo.yaml) and use this decorator to indicate the test method should use that rather than the tenant config file specified by the test class.

The decorator will cause that layout file to be added to a config-project called “common-config” and each “project” instance referenced in the layout file will have a git repo automatically initialized.

class tests.base.ZuulTestCase(*args, **kwargs)

A test case with a functioning Zuul.

The following class variables are used during test setup and can be overidden by subclasses but are effectively read-only once a test method starts running:

Variables:

  • config_file (str) – This points to the main zuul config file within the fixtures directory. Subclasses may override this to obtain a different behavior.

  • tenant_config_file (str) – This is the tenant config file (which specifies from what git repos the configuration should be loaded). It defaults to the value specified in config_file but can be overidden by subclasses to obtain a different tenant/project layout while using the standard main configuration. See also the simple_layout() decorator.

  • tenant_config_script_file (str) – This is the tenant config script file. This attribute has the same meaning than tenant_config_file except that the tenant configuration is loaded from a script. When this attribute is set then tenant_config_file is ignored by the scheduler.

  • create_project_keys (bool) – Indicates whether Zuul should auto-generate keys for each project, or whether the test infrastructure should insert dummy keys to save time during startup. Defaults to False.

  • log_console_port (int) – The zuul_stream/zuul_console port.

The following are instance variables that are useful within test methods:

Variables:

  • fake_<connection> (FakeGerritConnection) – A FakeGerritConnection will be instantiated for each connection present in the config file and stored here. For instance, fake_gerrit will hold the FakeGerritConnection object for a connection named gerrit.

  • executor_server (RecordingExecutorServer) – An instance of RecordingExecutorServer which is the Ansible execute server used to run jobs for this test.

  • builds (list) – A list of FakeBuild objects representing currently running builds. They are appended to the list in the order they are executed, and removed from this list upon completion.

  • history (list) – A list of BuildHistory objects representing completed builds. They are appended to the list in the order they complete.

addEvent(connection, event)

Inject a Fake (Gerrit) event.

This method accepts a JSON-encoded event and simulates Zuul having received it from Gerrit. It could (and should) eventually apply to any connection type, but is currently only used with Gerrit connections. The name of the connection is used to look up the corresponding server, and the event is simulated as having been received by all Zuul connections attached to that server. So if two Gerrit connections in Zuul are connected to the same Gerrit server, and you invoke this method specifying the name of one of them, the event will be received by both.

Note

“self.fake_gerrit.addEvent” calls should be migrated to this method.

Parameters:

  • connection (str) – The name of the connection corresponding to the gerrit server.

  • event (str) – The JSON-encoded event.

assertBuilds(builds)

Assert that the running builds are as described.

The list of running builds is examined and must match exactly the list of builds described by the input.

Parameters:

builds (list) – A list of dictionaries. Each item in the list must match the corresponding build in the build history, and each element of the dictionary must match the corresponding attribute of the build.

assertHistory(history, ordered=True)

Assert that the completed builds are as described.

The list of completed builds is examined and must match exactly the list of builds described by the input.

Parameters:

  • history (list) – A list of dictionaries. Each item in the list must match the corresponding build in the build history, and each element of the dictionary must match the corresponding attribute of the build.

  • ordered (bool) – If true, the history must match the order supplied, if false, the builds are permitted to have arrived in any order.

assertReportedStat(key, value=None, kind=None, timeout=5)

Check statsd output

Check statsd return values. A value should specify a kind, however a kind may be specified without a value for a generic match. Leave both empy to just check for key presence.

Parameters:

  • key (str) – The statsd key

  • value (str) – The expected value of the metric key

  • kind (str) –

    The expected type of the metric key For example

    • c counter

    • g gauge

    • ms timing

    • s set

  • timeout (int) – How long to wait for the stat to appear

Returns:

The value

commitConfigUpdate(project_name, source_name)

Commit an update to zuul.yaml

This overwrites the zuul.yaml in the specificed project with the contents specified.

Parameters:

  • project_name (str) – The name of the project containing zuul.yaml (e.g., common-config)

  • source_name (str) – The path to the file (underneath the test fixture directory) whose contents should be used to replace zuul.yaml.

getSortedBuilds()

Return the list of currently running builds sorted by name

getUpstreamRepos(projects)

Return upstream git repo objects for the listed projects

Parameters:

projects (list) – A list of strings, each the canonical name of a project.

Returns:

A dictionary of {name: repo} for every listed project.

Return type:

dict

logState()

Log the current state of the system

newTenantConfig(source_name)

Use this to update the tenant config file in tests

This will update self.tenant_config_file to point to a temporary file for the duration of this particular test. The content of that file will be taken from FIXTURE_DIR/source_name

After the test the original value of self.tenant_config_file will be restored.

Parameters:

source_name (str) – The path of the file under FIXTURE_DIR that will be used to populate the new tenant config file.

printHistory()

Log the build history.

This can be useful during tests to summarize what jobs have completed.

setUp()

Hook method for setting up the test fixture before exercising it.

waitUntilNodeCacheSync(zk_nodepool)

Wait until the node cache on the zk_nodepool object is in sync

class tests.base.FakeGerritConnection(driver, connection_name, connection_config, changes_db=None, upstream_root=None, poller_event=None, ref_watcher_event=None)

A Fake Gerrit connection for use in tests.

This subclasses GerritConnection to add the ability for tests to add changes to the fake Gerrit it represents.

addFakeChange(project, branch, subject, status='NEW', files=None, parent=None, merge_parents=None, merge_files=None, topic=None, empty=False)

Add a change to the fake Gerrit.

checkBranchCache(project_name, event, protected=None)

Clear the cache for a project when a branch event is processed

This method must be called when a branch event is processed: if the event references a branch and the unprotected branches are excluded, the branch protection status could have been changed.

Params str project_name:

The project name.

Params event:

The event, inherit from zuul.model.TriggerEvent class.

Params protected:

If defined the caller already knows if the branch is protected so the query can be skipped.

Return type:

None

cleanupCache()

Clean up the connection cache.

This allows a connection to perform periodic cleanup actions of the cache, e.g. garbage collection.

clearBranchCache(projects=None)

Clear the branch cache

In case the branch cache gets out of sync with the source, this method can be called to clear it and force querying the source the next time the cache is used.

clearConnectionCacheOnBranchEvent(event)

Update event and clear connection cache if needed.

This checks whether the event created or deleted a branch so that Zuul may know to perform a reconfiguration on the project. Drivers must call this method when a branch event is received.

Parameters:

event – The event, inherit from zuul.model.TriggerEvent class.

getEventQueue()

Return the event queue for this connection.

Returns:

A zuul.zk.event_queues.ConnectionEventQueue instance or None.

getProjectBranches(project, tenant, min_ltime=-1)

Get the branch names for the given project.

Parameters:

  • project (zuul.model.Project) – The project for which the branches are returned.

  • tenant (zuul.model.Tenant) – The related tenant.

  • min_ltime (int) – The minimum ltime to determine if we need to refresh the cache.

Returns:

The list of branch names.

getProjectMergeModes(project, tenant, min_ltime=-1)

Get the merge modes for the given project.

Parameters:

  • project (zuul.model.Project) – The project for which the merge modes are returned.

  • tenant (zuul.model.Tenant) – The related tenant.

  • min_ltime (int) – The minimum ltime to determine if we need to refresh the cache.

Returns:

The list of merge modes by model id.

getWebController(zuul_web)

Return a cherrypy web controller to register with zuul-web.

Parameters:

zuul_web (zuul.web.ZuulWeb) – Zuul Web instance.

Returns:

A zuul.web.handler.BaseWebController instance.

isBranchProtected(project_name, branch_name, zuul_event_id=None)

Return if the branch is protected or None if the branch is unknown.

Parameters:

  • project_name (str) – The name of the project.

  • branch_name (str) – The name of the branch.

  • zuul_event_id – Unique id associated to the handled event.

maintainCache(relevant, max_age)

Remove stale changes from the cache.

This lets the user supply a list of change cache keys that are still in use. Anything in our cache that isn’t in the supplied list and is older than the given max. age (in seconds) should be safe to remove from the cache.

toDict()

Return public information about the connection

updateProjectBranches(project)

Update the branch cache for the project.

Parameters:

project (zuul.model.Project) – The project for which the branches are returned.

validateWebConfig(config, connections)

Validate web config.

If there is a fatal error, the method should raise an exception.

Parameters:

  • config – The parsed config object.

  • connections (zuul.lib.connections.ConnectionRegistry) – Registry of all configured connections.

class tests.base.RecordingExecutorServer(*args, **kw)

An Ansible executor to be used in tests.

Variables:

hold_jobs_in_build (bool) – If true, when jobs are executed they will report that they have started but then pause until released before reporting completion. This attribute may be changed at any time and will take effect for subsequently executed builds, but previously held builds will still need to be explicitly released.

failJob(name, change)

Instruct the executor to report matching builds as failures.

Parameters:

  • name (str) – The name of the job to fail.

  • change (Change) – The FakeChange instance which should cause the job to fail. This job will also fail for changes depending on this change.

release(regex=None, change=None)

Release a held build.

Parameters:

regex (str) – A regular expression which, if supplied, will cause only builds with matching names to be released. If not supplied, all builds will be released.

retryJob(name, change, retries=None)

Instruct the executor to report matching builds as retries.

Parameters:

  • name (str) – The name of the job to fail.

  • change (Change) – The FakeChange instance which should cause the job to fail. This job will also fail for changes depending on this change.

returnData(name, change, data)

Instruct the executor to return data for this build.

Parameters:

  • name (str) – The name of the job to return data.

  • change (Change) – The FakeChange instance which should cause the job to return data.

  • data (dict) – The data to return

class tests.base.FakeBuild(executor_server, build_request, params)
getWorkspaceRepos(projects)

Return workspace git repo objects for the listed projects

Parameters:

projects (list) – A list of strings, each the canonical name of a project.

Returns:

A dictionary of {name: repo} for every listed project.

Return type:

dict

hasChanges(*changes)

Return whether this build has certain changes in its git repos.

Parameters:

changes (FakeChange) – One or more changes (varargs) that are expected to be present (in order) in the git repository of the active project.

Returns:

Whether the build has the indicated changes.

Return type:

bool

isWaiting()

Return whether this build is being held.

Returns:

Whether the build is being held.

Return type:

bool

release()

Release this build.

class tests.base.BuildHistory(**kw)