Integration tests are executed by
ansible-test and reside in directories under
Each test MUST have an
aliases file to control test execution.
Aliases are explained in the following sections. Each alias must be on a separate line in an
Tests must be configured to run in exactly one group. This is done by adding the appropriate group to the
The following are examples of some of the available groups:
Groups are used to balance tests across multiple CI jobs to minimize test run time. They also improve efficiency by keeping tests with similar requirements running together.
When selecting a group for a new test, use the same group as existing tests similar to the one being added. If more than one group is available, select one randomly.
Aliases can be used to express some test requirements:
--docker-privilegedwhen running tests with
needs/root- Requires running tests as
needs/ssh- Requires SSH connections to localhost (or the test container with
--docker) without a password.
Aliases can be used to skip platforms using one of the following:
skip/freebsd- Skip tests on FreeBSD.
skip/osx- Skip tests on macOS / OS X.
skip/rhel- Skip tests on RHEL.
Aliases can be used to skip Python major versions using one of the following:
skip/python2- Skip tests on Python 2.x.
skip/python3- Skip tests on Python 3.x.
For more fine grained skipping, use conditionals in integration test playbooks, such as:
when: ansible_distribution in ('Ubuntu')
Tests which fail sometimes should be marked with the
unstable alias until the instability has been fixed.
These tests will continue to run for pull requests which modify the test or the module under test.
This avoids unnecessary test failures for other pull requests, as well as tests on merge runs and nightly CI jobs.
There are two ways to run unstable tests manually:
unstable/when passing it to
Tests will be marked as unstable by a member of the Ansible Core Team. GitHub issues will be created to track each unstable test.
Tests which always fail should be marked with the
disabled alias until they can be fixed.
Disabled tests are automatically skipped.
There are two ways to run disabled tests manually:
disabled/when passing it to
Tests will be marked as disabled by a member of the Ansible Core Team. GitHub issues will be created to track each disabled test.
Tests which cannot be run in CI should be marked with the
Most tests can be supported through the use of simulators and/or cloud plugins.
However, if that is not possible then marking a test as unsupported will prevent it from running in CI.
There are two ways to run unsupported tests manually:
unsupported/when passing it to
Tests will be marked as unsupported by the contributor of the test.
Tests for cloud services and other modules that require access to external APIs usually require special support for testing in CI.
These require an additional alias to indicate the required test plugin.
Some of the available aliases are:
Every module and plugin should have integration tests, even if the tests cannot be run in CI.
For questions about integration tests reach out to @mattclay or @gundalow on GitHub or
#ansible-devel on IRC.