UTHelper Guide
Introduction
UTHelper
was written to reduce the boilerplate code used in unit tests for modules.
It was originally written to handle tests of modules that run external commands using AnsibleModule.run_command()
.
At the time of writing (Feb 2025) that remains the only type of tests you can use
UTHelper
for, but it aims to provide support for other types of interactions.
Until now, there are many different ways to implement unit tests that validate a module based on the execution of external commands. See some examples:
test_apk.py - A very simple one
test_bootc_manage.py - This one has more test cases, but do notice how the code is repeated amongst them.
test_modprobe.py - This one has 15 tests in it, but to achieve that it declares 8 classes repeating quite a lot of code.
As you can notice, there is no consistency in the way these tests are executed - they all do the same thing eventually, but each one is written in a very distinct way.
UTHelper
aims to:
provide a consistent idiom to define unit tests
reduce the code to a bare minimal, and
define tests as data instead
allow the test cases definition to be expressed not only as a Python data structure but also as YAML content
Quickstart
To use UTHelper, your test module will need only a bare minimal of code:
# tests/unit/plugin/modules/test_ansible_module.py
from ansible_collections.community.general.plugins.modules import ansible_module
from .uthelper import UTHelper, RunCommandMock
UTHelper.from_module(ansible_module, __name__, mocks=[RunCommandMock])
Then, in the test specification file, you have:
# tests/unit/plugin/modules/test_ansible_module.yaml
test_cases:
- id: test_ansible_module
flags:
diff: true
input:
state: present
name: Roger the Shrubber
output:
shrubbery:
looks: nice
price: not too expensive
changed: true
diff:
before:
shrubbery: null
after:
shrubbery:
looks: nice
price: not too expensive
mocks:
run_command:
- command: [/testbin/shrubber, --version]
rc: 0
out: "2.80.0\n"
err: ''
- command: [/testbin/shrubber, --make-shrubbery]
rc: 0
out: 'Shrubbery created'
err: ''
Note
If you prefer to pick a different YAML file for the test cases, or if you prefer to define them in plain Python,
you can use the convenience methods UTHelper.from_file()
and UTHelper.from_spec()
, respectively.
See more details below.
Using UTHelper
Test Module
UTHelper
is strictly for unit tests. To use it, you import the .uthelper.UTHelper
class.
As mentioned in different parts of this guide, there are three different mechanisms to load the test cases.
See also
See the UTHelper class reference below for API details on the three different mechanisms.
The easies and most recommended way of using UTHelper
is literally the example shown.
See a real world example at
test_gconftool2.py.
The from_module()
method will pick the filename of the test module up (in the example above, tests/unit/plugins/modules/test_gconftool2.py
)
and it will search for tests/unit/plugins/modules/test_gconftool2.yaml
(or .yml
if that is not found).
In that file it will expect to find the test specification expressed in YAML format, conforming to the structure described below LINK LINK LINK.
If you prefer to read the test specifications a different file path, use from_file()
passing the file handle for the YAML file.
And, if for any reason you prefer or need to pass the data structure rather than dealing with YAML files, use the from_spec()
method.
A real world example for that can be found at
test_snap.py.
Test Specification
The structure of the test specification data is described below.
Top level
At the top level there are two accepted keys:
anchors: dict
Optional. Placeholder for you to define YAML anchors that can be repeated in the test cases. Its contents are never accessed directly by test Helper.
test_cases: list
Mandatory. List of test cases, see below for definition.
Test cases
You write the test cases with five elements:
id: str
Mandatory. Used to identify the test case.
flags: dict
Optional. Flags controling the behavior of the test case. All flags are optional. Accepted flags:
check: bool
: set totrue
if the module is to be executed in check mode.diff: bool
: set totrue
if the module is to be executed in diff mode.skip: str
: set the test case to be skipped, providing the message forpytest.skip()
.xfail: str
: set the test case to expect failure, providing the message forpytest.xfail()
.
input: dict
Optional. Parameters for the Ansible module, it can be empty.
output: dict
Optional. Expected return values from the Ansible module. All RV names are used here are expected to be found in the module output, but not all RVs in the output must be here. It can include special RVs such as
changed
anddiff
. It can be empty.
mocks: dict
Optional. Mocked interactions,
run_command
being the only one supported for now. Each key in this dictionary refers to one subclass ofTestCaseMock
and its structure is dictated by theTestCaseMock
subclass implementation. All keys are expected to be named using snake case, as inrun_command
. TheTestCaseMock
subclass is responsible for defining the name used in the test specification. The structure for that specification is dependent on the implementing class. See more details below for the implementation ofRunCommandMock
Example using YAML
We recommend you use UTHelper
reading the test specifications from a YAML file.
See an example below of how one actually looks like (excerpt from test_opkg.yaml
):
---
anchors:
environ: &env-def {environ_update: {LANGUAGE: C, LC_ALL: C}, check_rc: false}
test_cases:
- id: install_zlibdev
input:
name: zlib-dev
state: present
output:
msg: installed 1 package(s)
mocks:
run_command:
- command: [/testbin/opkg, --version]
environ: *env-def
rc: 0
out: ''
err: ''
- command: [/testbin/opkg, list-installed, zlib-dev]
environ: *env-def
rc: 0
out: ''
err: ''
- command: [/testbin/opkg, install, zlib-dev]
environ: *env-def
rc: 0
out: |
Installing zlib-dev (1.2.11-6) to root...
Downloading https://downloads.openwrt.org/releases/22.03.0/packages/mips_24kc/base/zlib-dev_1.2.11-6_mips_24kc.ipk
Installing zlib (1.2.11-6) to root...
Downloading https://downloads.openwrt.org/releases/22.03.0/packages/mips_24kc/base/zlib_1.2.11-6_mips_24kc.ipk
Configuring zlib.
Configuring zlib-dev.
err: ''
- command: [/testbin/opkg, list-installed, zlib-dev]
environ: *env-def
rc: 0
out: |
zlib-dev - 1.2.11-6
err: ''
- id: install_zlibdev_present
input:
name: zlib-dev
state: present
output:
msg: package(s) already present
mocks:
run_command:
- command: [/testbin/opkg, --version]
environ: *env-def
rc: 0
out: ''
err: ''
- command: [/testbin/opkg, list-installed, zlib-dev]
environ: *env-def
rc: 0
out: |
zlib-dev - 1.2.11-6
err: ''
TestCaseMocks Specifications
The TestCaseMock
subclass is free to define the expected data structure.
RunCommandMock Specification
RunCommandMock
mocks can be specified with the key run_command
and it expects a list
in which elements follow the structure:
command: Union[list, str]
Mandatory. The command that is expected to be executed by the module. It corresponds to the parameter
args
of theAnsibleModule.run_command()
call. It can be either a list or a string, though the list form is generally recommended.
environ: dict
Mandatory. All other parameters passed to the
AnsibleModule.run_command()
call. Most commonly used areenviron_update
andcheck_rc
. Must include all parameters the Ansible module uses in theAnsibleModule.run_command()
call, otherwise the test will fail.
rc: int
Mandatory. The return code for the command execution. As per usual in bash scripting, a value of
0
means success, whereas any other number is an error code.
out: str
Mandatory. The stdout result of the command execution, as one single string containing zero or more lines.
err: str
Mandatory. The stderr result of the command execution, as one single string containing zero or more lines.
UTHelper
Reference
- class .uthelper.UTHelper
A class to encapsulate unit tests.
- static from_spec(ansible_module, test_module, test_spec, mocks=None)
Creates an
UTHelper
instance from a given test specification.- Parameters:
- Returns:
An
UTHelper
instance.- Return type:
Example usage of
from_spec()
:import sys from ansible_collections.community.general.plugins.modules import ansible_module from .uthelper import UTHelper, RunCommandMock TEST_SPEC = dict( test_cases=[ ... ] ) helper = UTHelper.from_spec(ansible_module, sys.modules[__name__], TEST_SPEC, mocks=[RunCommandMock])
- static from_file(ansible_module, test_module, test_spec_filehandle, mocks=None)
Creates an
UTHelper
instance from a test specification file.- Parameters:
ansible_module (module) – The Ansible module to be tested.
test_module (module) – The test module.
test_spec_filehandle (file) – A file handle to an file stream handle providing the test specification in YAML format.
mocks (list or None) – List of
TestCaseMocks
to be used during testing. Currently onlyRunCommandMock
exists.
- Returns:
An
UTHelper
instance.- Return type:
Example usage of
from_file()
:import sys from ansible_collections.community.general.plugins.modules import ansible_module from .uthelper import UTHelper, RunCommandMock with open("test_spec.yaml", "r") as test_spec_filehandle: helper = UTHelper.from_file(ansible_module, sys.modules[__name__], test_spec_filehandle, mocks=[RunCommandMock])
- static from_module(ansible_module, test_module_name, mocks=None)
Creates an
UTHelper
instance from a given Ansible module and test module.- Parameters:
- Returns:
An
UTHelper
instance.- Return type:
Example usage of
from_module()
:from ansible_collections.community.general.plugins.modules import ansible_module from .uthelper import UTHelper, RunCommandMock # Example usage helper = UTHelper.from_module(ansible_module, __name__, mocks=[RunCommandMock])
Creating TestCaseMocks
To create a new TestCaseMock
you must extend that class and implement the relevant parts:
class ShrubberyMock(TestCaseMock):
# this name is mandatory, it is the name used in the test specification
name = "shrubbery"
def setup(self, mocker):
# perform setup, commonly using mocker to patch some other piece of code
...
def check(self, test_case, results):
# verify the tst execution met the expectations of the test case
# for example the function was called as many times as it should
...
def fixtures(self):
# returns a dict mapping names to pytest fixtures that should be used for the test case
# for example, in RunCommandMock it creates a fixture that patches AnsibleModule.get_bin_path
...
Caveats
Known issues/opportunities for improvement:
Only one
UTHelper
per test module: UTHelper injects a test function with a fixed name into the module’s namespace, so placing a secondUTHelper
instance is going to overwrite the function created by the first one.Order of elements in module’s namespace is not consistent across executions in Python 3.5, so if adding more tests to the test module might make Test Helper add its function before or after the other test functions. In the community.general collection the CI processes uses
pytest-xdist
to paralellize and distribute the tests, and it requires the order of the tests to be consistent.
New in version 7.5.0.