Command Runner guide
Introduction
The ansible_collections.community.general.plugins.module_utils.cmd_runner
module util provides the
CmdRunner
class to help execute external commands. The class is a wrapper around
the standard AnsibleModule.run_command()
method, handling command arguments, localization setting,
output processing output, check mode, and other features.
It is even more useful when one command is used in multiple modules, so that you can define all options in a module util file, and each module uses the same runner with different arguments.
For the sake of clarity, throughout this guide, unless otherwise specified, we use the term option when referring to Ansible module options, and the term argument when referring to the command line arguments for the external command.
Quickstart
CmdRunner
defines a command and a set of coded instructions on how to format
the command-line arguments, in which specific order, for a particular execution.
It relies on ansible.module_utils.basic.AnsibleModule.run_command()
to actually execute the command.
There are other features, see more details throughout this document.
To use CmdRunner
you must start by creating an object. The example below is a simplified
version of the actual code in community.general.ansible_galaxy_install:
from ansible_collections.community.general.plugins.module_utils.cmd_runner import CmdRunner, cmd_runner_fmt
runner = CmdRunner(
module,
command="ansible-galaxy",
arg_formats=dict(
type=cmd_runner_fmt.as_func(lambda v: [] if v == 'both' else [v]),
galaxy_cmd=cmd_runner_fmt.as_list(),
upgrade=cmd_runner_fmt.as_bool("--upgrade"),
requirements_file=cmd_runner_fmt.as_opt_val('-r'),
dest=cmd_runner_fmt.as_opt_val('-p'),
force=cmd_runner_fmt.as_bool("--force"),
no_deps=cmd_runner_fmt.as_bool("--no-deps"),
version=cmd_runner_fmt.as_fixed("--version"),
name=cmd_runner_fmt.as_list(),
)
)
This is meant to be done once, then every time you need to execute the command you create a context and pass values as needed:
# Run the command with these arguments, when values exist for them
with runner("type galaxy_cmd upgrade force no_deps dest requirements_file name", output_process=process) as ctx:
ctx.run(galaxy_cmd="install", upgrade=upgrade)
# version is fixed, requires no value
with runner("version") as ctx:
dummy, stdout, dummy = ctx.run()
# passes arg 'data' to AnsibleModule.run_command()
with runner("type name", data=stdin_data) as ctx:
dummy, stdout, dummy = ctx.run()
# Another way of expressing it
dummy, stdout, dummy = runner("version").run()
Note that you can pass values for the arguments when calling run()
, otherwise CmdRunner
uses the module options with the exact same names to provide values for the runner arguments.
If no value is passed and no module option is found for the name specified, then an exception is raised, unless
the argument is using cmd_runner_fmt.as_fixed
as format function like the version
in the example above.
See more about it below.
In the first example, values of type
, force
, no_deps
and others
are taken straight from the module, whilst galaxy_cmd
and upgrade
are
passed explicitly.
Note
It is not possible to automatically retrieve values of suboptions.
That generates a resulting command line similar to (example taken from the output of an integration test):
[
"<venv>/bin/ansible-galaxy",
"collection",
"install",
"--upgrade",
"-p",
"<collection-install-path>",
"netbox.netbox",
]
Argument formats
As seen in the example, CmdRunner
expects a parameter named arg_formats
defining how to format each CLI named argument.
An “argument format” is nothing but a function to transform the value of a variable
into something formatted for the command line.
Argument format function
An arg_format
function is defined in the form similar to:
def func(value):
return ["--some-param-name", value]
The parameter value
can be of any type - although there are convenience
mechanisms to help handling sequence and mapping objects.
The result is expected to be of the type Sequence[str]
type (most commonly
list[str]
or tuple[str]
), otherwise it is considered to be a str
,
and it is coerced into list[str]
.
This resulting sequence of strings is added to the command line when that
argument is actually used.
For example, if func
returns:
["nee", 2, "shruberries"]
, the command line adds arguments"nee" "2" "shruberries"
.2 == 2
, the command line adds argumentTrue
.None
, the command line adds argumentNone
.[]
, the command line adds no command line argument for that particular argument.
Convenience format methods
In the same module as CmdRunner
there is a class cmd_runner_fmt
which
provides a set of convenience methods that return format functions for common cases.
In the first block of code in the Quickstart section you can see the importing of
that class:
from ansible_collections.community.general.plugins.module_utils.cmd_runner import CmdRunner, cmd_runner_fmt
The same example shows how to make use of some of them in the instantiation of the CmdRunner
object.
A description of each one of the convenience methods available and examples of how to use them is found below.
In these descriptions value
refers to the single parameter passed to the formatting function.
cmd_runner_fmt.as_list()
This method does not receive any parameter, function returns
value
as-is.- Creation:
cmd_runner_fmt.as_list()
- Examples:
Value
Outcome
["foo", "bar"]
["foo", "bar"]
"foobar"
["foobar"]
cmd_runner_fmt.as_bool()
This method receives two different parameters:
args_true
andargs_false
, latter being optional. If the boolean evaluation ofvalue
isTrue
, the format function returnsargs_true
. If the boolean evaluation isFalse
, then the function returnsargs_false
if it was provided, or[]
otherwise.- Creation (one arg):
cmd_runner_fmt.as_bool("--force")
- Examples:
Value
Outcome
True
["--force"]
False
[]
- Creation (two args,
None
treated asFalse
): cmd_runner_fmt.as_bool("--relax", "--dont-do-it")
- Creation (two args,
- Examples:
Value
Outcome
True
["--relax"]
False
["--dont-do-it"]
["--dont-do-it"]
- Creation (two args,
None
is ignored): cmd_runner_fmt.as_bool("--relax", "--dont-do-it", ignore_none=True)
- Creation (two args,
- Examples:
Value
Outcome
True
["--relax"]
False
["--dont-do-it"]
[]
cmd_runner_fmt.as_bool_not()
This method receives one parameter, which is returned by the function when the boolean evaluation of
value
isFalse
.- Creation:
cmd_runner_fmt.as_bool_not("--no-deps")
- Examples:
Value
Outcome
True
[]
False
["--no-deps"]
cmd_runner_fmt.as_optval()
This method receives one parameter
arg
, the function returns the string concatenation ofarg
andvalue
.- Creation:
cmd_runner_fmt.as_optval("-i")
- Examples:
Value
Outcome
3
["-i3"]
foobar
["-ifoobar"]
cmd_runner_fmt.as_opt_val()
This method receives one parameter
arg
, the function returns[arg, value]
.- Creation:
cmd_runner_fmt.as_opt_val("--name")
- Examples:
Value
Outcome
abc
["--name", "abc"]
cmd_runner_fmt.as_opt_eq_val()
This method receives one parameter
arg
, the function returns the string of the form{arg}={value}
.- Creation:
cmd_runner_fmt.as_opt_eq_val("--num-cpus")
- Examples:
Value
Outcome
10
["--num-cpus=10"]
cmd_runner_fmt.as_fixed()
This method receives one parameter
arg
, the function expects novalue
- if one is provided then it is ignored. The function returnsarg
as-is.- Creation:
cmd_runner_fmt.as_fixed("--version")
- Examples:
Value
Outcome
["--version"]
57
["--version"]
- Note:
This is the only special case in which a value can be missing for the formatting function. The example also comes from the code in Quickstart. In that case, the module has code to determine the command’s version so that it can assert compatibility. There is no value to be passed for that CLI argument.
cmd_runner_fmt.as_map()
This method receives one parameter
arg
which must be a dictionary, and an optional parameterdefault
. The function returns the evaluation ofarg[value]
. Ifvalue not in arg
, then it returnsdefault
if defined, otherwise[]
.- Creation:
cmd_runner_fmt.as_map(dict(a=1, b=2, c=3), default=42)
- Examples:
Value
Outcome
"b"
["2"]
"yabadabadoo"
["42"]
- Note:
If
default
is not specified, invalid values return an empty list, meaning they are silently ignored.
cmd_runner_fmt.as_func()
This method receives one parameter
arg
which is itself is a format function and it must abide by the rules described above.- Creation:
cmd_runner_fmt.as_func(lambda v: [] if v == 'stable' else ['--channel', '{0}'.format(v)])
- Note:
The outcome for that depends entirely on the function provided by the developer.
Other features for argument formatting
Some additional features are available as decorators:
cmd_runner_fmt.unpack args()
This decorator unpacks the incoming
value
as a list of elements.For example, in
ansible_collections.community.general.plugins.module_utils.puppet
, it is used as:@cmd_runner_fmt.unpack_args def execute_func(execute, manifest): if execute: return ["--execute", execute] else: return [manifest] runner = CmdRunner( module, command=_prepare_base_cmd(), path_prefix=_PUPPET_PATH_PREFIX, arg_formats=dict( # ... _execute=cmd_runner_fmt.as_func(execute_func), # ... ), )
Then, in community.general.puppet it is put to use with:
with runner(args_order) as ctx: rc, stdout, stderr = ctx.run(_execute=[p['execute'], p['manifest']])
cmd_runner_fmt.unpack_kwargs()
Conversely, this decorator unpacks the incoming
value
as adict
-like object.
cmd_runner_fmt.stack()
This decorator assumes
value
is a sequence and concatenates the output of the wrapped function applied to each element of the sequence.For example, in community.general.django_check, the argument format for
database
is defined as:arg_formats = dict( # ... database=cmd_runner_fmt.stack(cmd_runner_fmt.as_opt_val)("--database"), # ... )
When receiving a list
["abc", "def"]
, the output is:["--database", "abc", "--database", "def"]
Command Runner
Settings that can be passed to the CmdRunner
constructor are:
module: AnsibleModule
Module instance. Mandatory parameter.
command: str | list[str]
Command to be executed. It can be a single string, the executable name, or a list of strings containing the executable name as the first element and, optionally, fixed parameters. Those parameters are used in all executions of the runner. The executable pointed by this parameter (whether itself when
str
or its first element whenlist
) is processed usingAnsibleModule.get_bin_path()
unless it is an absolute path or contains the character/
.
arg_formats: dict
Mapping of argument names to formatting functions.
default_args_order: str
As the name suggests, a default ordering for the arguments. When this is passed, the context can be created without specifying
args_order
. Defaults to()
.
check_rc: bool
When
True
, if the return code from the command is not zero, the module exits with an error. Defaults toFalse
.
path_prefix: list[str]
If the command being executed is installed in a non-standard directory path, additional paths might be provided to search for the executable. Defaults to
None
.
environ_update: dict
Pass additional environment variables to be set during the command execution. Defaults to
None
.
force_lang: str
It is usually important to force the locale to one specific value, so that responses are consistent and, therefore, parseable. Please note that using this option (which is enabled by default) overwrites the environment variables
LANGUAGE
andLC_ALL
. To disable this mechanism, set this parameter toNone
. In community.general 9.1.0 a special valueauto
was introduced for this parameter, with the effect thatCmdRunner
then tries to determine the best parseable locale for the runtime. It should become the default value in the future, but for the time being the default value isC
.
When creating a context, the additional settings that can be passed to the call are:
args_order: str
Establishes the order in which the arguments are rendered in the command line. This parameter is mandatory unless
default_args_order
was provided to the runner instance.
output_process: func
Function to transform the output of the executable into different values or formats. See examples in section below.
check_mode_skip: bool
Whether to skip the actual execution of the command when the module is in check mode. Defaults to
False
.
check_mode_return: any
If
check_mode_skip=True
, then return this value instead.
- valid named arguments to
AnsibleModule.run_command()
Other than
args
, any valid argument torun_command()
can be passed when setting up the run context. For example,data
can be used to send information to the command’s standard input. Orcwd
can be used to run the command inside a specific working directory.
- valid named arguments to
Additionally, any other valid parameters for AnsibleModule.run_command()
may be passed, but unexpected behavior
might occur if redefining options already present in the runner or its context creation. Use with caution.
Processing results
As mentioned, CmdRunner
uses AnsibleModule.run_command()
to execute the external command,
and it passes the return value from that method back to caller. That means that,
by default, the result is going to be a tuple (rc, stdout, stderr)
.
If you need to transform or process that output, you can pass a function to the context,
as the output_process
parameter. It must be a function like:
def process(rc, stdout, stderr):
# do some magic
return processed_value # whatever that is
In that case, the return of run()
is the processed_value
returned by the function.
PythonRunner
The PythonRunner
class is a specialized version of CmdRunner
, geared towards the execution of
Python scripts. It features two extra and mutually exclusive parameters python
and venv
in its constructor:
from ansible_collections.community.general.plugins.module_utils.python_runner import PythonRunner
from ansible_collections.community.general.plugins.module_utils.cmd_runner import cmd_runner_fmt
runner = PythonRunner(
module,
command=["-m", "django"],
arg_formats=dict(...),
python="python",
venv="/path/to/some/venv",
)
The default value for python
is the string python
, and the for venv
it is None
.
The command line produced by such a command with python="python3.12"
is something like:
/usr/bin/python3.12 -m django <arg1> <arg2> ...
And the command line for venv="/work/venv"
is like:
/work/venv/bin/python -m django <arg1> <arg2> ...
You may provide the value of the command
argument as a string (in that case the string is used as a script name)
or as a list, in which case the elements of the list must be valid arguments for the Python interpreter, as in the example above.
See Command line and environment for more details.
If the parameter python
is an absolute path, or contains directory separators, such as /
, then it is used
as-is, otherwise the runtime PATH
is searched for that command name.
Other than that, everything else works as in CmdRunner
.
New in version 4.8.0.