This guide aims to help anyone who wishes to contribute to the
This guide can be improved with your help! Open a GitHub issue in the repository or contribute directly by following the instructions below.
Log into your GitHub account.
Fork the ansible-collections/community.hashi_vault repository by clicking the Fork button in the upper right corner. This will create a fork in your own account.
Clone the repository locally, following the example instructions here (but replace
hashi_vault). Pay special attention to the local path structure of the cloned repository as described in those instructions (for example
As mentioned on that page, commit your changes to a branch, push them to your fork, and create a pull request (GitHub will automatically prompt you to do so when you look at your repository).
Additions to the collection documentation are very welcome! We have three primary types of documentation, each with their own syntax and rules.
Markdown files (those with the extension
.md) can be found in several directories within the repository. These files are primarily aimed at developers and those browsing the repository, to explain or give context to the other files nearby.
The main exception to the above is the
README.md in the repository root. This file is more important because it provides introductory information and links for anyone browsing the repository, both on GitHub and on the collection’s Ansible Galaxy page.
Markdown files can be previewed natively in GitHub, so they are easy to validate by reviewers, and there are no specific tests that need to run against them.
Your IDE or code editor may also be able to preview these files. For example Visual Studio Code has built-in markdown preview.
This type of documentation gets generated from structured YAML, inside of a Python string. It is included in the same code that it’s documenting, or in a separate Python file, such as a doc fragment. Please see the module format and documentation guidance for more information.
This type of documentation is highly structured and tested with
ansible-test sanity. Full instructions are available on the testing module documentation page.
Additionally, the docsite build on pull requests (or built locally) will include module and plugin documentation as well. See the next section for details.
The collection docsite is what you are reading now. It is written in reStructuredText (RST) format and published on the ansible_documentation site. This is where we have long-form documentation that doesn’t fit into the other two categories.
If you are considering adding an entirely new document here it may be best to open a GitHub issue first to discuss the idea and how best to organize it.
Refer to the Ansible style guide for all submissions to the collection docsite.
RST files for the docsite are in the
docs/docsite/rst/ directory. Some submissions may also require edits to
When a pull request is submitted which changes the collection’s documentation, a new docsite will be generated and published to a temporary site, and a bot will post a comment on the PR with a link. This will let you see the rendered docs to help with spotting formatting errors.
It’s also possible to build the docs locally, by installing some extra Python requirements and running the build script:
$ pushd docs/preview $ pip install -r requirements.txt $ ./build.sh
You can then find the generated HTML in
docs/preview/build/html and can open them locally in your browser.
If you’re making anything more than very small or one-time changes, run the tests locally to avoid having to push a commit for each thing, and waiting for the CI to run tests.
First, review the guidance on testing collections, as it applies to this collection as well.
Unlike other collections, we require an integration_config.yml file for properly running integration tests, as the tests require external dependencies (like a Vault server) and they need to know where to find those dependencies.
If you have contributed to this collection or to the
hashi_vault lookup plugin in the past, you might remember that the integration tests used to download, extract, and run a Vault server during the course of the tests, by default. This legacy mode is no longer available.
The recommended way to run the tests has Vault and other dependencies running in their own containers, set up via docker-compose, and the integration tests run in their own container separately.
We have a pre-defined “localenv” setup role for this purpose.
For ease of typing / length of commands, we’ll enter the role directory first:
$ pushd tests/integration/targets/setup_localenv_docker
This localenv has both Ansible collection and Python requirements, so let’s get those out of the way:
$ pip install -r files/requirements/requirements.txt -c files/requirements/constraints.txt $ ansible-galaxy collection install -r files/requirements/requirements.yml
To set up your docker-compose environment with all the defaults:
The setup script does the following:
docker-compose.ymlfor the project.
Generate a private key and self-signed certificate for Vault.
Template a Vault config file.
Bring down the existing compose project.
Bring up the compose project as defined by the vars (specified or defaults).
integration_config.ymlfile that has all the right settings for integration tests to connect.
Copy the integration config to the correct location if there isn’t already one there (it won’t overwrite, in case you had customized changes).
With your containers running, you can now run the tests in docker (after returning back to the collection root):
$ popd $ ansible-test integration --docker default --docker-network hashi_vault_default -v
--docker-network part is important, because it ensures that the Ansible test container is in the same network as the dependency containers, that way the test container can reach them by their container names. The network name,
hashi_vault_default comes from the default docker-compose project name used by this role (
hashi_vault). See the customization section for more information.
setup.sh again can be used to re-deploy the containers, or if you prefer you can use the generated
files/.output/<project_name>/docker-compose.yml directly with local tools.
If running again, remember to manually copy the contents of newly generated
files/.output/integration_config.yml to the integration root, or delete the file in the root before re-running setup so that it copies the file automatically.
setup.sh passes any additional params you send it to the
ansible-playbook command it calls, so you can customize variables with the standard
-e) option. There are many advanced scenarios possible, but a few things you might want to override:
vault_version– can target any version of Vault for which a docker container exists (this is the container’s tag), defaults to
cleanbut could be set to
up– similar to running
docker-compose up(no op if the project is running as it should)
down– similar to
docker-compose down(destroys the project)
clean– (default) similar to
docker-compose downfollowed by
none– does the other tasks, including templating, but does not bring the project up or down. With this option, the
community.dockercollection is not required.
vault_crypto_force– by default this is
falseso if the cert and key exist they won’t be regenerated. Setting to
truewill overwrite them.
proxy_port– all of the ports are exposed to the host, so if you already have any of the default ports in use on your host, you may need to override these.
proxy_container_name– these are the names for their respective containers, which will also be the DNS names used within the container network. In case you have the default names in use you may need to override these.
docker_compose_project_name– unlikely to need to be changed, but it affects the name of the docker network which will be needed for your
ansible-testinvocation, so it’s worth mentioning. For example, if you set this to
ansible_hashi_vaultthen the docker network name will be
In this collection, auth methods are shared among all plugins and modules rather than being re-implemented in each one. This saves the effort of re-inventing the wheel, prevents test bloat by having to test functionality across auth methods, and provides a consistent experience.
Auth methods are implemented as classes in
module_utils, in a file named
plugins/module_utils/_auth_method_<method_name>.py. The leading underscore indicates that the module util is private to the collection and that it is not intended to be used outside the collection; this lets us make changes as needed without needing to release a new major version, and clearly indicates to would-be downstream users that they should not rely on these utils outside content within the collection.
In addition, all auth method module utils within the collection must contain a comment explaining this, such as:
# FOR INTERNAL COLLECTION USE ONLY # The interfaces in this file are meant for use within the community.hashi_vault collection # and may not remain stable to outside uses. Changes may be made in ANY release, even a bugfix release. # See also: https://github.com/ansible/community/issues/539#issuecomment-780839686 # Please open an issue if you have questions about this.
It is best to look at existing auth methods to get a feel for how they are implemented.
Each auth method class should be named
HashiVaultAuthMethod<MethodName> and inherit from
The base class provides some common functionality, like standardizing a way to emit warnings and providing a common function for validating required options.
An auth method must run the base class’s
It must implement two methods:
validate()– this method does everything it can to ensure that the requirements are met for performing authentication with this particular auth method. This may include checking for required options, validating the values of those options, pulling in additional information and context from the environment, preparing that information for use by
authenticate(), etc. Generally speaking, it should not contact Vault, and should minimize reliance on external sources and services, but that is a guideline and the details will depend on the specifics of the auth method in question.
validate()raises an exception if validation fails. If it succeeds, nothing is returned.
authenticate(client, use_token=False)– this method performs the actual authentication, and it returns the API result of the authentication (which will include the token, lease information, etc.). The HVAC client object is passed in, as well an optional parameter
use_tokenwhich specifies whether the client should have its token field set to the result of authentication (typically this is desired).
The auth method class should also contain two fields:
NAME– the name of the auth method.
OPTIONS– a list containing the name of every option that may be used by the auth method, including optional options; this list should not include the
Because auth methods are shared among both Ansible modules and Ansible plugins, any exceptions raised must be applicable to both. Standard Python exceptions like
KeyError can be raised if they appropriate.
In situations where you would normally raise
AnsibleError (in plugins), or call
module.fail_json() (in modules), you may raise
HashiVaultValueError with your error message. Plugins and modules in this collection should expect this type and act accordingly.
Similarly for warnings, because plugins and modules implement warnings differently, module util code that needs to warn takes a warning callback, and this is true for auth methods as well.
The base class provides a
warn() method that handles calling the callback specified at class init, so a simple
self.warn() can be used in auth method code.
Because auth methods are shared among both Ansible modules and Ansible plugins, which do not access options in the same way, this collection implements a class called
HashiVaultOptionAdapter. This class provides a standard interface for accessing option values in code that must work in both plugins and modules.
It implements the following methods:
get_option(key)– gets the option with the specified name. Raises
KeyErrorif the option is not present.
get_option_default(key, default=None)– gets the option with the specified name. If it’s not present, returns the value of
set_option(key, value)– sets the value of the specified option
set_option_default(key, default=None)– returns the value of the option
key. If the key is not present, sets its value to
defaultand returns that value.
Trueif the specified option is present (
Nonevalue counts as present).
set_options(**kwargs)– sets options to the key/value pairs specified in
get_options(*args)– returns a dict of the option names specified in
get_filtered_options(filter, *args)– returns a dict of the option names specified in
args, if the callable
valuepassed into it) returns
Truefor the given key/value pair.
get_filled_options(*args)– returns a dict of the option names specified in
*argsthat are not
HashiVaultAuthenticator class is how most of the content in the collection will handle authentication, rather than having to directly references each individual auth method. As a result,
_authenticator.py needs to be modified for every new auth method, because it imports and directly references each class. See the implementation of this class to find the places that need to be modified.
Because auth methods are shared among collection content, their options are documented in doc_fragment plugins. Because many options end up being shared among many auth methods (for example
password), we do not have a separate doc fragment for each auth method, as this would end up with duplicated option documentation.
Instead, all of the options for auth methods are in
This contains the standard
DOCUMENTATION field, as well as a
PLUGINS field. The reason for this split is that there are certain parts of the documentation that are only applicable to plugins; namely the
DOCUMENTATION should contains all fields common to both, like
required, etc., while anything plugin-specific goes in
Since plugins and modules will reference the doc fragments, it’s not usually required to modify the docstrings in the content directly; if it seems necessary, please raise an issue to discuss.
Wherever possible, we should provide
vars alternatives for specifying options, to give maximum flexibility for plugins. Occasionally, these won’t make sense, like providing
token (a sensitive value) in
When deciding to implement new options for an auth method, consider whether existing options can or should be reused. If a new option is needed, consider scoping its name to the auth method, in order to differentiate it from current or future option names that could be confusing in another context.
cert_auth_private_key were named that way to prevent them being confused with other certificate options that relate to the Vault connection, or other contexts where specific plugins or modules might need key pairs.
Because auth methods are shared across the collection, we want them to be very well tested. Auth methods have both unit and integration tests, and the combination of those should give us high confidence that the methods work as intended.
Unit tests allow us to check some of the functionality that is difficult to test effectively in integration tests, like checking that every possible combination of options behaves as it should, or simulating conditions that we can’t easily reproduce. The coverage of various scenarios should be extensive, and the details of which, or how complex they are, will depend on the intricacies of the auth method itself. Looking at existing examples is highly recommended.
A pytest fixture is provided to load fixtures files that contain sample Vault API responses. Using these allows for mocking of the HVAC authentication calls within the unit tests.
Our integration tests provide a running Vault server, and with that we can set up any auth methods we want (in theory). In practice, auth methods often require external services to integrate with. When possible, we should consider setting up such external services so that we can create a meaningful, real life integration and test it.
Often however, this is not possible, or difficult. We must consider that tests are not only run in CI, but should be able to be run locally as well.
We have implemented MMock (Monster Mock) in our integration test setup to help with this. This server is setup to proxy its requests to the test Vault server, but you can write configurations that allow it to return different data for specific requests. By carefully constructing these responses, we can simulate the Vault API’s response to login requests for specific auth methods, and also simulate its failures. With that, we can then run integration tests that hopefully provide us some assurance that we are implementing it correctly.
Testing plugin and module Usage
Auth methods are usable from modules and plugins, so integration tests for an auth method must test it via both plugins and modules.
We provide custom modules and plugins specifically for testing auth methods within the integration tests. These are simplified implementations but they use the common code that should be used by all content, and they can be set to return some useful information about the login process. See the existing tests for details.
In CI, we use CodeCov to track coverage. We also set some specific “tags” in coverage, and one of those is to tag individual auth methods as targets for integration tests. This happens automatically in CI, however new auth methods need an entry into
codecov.yml that maps the coverage flag to the file where the auth method is implemented. For example:
flags: target_auth_aws_iam: paths: - plugins/module_utils/_auth_method_aws_iam.py