Distributing collections
A collection is a distribution format for Ansible content. A typical collection contains modules and other plugins that address a set of related use cases. For example, a collection might automate administering a particular database. A collection can also contain roles and playbooks.
To distribute your collection and allow others to use it, you can publish your collection on one or more distribution servers. Distribution servers include:
Distribution server |
Collections accepted |
---|---|
Ansible Galaxy |
All collections |
Red Hat Automation Hub |
Only collections certified by Red Hat |
Privately hosted Automation Hub |
Collections authorized by the owners |
Distributing collections involves four major steps:
Initial configuration of your distribution server or servers
Building your collection tarball
Preparing to publish your collection
Publishing your collection
Initial configuration of your distribution server or servers
Configure a connection to one or more distribution servers so you can publish collections there. You only need to configure each distribution server once. You must repeat the other steps (building your collection tarball, preparing to publish, and publishing your collection) every time you publish a new collection or a new version of an existing collection.
Create a namespace on each distribution server you want to use.
Get an API token for each distribution server you want to use.
Specify the API token for each distribution server you want to use.
Creating a namespace
You must upload your collection into a namespace on each distribution server. If you have a login for Ansible Galaxy, your Ansible Galaxy username is usually also an Ansible Galaxy namespace.
Warning
Namespaces on Ansible Galaxy cannot include hyphens. If you have a login for Ansible Galaxy that includes a hyphen, your Galaxy username is not also a Galaxy namespace. For example, awesome-user
is a valid username for Ansible Galaxy, but it is not a valid namespace.
You can create additional namespaces on Ansible Galaxy if you choose. For Red Hat Automation Hub and private Automation Hub you must create a namespace before you can upload your collection. To create a namespace:
To create a namespace on Galaxy, see Galaxy namespaces on the Galaxy docsite for details.
To create a namespace on Red Hat Automation Hub, see the Ansible Certified Content FAQ.
Specify the namespace in the galaxy.yaml
file for each collection. For more information on the galaxy.yaml
file, see Collection Galaxy metadata structure.
Getting your API token
An API token authenticates your connection to each distribution server. You need a separate API token for each distribution server. Use the correct API token to connect to each distribution server securely and protect your content.
To get your API token:
To get an API token for Galaxy, go to the Galaxy profile preferences page and click API Key.
To get an API token for Automation Hub, go to the token page and click Load token.
Specifying your API token and distribution server
Each time you publish a collection, you must specify the API token and the distribution server to create a secure connection. You have two options for specifying the token and distribution server:
You can configure the token in configuration, as part of a
galaxy_server_list
entry in youransible.cfg
file. Using configuration is the most secure option.You can pass the token at the command line as an argument to the
ansible-galaxy
command. If you pass the token at the command line, you can specify the server at the command line, by using the default setting, or by setting the server in configuration. Passing the token at the command line is insecure, because typing secrets at the command line may expose them to other users on the system.
Specifying the token and distribution server in configuration
By default, Ansible Galaxy is configured as the only distribution server. You can add other distribution servers and specify your API token or tokens in configuration by editing the galaxy_server_list
section of your ansible.cfg
file. This is the most secure way to manage authentication for distribution servers. Specify a URL and token for each server. For example:
[galaxy]
server_list = release_galaxy
[galaxy_server.release_galaxy]
url=https://galaxy.ansible.com/
token=abcdefghijklmnopqrtuvwxyz
You cannot use apt-key
with any servers defined in your galaxy_server_list. See Configuring the ansible-galaxy client for complete details.
Specifying the token at the command line
You can specify the API token at the command line using the --token
argument of the ansible-galaxy command. There are three ways to specify the distribution server when passing the token at the command line:
using the
--server
argument of the ansible-galaxy commandrelying on the default (https://galaxy.ansible.com)
setting a server in configuration by creating a GALAXY_SERVER setting in your
ansible.cfg
file
For example:
ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz --token abcdefghijklmnopqrtuvwxyz
Warning
Using the --token
argument is insecure. Passing secrets at the command line may expose them to others on the system.
Building your collection tarball
After configuring one or more distribution servers, build a collection tarball. The collection tarball is the published artifact, the object that you upload and other users download to install your collection. To build a collection tarball:
Review the version number in your
galaxy.yaml
file. Each time you publish your collection, it must have a new version number. You cannot make changes to existing versions of your collection on a distribution server. If you try to upload the same collection version more than once, the distribution server returns the errorCode: conflict.collection_exists
. Collections follow semantic versioning rules. For more information on versions, see Understanding collection versioning. For more information on thegalaxy.yaml
file, see Collection Galaxy metadata structure.Run
ansible-galaxy collection build
from inside the top-level directory of the collection. For example:
collection_dir#> ansible-galaxy collection build
This command builds a tarball of the collection in the current directory, which you can upload to your selected distribution server:
my_collection/
├── galaxy.yml
├── ...
├── my_namespace-my_collection-1.0.0.tar.gz
└── ...
Note
To reduce the size of collections, certain files and folders are excluded from the collection tarball by default. See Ignoring files and folders if your collection directory contains other files you want to exclude.
The current Galaxy maximum tarball size is 2 MB.
You can upload your tarball to one or more distribution servers. You can also distribute your collection locally by copying the tarball to install your collection directly on target systems.
Ignoring files and folders
By default the build step includes all the files in the collection directory in the tarball except for the following:
galaxy.yml
*.pyc
*.retry
tests/output
previously built tarballs in the root directory
various version control directories such as
.git/
To exclude other files and folders from your collection tarball, set a list of file glob-like patterns in the build_ignore
key in the collection’s galaxy.yml
file. These patterns use the following special characters for wildcard matching:
*
: Matches everything?
: Matches any single character[seq]
: Matches any character in sequence[!seq]
:Matches any character not in sequence
For example, to exclude the sensitive
folder within the playbooks
folder as well any .tar.gz
archives, set the following in your galaxy.yml
file:
build_ignore:
- playbooks/sensitive
- '*.tar.gz'
For more information on the galaxy.yml
file, see Collection Galaxy metadata structure.
Note
The build_ignore
feature is only supported with ansible-galaxy collection build
in Ansible 2.10 or newer.
Preparing to publish your collection
Each time you publish your collection, you must create a new version on the distribution server. After you publish a version of a collection, you cannot delete or modify that version. To avoid unnecessary extra versions, check your collection for bugs, typos, and other issues locally before publishing:
Install the collection locally.
Review the locally installed collection before publishing a new version.
Installing your collection locally
You have two options for installing your collection locally:
Install your collection locally from the tarball.
Install your collection locally from your git repository.
Installing your collection locally from the tarball
To install your collection locally from the tarball, run ansible-galaxy collection install
and specify the collection tarball. You can optionally specify a location using the -p
flag. For example:
collection_dir#> ansible-galaxy collection install my_namespace-my_collection-1.0.0.tar.gz -p ./collections
Install the tarball into a directory configured in COLLECTIONS_PATHS so Ansible can easily find and load the collection. If you do not specify a path value, ansible-galaxy collection install
installs the collection in the first path defined in COLLECTIONS_PATHS.
Installing your collection locally from a git repository
To install your collection locally from a git repository, specify the repository and the branch you want to install:
collection_dir#> ansible-galaxy collection install git+https://github.com/org/repo.git,devel
You can install a collection from a git repository instead of from Galaxy or Automation Hub. As a developer, installing from a git repository lets you review your collection before you create the tarball and publish the collection. As a user, installing from a git repository lets you use collections or versions that are not in Galaxy or Automation Hub yet.
The repository must contain a galaxy.yml
or MANIFEST.json
file. This file provides metadata such as the version number and namespace of the collection.
Installing a collection from a git repository at the command line
To install a collection from a git repository at the command line, use the URI of the repository instead of a collection name or path to a tar.gz
file. Prefix the URI with git+
(or with git@
to use a private repository with ssh authentication). You can specify a branch, commit, or tag using the comma-separated git commit-ish syntax.
For example:
# Install a collection in a repository using the latest commit on the branch 'devel'
ansible-galaxy collection install git+https://github.com/organization/repo_name.git,devel
# Install a collection from a private github repository
ansible-galaxy collection install [email protected]:organization/repo_name.git
# Install a collection from a local git repository
ansible-galaxy collection install git+file:///home/user/path/to/repo_name.git
Warning
Embedding credentials into a git URI is not secure. Use safe authentication options to prevent your credentials from being exposed in logs or elsewhere.
Use SSH authentication
Use netrc authentication
Use http.extraHeader in your git configuration
Use url.<base>.pushInsteadOf in your git configuration
Specifying the collection location within the git repository
When you install a collection from a git repository, Ansible uses the collection galaxy.yml
or MANIFEST.json
metadata file to build the collection. By default, Ansible searches two paths for collection galaxy.yml
or MANIFEST.json
metadata files:
The top level of the repository.
Each directory in the repository path (one level deep).
If a galaxy.yml
or MANIFEST.json
file exists in the top level of the repository, Ansible uses the collection metadata in that file to install an individual collection.
├── galaxy.yml
├── plugins/
│ ├── lookup/
│ ├── modules/
│ └── module_utils/
└─── README.md
If a galaxy.yml
or MANIFEST.json
file exists in one or more directories in the repository path (one level deep), Ansible installs each directory with a metadata file as a collection. For example, Ansible installs both collection1 and collection2 from this repository structure by default:
├── collection1
│ ├── docs/
│ ├── galaxy.yml
│ └── plugins/
│ ├── inventory/
│ └── modules/
└── collection2
├── docs/
├── galaxy.yml
├── plugins/
| ├── filter/
| └── modules/
└── roles/
If you have a different repository structure or only want to install a subset of collections, you can add a fragment to the end of your URI (before the optional comma-separated version) to indicate the location of the metadata file or files. The path should be a directory, not the metadata file itself. For example, to install only collection2 from the example repository with two collections:
ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/collection2/
In some repositories, the main directory corresponds to the namespace:
namespace/
├── collectionA/
| ├── docs/
| ├── galaxy.yml
| ├── plugins/
| │ ├── README.md
| │ └── modules/
| ├── README.md
| └── roles/
└── collectionB/
├── docs/
├── galaxy.yml
├── plugins/
│ ├── connection/
│ └── modules/
├── README.md
└── roles/
You can install all collections in this repository, or install one collection from a specific commit:
# Install all collections in the namespace
ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/namespace/
# Install an individual collection using a specific commit
ansible-galaxy collection install git+https://github.com/organization/repo_name.git#/namespace/collectionA/,7b60ddc245bc416b72d8ea6ed7b799885110f5e5
Reviewing your collection
Review the collection:
Run a playbook that uses the modules and plugins in your collection. Verify that new features and functionality work as expected. For examples and more details see Using collections.
Check the documentation for typos.
Check that the version number of your tarball is higher than the latest published version on the distribution server or servers.
If you find any issues, fix them and rebuild the collection tarball.
Understanding collection versioning
The only way to change a collection is to release a new version. The latest version of a collection (by highest version number) is the version displayed everywhere in Galaxy and Automation Hub. Users can still download older versions.
Follow semantic versioning when setting the version for your collection. In summary:
Increment the major version number,
x
ofx.y.z
, for an incompatible API change.Increment the minor version number,
y
ofx.y.z
, for new functionality in a backwards compatible manner (for example new modules/plugins, parameters, return values).Increment the patch version number,
z
ofx.y.z
, for backwards compatible bug fixes.
Read the official Semantic Versioning documentation for details and examples.
Publishing your collection
The last step in distributing your collection is publishing the tarball to Ansible Galaxy, Red Hat Automation Hub, or a privately hosted Automation Hub instance. You can publish your collection in two ways:
from the command line using the
ansible-galaxy collection publish
commandfrom the website of the distribution server (Galaxy, Automation Hub) itself
Publishing a collection from the command line
To upload the collection tarball from the command line using ansible-galaxy
:
ansible-galaxy collection publish path/to/my_namespace-my_collection-1.0.0.tar.gz
Note
This ansible-galaxy command assumes you have retrieved and stored your API token in configuration. See Specifying your API token and distribution server for details.
The ansible-galaxy collection publish
command triggers an import process, just as if you uploaded the collection through the Galaxy website. The command waits until the import process completes before reporting the status back. If you want to continue without waiting for the import result, use the --no-wait
argument and manually look at the import progress in your My Imports page.
Publishing a collection from the website
To publish your collection directly on the Galaxy website:
Go to the My Content page, and click the Add Content button on one of your namespaces.
From the Add Content dialogue, click Upload New Collection, and select the collection archive file from your local filesystem.
When you upload a collection, Ansible always uploads the tarball to the namespace specified in the collection metadata in the galaxy.yml
file, no matter which namespace you select on the website. If you are not an owner of the namespace specified in your collection metadata, the upload request fails.
After Galaxy uploads and accepts a collection, the website shows you the My Imports page. This page shows import process information. You can review any errors or warnings about your upload there.
See also
- Using collections
Learn how to install and use collections.
- Collection Galaxy metadata structure
Table of fields used in the
galaxy.yml
file- Mailing List
The development mailing list
- irc.libera.chat
#ansible IRC chat channel