community.aws.msk_cluster module – Manage Amazon MSK clusters
Note
This module is part of the community.aws collection (version 7.1.0).
You might already have this collection installed if you are using the ansible package.
It is not included in ansible-core.
To check whether it is installed, run ansible-galaxy collection list.
To install it, use: ansible-galaxy collection install community.aws.
You need further requirements to be able to use this module,
see Requirements for details.
To use it in a playbook, specify: community.aws.msk_cluster.
New in community.aws 2.0.0
Synopsis
Create, delete and modify Amazon MSK (Managed Streaming for Apache Kafka) clusters.
Prior to release 5.0.0 this module was called
community.aws.aws_msk_cluster. The usage did not change.
Aliases: aws_msk_cluster
Requirements
The below requirements are needed on the host that executes this module.
python >= 3.6
boto3 >= 1.26.0
botocore >= 1.29.0
Parameters
Parameter |
Comments |
|---|---|
AWS access key ID. See the AWS documentation for more information about access tokens https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys. The The aws_access_key and profile options are mutually exclusive. The aws_access_key_id alias was added in release 5.1.0 for consistency with the AWS botocore SDK. The ec2_access_key alias has been deprecated and will be removed in a release after 2024-12-01. Support for the |
|
Includes all client authentication related information. Effective only for new cluster and can not be updated. |
|
IAM authentication is enabled or not. Choices:
|
|
SASL/SCRAM authentication is enabled or not. Choices:
|
|
List of ACM Certificate Authority ARNs. |
|
Option to explicitly turn on or off authentication Choices:
|
|
The location of a CA Bundle to use when validating SSL certificates. The |
|
A dictionary to modify the botocore configuration. Parameters can be found in the AWS documentation https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config. |
|
ARN of the configuration to use. This parameter is required when state=present. |
|
The revision of the configuration to use. This parameter is required when state=present. |
|
Use a The Choices:
|
|
The size in GiB of the EBS volume for the data drive on each broker node. Default: |
|
Includes all encryption-related information. Effective only for new cluster and can not be updated. |
|
The details for encryption in transit. |
|
Indicates the encryption setting for data in transit between clients and brokers. The following are the possible values. TLS means that client-broker communication is enabled with TLS only. TLS_PLAINTEXT means that client-broker communication is enabled for both TLS-encrypted, as well as plaintext data. PLAINTEXT means that client-broker communication is enabled in plaintext only. Choices:
|
|
When set to true, it indicates that data communication among the broker nodes of the cluster is encrypted. When set to false, the communication happens in plaintext. Choices:
|
|
The ARN of the AWS KMS key for encrypting data at rest. If you don’t specify a KMS key, MSK creates one for you and uses it. |
|
URL to connect to instead of the default AWS endpoints. While this can be used to connection to other AWS-compatible services the amazon.aws and community.aws collections are only tested against AWS. The The ec2_url and s3_url aliases have been deprecated and will be removed in a release after 2024-12-01. Support for the |
|
Specifies the level of monitoring for the MSK cluster. Choices:
|
|
The type of Amazon EC2 instances to use for Kafka brokers. Choices:
|
|
Logging configuration. |
|
Details of the CloudWatch Logs destination for broker logs. |
|
Specifies whether broker logs get sent to the specified CloudWatch Logs destination. Choices:
|
|
The CloudWatch log group that is the destination for broker logs. |
|
Details of the Kinesis Data Firehose delivery stream that is the destination for broker logs. |
|
The Kinesis Data Firehose delivery stream that is the destination for broker logs. |
|
Specifies whether broker logs get send to the specified Kinesis Data Firehose delivery stream. Choices:
|
|
Details of the Amazon S3 destination for broker logs. |
|
The name of the S3 bucket that is the destination for broker logs. |
|
Specifies whether broker logs get sent to the specified Amazon S3 destination. Choices:
|
|
The S3 prefix that is the destination for broker logs. |
|
The name of the cluster. |
|
The number of broker nodes in the cluster. Should be greater or equal to two. Default: |
|
The settings for open monitoring. |
|
Indicates whether you want to enable or disable the JMX Exporter. Choices:
|
|
Indicates whether you want to enable or disable the Node Exporter. Choices:
|
|
A named AWS profile to use for authentication. See the AWS documentation for more information about named profiles https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html. The The profile option is mutually exclusive with the aws_access_key, aws_secret_key and security_token options. |
|
If purge_tags=true and tags is set, existing tags will be purged from the resource to match exactly what is defined by tags parameter. If the tags parameter is not set then tags will not be modified, even if purge_tags=True. Tag keys beginning with Choices:
|
|
The AWS region to use. For global services such as IAM, Route53 and CloudFront, region is ignored. The See the Amazon AWS documentation for more information http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region. The Support for the |
|
AWS secret access key. See the AWS documentation for more information about access tokens https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys. The The secret_key and profile options are mutually exclusive. The aws_secret_access_key alias was added in release 5.1.0 for consistency with the AWS botocore SDK. The ec2_secret_key alias has been deprecated and will be removed in a release after 2024-12-01. Support for the |
|
The AWS security groups to associate with the elastic network interfaces in order to specify who can connect to and communicate with the Amazon MSK cluster. If you don’t specify a security group, Amazon MSK uses the default security group associated with the VPC. |
|
AWS STS session token for use with temporary credentials. See the AWS documentation for more information about access tokens https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys. The The security_token and profile options are mutually exclusive. Aliases aws_session_token and session_token were added in release 3.2.0, with the parameter being renamed from security_token to session_token in release 6.0.0. The security_token, aws_security_token, and access_token aliases have been deprecated and will be removed in a release after 2024-12-01. Support for the |
|
Create ( Choices:
|
|
The list of subnets to connect to in the client virtual private cloud (VPC). AWS creates elastic network interfaces inside these subnets. Client applications use elastic network interfaces to produce and consume data. Client subnets can’t be in Availability Zone us-east-1e. This parameter is required when state=present. |
|
A dictionary representing the tags to be applied to the resource. If the tags parameter is not set then tags will not be modified. |
|
When set to Setting validate_certs=false is strongly discouraged, as an alternative, consider setting aws_ca_bundle instead. Choices:
|
|
The version of Apache Kafka. This version should exist in given configuration. This parameter is required when state=present. |
|
Whether to wait for the cluster to be available or deleted. Choices:
|
|
How many seconds to wait. Cluster creation can take up to 20-30 minutes. Default: |
Notes
Note
All operations are time consuming, for example create takes 20-30 minutes, update kafka version – more than one hour, update configuration – 10-15 minutes;
Cluster’s brokers get evenly distributed over a number of availability zones that’s equal to the number of subnets.
Caution: For modules, environment variables and configuration files are read from the Ansible ‘host’ context and not the ‘controller’ context. As such, files may need to be explicitly copied to the ‘host’. For lookup and connection plugins, environment variables and configuration files are read from the Ansible ‘controller’ context and not the ‘host’ context.
The AWS SDK (boto3) that Ansible uses may also read defaults for credentials and other settings, such as the region, from its configuration files in the Ansible ‘host’ context (typically
~/.aws/credentials). See https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html for more information.
Examples
# Note: These examples do not set authentication details, see the AWS Guide for details.
- community.aws.msk_cluster:
name: kafka-cluster
state: present
version: 2.6.1
nodes: 6
ebs_volume_size: "{{ aws_msk_options.ebs_volume_size }}"
subnets:
- subnet-e3b48ce7c25861eeb
- subnet-2990c8b25b07ddd43
- subnet-d9fbeaf46c54bfab6
wait: true
wait_timeout: 1800
configuration_arn: arn:aws:kafka:us-east-1:123456789012:configuration/kafka-cluster-configuration/aaaaaaaa-bbbb-4444-3333-ccccccccc-1
configuration_revision: 1
- community.aws.msk_cluster:
name: kafka-cluster
state: absent
Return Values
Common return values are documented here, the following are the fields unique to this module:
Key |
Description |
|---|---|
A list of brokers that a client application can use to bootstrap. Returned: state=present and cluster state is ACTIVE |
|
A string containing one or more hostname:port pairs. Returned: success |
|
A string containing one or more DNS names (or IP) and TLS port pairs. Returned: success |
|
Description of the MSK cluster. Returned: state=present |
|
The response from actual API call. Returned: always Sample: |