google.cloud.gcp_bigquery_table module – Creates a GCP Table
Note
This module is part of the google.cloud collection (version 1.4.1).
You might already have this collection installed if you are using the ansible package.
It is not included in ansible-core.
To check whether it is installed, run ansible-galaxy collection list.
To install it, use: ansible-galaxy collection install google.cloud.
You need further requirements to be able to use this module,
see Requirements for details.
To use it in a playbook, specify: google.cloud.gcp_bigquery_table.
Note
The google.cloud collection will be removed from Ansible 12 due to violations of the Ansible inclusion requirements. The collection has unresolved sanity test failures. See the discussion thread for more information.
Synopsis
A Table that belongs to a Dataset .
Requirements
The below requirements are needed on the host that executes this module.
python >= 2.6
requests >= 2.18.4
google-auth >= 1.3.0
Parameters
Parameter |
Comments |
|---|---|
An OAuth2 access token if credential type is accesstoken. |
|
The type of credential used. Choices:
|
|
One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. When you cluster a table using multiple columns, the order of columns you specify is important. The order of the specified columns determines the sort order of the data. |
|
Name of the dataset. |
|
A user-friendly description of the dataset. |
|
Custom encryption configuration. |
|
Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key. |
|
Specifies which Ansible environment you’re running this module within. This should not be set unless you know what you’re doing. This only alters the User Agent string for any API requests. |
|
The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. |
|
Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. |
|
Try to detect schema and format options automatically. Any option specified explicitly will be honored. Choices:
|
|
Additional options if sourceFormat is set to BIGTABLE. |
|
List of column families to expose in the table schema along with their types. |
|
Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. |
|
The encoding of the values when the type is not STRING. Some valid choices include: “TEXT”, “BINARY” |
|
If the qualifier is not a valid BigQuery field identifier, a valid identifier must be provided as the column field name and is used as field name in queries. |
|
If this is set, only the latest version of value in this column are exposed . Choices:
|
|
Qualifier of the column. |
|
The type to convert the value in cells of this column. Some valid choices include: “BYTES”, “STRING”, “INTEGER”, “FLOAT”, “BOOLEAN” |
|
The encoding of the values when the type is not STRING. Some valid choices include: “TEXT”, “BINARY” |
|
Identifier of the column family. |
|
If this is set only the latest version of value are exposed for all columns in this column family . Choices:
|
|
The type to convert the value in cells of this column family. Some valid choices include: “BYTES”, “STRING”, “INTEGER”, “FLOAT”, “BOOLEAN” |
|
If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema . Choices:
|
|
If field is true, then the rowkey column families will be read and converted to string. Choices:
|
|
The compression type of the data source. Some valid choices include: “GZIP”, “NONE” |
|
Additional properties to set if sourceFormat is set to CSV. |
|
Indicates if BigQuery should accept rows that are missing trailing optional columns . Choices:
|
|
Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file . Choices:
|
|
The character encoding of the data. Some valid choices include: “UTF-8”, “ISO-8859-1” |
|
The separator for fields in a CSV file. |
|
The value that is used to quote data sections in a CSV file. |
|
The number of rows at the top of a CSV file that BigQuery will skip when reading the data. |
|
Additional options if sourceFormat is set to GOOGLE_SHEETS. |
|
The number of rows at the top of a Google Sheet that BigQuery will skip when reading the data. |
|
Indicates if BigQuery should allow extra values that are not represented in the table schema . Choices:
|
|
The maximum number of bad records that BigQuery can ignore when reading data . |
|
The schema for the data. Schema is required for CSV and JSON formats. |
|
Describes the fields in a table. |
|
The field description. |
|
Describes the nested schema fields if the type property is set to RECORD . |
|
Field mode. Some valid choices include: “NULLABLE”, “REQUIRED”, “REPEATED” |
|
Field name. |
|
Field data type. Some valid choices include: “STRING”, “BYTES”, “INTEGER”, “FLOAT”, “TIMESTAMP”, “DATE”, “TIME”, “DATETIME”, “RECORD” |
|
The data format. Some valid choices include: “CSV”, “GOOGLE_SHEETS”, “NEWLINE_DELIMITED_JSON”, “AVRO”, “DATASTORE_BACKUP”, “BIGTABLE”, “ORC” |
|
The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘*’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the ‘*’ wildcard character is not allowed. |
|
A descriptive name for this table. |
|
The labels associated with this dataset. You can use these to organize and group your datasets . |
|
Name of the table. |
|
The number of rows of data in this table, excluding any data in the streaming buffer. |
|
The Google Cloud Platform project to use. |
|
Describes the schema of this table. |
|
Describes the fields in a table. |
|
The field description. The maximum length is 1,024 characters. |
|
Describes the nested schema fields if the type property is set to RECORD. |
|
The field mode. Some valid choices include: “NULLABLE”, “REQUIRED”, “REPEATED” |
|
The field name. |
|
The field data type. Some valid choices include: “STRING”, “BYTES”, “INTEGER”, “FLOAT”, “TIMESTAMP”, “DATE”, “TIME”, “DATETIME”, “RECORD” |
|
Array of scopes to be used |
|
The contents of a Service Account JSON file, either in a dictionary or as a JSON string that represents it. |
|
An optional service account email address if machineaccount is selected and the user does not wish to use the default email. |
|
The path of a Service Account JSON file if serviceaccount is selected as type. |
|
Whether the given object should exist in GCP Choices:
|
|
Reference describing the ID of this table. |
|
The ID of the dataset containing this table. |
|
The ID of the project containing this table. |
|
The ID of the the table. |
|
If specified, configures time-based partitioning for this table. |
|
Number of milliseconds for which to keep the storage for a partition. |
|
If not set, the table is partitioned by pseudo column, referenced via either ‘_PARTITIONTIME’ as TIMESTAMP type, or ‘_PARTITIONDATE’ as DATE type. If field is specified, the table is instead partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. |
|
The only type supported is DAY, which will generate one partition per day. Some valid choices include: “DAY” |
|
The view definition. |
|
Specifies whether to use BigQuery’s legacy SQL for this view . Choices:
|
|
Describes user-defined function resources used in the query. |
|
An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code. |
|
A code resource to load from a Google Cloud Storage URI (gs://bucket/path). |
Examples
- name: create a dataset
google.cloud.gcp_bigquery_dataset:
name: example_dataset
dataset_reference:
dataset_id: example_dataset
project: "{{ gcp_project }}"
auth_kind: "{{ gcp_cred_kind }}"
service_account_file: "{{ gcp_cred_file }}"
state: present
register: dataset
- name: create a table
google.cloud.gcp_bigquery_table:
name: example_table
dataset: example_dataset
table_reference:
dataset_id: example_dataset
project_id: test_project
table_id: example_table
project: test_project
auth_kind: serviceaccount
service_account_file: "/tmp/auth.pem"
state: present
Return Values
Common return values are documented here, the following are the fields unique to this module:
Key |
Description |
|---|---|
One or more fields on which data should be clustered. Only top-level, non-repeated, simple-type fields are supported. When you cluster a table using multiple columns, the order of columns you specify is important. The order of the specified columns determines the sort order of the data. Returned: success |
|
The time when this dataset was created, in milliseconds since the epoch. Returned: success |
|
Name of the dataset. Returned: success |
|
A user-friendly description of the dataset. Returned: success |
|
Custom encryption configuration. Returned: success |
|
Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key. Returned: success |
|
The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Returned: success |
|
Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. Returned: success |
|
Try to detect schema and format options automatically. Any option specified explicitly will be honored. Returned: success |
|
Additional options if sourceFormat is set to BIGTABLE. Returned: success |
|
List of column families to expose in the table schema along with their types. Returned: success |
|
Lists of columns that should be exposed as individual fields as opposed to a list of (column name, value) pairs. Returned: success |
|
The encoding of the values when the type is not STRING. Returned: success |
|
If the qualifier is not a valid BigQuery field identifier, a valid identifier must be provided as the column field name and is used as field name in queries. Returned: success |
|
If this is set, only the latest version of value in this column are exposed . Returned: success |
|
Qualifier of the column. Returned: success |
|
The type to convert the value in cells of this column. Returned: success |
|
The encoding of the values when the type is not STRING. Returned: success |
|
Identifier of the column family. Returned: success |
|
If this is set only the latest version of value are exposed for all columns in this column family . Returned: success |
|
The type to convert the value in cells of this column family. Returned: success |
|
If field is true, then the column families that are not specified in columnFamilies list are not exposed in the table schema . Returned: success |
|
If field is true, then the rowkey column families will be read and converted to string. Returned: success |
|
The compression type of the data source. Returned: success |
|
Additional properties to set if sourceFormat is set to CSV. Returned: success |
|
Indicates if BigQuery should accept rows that are missing trailing optional columns . Returned: success |
|
Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file . Returned: success |
|
The character encoding of the data. Returned: success |
|
The separator for fields in a CSV file. Returned: success |
|
The value that is used to quote data sections in a CSV file. Returned: success |
|
The number of rows at the top of a CSV file that BigQuery will skip when reading the data. Returned: success |
|
Additional options if sourceFormat is set to GOOGLE_SHEETS. Returned: success |
|
The number of rows at the top of a Google Sheet that BigQuery will skip when reading the data. Returned: success |
|
Indicates if BigQuery should allow extra values that are not represented in the table schema . Returned: success |
|
The maximum number of bad records that BigQuery can ignore when reading data . Returned: success |
|
The schema for the data. Schema is required for CSV and JSON formats. Returned: success |
|
Describes the fields in a table. Returned: success |
|
The field description. Returned: success |
|
Describes the nested schema fields if the type property is set to RECORD . Returned: success |
|
Field mode. Returned: success |
|
Field name. Returned: success |
|
Field data type. Returned: success |
|
The data format. Returned: success |
|
The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘*’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one URI can be specified. Also, the ‘*’ wildcard character is not allowed. Returned: success |
|
A descriptive name for this table. Returned: success |
|
An opaque ID uniquely identifying the table. Returned: success |
|
The labels associated with this dataset. You can use these to organize and group your datasets . Returned: success |
|
The time when this table was last modified, in milliseconds since the epoch. Returned: success |
|
The geographic location where the table resides. This value is inherited from the dataset. Returned: success |
|
Name of the table. Returned: success |
|
The size of this table in bytes, excluding any data in the streaming buffer. Returned: success |
|
The number of bytes in the table that are considered “long-term storage”. Returned: success |
|
The number of rows of data in this table, excluding any data in the streaming buffer. Returned: success |
|
If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified. Returned: success |
|
Describes the schema of this table. Returned: success |
|
Describes the fields in a table. Returned: success |
|
The field description. The maximum length is 1,024 characters. Returned: success |
|
Describes the nested schema fields if the type property is set to RECORD. Returned: success |
|
The field mode. Returned: success |
|
The field name. Returned: success |
|
The field data type. Returned: success |
|
Contains information regarding this table’s streaming buffer, if one is present. This field will be absent if the table is not being streamed to or if there is no data in the streaming buffer. Returned: success |
|
A lower-bound estimate of the number of bytes currently in the streaming buffer. Returned: success |
|
A lower-bound estimate of the number of rows currently in the streaming buffer. Returned: success |
|
Contains the timestamp of the oldest entry in the streaming buffer, in milliseconds since the epoch, if the streaming buffer is available. Returned: success |
|
Reference describing the ID of this table. Returned: success |
|
The ID of the dataset containing this table. Returned: success |
|
The ID of the project containing this table. Returned: success |
|
The ID of the the table. Returned: success |
|
If specified, configures time-based partitioning for this table. Returned: success |
|
Number of milliseconds for which to keep the storage for a partition. Returned: success |
|
If not set, the table is partitioned by pseudo column, referenced via either ‘_PARTITIONTIME’ as TIMESTAMP type, or ‘_PARTITIONDATE’ as DATE type. If field is specified, the table is instead partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. Returned: success |
|
The only type supported is DAY, which will generate one partition per day. Returned: success |
|
Describes the table type. Returned: success |
|
The view definition. Returned: success |
|
Specifies whether to use BigQuery’s legacy SQL for this view . Returned: success |
|
Describes user-defined function resources used in the query. Returned: success |
|
An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code. Returned: success |
|
A code resource to load from a Google Cloud Storage URI (gs://bucket/path). Returned: success |