1
0
Fork 0
mirror of https://github.com/imjasonh/gcloud-help synced 2026-07-11 07:29:40 +00:00
gcloud-help/gcloud/beta/datastream/streams/create

371 lines
13 KiB
Text
Raw Normal View History

2022-03-01 04:29:52 +00:00
NAME
gcloud beta datastream streams create - creates a Datastream stream
SYNOPSIS
gcloud beta datastream streams create (STREAM : --location=LOCATION)
--display-name=DISPLAY_NAME
(--backfill-none
| --backfill-all --mysql-excluded-objects=MYSQL_EXCLUDED_OBJECTS
2022-10-26 10:52:11 +00:00
| --oracle-excluded-objects=ORACLE_EXCLUDED_OBJECTS
| --postgresql-excluded-objects=POSTGRESQL_EXCLUDED_OBJECTS)
2022-03-01 04:29:52 +00:00
(--destination-name=DESTINATION_NAME
2022-10-18 11:38:19 +00:00
(--bigquery-destination-config=BIGQUERY_DESTINATION_CONFIG
| --gcs-destination-config=GCS_DESTINATION_CONFIG))
2022-03-01 04:29:52 +00:00
(--source-name=SOURCE_NAME (--mysql-source-config=MYSQL_SOURCE_CONFIG
2022-10-26 10:52:11 +00:00
| --oracle-source-config=ORACLE_SOURCE_CONFIG
| --postgresql-source-config=POSTGRESQL_SOURCE_CONFIG))
2022-03-01 04:29:52 +00:00
[--labels=[KEY=VALUE,...]] [--force | --validate-only]
[GCLOUD_WIDE_FLAG ...]
DESCRIPTION
2022-04-13 08:33:42 +00:00
(BETA) (DEPRECATED) Datastream beta version is deprecated. Please
use`gcloud datastream streams create command instead.
2023-04-26 10:24:37 +00:00
Create a Datastream stream. If successful, the response body contains a
newly created instance of Operation. To get the operation result, call:
describe OPERATION
2022-03-01 04:29:52 +00:00
EXAMPLES
2022-05-04 09:15:29 +00:00
To create a stream with an Oracle source and a Google Cloud Storage
2022-03-01 04:29:52 +00:00
destination:
$ gcloud beta datastream streams create STREAM \
--location=us-central1 --display-name=my-stream \
--source-name=source --oracle-source-config=source_config.json \
--destination-name=destination \
--gcs-destination-config=destination_config.json --backfill-none
2022-05-04 09:15:29 +00:00
To create a stream with a MySQL source and a Cloud Storage destination and
that excludes some objects from being backfilled:
2022-03-01 04:29:52 +00:00
$ gcloud beta datastream streams create STREAM \
--location=us-central1 --display-name=my-stream \
--source-name=source --mysql-source-config=source_config.json \
--destination-name=destination \
--gcs-destination-config=destination_config.json \
--backfill-all --mysql-excluded-objects=excluded_objects.json
POSITIONAL ARGUMENTS
Stream resource - The stream to create. The arguments in this group can be
used to specify the attributes of this resource. (NOTE) Some attributes
2023-05-04 10:43:54 +00:00
are not given arguments in this group but can be set in other ways.
To set the project attribute:
2022-03-01 04:29:52 +00:00
◆ provide the argument stream on the command line with a fully
specified name;
◆ provide the argument --project on the command line;
◆ set the property core/project.
This must be specified.
STREAM
2023-05-04 10:43:54 +00:00
ID of the stream or fully qualified identifier for the stream.
To set the stream attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument stream on the command line.
2022-08-10 08:48:58 +00:00
This positional argument must be specified if any of the other
arguments in this group are specified.
2022-03-01 04:29:52 +00:00
--location=LOCATION
2023-05-04 10:43:54 +00:00
The Cloud location for the stream.
To set the location attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument stream on the command line with a fully
specified name;
▸ provide the argument --location on the command line.
REQUIRED FLAGS
--display-name=DISPLAY_NAME
Friendly name for the stream.
Exactly one of these must be specified:
--backfill-none
Do not automatically backfill any objects.
--backfill-all
Automatically backfill objects included in the stream source
configuration. Specific objects can be excluded.
At most one of these can be specified:
--mysql-excluded-objects=MYSQL_EXCLUDED_OBJECTS
2023-09-20 10:45:25 +00:00
Path to a YAML (or JSON) file containing the MySQL data sources to
avoid backfilling.
The JSON file is formatted as follows, with camelCase field naming:
{
"mysqlDatabases": [
{
"database":"sample_database",
"mysqlTables": [
{
"table": "sample_table",
"mysqlColumns": [
{
"column": "sample_column",
}
]
}
]
}
]
}
}
2022-03-01 04:29:52 +00:00
--oracle-excluded-objects=ORACLE_EXCLUDED_OBJECTS
2023-09-20 10:45:25 +00:00
Path to a YAML (or JSON) file containing the Oracle data sources to
avoid backfilling.
The JSON file is formatted as follows, with camelCase field naming:
{
"oracleSchemas": [
{
"schema": "SAMPLE",
"oracleTables": [
{
"table": "SAMPLE_TABLE",
"oracleColumns": [
{
"column": "COL",
}
]
}
]
}
]
}
}
2022-03-01 04:29:52 +00:00
2022-10-26 10:52:11 +00:00
--postgresql-excluded-objects=POSTGRESQL_EXCLUDED_OBJECTS
2023-09-20 10:45:25 +00:00
Path to a YAML (or JSON) file containing the PostgreSQL data
sources to avoid backfilling.
The JSON file is formatted as follows, with camelCase field naming:
{
"postgresqlSchemas": [
{
"schema": "SAMPLE",
"postgresqlTables": [
{
"table": "SAMPLE_TABLE",
"postgresqlColumns": [
{
"column": "COL",
}
]
}
]
}
]
}
}
2022-10-26 10:52:11 +00:00
2022-03-01 04:29:52 +00:00
This must be specified.
Connection profile resource - Resource ID of the destination connection
profile. This represents a Cloud resource. (NOTE) Some attributes are
2023-05-04 10:43:54 +00:00
not given arguments in this group but can be set in other ways.
To set the project attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument --destination-name on the command line with a
fully specified name;
▸ provide the argument --project on the command line;
2023-05-04 10:43:54 +00:00
▸ set the property core/project.
To set the location attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument --destination-name on the command line with a
fully specified name;
▸ provide the argument --location on the command line.
This must be specified.
--destination-name=DESTINATION_NAME
ID of the connection_profile or fully qualified identifier for the
2023-05-04 10:43:54 +00:00
connection_profile.
To set the connection_profile attribute:
2022-03-01 04:29:52 +00:00
▫ provide the argument --destination-name on the command line.
2022-10-18 11:38:19 +00:00
Exactly one of these must be specified:
2022-03-01 04:29:52 +00:00
2022-10-18 11:38:19 +00:00
--bigquery-destination-config=BIGQUERY_DESTINATION_CONFIG
Path to a YAML (or JSON) file containing the configuration for
Google BigQuery Destination Config.
The JSON file is formatted as follows:
{
2023-02-15 10:13:15 +00:00
"sourceHierarchyDatasets": {
"datasetTemplate": {
2022-10-18 11:38:19 +00:00
"location": "us-central1",
2023-02-15 10:13:15 +00:00
"datasetIdPrefix": "my_prefix",
"kmsKeyName": "projects/{project}/locations/{location}/keyRings/{key_ring}/cryptoKeys/{cryptoKey}"
2022-10-18 11:38:19 +00:00
}
},
2023-05-04 10:43:54 +00:00
"dataFreshness": "3600s"
2022-10-18 11:38:19 +00:00
}
--gcs-destination-config=GCS_DESTINATION_CONFIG
Path to a YAML (or JSON) file containing the configuration for
Google Cloud Storage Destination Config.
The JSON file is formatted as follows:
{
"path": "some/path",
2023-02-15 10:13:15 +00:00
"fileRotationMb":5,
"fileRotationInterval":"15s",
"avroFileFormat": {}
2022-10-18 11:38:19 +00:00
}
2022-03-01 04:29:52 +00:00
This must be specified.
Connection profile resource - Resource ID of the source connection
profile. This represents a Cloud resource. (NOTE) Some attributes are
2023-05-04 10:43:54 +00:00
not given arguments in this group but can be set in other ways.
To set the project attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument --source-name on the command line with a fully
specified name;
▸ provide the argument --project on the command line;
2023-05-04 10:43:54 +00:00
▸ set the property core/project.
To set the location attribute:
2022-03-01 04:29:52 +00:00
▸ provide the argument --source-name on the command line with a fully
specified name;
▸ provide the argument --location on the command line.
This must be specified.
--source-name=SOURCE_NAME
ID of the connection_profile or fully qualified identifier for the
2023-05-04 10:43:54 +00:00
connection_profile.
To set the connection_profile attribute:
2022-03-01 04:29:52 +00:00
▫ provide the argument --source-name on the command line.
Exactly one of these must be specified:
--mysql-source-config=MYSQL_SOURCE_CONFIG
Path to a YAML (or JSON) file containing the configuration for
2022-10-26 10:52:11 +00:00
MySQL Source Config.
2022-03-01 04:29:52 +00:00
The JSON file is formatted as follows, with snake_case field
naming:
{
"allowlist": {},
"rejectlist": {
"mysql_databases": [
{
"database_name":"sample_database",
"mysql_tables": [
{
"table_name": "sample_table",
"mysql_columns": [
{
"column_name": "sample_column",
}
]
}
]
}
]
}
}
--oracle-source-config=ORACLE_SOURCE_CONFIG
Path to a YAML (or JSON) file containing the configuration for
Oracle Source Config.
The JSON file is formatted as follows, with snake_case field
naming:
{
"allowlist": {},
"rejectlist": {
"oracle_schemas": [
{
"schema_name": "SAMPLE",
"oracle_tables": [
{
"table_name": "SAMPLE_TABLE",
"oracle_columns": [
{
"column_name": "COL",
}
]
}
]
}
]
}
}
2022-10-26 10:52:11 +00:00
--postgresql-source-config=POSTGRESQL_SOURCE_CONFIG
Path to a YAML (or JSON) file containing the configuration for
PostgreSQL Source Config.
2023-02-15 10:13:15 +00:00
The JSON file is formatted as follows, with camelCase field naming:
2022-10-26 10:52:11 +00:00
{
2023-02-15 10:13:15 +00:00
"includeObjects": {},
"excludeObjects": {
"postgresqlSchemas": [
2022-10-26 10:52:11 +00:00
{
"schema": "SAMPLE",
2023-02-15 10:13:15 +00:00
"postgresqlTables": [
2022-10-26 10:52:11 +00:00
{
"table": "SAMPLE_TABLE",
2023-02-15 10:13:15 +00:00
"postgresqlColumns": [
2022-10-26 10:52:11 +00:00
{
"column": "COL",
}
]
}
]
}
]
},
2023-02-15 10:13:15 +00:00
"replicationSlot": "SAMPLE_REPLICATION_SLOT",
2022-10-26 10:52:11 +00:00
"publication": "SAMPLE_PUBLICATION"
}
2022-03-01 04:29:52 +00:00
OPTIONAL FLAGS
--labels=[KEY=VALUE,...]
List of label KEY=VALUE pairs to add.
Keys must start with a lowercase character and contain only hyphens
(-), underscores (_), lowercase characters, and numbers. Values must
contain only hyphens (-), underscores (_), lowercase characters, and
numbers.
At most one of these can be specified:
--force
Create the stream without validating it.
--validate-only
Only validate the stream, but do not create any resources. The
default is false.
GCLOUD WIDE FLAGS
These flags are available to all commands: --access-token-file, --account,
--billing-project, --configuration, --flags-file, --flatten, --format,
--help, --impersonate-service-account, --log-http, --project, --quiet,
--trace-token, --user-output-enabled, --verbosity.
Run $ gcloud help for details.
NOTES
This command is currently in beta and might change without notice. This
variant is also available:
$ gcloud datastream streams create