imjasonh/gcloud-help
1
0
Fork 0
mirror of https://github.com/imjasonh/gcloud-help synced 2026-07-16 20:36:39 +00:00
Code Activity
gcloud-help/gcloud/dataplex/metadata-jobs/create
Automated fa40c10f6a gcloud: Sat Jun 14 05:31:37 UTC 2025
2025-06-14 05:31:37 +00:00

269 lines
12 KiB
Text
Raw Blame History

NAME
gcloud dataplex metadata-jobs create - create a Dataplex Metadata Job
SYNOPSIS
gcloud dataplex metadata-jobs create [[METADATA_JOB] --location=LOCATION]
--type=TYPE
(--export-output-path=EXPORT_OUTPUT_PATH
((--export-entry-groups=[EXPORT_ENTRY_GROUPS,...]
| --export-organization-level=EXPORT_ORGANIZATION_LEVEL
| --export-projects=[EXPORT_PROJECTS,...])
: --export-aspect-types=[EXPORT_ASPECT_TYPES,...]
--export-entry-types=[EXPORT_ENTRY_TYPES,...])
| [--import-aspect-sync-mode=IMPORT_ASPECT_SYNC_MODE
--import-entry-sync-mode=IMPORT_ENTRY_SYNC_MODE
--import-source-storage-uri=IMPORT_SOURCE_STORAGE_URI
(--import-aspect-types=[IMPORT_ASPECT_TYPES,...]
--import-entry-groups=[IMPORT_ENTRY_GROUPS,...]
--import-entry-link-types=[IMPORT_ENTRY_LINK_TYPES,...]
--import-entry-types=[IMPORT_ENTRY_TYPES,...]
--import-glossaries=[IMPORT_GLOSSARIES,...]
--import-referenced-entry-scopes=[IMPORT_REFERENCED_ENTRY_SCOPES,
...]) : --import-log-level=IMPORT_LOG_LEVEL
--import-source-create-time=IMPORT_SOURCE_CREATE_TIME]) [--async]
[--labels=[KEY=VALUE,...]] [--validate-only] [GCLOUD_WIDE_FLAG ...]
DESCRIPTION
A metadata job represents a long running job on Dataplex Catalog metadata
entries. Some operations include importing and exporting metadata into
entry groups through the usage of entry types and aspect types.
The Metadata Job ID will be used to identify each configuration run. The
Metadata Job id must follow these rules:
▪ Must contain only lowercase letters, numbers, and hyphens.
▪ Must start with a letter.
▪ Must end with a number or a letter.
▪ Must be between 1-63 characters.
▪ Must be unique within the customer project / location.
EXAMPLES
To create a Dataplex Metadata Job with type IMPORT and name my-metadata-job
in location us-central1 with additional parameters, run:
$ gcloud dataplex metadata-jobs create my-metadata-job \
--location=us-central --project=test-project --type=import \
--import-source-storage-uri=gs://test-storage/ \
--import-source-create-time="2019-01-23T12:34:56.123456789Z" \
--import-entry-sync-mode=FULL \
--import-aspect-sync-mode=INCREMENTAL \
--import-log-level="debug" \
--import-entry-groups=projects/test-project/locations/\
us-central1/entryGroups/eg1 \
--import-entry-types="projects/test-project/locations/us-central\
1/entryTypes/et1", \
"projects/test-project/locations/us-central1/entryTypes/et2" \
--import-aspect-types="projects/test-project/locations/us-centra\
l1/aspectTypes/at1", \
"projects/test-project/locations/us-central1/aspectTypes/at2"
To create a Dataplex Metadata Job with type EXPORT and name my-metadata-job
in location us-central1 with additional parameters, run:
$ gcloud dataplex metadata-jobs create my-metadata-job \
--location=us-central --project=test-project --type=export \
--export-output-path=gs://test-storage/ \
--export-entry-groups=projects/test-project/locations/\
us-central1/entryGroups/eg1 \
--export-entry-types="projects/test-project/locations/us-central\
1/entryTypes/et1", \
"projects/test-project/locations/us-central1/entryTypes/et2" \
--export-aspect-types="projects/test-project/locations/us-centra\
l1/aspectTypes/at1", \
"projects/test-project/locations/us-central1/aspectTypes/at2"
POSITIONAL ARGUMENTS
Metadata job resource - Arguments and flags that define the Dataplex
metdata job you want to create. The arguments in this group can be used to
specify the attributes of this resource. (NOTE) Some attributes are not
given arguments in this group but can be set in other ways.
To set the project attribute:
◆ provide the argument metadata_job on the command line with a fully
specified name;
◆ job ID is optional and will be generated if not specified with a
fully specified name;
◆ provide the argument --project on the command line;
◆ set the property core/project.
[METADATA_JOB]
ID of the metadata job or fully qualified identifier for the metadata
job.
To set the metadata_job attribute:
▸ provide the argument metadata_job on the command line;
▸ job ID is optional and will be generated if not specified.
--location=LOCATION
The location of the Dataplex resource.
To set the location attribute:
▸ provide the argument metadata_job on the command line with a
fully specified name;
▸ job ID is optional and will be generated if not specified with a
fully specified name;
▸ provide the argument --location on the command line;
▸ set the property dataplex/location.
REQUIRED FLAGS
--type=TYPE
Type. TYPE must be one of:
EXPORT
A Metadata Export Job will export entries and aspects from the
declared Dataplex scope to the specified Cloud Storage location.
IMPORT
A Metadata Import Job will ingest, update, or delete entries and
aspects into the declared Dataplex entry group.
Settings for metadata job operation.
Exactly one of these must be specified:
Settings for metadata export job operation.
--export-output-path=EXPORT_OUTPUT_PATH
The Cloud Storage location to export metadata to.
This flag argument must be specified if any of the other arguments
in this group are specified.
A boundary on the scope of impact that the metadata export job can
have.
At least one of these must be specified:
--export-aspect-types=[EXPORT_ASPECT_TYPES,...]
The list of aspect types to export metadata from.
--export-entry-types=[EXPORT_ENTRY_TYPES,...]
The list of entry types to export metadata from.
The scope of resources to export metadata from.
Exactly one of these must be specified:
--export-entry-groups=[EXPORT_ENTRY_GROUPS,...]
The list of entry groups to export metadata from.
--export-organization-level=EXPORT_ORGANIZATION_LEVEL
Whether to export metadata at the organization level.
--export-projects=[EXPORT_PROJECTS,...]
The list of projects to export metadata from.
Settings for metadata import job operation.
--import-aspect-sync-mode=IMPORT_ASPECT_SYNC_MODE
Type. IMPORT_ASPECT_SYNC_MODE must be one of:
FULL
All resources in the job's scope are modified. If a resource
exists in Dataplex but isn't included in the metadata import
file, the resource is deleted when you run the metadata job.
Use this mode to perform a full sync of the set of entries in
the job scope.
INCREMENTAL
Only the entries and aspects that are explicitly included in
the metadata import file are modified. Use this mode to modify
a subset of resources while leaving unreferenced resources
unchanged.
This flag argument must be specified if any of the other arguments
in this group are specified.
--import-entry-sync-mode=IMPORT_ENTRY_SYNC_MODE
Type. IMPORT_ENTRY_SYNC_MODE must be one of:
FULL
All resources in the job's scope are modified. If a resource
exists in Dataplex but isn't included in the metadata import
file, the resource is deleted when you run the metadata job.
Use this mode to perform a full sync of the set of entries in
the job scope.
INCREMENTAL
Only the entries and aspects that are explicitly included in
the metadata import file are modified. Use this mode to modify
a subset of resources while leaving unreferenced resources
unchanged.
This flag argument must be specified if any of the other arguments
in this group are specified.
--import-source-storage-uri=IMPORT_SOURCE_STORAGE_URI
The Dataplex source storage URI to import metadata from.
This flag argument must be specified if any of the other arguments
in this group are specified.
--import-log-level=IMPORT_LOG_LEVEL
Type. IMPORT_LOG_LEVEL must be one of:
DEBUG
Debug-level logging. Captures detailed logs for each import
item. Use debug-level logging to troubleshoot issues with
specific import items. For example, use debug-level logging to
identify resources that are missing from the job scope, entries
or aspects that don't conform to the associated entry type or
aspect type, or other misconfigurations with the metadata
import file..
INFO
Info-level logging. Captures logs at the overall job level.
Includes aggregate logs about import items, but doesn't specify
which import item has an error..
--import-source-create-time=IMPORT_SOURCE_CREATE_TIME
Time at which the event took place. See $ gcloud topic datetimes
for information on supported time formats.
A boundary on the scope of impact that the metadata import job can
have.
At least one of these must be specified:
--import-aspect-types=[IMPORT_ASPECT_TYPES,...]
The list of aspect types to import metadata jobs into.
--import-entry-groups=[IMPORT_ENTRY_GROUPS,...]
The list of entry groups to import metadata jobs into.
--import-entry-link-types=[IMPORT_ENTRY_LINK_TYPES,...]
The list of entry link types to import metadata jobs into.
--import-entry-types=[IMPORT_ENTRY_TYPES,...]
The list of entry types to import metadata jobs into.
--import-glossaries=[IMPORT_GLOSSARIES,...]
The list of glossaries to import metadata jobs into.
--import-referenced-entry-scopes=[IMPORT_REFERENCED_ENTRY_SCOPES,...]
The list of referenced entry scopes to import metadata jobs into.
OPTIONAL FLAGS
--async
Return immediately, without waiting for the operation in progress to
complete.
--labels=[KEY=VALUE,...]
List of label KEY=VALUE pairs to add.
Keys must start with a lowercase character and contain only hyphens
(-), underscores (_), lowercase characters, and numbers. Values must
contain only hyphens (-), underscores (_), lowercase characters, and
numbers.
--validate-only
Validate the create action, but don't actually perform it.
GCLOUD WIDE FLAGS
These flags are available to all commands: --access-token-file, --account,
--billing-project, --configuration, --flags-file, --flatten, --format,
--help, --impersonate-service-account, --log-http, --project, --quiet,
--trace-token, --user-output-enabled, --verbosity.
Run $ gcloud help for details.
NOTES
This variant is also available:
$ gcloud alpha dataplex metadata-jobs create
View git blame Copy permalink