imjasonh/gcloud-help
1
0
Fork 0
mirror of https://github.com/imjasonh/gcloud-help synced 2026-07-18 06:47:12 +00:00
Code Activity

gcloud: Wed Oct 25 11:42:23 UTC 2023

Browse source
This commit is contained in:
Automated 2023-10-25 11:42:23 +00:00
parent a15665661a
commit d34c41a2ce
305 changed files with 5549 additions and 699 deletions
Download patch file Download diff file Expand all files Collapse all files

14
gcloud/dataproc/jobs/submit/flink
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit flink (--class=MAIN_CLASS | --jar=MAIN_JAR)
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...]) [--async]
[--bucket=BUCKET] [--driver-log-levels=[PACKAGE=LEVEL,...]]
[--jars=[JAR,...]] [--labels=[KEY=VALUE,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--jars=[JAR,...]]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
[--properties=[PROPERTY=VALUE,...]] [--properties-file=PROPERTIES_FILE]
@ -78,6 +80,16 @@ OPTIONAL FLAGS
List of package to log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO.
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--jars=[JAR,...]
Comma-separated list of jar files to provide to the task manager
classpaths.

14
gcloud/dataproc/jobs/submit/hadoop
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit hadoop (--class=MAIN_CLASS | --jar=MAIN_JAR)
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
[--archives=[ARCHIVE,...]] [--async] [--bucket=BUCKET]
[--driver-log-levels=[PACKAGE=LEVEL,...]] [--files=[FILE,...]]
[--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--files=[FILE,...]]
[--jars=[JAR,...]] [--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -82,6 +84,16 @@ OPTIONAL FLAGS
A list of package to log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--files=[FILE,...]
Comma separated list of file paths to be provided to the job. A file
path can either be a path to a local file or a path to a file already

14
gcloud/dataproc/jobs/submit/hive
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit hive
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
(--execute=QUERY, -e QUERY | --file=FILE, -f FILE) [--async]
[--bucket=BUCKET] [--continue-on-failure] [--jars=[JAR,...]]
[--bucket=BUCKET] [--continue-on-failure]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--jars=[JAR,...]]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL] [--params=[PARAM=VALUE,...]]
@ -63,6 +65,16 @@ OPTIONAL FLAGS
--continue-on-failure
Whether to continue if a single query fails.
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--jars=[JAR,...]
Comma separated list of jar files to be provided to the Hive and MR.
May contain UDFs.

14
gcloud/dataproc/jobs/submit/pig
View file

@ -6,7 +6,9 @@ SYNOPSIS
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
(--execute=QUERY, -e QUERY | --file=FILE, -f FILE) [--async]
[--bucket=BUCKET] [--continue-on-failure]
[--driver-log-levels=[PACKAGE=LEVEL,...]] [--jars=[JAR,...]]
[--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--jars=[JAR,...]]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL] [--params=[PARAM=VALUE,...]]
@ -71,6 +73,16 @@ OPTIONAL FLAGS
A list of package to log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--jars=[JAR,...]
Comma separated list of jar files to be provided to Pig and MR. May
contain UDFs.

12
gcloud/dataproc/jobs/submit/presto
View file

@ -7,6 +7,8 @@ SYNOPSIS
(--execute=QUERY, -e QUERY | --file=FILE, -f FILE) [--async]
[--bucket=BUCKET] [--client-tags=[CLIENT_TAG,...]]
[--continue-on-failure] [--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -71,6 +73,16 @@ OPTIONAL FLAGS
A list of package-to-log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--labels=[KEY=VALUE,...]
List of label KEY=VALUE pairs to add.

14
gcloud/dataproc/jobs/submit/pyspark
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit pyspark PY_FILE
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
[--archives=[ARCHIVE,...]] [--async] [--bucket=BUCKET]
[--driver-log-levels=[PACKAGE=LEVEL,...]] [--files=[FILE,...]]
[--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--files=[FILE,...]]
[--jars=[JAR,...]] [--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -73,6 +75,16 @@ OPTIONAL FLAGS
package and value is the log4j log level. For example:
root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--files=[FILE,...]
Comma separated list of files to be placed in the working directory of
both the app master and executors.

14
gcloud/dataproc/jobs/submit/spark
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit spark (--class=MAIN_CLASS | --jar=MAIN_JAR)
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
[--archives=[ARCHIVE,...]] [--async] [--bucket=BUCKET]
[--driver-log-levels=[PACKAGE=LEVEL,...]] [--files=[FILE,...]]
[--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--files=[FILE,...]]
[--jars=[JAR,...]] [--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -84,6 +86,16 @@ OPTIONAL FLAGS
List of package to log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--files=[FILE,...]
Comma separated list of files to be placed in the working directory of
both the app master and executors.

14
gcloud/dataproc/jobs/submit/spark-r
View file

@ -5,7 +5,9 @@ SYNOPSIS
gcloud dataproc jobs submit spark-r R_FILE
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
[--archives=[ARCHIVE,...]] [--async] [--bucket=BUCKET]
[--driver-log-levels=[PACKAGE=LEVEL,...]] [--files=[FILE,...]]
[--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--files=[FILE,...]]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -71,6 +73,16 @@ OPTIONAL FLAGS
package and value is the log4j log level. For example:
root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--files=[FILE,...]
Comma separated list of files to be placed in the working directory of
both the app master and executors.

14
gcloud/dataproc/jobs/submit/spark-sql
View file

@ -6,7 +6,9 @@ SYNOPSIS
(--cluster=CLUSTER | --cluster-labels=[KEY=VALUE,...])
(--execute=QUERY, -e QUERY | --file=FILE, -f FILE) [--async]
[--bucket=BUCKET] [--driver-log-levels=[PACKAGE=LEVEL,...]]
[--jars=[JAR,...]] [--labels=[KEY=VALUE,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES] [--jars=[JAR,...]]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL] [--params=[PARAM=VALUE,...]]
[--properties=[PROPERTY=VALUE,...]] [--properties-file=PROPERTIES_FILE]
@ -64,6 +66,16 @@ OPTIONAL FLAGS
A list of package to log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--jars=[JAR,...]
Comma separated list of jar files to be provided to the executor and
driver classpaths. May contain UDFs.

12
gcloud/dataproc/jobs/submit/trino
View file

@ -7,6 +7,8 @@ SYNOPSIS
(--execute=QUERY, -e QUERY | --file=FILE, -f FILE) [--async]
[--bucket=BUCKET] [--client-tags=[CLIENT_TAG,...]]
[--continue-on-failure] [--driver-log-levels=[PACKAGE=LEVEL,...]]
[--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB]
[--driver-required-vcores=DRIVER_REQUIRED_VCORES]
[--labels=[KEY=VALUE,...]]
[--max-failures-per-hour=MAX_FAILURES_PER_HOUR]
[--max-failures-total=MAX_FAILURES_TOTAL]
@ -71,6 +73,16 @@ OPTIONAL FLAGS
A list of package-to-log4j log level pairs to configure driver logging.
For example: root=FATAL,com.example=INFO
--driver-required-memory-mb=DRIVER_REQUIRED_MEMORY_MB
The memory allocation requested by the job driver in megabytes (MB) for
execution on the driver node group (it is used only by clusters with a
driver node group).
--driver-required-vcores=DRIVER_REQUIRED_VCORES
The vCPU allocation requested by the job driver for execution on the
driver node group (it is used only by clusters with a driver node
group).
--labels=[KEY=VALUE,...]
List of label KEY=VALUE pairs to add.