1
0
Fork 0
mirror of https://github.com/imjasonh/gcloud-help synced 2026-07-08 02:25:19 +00:00
gcloud-help/gcloud/transfer/agents/install

195 lines
8.7 KiB
Text
Raw Normal View History

2022-03-16 08:47:18 +00:00
NAME
gcloud transfer agents install - install Transfer Service agents
SYNOPSIS
gcloud transfer agents install --pool=POOL [--count=COUNT]
2024-11-20 10:03:22 +00:00
[--creds-file=CREDS_FILE] [--docker-network=NETWORK]
2023-06-02 10:25:45 +00:00
[--[no-]enable-multipart] [--id-prefix=ID_PREFIX]
2022-03-16 08:47:18 +00:00
[--logs-directory=LOGS_DIRECTORY; default="/tmp"]
[--memlock-limit=MEMLOCK_LIMIT; default=64000000]
[--mount-directories=[MOUNT-DIRECTORIES,...]] [--proxy=PROXY]
2024-03-06 09:53:10 +00:00
[--s3-compatible-mode]
[--hdfs-namenode-uri=HDFS_NAMENODE_URI --hdfs-username=HDFS_USERNAME
--hdfs-data-transfer-protection=HDFS_DATA_TRANSFER_PROTECTION]
[--kerberos-config-file=KERBEROS_CONFIG_FILE
--kerberos-keytab-file=KERBEROS_KEYTAB_FILE
--kerberos-user-principal=KERBEROS_USER_PRINCIPAL
--kerberos-service-principal=KERBEROS_SERVICE_PRINCIPAL]
[GCLOUD_WIDE_FLAG ...]
2022-03-16 08:47:18 +00:00
DESCRIPTION
Install Transfer Service agents to enable you to transfer data to or from
POSIX filesystems, such as on-premises filesystems. Agents are installed
locally on your machine and run inside Docker containers.
EXAMPLES
2023-01-19 00:56:33 +00:00
To create an agent pool for your agent, see the gcloud transfer agent-pools
create command.
2022-03-16 08:47:18 +00:00
To install an agent that authenticates with your user account credentials
and has default agent parameters, run:
2023-01-19 00:56:33 +00:00
$ gcloud transfer agents install --pool=AGENT_POOL
You will be prompted to run a command to generate a credentials file if one
does not already exist.
2022-03-16 08:47:18 +00:00
To install an agent that authenticates with a service account with
credentials stored at '/example/path.json', run:
2023-01-19 00:56:33 +00:00
$ gcloud transfer agents install --creds-file=/example/path.json \
--pool=AGENT_POOL
2022-03-16 08:47:18 +00:00
2025-09-10 11:12:24 +00:00
To install an agent using service account impersonation, run:
$ gcloud transfer agents install --creds-file=/example/path.json \
--pool=CUSTOM_AGENT_POOL \
--impersonate-service-account=impersonated-account@project-id.ia\
m.gserviceaccount.com
Note : The --impersonate-service-account flag only applies to the API calls
made by gcloud during the agent installation and authorization process. The
impersonated credentials are not passed to the transfer agent's runtime
environment. The agent itself does not support impersonation and will use
the credentials provided via the --creds-file flag or the default gcloud
authenticated account for all of its operations. To grant the agent
permissions, you must provide a service account key with the required
direct roles (e.g., Storage Transfer Agent, Storage Object User)
2022-03-16 08:47:18 +00:00
REQUIRED FLAGS
--pool=POOL
The agent pool to associate with the newly installed agent. When
creating transfer jobs, the agent pool parameter will determine which
agents are activated.
2024-03-06 09:53:10 +00:00
FLAGS
2022-03-16 08:47:18 +00:00
--count=COUNT
Specify the number of agents to install on your current machine. System
requirements: 8 GB of memory and 4 CPUs per agent.
Note: If the 'id-prefix' flag is specified, Transfer Service increments
a number value after each prefix. Example: prefix1, prefix2, etc.
--creds-file=CREDS_FILE
Specify the path to the service account's credentials file.
No input required if authenticating with your user account credentials,
which Transfer Service will look for in your system.
Note that the credentials location will be mounted to the agent
container.
2024-11-20 10:03:22 +00:00
--docker-network=NETWORK
Specify the network to connect the container to. This flag maps
2023-06-02 10:25:45 +00:00
directly to the --network flag in the underlying 'docker run' command.
2024-11-20 10:03:22 +00:00
If binding directly to the host's network is an option, then setting
this value to 'host' can dramatically improve transfer performance.
2023-06-02 10:25:45 +00:00
2023-01-19 00:56:33 +00:00
--[no-]enable-multipart
2022-08-24 09:45:53 +00:00
Split up files and transfer the resulting chunks in parallel before
merging them at the destination. Can be used make transfers of large
files faster as long as the network and disk speed are not limiting
2023-01-19 00:56:33 +00:00
factors. If unset, agent decides when to use the feature. Use
--enable-multipart to enable and --no-enable-multipart to disable.
2022-08-24 09:45:53 +00:00
2022-03-16 08:47:18 +00:00
--id-prefix=ID_PREFIX
An optional prefix to add to the agent ID to help identify the agent.
--logs-directory=LOGS_DIRECTORY; default="/tmp"
Specify the absolute path to the directory you want to store transfer
logs in. If not specified, gcloud transfer will mount your /tmp
directory for logs.
--memlock-limit=MEMLOCK_LIMIT; default=64000000
Set the agent container's memlock limit. A value of 64000000 (default)
or higher is required to ensure that agent versions 1.14 or later have
enough locked memory to be able to start.
--mount-directories=[MOUNT-DIRECTORIES,...]
If you want to grant agents access to specific parts of your filesystem
instead of the entire filesystem, specify which directory paths to
mount to the agent container. Multiple paths must be separated by
commas with no spaces (e.g.,
--mount-directories=/system/path/to/dir1,/path/to/dir2). When mounting
specific directories, gcloud transfer will also mount a directory for
logs (either /tmp or what you've specified for --logs-directory) and
your Google credentials file for agent authentication.
2022-10-26 10:52:11 +00:00
It is strongly recommended that you use this flag. If this flag isn't
specified, gcloud transfer will mount your entire filesystem to the
agent container and give the agent root access.
2022-03-16 08:47:18 +00:00
--proxy=PROXY
Specify the HTTP URL and port of a proxy server if you want to use a
forward proxy. For example, to use the URL 'example.com' and port
'8080' specify 'http://www.example.com:8080/'
Ensure that you specify the HTTP URL and not an HTTPS URL to avoid
double-wrapping requests in TLS encryption. Double-wrapped requests
prevent the proxy server from sending valid outbound requests.
2022-07-13 09:59:38 +00:00
--s3-compatible-mode
Allow the agent to work with S3-compatible sources. This flag blocks
2022-08-10 08:48:58 +00:00
the agent's ability to work with other source types (e.g., file
systems).
When using this flag, you must provide source credentials either as
environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY or as
default credentials in your system's configuration files.
To provide credentials as environment variables, run:
AWS_ACCESS_KEY_ID="id" AWS_SECRET_ACCESS_KEY="secret" gcloud transfer agents install --s3-compatible-mode
2022-07-13 09:59:38 +00:00
2024-03-06 09:53:10 +00:00
HDFS FLAGS
--hdfs-namenode-uri=HDFS_NAMENODE_URI
A URI representing an HDFS cluster including a schema, namenode, and
port. Examples: "rpc://my-namenode:8020", "http://my-namenode:9870".
Use "http" or "https" for WebHDFS. If no schema is provided, the CLI
assumes native "rpc". If no port is provided, the default is 8020 for
RPC, 9870 for HTTP, and 9871 for HTTPS. For example, the input
"my-namenode" becomes "rpc://my-namenode:8020".
--hdfs-username=HDFS_USERNAME
Username for connecting to an HDFS cluster with simple auth.
--hdfs-data-transfer-protection=HDFS_DATA_TRANSFER_PROTECTION
Client-side quality of protection setting for Kerberized clusters.
Client-side QOP value cannot be more restrictive than the server-side
QOP value. HDFS_DATA_TRANSFER_PROTECTION must be one of:
authentication, integrity, privacy.
Kerberos FLAGS
--kerberos-config-file=KERBEROS_CONFIG_FILE
Path to Kerberos config file.
--kerberos-keytab-file=KERBEROS_KEYTAB_FILE
Path to a Keytab file containing the user principal specified with the
--kerberos-user-principal flag.
--kerberos-user-principal=KERBEROS_USER_PRINCIPAL
Kerberos user principal to use when connecting to an HDFS cluster via
Kerberos auth.
--kerberos-service-principal=KERBEROS_SERVICE_PRINCIPAL
Kerberos service principal to use, of the form "<primary>/<instance>".
Realm is mapped from your Kerberos config. Any supplied realm is
ignored. If not passed in, it will default to "hdfs/<namenode_fqdn>"
(fqdn = fully qualified domain name).
2022-03-16 08:47:18 +00:00
GCLOUD WIDE FLAGS
These flags are available to all commands: --access-token-file, --account,
--billing-project, --configuration, --flags-file, --flatten, --format,
--help, --impersonate-service-account, --log-http, --project, --quiet,
--trace-token, --user-output-enabled, --verbosity.
Run $ gcloud help for details.
NOTES
This variant is also available:
$ gcloud alpha transfer agents install