Dataflow V1beta3 API - Class Google::Cloud::Dataflow::V1beta3::WorkerPool (v0.4.0)

Reference documentation and code samples for the Dataflow V1beta3 API class Google::Cloud::Dataflow::V1beta3::WorkerPool.

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Inherits

  • Object

Extended By

  • Google::Protobuf::MessageExts::ClassMethods

Includes

  • Google::Protobuf::MessageExts

Methods

#autoscaling_settings

def autoscaling_settings() -> ::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings
Returns

#autoscaling_settings=

def autoscaling_settings=(value) -> ::Google::Cloud::Dataflow::V1beta3::AutoscalingSettings
Parameter
Returns

#data_disks

def data_disks() -> ::Array<::Google::Cloud::Dataflow::V1beta3::Disk>
Returns

#data_disks=

def data_disks=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::Disk>
Parameter
Returns

#default_package_set

def default_package_set() -> ::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet
Returns

#default_package_set=

def default_package_set=(value) -> ::Google::Cloud::Dataflow::V1beta3::DefaultPackageSet
Parameter
Returns

#disk_size_gb

def disk_size_gb() -> ::Integer
Returns
  • (::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

#disk_size_gb=

def disk_size_gb=(value) -> ::Integer
Parameter
  • value (::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.
Returns
  • (::Integer) — Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

#disk_source_image

def disk_source_image() -> ::String
Returns
  • (::String) — Fully qualified source image for disks.

#disk_source_image=

def disk_source_image=(value) -> ::String
Parameter
  • value (::String) — Fully qualified source image for disks.
Returns
  • (::String) — Fully qualified source image for disks.

#disk_type

def disk_type() -> ::String
Returns
  • (::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

#disk_type=

def disk_type=(value) -> ::String
Parameter
  • value (::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.
Returns
  • (::String) — Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

#ip_configuration

def ip_configuration() -> ::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration
Returns

#ip_configuration=

def ip_configuration=(value) -> ::Google::Cloud::Dataflow::V1beta3::WorkerIPAddressConfiguration
Parameter
Returns

#kind

def kind() -> ::String
Returns
  • (::String) — The kind of the worker pool; currently only harness and shuffle are supported.

#kind=

def kind=(value) -> ::String
Parameter
  • value (::String) — The kind of the worker pool; currently only harness and shuffle are supported.
Returns
  • (::String) — The kind of the worker pool; currently only harness and shuffle are supported.

#machine_type

def machine_type() -> ::String
Returns
  • (::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

#machine_type=

def machine_type=(value) -> ::String
Parameter
  • value (::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.
Returns
  • (::String) — Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

#metadata

def metadata() -> ::Google::Protobuf::Map{::String => ::String}
Returns
  • (::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.

#metadata=

def metadata=(value) -> ::Google::Protobuf::Map{::String => ::String}
Parameter
  • value (::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.
Returns
  • (::Google::Protobuf::Map{::String => ::String}) — Metadata to set on the Google Compute Engine VMs.

#network

def network() -> ::String
Returns
  • (::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

#network=

def network=(value) -> ::String
Parameter
  • value (::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".
Returns
  • (::String) — Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

#num_threads_per_worker

def num_threads_per_worker() -> ::Integer
Returns
  • (::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

#num_threads_per_worker=

def num_threads_per_worker=(value) -> ::Integer
Parameter
  • value (::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).
Returns
  • (::Integer) — The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

#num_workers

def num_workers() -> ::Integer
Returns
  • (::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

#num_workers=

def num_workers=(value) -> ::Integer
Parameter
  • value (::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.
Returns
  • (::Integer) — Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

#on_host_maintenance

def on_host_maintenance() -> ::String
Returns
  • (::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.

#on_host_maintenance=

def on_host_maintenance=(value) -> ::String
Parameter
  • value (::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.
Returns
  • (::String) — The action to take on host maintenance, as defined by the Google Compute Engine API.

#packages

def packages() -> ::Array<::Google::Cloud::Dataflow::V1beta3::Package>
Returns

#packages=

def packages=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::Package>
Parameter
Returns

#pool_args

def pool_args() -> ::Google::Protobuf::Any
Returns

#pool_args=

def pool_args=(value) -> ::Google::Protobuf::Any
Parameter
Returns

#sdk_harness_container_images

def sdk_harness_container_images() -> ::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>
Returns
  • (::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

#sdk_harness_container_images=

def sdk_harness_container_images=(value) -> ::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>
Parameter
  • value (::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.
Returns
  • (::Array<::Google::Cloud::Dataflow::V1beta3::SdkHarnessContainerImage>) — Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

#subnetwork

def subnetwork() -> ::String
Returns
  • (::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

#subnetwork=

def subnetwork=(value) -> ::String
Parameter
  • value (::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".
Returns
  • (::String) — Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

#taskrunner_settings

def taskrunner_settings() -> ::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings
Returns

#taskrunner_settings=

def taskrunner_settings=(value) -> ::Google::Cloud::Dataflow::V1beta3::TaskRunnerSettings
Parameter
Returns

#teardown_policy

def teardown_policy() -> ::Google::Cloud::Dataflow::V1beta3::TeardownPolicy
Returns
  • (::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

    If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

    If unknown or unspecified, the service will attempt to choose a reasonable default.

#teardown_policy=

def teardown_policy=(value) -> ::Google::Cloud::Dataflow::V1beta3::TeardownPolicy
Parameter
  • value (::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

    If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

    If unknown or unspecified, the service will attempt to choose a reasonable default.

Returns
  • (::Google::Cloud::Dataflow::V1beta3::TeardownPolicy) — Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down.

    If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs.

    If unknown or unspecified, the service will attempt to choose a reasonable default.

#worker_harness_container_image

def worker_harness_container_image() -> ::String
Returns
  • (::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

    Deprecated for the Fn API path. Use sdk_harness_container_images instead.

#worker_harness_container_image=

def worker_harness_container_image=(value) -> ::String
Parameter
  • value (::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

    Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Returns
  • (::String) — Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry.

    Deprecated for the Fn API path. Use sdk_harness_container_images instead.

#zone

def zone() -> ::String
Returns
  • (::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

#zone=

def zone=(value) -> ::String
Parameter
  • value (::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.
Returns
  • (::String) — Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.