InstanceGroupConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)
The config settings for Compute Engine resources in an instance group, such as a master or worker group.
Attributes | |
---|---|
Name | Description |
num_instances |
int
Optional. The number of VM instances in the instance group. For `HA cluster `__ master_config <#fields.master_config> __ groups, **must be
set to 3**. For standard cluster
master_config <#fields.master_config> __ groups, **must be
set to 1**.
|
instance_names |
MutableSequence[str]
Output only. The list of instance names. Dataproc derives the names from cluster_name , num_instances , and the
instance group.
|
instance_references |
MutableSequence[google.cloud.dataproc_v1.types.InstanceReference]
Output only. List of references to Compute Engine instances. |
image_uri |
str
Optional. The Compute Engine image resource used for cluster instances. The URI can represent an image or image family. Image examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/[image-id]
- projects/[project_id]/global/images/[image-id]
- image-id
Image family examples. Dataproc will use the most recent
image from the family:
- https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/family/[custom-image-family-name]
- projects/[project_id]/global/images/family/[custom-image-family-name]
If the URI is unspecified, it will be inferred from
SoftwareConfig.image_version or the system default.
|
machine_type_uri |
str
Optional. The Compute Engine machine type used for cluster instances. A full URL, partial URI, or short name are valid. Examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2
- projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2
- n1-standard-2
**Auto Zone Exception**: If you are using the Dataproc `Auto
Zone
Placement |
disk_config |
google.cloud.dataproc_v1.types.DiskConfig
Optional. Disk option config settings. |
is_preemptible |
bool
Output only. Specifies that this instance group contains preemptible instances. |
preemptibility |
google.cloud.dataproc_v1.types.InstanceGroupConfig.Preemptibility
Optional. Specifies the preemptibility of the instance group. The default value for master and worker groups is NON_PREEMPTIBLE . This default cannot be changed.
The default value for secondary instances is
PREEMPTIBLE .
|
managed_group_config |
google.cloud.dataproc_v1.types.ManagedGroupConfig
Output only. The config for Compute Engine Instance Group Manager that manages this group. This is only used for preemptible instance groups. |
accelerators |
MutableSequence[google.cloud.dataproc_v1.types.AcceleratorConfig]
Optional. The Compute Engine accelerator configuration for these instances. |
min_cpu_platform |
str
Optional. Specifies the minimum cpu platform for the Instance Group. See `Dataproc -> Minimum CPU Platform |
min_num_instances |
int
Optional. The minimum number of instances to create. If min_num_instances is set, min_num_instances is used for a criteria to decide the cluster. Cluster creation will be failed by being an error state if the total number of instances created is less than the min_num_instances. For example, given that num_instances = 5 and min_num_instances = 3, - if 4 instances are created and then registered successfully but one instance is failed, the failed VM will be deleted and the cluster will be resized to 4 instances in running state. - if 2 instances are created successfully and 3 instances are failed, the cluster will be in an error state and does not delete failed VMs for debugging. - if 2 instance are created and then registered successfully but 3 instances are failed to initialize, the cluster will be in an error state and does not delete failed VMs for debugging. NB: This can only be set for primary workers now. |
instance_flexibility_policy |
google.cloud.dataproc_v1.types.InstanceFlexibilityPolicy
Optional. Instance flexibility Policy allowing a mixture of VM shapes and provisioning models. |
Classes
Preemptibility
Preemptibility(value)
Controls the use of preemptible instances within the group.
Values: PREEMPTIBILITY_UNSPECIFIED (0): Preemptibility is unspecified, the system will choose the appropriate setting for each instance group. NON_PREEMPTIBLE (1): Instances are non-preemptible.
This option is allowed for all instance groups
and is the only valid value for Master and
Worker instance groups.
PREEMPTIBLE (2):
Instances are [preemptible]
(https://cloud.google.com/compute/docs/instances/preemptible).
This option is allowed only for [secondary worker]
(https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
groups.
SPOT (3):
Instances are [Spot VMs]
(https://cloud.google.com/compute/docs/instances/spot).
This option is allowed only for [secondary worker]
(https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
groups. Spot VMs are the latest version of [preemptible VMs]
(https://cloud.google.com/compute/docs/instances/preemptible),
and provide additional features.