Class InstanceGroupConfig (5.5.1)

InstanceGroupConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The config settings for Compute Engine resources in an instance group, such as a master or worker group.

Attributes

NameDescription
num_instances int
Optional. The number of VM instances in the instance group. For `HA cluster `__ master_config <#fields.master_config>__ groups, **must be set to 3**. For standard cluster master_config <#fields.master_config>__ groups, **must be set to 1**.
instance_names MutableSequence[str]
Output only. The list of instance names. Dataproc derives the names from cluster_name, num_instances, and the instance group.
instance_references MutableSequence[google.cloud.dataproc_v1.types.InstanceReference]
Output only. List of references to Compute Engine instances.
image_uri str
Optional. The Compute Engine image resource used for cluster instances. The URI can represent an image or image family. Image examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/[image-id] - projects/[project_id]/global/images/[image-id] - image-id Image family examples. Dataproc will use the most recent image from the family: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/family/[custom-image-family-name] - projects/[project_id]/global/images/family/[custom-image-family-name] If the URI is unspecified, it will be inferred from SoftwareConfig.image_version or the system default.
machine_type_uri str
Optional. The Compute Engine machine type used for cluster instances. A full URL, partial URI, or short name are valid. Examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - n1-standard-2 **Auto Zone Exception**: If you are using the Dataproc `Auto Zone Placement
disk_config google.cloud.dataproc_v1.types.DiskConfig
Optional. Disk option config settings.
is_preemptible bool
Output only. Specifies that this instance group contains preemptible instances.
preemptibility google.cloud.dataproc_v1.types.InstanceGroupConfig.Preemptibility
Optional. Specifies the preemptibility of the instance group. The default value for master and worker groups is NON_PREEMPTIBLE. This default cannot be changed. The default value for secondary instances is PREEMPTIBLE.
managed_group_config google.cloud.dataproc_v1.types.ManagedGroupConfig
Output only. The config for Compute Engine Instance Group Manager that manages this group. This is only used for preemptible instance groups.
accelerators MutableSequence[google.cloud.dataproc_v1.types.AcceleratorConfig]
Optional. The Compute Engine accelerator configuration for these instances.
min_cpu_platform str
Optional. Specifies the minimum cpu platform for the Instance Group. See `Dataproc -> Minimum CPU Platform
min_num_instances int
Optional. The minimum number of instances to create. If min_num_instances is set, min_num_instances is used for a criteria to decide the cluster. Cluster creation will be failed by being an error state if the total number of instances created is less than the min_num_instances. For example, given that num_instances = 5 and min_num_instances = 3, - if 4 instances are created and then registered successfully but one instance is failed, the failed VM will be deleted and the cluster will be resized to 4 instances in running state. - if 2 instances are created successfully and 3 instances are failed, the cluster will be in an error state and does not delete failed VMs for debugging. - if 2 instance are created and then registered successfully but 3 instances are failed to initialize, the cluster will be in an error state and does not delete failed VMs for debugging. NB: This can only be set for primary workers now.
instance_flexibility_policy google.cloud.dataproc_v1.types.InstanceFlexibilityPolicy
Optional. Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.

Classes

Preemptibility

Preemptibility(value)

Controls the use of preemptible instances within the group.

Values: PREEMPTIBILITY_UNSPECIFIED (0): Preemptibility is unspecified, the system will choose the appropriate setting for each instance group. NON_PREEMPTIBLE (1): Instances are non-preemptible.

    This option is allowed for all instance groups
    and is the only valid value for Master and
    Worker instance groups.
PREEMPTIBLE (2):
    Instances are [preemptible]
    (https://cloud.google.com/compute/docs/instances/preemptible).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups.
SPOT (3):
    Instances are [Spot VMs]
    (https://cloud.google.com/compute/docs/instances/spot).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups. Spot VMs are the latest version of [preemptible VMs]
    (https://cloud.google.com/compute/docs/instances/preemptible),
    and provide additional features.