Class InstanceGroupConfig (5.7.0)

InstanceGroupConfig(mapping=None, *, ignore_unknown_fields=False, **kwargs)

The config settings for Compute Engine resources in an instance group, such as a master or worker group.

Attributes

NameDescription
num_instances int
Optional. The number of VM instances in the instance group. For `HA cluster `__ master_config <#fields.master_config>__ groups, **must be set to 3**. For standard cluster master_config <#fields.master_config>__ groups, **must be set to 1**.
instance_names MutableSequence[str]
Output only. The list of instance names. Dataproc derives the names from cluster_name, num_instances, and the instance group.
instance_references MutableSequence[google.cloud.dataproc_v1.types.InstanceReference]
Output only. List of references to Compute Engine instances.
image_uri str
Optional. The Compute Engine image resource used for cluster instances. The URI can represent an image or image family. Image examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/[image-id] - projects/[project_id]/global/images/[image-id] - image-id Image family examples. Dataproc will use the most recent image from the family: - https://www.googleapis.com/compute/v1/projects/[project_id]/global/images/family/[custom-image-family-name] - projects/[project_id]/global/images/family/[custom-image-family-name] If the URI is unspecified, it will be inferred from SoftwareConfig.image_version or the system default.
machine_type_uri str
Optional. The Compute Engine machine type used for cluster instances. A full URL, partial URI, or short name are valid. Examples: - https://www.googleapis.com/compute/v1/projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - projects/[project_id]/zones/[zone]/machineTypes/n1-standard-2 - n1-standard-2 **Auto Zone Exception**: If you are using the Dataproc `Auto Zone Placement
disk_config google.cloud.dataproc_v1.types.DiskConfig
Optional. Disk option config settings.
is_preemptible bool
Output only. Specifies that this instance group contains preemptible instances.
preemptibility google.cloud.dataproc_v1.types.InstanceGroupConfig.Preemptibility
Optional. Specifies the preemptibility of the instance group. The default value for master and worker groups is NON_PREEMPTIBLE. This default cannot be changed. The default value for secondary instances is PREEMPTIBLE.
managed_group_config google.cloud.dataproc_v1.types.ManagedGroupConfig
Output only. The config for Compute Engine Instance Group Manager that manages this group. This is only used for preemptible instance groups.
accelerators MutableSequence[google.cloud.dataproc_v1.types.AcceleratorConfig]
Optional. The Compute Engine accelerator configuration for these instances.
min_cpu_platform str
Optional. Specifies the minimum cpu platform for the Instance Group. See `Dataproc -> Minimum CPU Platform
min_num_instances int
Optional. The minimum number of primary worker instances to create. If min_num_instances is set, cluster creation will succeed if the number of primary workers created is at least equal to the min_num_instances number. Example: Cluster creation request with num_instances = 5 and min_num_instances = 3: - If 4 VMs are created and 1 instance fails, the failed VM is deleted. The cluster is resized to 4 instances and placed in a RUNNING state. - If 2 instances are created and 3 instances fail, the cluster in placed in an ERROR state. The failed VMs are not deleted.
instance_flexibility_policy google.cloud.dataproc_v1.types.InstanceFlexibilityPolicy
Optional. Instance flexibility Policy allowing a mixture of VM shapes and provisioning models.
startup_config google.cloud.dataproc_v1.types.StartupConfig
Optional. Configuration to handle the startup of instances during cluster create and update process.

Classes

Preemptibility

Preemptibility(value)

Controls the use of preemptible instances within the group.

Values: PREEMPTIBILITY_UNSPECIFIED (0): Preemptibility is unspecified, the system will choose the appropriate setting for each instance group. NON_PREEMPTIBLE (1): Instances are non-preemptible.

    This option is allowed for all instance groups
    and is the only valid value for Master and
    Worker instance groups.
PREEMPTIBLE (2):
    Instances are [preemptible]
    (https://cloud.google.com/compute/docs/instances/preemptible).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups.
SPOT (3):
    Instances are [Spot VMs]
    (https://cloud.google.com/compute/docs/instances/spot).

    This option is allowed only for [secondary worker]
    (https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms)
    groups. Spot VMs are the latest version of [preemptible VMs]
    (https://cloud.google.com/compute/docs/instances/preemptible),
    and provide additional features.