Package google.cloud.metastore.v1

Index

DataprocMetastore

Configures and manages metastore services. Metastore services are fully managed, highly available, autoscaled, autohealing, OSS-native deployments of technical metadata management software. Each metastore service exposes a network endpoint through which metadata queries are served. Metadata queries can originate from a variety of sources, including Apache Hive, Apache Presto, and Apache Spark.

The Dataproc Metastore API defines the following resource model:

  • The service works with a collection of Google Cloud projects, named: /projects/*
  • Each project has a collection of available locations, named: /locations/* (a location must refer to a Google Cloud region)
  • Each location has a collection of services, named: /services/*
  • Dataproc Metastore services are resources with names of the form:

/projects/{project_number}/locations/{location_id}/services/{service_id}.

AlterMetadataResourceLocation

rpc AlterMetadataResourceLocation(AlterMetadataResourceLocationRequest) returns (Operation)

Alter metadata resource location. The metadata resource can be a database, table, or partition. This functionality only updates the parent directory for the respective metadata resource and does not transfer any existing data to the new location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
AlterTableProperties

rpc AlterTableProperties(AlterTablePropertiesRequest) returns (Operation)

Alter metadata table properties.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CancelMigration

rpc CancelMigration(CancelMigrationRequest) returns (Operation)

Cancels the ongoing Managed Migration process.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CompleteMigration

rpc CompleteMigration(CompleteMigrationRequest) returns (Operation)

Completes the managed migration process. The Dataproc Metastore service will switch to using its own backend database after successful migration.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateBackup

rpc CreateBackup(CreateBackupRequest) returns (Operation)

Creates a new backup in a given project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateMetadataImport

rpc CreateMetadataImport(CreateMetadataImportRequest) returns (Operation)

Creates a new MetadataImport in a given project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
CreateService

rpc CreateService(CreateServiceRequest) returns (Operation)

Creates a metastore service in a project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteBackup

rpc DeleteBackup(DeleteBackupRequest) returns (Operation)

Deletes a single backup.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteMigrationExecution

rpc DeleteMigrationExecution(DeleteMigrationExecutionRequest) returns (Operation)

Deletes a single migration execution.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteService

rpc DeleteService(DeleteServiceRequest) returns (Operation)

Deletes a single service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ExportMetadata

rpc ExportMetadata(ExportMetadataRequest) returns (Operation)

Exports metadata from a service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetBackup

rpc GetBackup(GetBackupRequest) returns (Backup)

Gets details of a single backup.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetMetadataImport

rpc GetMetadataImport(GetMetadataImportRequest) returns (MetadataImport)

Gets details of a single import.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetMigrationExecution

rpc GetMigrationExecution(GetMigrationExecutionRequest) returns (MigrationExecution)

Gets details of a single migration execution.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetService

rpc GetService(GetServiceRequest) returns (Service)

Gets the details of a single service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListBackups

rpc ListBackups(ListBackupsRequest) returns (ListBackupsResponse)

Lists backups in a service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListMetadataImports

rpc ListMetadataImports(ListMetadataImportsRequest) returns (ListMetadataImportsResponse)

Lists imports in a service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListMigrationExecutions

rpc ListMigrationExecutions(ListMigrationExecutionsRequest) returns (ListMigrationExecutionsResponse)

Lists migration executions on a service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListServices

rpc ListServices(ListServicesRequest) returns (ListServicesResponse)

Lists services in a project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
MoveTableToDatabase

rpc MoveTableToDatabase(MoveTableToDatabaseRequest) returns (Operation)

Move a table to another database.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
QueryMetadata

rpc QueryMetadata(QueryMetadataRequest) returns (Operation)

Query Dataproc Metastore metadata.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
RestoreService

rpc RestoreService(RestoreServiceRequest) returns (Operation)

Restores a service from a backup.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
StartMigration

rpc StartMigration(StartMigrationRequest) returns (Operation)

Starts the Managed Migration process.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateMetadataImport

rpc UpdateMetadataImport(UpdateMetadataImportRequest) returns (Operation)

Updates a single import. Only the description field of MetadataImport is supported to be updated.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateService

rpc UpdateService(UpdateServiceRequest) returns (Operation)

Updates the parameters of a single service.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

DataprocMetastoreFederation

Configures and manages metastore federation services. Dataproc Metastore Federation Service allows federating a collection of backend metastores like BigQuery, Dataplex Lakes, and other Dataproc Metastores. The Federation Service exposes a gRPC URL through which metadata from the backend metastores are served at query time.

The Dataproc Metastore Federation API defines the following resource model: * The service works with a collection of Google Cloud projects. * Each project has a collection of available locations. * Each location has a collection of federations. * Dataproc Metastore Federations are resources with names of the form: projects/{project_number}/locations/{location_id}/federations/{federation_id}.

CreateFederation

rpc CreateFederation(CreateFederationRequest) returns (Operation)

Creates a metastore federation in a project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
DeleteFederation

rpc DeleteFederation(DeleteFederationRequest) returns (Operation)

Deletes a single federation.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
GetFederation

rpc GetFederation(GetFederationRequest) returns (Federation)

Gets the details of a single federation.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
ListFederations

rpc ListFederations(ListFederationsRequest) returns (ListFederationsResponse)

Lists federations in a project and location.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.
UpdateFederation

rpc UpdateFederation(UpdateFederationRequest) returns (Operation)

Updates the fields of a federation.

Authorization scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

AlterMetadataResourceLocationRequest

Request message for DataprocMetastore.AlterMetadataResourceLocation.

Fields
service

string

Required. The relative resource name of the metastore service to mutate metadata, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.mutateMetadata
resource_name

string

Required. The relative metadata resource name in the following format.

databases/{database_id} or databases/{database_id}/tables/{table_id} or databases/{database_id}/tables/{table_id}/partitions/{partition_id}
location_uri

string

Required. The new location URI for the metadata resource.

AlterMetadataResourceLocationResponse

This type has no fields.

Response message for DataprocMetastore.AlterMetadataResourceLocation.

AlterTablePropertiesRequest

Request message for DataprocMetastore.AlterTableProperties.

Fields
service

string

Required. The relative resource name of the Dataproc Metastore service that's being used to mutate metadata table properties, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.mutateMetadata
table_name

string

Required. The name of the table containing the properties you're altering in the following format.

databases/{database_id}/tables/{table_id}
update_mask

FieldMask

A field mask that specifies the metadata table properties that are overwritten by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.

For example, given the target properties:

properties {
  a: 1
  b: 2
}

And an update properties:

properties {
  a: 2
  b: 3
  c: 4
}

then if the field mask is:

paths: ["properties.b", "properties.c"]

then the result will be:

properties {
  a: 1
  b: 3
  c: 4
}
properties

map<string, string>

A map that describes the desired values to mutate. If update_mask is empty, the properties will not update. Otherwise, the properties only alters the value whose associated paths exist in the update mask

AlterTablePropertiesResponse

This type has no fields.

Response message for DataprocMetastore.AlterTableProperties.

AutoscalingConfig

Represents the autoscaling configuration of a metastore service.

Fields
autoscaling_factor

float

Output only. The scaling factor of a service with autoscaling enabled.
autoscaling_enabled

bool

Optional. Whether or not autoscaling is enabled for this service.
limit_config

LimitConfig

Optional. The LimitConfig of the service.

AuxiliaryVersionConfig

Configuration information for the auxiliary service versions.

Fields
version

string

The Hive metastore version of the auxiliary service. It must be less than the primary Hive metastore service's version.
config_overrides

map<string, string>

A mapping of Hive metastore configuration key-value pairs to apply to the auxiliary Hive metastore (configured in hive-site.xml) in addition to the primary version's overrides. If keys are present in both the auxiliary version's overrides and the primary version's overrides, the value from the auxiliary version's overrides takes precedence.
network_config

NetworkConfig

Output only. The network configuration contains the endpoint URI(s) of the auxiliary Hive metastore service.

BackendMetastore

Represents a backend metastore for the federation.

Fields
name

string

The relative resource name of the metastore that is being federated. The formats of the relative resource names for the currently supported metastores are listed below:

  • BigQuery
    • projects/{project_id}
  • Dataproc Metastore
    • projects/{project_id}/locations/{location}/services/{service_id}
metastore_type

MetastoreType

The type of the backend metastore.

MetastoreType

The type of the backend metastore.

Enums
METASTORE_TYPE_UNSPECIFIED The metastore type is not set.
BIGQUERY The backend metastore is BigQuery.
DATAPROC_METASTORE The backend metastore is Dataproc Metastore.

Backup

The details of a backup resource.

Fields
name

string

Immutable. The relative resource name of the backup, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/backups/{backup_id}
create_time

Timestamp

Output only. The time when the backup was started.
end_time

Timestamp

Output only. The time when the backup finished creating.
state

State

Output only. The current state of the backup.
service_revision

Service

Output only. The revision of the service at the time of backup.
description

string

The description of the backup.
restoring_services[]

string

Output only. Services that are restoring from the backup.

State

The current state of the backup.

Enums
STATE_UNSPECIFIED The state of the backup is unknown.
CREATING The backup is being created.
DELETING The backup is being deleted.
ACTIVE The backup is active and ready to use.
FAILED The backup failed.
RESTORING The backup is being restored.

CancelMigrationRequest

Request message for DataprocMetastore.CancelMigration.

Fields
service

string

Required. The relative resource name of the metastore service to cancel the ongoing migration to, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.migrations.cancel

CancelMigrationResponse

Response message for DataprocMetastore.CancelMigration.

Fields
migration_execution

string

The relative resource name of the migration execution, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}.

CompleteMigrationRequest

Request message for DataprocMetastore.CompleteMigration.

Fields
service

string

Required. The relative resource name of the metastore service to complete the migration to, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.migrations.complete

CompleteMigrationResponse

Response message for DataprocMetastore.CompleteMigration.

Fields
migration_execution

string

The relative resource name of the migration execution, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}.

CreateBackupRequest

Request message for DataprocMetastore.CreateBackup.

Fields
parent

string

Required. The relative resource name of the service in which to create a backup of the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.backups.create
backup_id

string

Required. The ID of the backup, which is used as the final component of the backup's name.

This value must be between 1 and 64 characters long, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
backup

Backup

Required. The backup to create. The name field is ignored. The ID of the created backup must be provided in the request's backup_id field.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

CreateFederationRequest

Request message for CreateFederation.

Fields
parent

string

Required. The relative resource name of the location in which to create a federation service, in the following form:

projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.federations.create
federation_id

string

Required. The ID of the metastore federation, which is used as the final component of the metastore federation's name.

This value must be between 2 and 63 characters long inclusive, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
federation

Federation

Required. The Metastore Federation to create. The name field is ignored. The ID of the created metastore federation must be provided in the request's federation_id field.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

CreateMetadataImportRequest

Request message for DataprocMetastore.CreateMetadataImport.

Fields
parent

string

Required. The relative resource name of the service in which to create a metastore import, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.imports.create
metadata_import_id

string

Required. The ID of the metadata import, which is used as the final component of the metadata import's name.

This value must be between 1 and 64 characters long, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
metadata_import

MetadataImport

Required. The metadata import to create. The name field is ignored. The ID of the created metadata import must be provided in the request's metadata_import_id field.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

CreateServiceRequest

Request message for DataprocMetastore.CreateService.

Fields
parent

string

Required. The relative resource name of the location in which to create a metastore service, in the following form:

projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.services.create
service_id

string

Required. The ID of the metastore service, which is used as the final component of the metastore service's name.

This value must be between 2 and 63 characters long inclusive, begin with a letter, end with a letter or number, and consist of alpha-numeric ASCII characters or hyphens.
service

Service

Required. The Metastore service to create. The name field is ignored. The ID of the created metastore service must be provided in the request's service_id field.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

DataCatalogConfig

Specifies how metastore metadata should be integrated with the Data Catalog service.

Fields
enabled

bool

Optional. Defines whether the metastore metadata should be synced to Data Catalog. The default value is to disable syncing metastore metadata to Data Catalog.

DatabaseDumpSpec

This type has no fields.

The specification of database dump to import from or export to.

Type

The type of the database dump.

Enums
TYPE_UNSPECIFIED The type of the database dump is unknown.
MYSQL Database dump is a MySQL dump file.
AVRO Database dump contains Avro files.

DeleteBackupRequest

Request message for DataprocMetastore.DeleteBackup.

Fields
name

string

Required. The relative resource name of the backup to delete, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/backups/{backup_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.backups.delete
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

DeleteFederationRequest

Request message for DeleteFederation.

Fields
name

string

Required. The relative resource name of the metastore federation to delete, in the following form:

projects/{project_number}/locations/{location_id}/federations/{federation_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.federations.delete
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

DeleteMigrationExecutionRequest

Request message for DataprocMetastore.DeleteMigrationExecution.

Fields
name

string

Required. The relative resource name of the migrationExecution to delete, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.migrations.delete
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

DeleteServiceRequest

Request message for DataprocMetastore.DeleteService.

Fields
name

string

Required. The relative resource name of the metastore service to delete, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.services.delete
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

EncryptionConfig

Encryption settings for the service.

Fields
kms_key

string

The fully qualified customer provided Cloud KMS key name to use for customer data encryption, in the following format:

projects/{project_number}/locations/{location_id}/keyRings/{key_ring_id}/cryptoKeys/{crypto_key_id}.

ErrorDetails

Error details in public error message for DataprocMetastore.QueryMetadata.

Fields
details

map<string, string>

Additional structured details about this error.

Keys define the failure items. Value describes the exception or details of the item.

ExportMetadataRequest

Request message for DataprocMetastore.ExportMetadata.

Fields
service

string

Required. The relative resource name of the metastore service to run export, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.export
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID. A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.
database_dump_type

Type

Optional. The type of the database dump. If unspecified, defaults to MYSQL.
Union field destination. Required. Destination that metadata is exported to. destination can be only one of the following:
destination_gcs_folder

string

A Cloud Storage URI of a folder, in the format gs://<bucket_name>/<path_inside_bucket>. A sub-folder <export_folder> containing exported files will be created below it.

Federation

Represents a federation of multiple backend metastores.

Fields
name

string

Immutable. The relative resource name of the federation, of the form: projects/{project_number}/locations/{location_id}/federations/{federation_id}`.
create_time

Timestamp

Output only. The time when the metastore federation was created.
update_time

Timestamp

Output only. The time when the metastore federation was last updated.
labels

map<string, string>

User-defined labels for the metastore federation.
version

string

Immutable. The Apache Hive metastore version of the federation. All backend metastore versions must be compatible with the federation version.
backend_metastores

map<int32, BackendMetastore>

A map from BackendMetastore rank to BackendMetastores from which the federation service serves metadata at query time. The map key represents the order in which BackendMetastores should be evaluated to resolve database names at query time and should be greater than or equal to zero. A BackendMetastore with a lower number will be evaluated before a BackendMetastore with a higher number.
endpoint_uri

string

Output only. The federation endpoint.
state

State

Output only. The current state of the federation.
state_message

string

Output only. Additional information about the current state of the metastore federation, if available.
uid

string

Output only. The globally unique resource identifier of the metastore federation.

State

The current state of the federation.

Enums
STATE_UNSPECIFIED The state of the metastore federation is unknown.
CREATING The metastore federation is in the process of being created.
ACTIVE The metastore federation is running and ready to serve queries.
UPDATING The metastore federation is being updated. It remains usable but cannot accept additional update requests or be deleted at this time.
DELETING The metastore federation is undergoing deletion. It cannot be used.
ERROR The metastore federation has encountered an error and cannot be used. The metastore federation should be deleted.

GetBackupRequest

Request message for DataprocMetastore.GetBackup.

Fields
name

string

Required. The relative resource name of the backup to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/backups/{backup_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.backups.get

GetFederationRequest

Request message for GetFederation.

Fields
name

string

Required. The relative resource name of the metastore federation to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/federations/{federation_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.federations.get

GetMetadataImportRequest

Request message for DataprocMetastore.GetMetadataImport.

Fields
name

string

Required. The relative resource name of the metadata import to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{import_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.imports.get

GetMigrationExecutionRequest

Request message for DataprocMetastore.GetMigrationExecution.

Fields
name

string

Required. The relative resource name of the migration execution to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.migrations.get

GetServiceRequest

Request message for DataprocMetastore.GetService.

Fields
name

string

Required. The relative resource name of the metastore service to retrieve, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource name:

  • metastore.services.get

HiveMetastoreConfig

Specifies configuration information specific to running Hive metastore software as the metastore service.

Fields
version

string

Immutable. The Hive metastore schema version.
config_overrides

map<string, string>

A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore (configured in hive-site.xml). The mappings override system defaults (some keys cannot be overridden). These overrides are also applied to auxiliary versions and can be further customized in the auxiliary version's AuxiliaryVersionConfig.
kerberos_config

KerberosConfig

Information used to configure the Hive metastore service as a service principal in a Kerberos realm. To disable Kerberos, use the UpdateService method and specify this field's path (hive_metastore_config.kerberos_config) in the request's update_mask while omitting this field from the request's service.
endpoint_protocol

EndpointProtocol

The protocol to use for the metastore service endpoint. If unspecified, defaults to THRIFT.
auxiliary_versions

map<string, AuxiliaryVersionConfig>

A mapping of Hive metastore version to the auxiliary version configuration. When specified, a secondary Hive metastore service is created along with the primary service. All auxiliary versions must be less than the service's primary version. The key is the auxiliary service name and it must match the regular expression a-z?. This means that the first character must be a lowercase letter, and all the following characters must be hyphens, lowercase letters, or digits, except the last character, which cannot be a hyphen.

EndpointProtocol

Protocols available for serving the metastore service endpoint.

Enums
ENDPOINT_PROTOCOL_UNSPECIFIED The protocol is not set.
THRIFT Use the legacy Apache Thrift protocol for the metastore service endpoint.
GRPC Use the modernized gRPC protocol for the metastore service endpoint.

KerberosConfig

Configuration information for a Kerberos principal.

Fields
keytab

Secret

A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center (KDC).
principal

string

A Kerberos principal that exists in the both the keytab the KDC to authenticate as. A typical principal is of the form primary/instance@REALM, but there is no exact format.
krb5_config_gcs_uri

string

A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/to/krb5.conf, although the file does not need to be named krb5.conf explicitly.

LatestBackup

The details of the latest scheduled backup.

Fields
backup_id

string

Output only. The ID of an in-progress scheduled backup. Empty if no backup is in progress.
start_time

Timestamp

Output only. The time when the backup was started.
state

State

Output only. The current state of the backup.
duration

Duration

Output only. The duration of the backup completion.

State

The current state of the backup.

Enums
STATE_UNSPECIFIED The state of the backup is unknown.
IN_PROGRESS The backup is in progress.
SUCCEEDED The backup completed.
FAILED The backup failed.

LimitConfig

Represents the autoscaling limit configuration of a metastore service.

Fields
max_scaling_factor

float

Optional. The highest scaling factor that the service should be autoscaled to.
min_scaling_factor

float

Optional. The lowest scaling factor that the service should be autoscaled to.

ListBackupsRequest

Request message for DataprocMetastore.ListBackups.

Fields
parent

string

Required. The relative resource name of the service whose backups to list, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/backups.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.backups.list
page_size

int32

Optional. The maximum number of backups to return. The response may contain less than the maximum number. If unspecified, no more than 500 backups are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListBackups call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListBackups must match the call that provided the page token.
filter

string

Optional. The filter to apply to list results.
order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListBackupsResponse

Response message for DataprocMetastore.ListBackups.

Fields
backups[]

Backup

The backups of the specified service.
next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
unreachable[]

string

Locations that could not be reached.

ListFederationsRequest

Request message for ListFederations.

Fields
parent

string

Required. The relative resource name of the location of metastore federations to list, in the following form: projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.federations.list
page_size

int32

Optional. The maximum number of federations to return. The response may contain less than the maximum number. If unspecified, no more than 500 services are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token

string

Optional. A page token, received from a previous ListFederationServices call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to ListFederationServices must match the call that provided the page token.
filter

string

Optional. The filter to apply to list results.
order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListFederationsResponse

Response message for ListFederations

Fields
federations[]

Federation

The services in the specified location.
next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
unreachable[]

string

Locations that could not be reached.

ListMetadataImportsRequest

Request message for DataprocMetastore.ListMetadataImports.

Fields
parent

string

Required. The relative resource name of the service whose metadata imports to list, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.imports.list
page_size

int32

Optional. The maximum number of imports to return. The response may contain less than the maximum number. If unspecified, no more than 500 imports are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListServices call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListServices must match the call that provided the page token.
filter

string

Optional. The filter to apply to list results.
order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListMetadataImportsResponse

Response message for DataprocMetastore.ListMetadataImports.

Fields
metadata_imports[]

MetadataImport

The imports in the specified service.
next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
unreachable[]

string

Locations that could not be reached.

ListMigrationExecutionsRequest

Request message for DataprocMetastore.ListMigrationExecutions.

Fields
parent

string

Required. The relative resource name of the service whose migration executions to list, in the following form:

projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.migrations.list
page_size

int32

Optional. The maximum number of migration executions to return. The response may contain less than the maximum number. If unspecified, no more than 500 migration executions are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListMigrationExecutions call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListMigrationExecutions must match the call that provided the page token.
filter

string

Optional. The filter to apply to list results.
order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListMigrationExecutionsResponse

Response message for DataprocMetastore.ListMigrationExecutions.

Fields
migration_executions[]

MigrationExecution

The migration executions on the specified service.
next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
unreachable[]

string

Locations that could not be reached.

ListServicesRequest

Request message for DataprocMetastore.ListServices.

Fields
parent

string

Required. The relative resource name of the location of metastore services to list, in the following form:

projects/{project_number}/locations/{location_id}.

Authorization requires the following IAM permission on the specified resource parent:

  • metastore.services.list
page_size

int32

Optional. The maximum number of services to return. The response may contain less than the maximum number. If unspecified, no more than 500 services are returned. The maximum value is 1000; values above 1000 are changed to 1000.
page_token

string

Optional. A page token, received from a previous DataprocMetastore.ListServices call. Provide this token to retrieve the subsequent page.

To retrieve the first page, supply an empty page token.

When paginating, other parameters provided to DataprocMetastore.ListServices must match the call that provided the page token.
filter

string

Optional. The filter to apply to list results.
order_by

string

Optional. Specify the ordering of results as described in Sorting Order. If not specified, the results will be sorted in the default order.

ListServicesResponse

Response message for DataprocMetastore.ListServices.

Fields
services[]

Service

The services in the specified location.
next_page_token

string

A token that can be sent as page_token to retrieve the next page. If this field is omitted, there are no subsequent pages.
unreachable[]

string

Locations that could not be reached.

LocationMetadata

Metadata about the service in a location.

Fields
supported_hive_metastore_versions[]

HiveMetastoreVersion

The versions of Hive Metastore that can be used when creating a new metastore service in this location. The server guarantees that exactly one HiveMetastoreVersion in the list will set is_default.
multi_region_metadata

MultiRegionMetadata

The multi-region metadata if the current region is a multi-region.
custom_region_metadata[]

CustomRegionMetadata

Possible configurations supported if the current region is a custom region.

CustomRegionMetadata

Metadata about a custom region. This is only populated if the region is a custom region. For single/multi regions, it will be empty.

Fields
required_read_write_regions[]

string

The read-write regions for this custom region.
optional_read_only_regions[]

string

The read-only regions for this custom region.
witness_region

string

The Spanner witness region for this custom region.

HiveMetastoreVersion

A specification of a supported version of the Hive Metastore software.

Fields
version

string

The semantic version of the Hive Metastore software.
is_default

bool

Whether version will be chosen by the server if a metastore service is created with a HiveMetastoreConfig that omits the version.

MultiRegionMetadata

The metadata for the multi-region that includes the constituent regions. The metadata is only populated if the region is multi-region. For single region or custom dual region, it will be empty.

Fields
constituent_regions[]

string

The regions constituting the multi-region.

MaintenanceWindow

Maintenance window. This specifies when Dataproc Metastore may perform system maintenance operation to the service.

Fields
hour_of_day

Int32Value

The hour of day (0-23) when the window starts.
day_of_week

DayOfWeek

The day of week, when the window starts.

MetadataExport

The details of a metadata export operation.

Fields
start_time

Timestamp

Output only. The time when the export started.
end_time

Timestamp

Output only. The time when the export ended.
state

State

Output only. The current state of the export.
database_dump_type

Type

Output only. The type of the database dump.

Union field destination.

destination can be only one of the following:
destination_gcs_uri

string

Output only. A Cloud Storage URI of a folder that metadata are exported to, in the form of gs://<bucket_name>/<path_inside_bucket>/<export_folder>, where <export_folder> is automatically generated.

State

The current state of the metadata export.

Enums
STATE_UNSPECIFIED The state of the metadata export is unknown.
RUNNING The metadata export is running.
SUCCEEDED The metadata export completed successfully.
FAILED The metadata export failed.
CANCELLED The metadata export is cancelled.

MetadataImport

A metastore resource that imports metadata.

Fields
name

string

Immutable. The relative resource name of the metadata import, of the form:

projects/{project_number}/locations/{location_id}/services/{service_id}/metadataImports/{metadata_import_id}.
description

string

The description of the metadata import.
create_time

Timestamp

Output only. The time when the metadata import was started.
update_time

Timestamp

Output only. The time when the metadata import was last updated.
end_time

Timestamp

Output only. The time when the metadata import finished.
state

State

Output only. The current state of the metadata import.
Union field metadata. The metadata to be imported. metadata can be only one of the following:
database_dump

DatabaseDump

Immutable. A database dump from a pre-existing metastore's database.

DatabaseDump

A specification of the location of and metadata about a database dump from a relational database management system.

Fields
database_type
(deprecated)

DatabaseType

The type of the database.
gcs_uri

string

A Cloud Storage object or folder URI that specifies the source from which to import metadata. It must begin with gs://.
source_database
(deprecated)

string

The name of the source database.
type

Type

Optional. The type of the database dump. If unspecified, defaults to MYSQL.

DatabaseType

The type of the database.

Enums
DATABASE_TYPE_UNSPECIFIED The type of the source database is unknown.
MYSQL The type of the source database is MySQL.

State

The current state of the metadata import.

Enums
STATE_UNSPECIFIED The state of the metadata import is unknown.
RUNNING The metadata import is running.
SUCCEEDED The metadata import completed successfully.
UPDATING The metadata import is being updated.
FAILED The metadata import failed, and attempted metadata changes were rolled back.

MetadataIntegration

Specifies how metastore metadata should be integrated with external services.

Fields
data_catalog_config

DataCatalogConfig

Optional. The integration config for the Data Catalog service.

MetadataManagementActivity

The metadata management activities of the metastore service.

Fields
metadata_exports[]

MetadataExport

Output only. The latest metadata exports of the metastore service.
restores[]

Restore

Output only. The latest restores of the metastore service.

MigrationExecution

The details of a migration execution resource.

Fields
name

string

Output only. The relative resource name of the migration execution, in the following form: projects/{project_number}/locations/{location_id}/services/{service_id}/migrationExecutions/{migration_execution_id}
create_time

Timestamp

Output only. The time when the migration execution was started.
end_time

Timestamp

Output only. The time when the migration execution finished.
state

State

Output only. The current state of the migration execution.
phase

Phase

Output only. The current phase of the migration execution.
state_message

string

Output only. Additional information about the current state of the migration execution.
Union field migration_config. Configuration information required by Dataproc Metastore for starting the migration process migration_config can be only one of the following:
cloud_sql_migration_config

CloudSQLMigrationConfig

Configuration information specific to migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.

CloudSQLMigrationConfig

Configuration information for migrating from self-managed hive metastore on Google Cloud using Cloud SQL as the backend database to Dataproc Metastore.

Fields
cdc_config

CdcConfig

Required. Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore. Dataproc Metastore switches to using its backend database after the cutover phase of migration.
cloud_sql_connection_config

CloudSQLConnectionConfig

Required. Configuration information to establish customer database connection before the cutover phase of migration

CdcConfig

Configuration information to start the Change Data Capture (CDC) streams from customer database to backend database of Dataproc Metastore.

Fields
vpc_network

string

Required. Fully qualified name of the Cloud SQL instance's VPC network or the shared VPC network that Datastream will peer to, in the following format: projects/{project_id}/locations/global/networks/{network_id}. More context in https://cloud.google.com/datastream/docs/network-connectivity-options#privateconnectivity
subnet_ip_range

string

Required. A /29 CIDR IP range for peering with datastream.
username

string

Required. The username that the Datastream service should use for the MySQL connection.
password

string

Required. Input only. The password for the user that Datastream service should use for the MySQL connection. This field is not returned on request.
reverse_proxy_subnet

string

Required. The URL of the subnetwork resource to create the VM instance hosting the reverse proxy in. More context in https://cloud.google.com/datastream/docs/private-connectivity#reverse-csql-proxy The subnetwork should reside in the network provided in the request that Datastream will peer to and should be in the same region as Datastream, in the following format. projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
bucket

string

Optional. The bucket to write the intermediate stream event data in. The bucket name must be without any prefix like "gs://". See the bucket naming requirements. This field is optional. If not set, the Artifacts Cloud Storage bucket will be used.
root_path

string

Optional. The root path inside the Cloud Storage bucket. The stream event data will be written to this path. The default value is /migration.

CloudSQLConnectionConfig

Configuration information to establish customer database connection before the cutover phase of migration

Fields
instance_connection_name

string

Required. Cloud SQL database connection name (project_id:region:instance_name)
ip_address

string

Required. The private IP address of the Cloud SQL instance.
port

int32

Required. The network port of the database.
hive_database_name

string

Required. The hive database name.
username

string

Required. The username that Dataproc Metastore service will use to connect to the database.
password

string

Required. Input only. The password for the user that Dataproc Metastore service will be using to connect to the database. This field is not returned on request.
proxy_subnet

string

Required. The relative resource name of the subnetwork to deploy the SOCKS5 proxy service in. The subnetwork should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
nat_subnet

string

Required. The relative resource name of the subnetwork to be used for Private Service Connect. Note that this cannot be a regular subnet and is used only for NAT. (https://cloud.google.com/vpc/docs/about-vpc-hosted-services#psc-subnets) This subnet is used to publish the SOCKS5 proxy service. The subnet size must be at least /29 and it should reside in a network through which the Cloud SQL instance is accessible. The resource name should be in the format, projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}

Phase

The current phase of the migration execution.

Enums
PHASE_UNSPECIFIED The phase of the migration execution is unknown.
REPLICATION Replication phase refers to the migration phase when Dataproc Metastore is running a pipeline to replicate changes in the customer database to its backend database. During this phase, Dataproc Metastore uses the customer database as the hive metastore backend database.
CUTOVER Cutover phase refers to the migration phase when Dataproc Metastore switches to using its own backend database. Migration enters this phase when customer is done migrating all their clusters/workloads to Dataproc Metastore and triggers CompleteMigration.

State

The current state of the migration execution.

Enums
STATE_UNSPECIFIED The state of the migration execution is unknown.
STARTING The migration execution is starting.
RUNNING The migration execution is running.
CANCELLING The migration execution is in the process of being cancelled.
AWAITING_USER_ACTION The migration execution is awaiting user action.
SUCCEEDED The migration execution has completed successfully.
FAILED The migration execution has failed.
CANCELLED The migration execution is cancelled.
DELETING The migration execution is being deleted.

MoveTableToDatabaseRequest

Request message for DataprocMetastore.MoveTableToDatabase.

Fields
service

string

Required. The relative resource name of the metastore service to mutate metadata, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.mutateMetadata
table_name

string

Required. The name of the table to be moved.
db_name

string

Required. The name of the database where the table resides.
destination_db_name

string

Required. The name of the database where the table should be moved.

MoveTableToDatabaseResponse

This type has no fields.

Response message for DataprocMetastore.MoveTableToDatabase.

NetworkConfig

Network configuration for the Dataproc Metastore service.

Fields
consumers[]

Consumer

Immutable. The consumer-side network configuration for the Dataproc Metastore instance.

Consumer

Contains information of the customer's network configurations.

Fields
endpoint_uri

string

Output only. The URI of the endpoint used to access the metastore service.
endpoint_location

string

Output only. The location of the endpoint URI. Format: projects/{project}/locations/{location}.

Union field vpc_resource.

vpc_resource can be only one of the following:
subnetwork

string

Immutable. The subnetwork of the customer project from which an IP address is reserved and used as the Dataproc Metastore service's endpoint. It is accessible to hosts in the subnet and to all hosts in a subnet in the same region and same network. There must be at least one IP address available in the subnet's primary range. The subnet is specified in the following form:

projects/{project_number}/regions/{region_id}/subnetworks/{subnetwork_id}

OperationMetadata

Represents the metadata of a long-running operation.

Fields
create_time

Timestamp

Output only. The time the operation was created.
end_time

Timestamp

Output only. The time the operation finished running.
target

string

Output only. Server-defined resource path for the target of the operation.
verb

string

Output only. Name of the verb executed by the operation.
status_message

string

Output only. Human-readable status of the operation, if any.
requested_cancellation

bool

Output only. Identifies whether the caller has requested cancellation of the operation. Operations that have successfully been cancelled have [Operation.error][] value with a google.rpc.Status.code of 1, corresponding to Code.CANCELLED.
api_version

string

Output only. API version used to start the operation.

QueryMetadataRequest

Request message for DataprocMetastore.QueryMetadata.

Fields
service

string

Required. The relative resource name of the metastore service to query metadata, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.queryMetadata
query

string

Required. A read-only SQL query to execute against the metadata database. The query cannot change or mutate the data.

QueryMetadataResponse

Response message for DataprocMetastore.QueryMetadata.

Fields
result_manifest_uri

string

The manifest URI is link to a JSON instance in Cloud Storage. This instance manifests immediately along with QueryMetadataResponse. The content of the URI is not retriable until the long-running operation query against the metadata finishes.

Restore

The details of a metadata restore operation.

Fields
start_time

Timestamp

Output only. The time when the restore started.
end_time

Timestamp

Output only. The time when the restore ended.
state

State

Output only. The current state of the restore.
backup

string

Output only. The relative resource name of the metastore service backup to restore from, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}/backups/{backup_id}.
type

RestoreType

Output only. The type of restore.
details

string

Output only. The restore details containing the revision of the service to be restored to, in format of JSON.
backup_location

string

Optional. A Cloud Storage URI specifying where the backup artifacts are stored, in the format gs://<bucket_name>/<path_inside_bucket>.

RestoreType

The type of restore. If unspecified, defaults to METADATA_ONLY.

Enums
RESTORE_TYPE_UNSPECIFIED The restore type is unknown.
FULL The service's metadata and configuration are restored.
METADATA_ONLY Only the service's metadata is restored.

State

The current state of the restore.

Enums
STATE_UNSPECIFIED The state of the metadata restore is unknown.
RUNNING The metadata restore is running.
SUCCEEDED The metadata restore completed successfully.
FAILED The metadata restore failed.
CANCELLED The metadata restore is cancelled.

RestoreServiceRequest

Request message for [DataprocMetastore.Restore][].

Fields
service

string

Required. The relative resource name of the metastore service to run restore, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.restore
backup

string

Optional. The relative resource name of the metastore service backup to restore from, in the following form:

projects/{project_id}/locations/{location_id}/services/{service_id}/backups/{backup_id}. Mutually exclusive with backup_location, and exactly one of the two must be set.

Authorization requires the following IAM permission on the specified resource backup:

  • metastore.backups.use
backup_location

string

Optional. A Cloud Storage URI specifying the location of the backup artifacts, namely - backup avro files under "avro/", backup_metastore.json and service.json, in the following form:

gs://<path_to_backup>. Mutually exclusive with backup, and exactly one of the two must be set.
restore_type

RestoreType

Optional. The type of restore. If unspecified, defaults to METADATA_ONLY.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID. A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

ScalingConfig

Represents the scaling configuration of a metastore service.

Fields
autoscaling_config

AutoscalingConfig

Optional. The autoscaling configuration.
Union field scaling_model. Represents either a predetermined instance size or a numeric scaling factor. scaling_model can be only one of the following:
instance_size

InstanceSize

An enum of readable instance sizes, with each instance size mapping to a float value (e.g. InstanceSize.EXTRA_SMALL = scaling_factor(0.1))
scaling_factor

float

Scaling factor, increments of 0.1 for values less than 1.0, and increments of 1.0 for values greater than 1.0.

InstanceSize

Metastore instance sizes.

Enums
INSTANCE_SIZE_UNSPECIFIED Unspecified instance size
EXTRA_SMALL Extra small instance size, maps to a scaling factor of 0.1.
SMALL Small instance size, maps to a scaling factor of 0.5.
MEDIUM Medium instance size, maps to a scaling factor of 1.0.
LARGE Large instance size, maps to a scaling factor of 3.0.
EXTRA_LARGE Extra large instance size, maps to a scaling factor of 6.0.

ScheduledBackup

This specifies the configuration of scheduled backup.

Fields
enabled

bool

Optional. Defines whether the scheduled backup is enabled. The default value is false.
cron_schedule

string

Optional. The scheduled interval in Cron format, see https://en.wikipedia.org/wiki/Cron The default is empty: scheduled backup is not enabled. Must be specified to enable scheduled backups.
time_zone

string

Optional. Specifies the time zone to be used when interpreting cron_schedule. Must be a time zone name from the time zone database (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g. America/Los_Angeles or Africa/Abidjan. If left unspecified, the default is UTC.
next_scheduled_time

Timestamp

Output only. The time when the next backups execution is scheduled to start.
backup_location

string

Optional. A Cloud Storage URI of a folder, in the format gs://<bucket_name>/<path_inside_bucket>. A sub-folder <backup_folder> containing backup files will be stored below it.
latest_backup

LatestBackup

Output only. The details of the latest scheduled backup.

Secret

A securely stored value.

Fields

Union field value.

value can be only one of the following:
cloud_secret

string

The relative resource name of a Secret Manager secret version, in the following form:

projects/{project_number}/secrets/{secret_id}/versions/{version_id}.

Service

A managed metastore service that serves metadata queries.

Fields
name

string

Immutable. The relative resource name of the metastore service, in the following format:

projects/{project_number}/locations/{location_id}/services/{service_id}.
create_time

Timestamp

Output only. The time when the metastore service was created.
update_time

Timestamp

Output only. The time when the metastore service was last updated.
labels

map<string, string>

User-defined labels for the metastore service.
network

string

Immutable. The relative resource name of the VPC network on which the instance can be accessed. It is specified in the following form:

projects/{project_number}/global/networks/{network_id}.
endpoint_uri

string

Output only. The URI of the endpoint used to access the metastore service.
port

int32

The TCP port at which the metastore service is reached. Default: 9083.
state

State

Output only. The current state of the metastore service.
state_message

string

Output only. Additional information about the current state of the metastore service, if available.
artifact_gcs_uri

string

Output only. A Cloud Storage URI (starting with gs://) that specifies where artifacts related to the metastore service are stored.
tier

Tier

The tier of the service.
metadata_integration

MetadataIntegration

Optional. The setting that defines how metastore metadata should be integrated with external services and systems.
maintenance_window

MaintenanceWindow

The one hour maintenance window of the metastore service. This specifies when the service can be restarted for maintenance purposes in UTC time. Maintenance window is not needed for services with the SPANNER database type.
uid

string

Output only. The globally unique resource identifier of the metastore service.
metadata_management_activity

MetadataManagementActivity

Output only. The metadata management activities of the metastore service.
release_channel

ReleaseChannel

Immutable. The release channel of the service. If unspecified, defaults to STABLE.
encryption_config

EncryptionConfig

Immutable. Information used to configure the Dataproc Metastore service to encrypt customer data at rest. Cannot be updated.
network_config

NetworkConfig

The configuration specifying the network settings for the Dataproc Metastore service.
database_type

DatabaseType

Immutable. The database type that the Metastore service stores its data.
telemetry_config

TelemetryConfig

The configuration specifying telemetry settings for the Dataproc Metastore service. If unspecified defaults to JSON.
scaling_config

ScalingConfig

Scaling configuration of the metastore service.
scheduled_backup

ScheduledBackup

Optional. The configuration of scheduled backup for the metastore service.
deletion_protection

bool

Optional. Indicates if the dataproc metastore should be protected against accidental deletions.
Union field metastore_config. Configuration properties specific to the underlying metastore service technology (the software that serves metastore queries). metastore_config can be only one of the following:
hive_metastore_config

HiveMetastoreConfig

Configuration information specific to running Hive metastore software as the metastore service.

DatabaseType

The backend database type for the metastore service.

Enums
DATABASE_TYPE_UNSPECIFIED The DATABASE_TYPE is not set.
MYSQL MySQL is used to persist the metastore data.
SPANNER Spanner is used to persist the metastore data.

ReleaseChannel

Release channels bundle features of varying levels of stability. Newer features may be introduced initially into less stable release channels and can be automatically promoted into more stable release channels.

Enums
RELEASE_CHANNEL_UNSPECIFIED Release channel is not specified.
CANARY The CANARY release channel contains the newest features, which may be unstable and subject to unresolved issues with no known workarounds. Services using the CANARY release channel are not subject to any SLAs.
STABLE The STABLE release channel contains features that are considered stable and have been validated for production use.

State

The current state of the metastore service.

Enums
STATE_UNSPECIFIED The state of the metastore service is unknown.
CREATING The metastore service is in the process of being created.
ACTIVE The metastore service is running and ready to serve queries.
SUSPENDING The metastore service is entering suspension. Its query-serving availability may cease unexpectedly.
SUSPENDED The metastore service is suspended and unable to serve queries.
UPDATING The metastore service is being updated. It remains usable but cannot accept additional update requests or be deleted at this time.
DELETING The metastore service is undergoing deletion. It cannot be used.
ERROR The metastore service has encountered an error and cannot be used. The metastore service should be deleted.
AUTOSCALING The Dataproc Metastore service 2 is being scaled up or down.
MIGRATING The metastore service is processing a managed migration.

Tier

Available service tiers.

Enums
TIER_UNSPECIFIED The tier is not set.
DEVELOPER The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept.
ENTERPRISE The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads.

StartMigrationRequest

Request message for DataprocMetastore.StartMigration.

Fields
service

string

Required. The relative resource name of the metastore service to start migrating to, in the following format:

projects/{project_id}/locations/{location_id}/services/{service_id}.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.migrations.start
migration_execution

MigrationExecution

Required. The configuration details for the migration.
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

TelemetryConfig

Telemetry Configuration for the Dataproc Metastore service.

Fields
log_format

LogFormat

The output format of the Dataproc Metastore service's logs.

LogFormat

Enums
LOG_FORMAT_UNSPECIFIED The LOG_FORMAT is not set.
LEGACY Logging output uses the legacy textPayload format.
JSON Logging output uses the jsonPayload format.

UpdateFederationRequest

Request message for UpdateFederation.

Fields
update_mask

FieldMask

Required. A field mask used to specify the fields to be overwritten in the metastore federation resource by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
federation

Federation

Required. The metastore federation to update. The server only merges fields in the service if they are specified in update_mask.

The metastore federation's name field is used to identify the metastore service to be updated.

Authorization requires the following IAM permission on the specified resource federation:

  • metastore.federations.update
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

UpdateMetadataImportRequest

Request message for DataprocMetastore.UpdateMetadataImport.

Fields
update_mask

FieldMask

Required. A field mask used to specify the fields to be overwritten in the metadata import resource by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
metadata_import

MetadataImport

Required. The metadata import to update. The server only merges fields in the import if they are specified in update_mask.

The metadata import's name field is used to identify the metastore import to be updated.

Authorization requires the following IAM permission on the specified resource metadataImport:

  • metastore.imports.update
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.

UpdateServiceRequest

Request message for DataprocMetastore.UpdateService.

Fields
update_mask

FieldMask

Required. A field mask used to specify the fields to be overwritten in the metastore service resource by the update. Fields specified in the update_mask are relative to the resource (not to the full request). A field is overwritten if it is in the mask.
service

Service

Required. The metastore service to update. The server only merges fields in the service if they are specified in update_mask.

The metastore service's name field is used to identify the metastore service to be updated.

Authorization requires the following IAM permission on the specified resource service:

  • metastore.services.update
request_id

string

Optional. A request ID. Specify a unique request ID to allow the server to ignore the request if it has completed. The server will ignore subsequent requests that provide a duplicate request ID for at least 60 minutes after the first request.

For example, if an initial request times out, followed by another request with the same request ID, the server ignores the second request to prevent the creation of duplicate commitments.

The request ID must be a valid UUID A zero UUID (00000000-0000-0000-0000-000000000000) is not supported.