Class JobControllerClient (1.0.1)

JobControllerClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

The JobController provides methods to manage jobs.

Methods

JobControllerClient

JobControllerClient(
    transport=None,
    channel=None,
    credentials=None,
    client_config=None,
    client_info=None,
    client_options=None,
)

Constructor.

Parameters
Name	Description
`channel`	`grpc.Channel` DEPRECATED. A `Channel` instance through which to make calls. This argument is mutually exclusive with `credentials`; providing both will raise an exception.
`credentials`	`google.auth.credentials.Credentials` The authorization credentials to attach to requests. These credentials identify this application to the service. If none are specified, the client will attempt to ascertain the credentials from the environment. This argument is mutually exclusive with providing a transport instance to `transport`; doing so will raise an exception.
`client_config`	`dict` DEPRECATED. A dictionary of call options for each method. If not specified, the default configuration is used.
`client_info`	`google.api_core.gapic_v1.client_info.ClientInfo` The client info used to send a user-agent string along with API requests. If `None`, then default info will be used. Generally, you only need to set this if you're developing your own client library.
`client_options`	`Union[dict, google.api_core.client_options.ClientOptions]` Client options used to set user options on the client. API Endpoint should be set through client_options.

cancel_job

cancel_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list <https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs/list> or regions/{region}/jobs.get <https://cloud.google.com/dataproc/docs/reference/rest/v1/projects.regions.jobs/get>.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

response = client.cancel_job(project_id, region, job_id)

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job_id`	`str` Required. The job ID.
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

delete_job

delete_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

client.delete_job(project_id, region, job_id)

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job_id`	`str` Required. The job ID.
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

from_service_account_file

from_service_account_file(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
Name	Description
`filename`	`str` The path to the service account private key json file.

Returns
Type	Description
`JobControllerClient`	The constructed client.

from_service_account_json

from_service_account_json(filename, *args, **kwargs)

Creates an instance of this client using the provided credentials file.

Parameter
Name	Description
`filename`	`str` The path to the service account private key json file.

Returns
Type	Description
`JobControllerClient`	The constructed client.

get_job

get_job(project_id, region, job_id, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Gets the resource representation for a job in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

response = client.get_job(project_id, region, job_id)

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job_id`	`str` Required. The job ID.
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

list_jobs

list_jobs(project_id, region, page_size=None, cluster_name=None, job_state_matcher=None, filter_=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Lists regions/{region}/jobs in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

Iterate over all results

for element in client.list_jobs(project_id, region): ... # process element ... pass

Alternatively:

Iterate over results one page at a time

for page in client.list_jobs(project_id, region).pages: ... for element in page: ... # process element ... pass

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`page_size`	`int` The maximum number of resources contained in the underlying API response. If page streaming is performed per- resource, this parameter does not affect the return value. If page streaming is performed per-page, this determines the maximum number of resources in a page.
`cluster_name`	`str` Optional. If set, the returned jobs list includes only jobs that were submitted to the named cluster.
`job_state_matcher`	`JobStateMatcher` Optional. Specifies enumerated categories of jobs to list. (default = match ALL jobs). If `filter` is provided, `jobStateMatcher` will be ignored.
`filter_`	`str` Optional. A filter constraining the jobs to list. Filters are case-sensitive and have the following syntax: [field = value] AND [field [= value]] ... where field is `status.state` or `labels.[KEY]`, and `[KEY]` is a label key. value can be `` to match all values. `status.state` can be either `ACTIVE` or `NON_ACTIVE`. Only the logical `AND` operator is supported; space-separated items are treated as having an implicit `AND` operator. Example filter: status.state = ACTIVE AND labels.env = staging AND labels.starred =
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

submit_job

submit_job(project_id, region, job, request_id=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Submits a job to a cluster.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job:

job = {}

response = client.submit_job(project_id, region, job)

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job`	`Union[dict, Job]` Required. The job resource. If a dict is provided, it must be of the same form as the protobuf message Job
`request_id`	`str` Optional. A unique id used to identify the request. If the server receives two `SubmitJobRequest` requests with the same id, then the second request will be ignored and the first `Job` created and stored in the backend is returned. It is recommended to always set this value to a `UUID https://en.wikipedia.org/wiki/Universally_unique_identifier`_. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (), and hyphens (-). The maximum length is 40 characters.
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

submit_job_as_operation

submit_job_as_operation(project_id, region, job, request_id=None, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Submits job to a cluster.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job:

job = {}

response = client.submit_job_as_operation(project_id, region, job)

def callback(operation_future): ... # Handle result. ... result = operation_future.result()

response.add_done_callback(callback)

Handle metadata.

metadata = response.metadata()

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job`	`Union[dict, Job]` Required. The job resource. If a dict is provided, it must be of the same form as the protobuf message Job
`request_id`	`str` Optional. A unique id used to identify the request. If the server receives two `SubmitJobRequest` requests with the same id, then the second request will be ignored and the first `Job` created and stored in the backend is returned. It is recommended to always set this value to a `UUID https://en.wikipedia.org/wiki/Universally_unique_identifier`_. The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (), and hyphens (-). The maximum length is 40 characters.
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

update_job

update_job(project_id, region, job_id, job, update_mask, retry=<_MethodDefault._DEFAULT_VALUE: <object object>>, timeout=<_MethodDefault._DEFAULT_VALUE: <object object>>, metadata=None)

Updates a job in a project.

.. rubric:: Example

from google.cloud import dataproc_v1

client = dataproc_v1.JobControllerClient()

TODO: Initialize project_id:

project_id = ''

TODO: Initialize region:

region = ''

TODO: Initialize job_id:

job_id = ''

TODO: Initialize job:

job = {}

TODO: Initialize update_mask:

update_mask = {}

response = client.update_job(project_id, region, job_id, job, update_mask)

Parameters
Name	Description
`project_id`	`str` Required. The ID of the Google Cloud Platform project that the job belongs to.
`region`	`str` Required. The Dataproc region in which to handle the request.
`job_id`	`str` Required. The job ID.
`job`	`Union[dict, Job]` Required. The changes to the job. If a dict is provided, it must be of the same form as the protobuf message Job
`update_mask`	`Union[dict, FieldMask]` Required. Specifies the path, relative to Job, of the field to update. For example, to update the labels of a Job the update_mask parameter would be specified as labels, and the `PATCH` request body would specify the new value. Note: Currently, labels is the only field that can be updated. If a dict is provided, it must be of the same form as the protobuf message FieldMask
`retry`	`Optional[google.api_core.retry.Retry]` A retry object used to retry requests. If `None` is specified, requests will be retried using a default configuration.
`timeout`	`Optional[float]` The amount of time, in seconds, to wait for the request to complete. Note that if `retry` is specified, the timeout applies to each individual attempt.
`metadata`	`Optional[Sequence[Tuple[str, str]]]` Additional metadata that is provided to the method.

Exceptions
Type	Description
`google.api_core.exceptions.GoogleAPICallError`	If the request failed for any reason.
`google.api_core.exceptions.RetryError`	If the request failed due to a retryable error and retry attempts failed.
`ValueError`	If the parameters are invalid.

Class JobControllerClient (1.0.1)

Methods

JobControllerClient

cancel_job

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

delete_job

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

from_service_account_file

from_service_account_json

get_job

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

list_jobs

TODO: Initialize `project_id`:

TODO: Initialize `region`:

Iterate over all results

Alternatively:

Iterate over results one page at a time

submit_job

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job`:

submit_job_as_operation

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job`:

Handle metadata.

update_job

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

TODO: Initialize `job`:

TODO: Initialize `update_mask`:

Class JobControllerClient (1.0.1)

Methods

JobControllerClient

cancel_job

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job_id:

delete_job

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job_id:

from_service_account_file

from_service_account_json

get_job

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job_id:

list_jobs

TODO: Initialize project_id:

TODO: Initialize region:

Iterate over all results

Alternatively:

Iterate over results one page at a time

submit_job

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job:

submit_job_as_operation

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job:

Handle metadata.

update_job

TODO: Initialize project_id:

TODO: Initialize region:

TODO: Initialize job_id:

TODO: Initialize job:

TODO: Initialize update_mask:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job`:

TODO: Initialize `project_id`:

TODO: Initialize `region`:

TODO: Initialize `job_id`:

TODO: Initialize `job`:

TODO: Initialize `update_mask`: