Dataflow API v1beta3 - Package cloud.google.com/go/dataflow/apiv1beta3 (v0.86.0)

Version 0.86.0

• 0.116.0 (latest)
• 0.115.1
• 0.114.0
• 0.113.0
• 0.112.2
• 0.111.0
• 0.110.10
• 0.109.0
• 0.108.0
• 0.107.0
• 0.106.0
• 0.105.0
• 0.104.0
• 0.103.0
• 0.102.1
• 0.101.1
• 0.100.2
• 0.99.0
• 0.98.0
• 0.97.0
• 0.96.0
• 0.95.0
• 0.94.1
• 0.93.3
• 0.92.3
• 0.91.1
• 0.90.0
• 0.89.0
• 0.88.0
• 0.87.0
• 0.86.0
• 0.85.0
• 0.84.0
• 0.83.0
• 0.82.0
• 0.81.0
• 0.80.0
• 0.79.0
• 0.78.0
• 0.77.0
• 0.76.0
• 0.75.0

Beta

This library is covered by the Pre-GA Offerings Terms of the Terms of Service. Pre-GA libraries might have limited support, and changes to pre-GA libraries might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

Note: To get more information about this package, such as access to older versions, view this package on pkg.go.dev.

Package dataflow is an auto-generated package for the Dataflow API.

Manages Google Cloud Dataflow projects on Google Cloud Platform.

NOTE: This package is in beta. It is not stable, and may be subject to changes.

Use of Context

The ctx passed to NewClient is used for authentication requests and for creating the underlying connection, but is not used for subsequent calls. Individual methods on the client use the ctx given to them.

To close the open connection, use the Close() method.

For information about setting deadlines, reusing contexts, and more please visit https://pkg.go.dev/cloud.google.com/go.

Functions

func DefaultAuthScopes

func DefaultAuthScopes() []string

DefaultAuthScopes reports the default set of authentication scopes to use with this package.

FlexTemplatesCallOptions

type FlexTemplatesCallOptions struct {
	LaunchFlexTemplate []gax.CallOption
}

FlexTemplatesCallOptions contains the retry settings for each method of FlexTemplatesClient.

FlexTemplatesClient

type FlexTemplatesClient struct {

	// The call options for this service.
	CallOptions *FlexTemplatesCallOptions
	// contains filtered or unexported fields
}

FlexTemplatesClient is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

Provides a service for Flex templates. This feature is not ready yet.

func NewFlexTemplatesClient

func NewFlexTemplatesClient(ctx context.Context, opts ...option.ClientOption) (*FlexTemplatesClient, error)

NewFlexTemplatesClient creates a new flex templates service client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

Provides a service for Flex templates. This feature is not ready yet.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewFlexTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*FlexTemplatesClient) Close

func (c *FlexTemplatesClient) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*FlexTemplatesClient) Connection

func (c *FlexTemplatesClient) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*FlexTemplatesClient) LaunchFlexTemplate

func (c *FlexTemplatesClient) LaunchFlexTemplate(ctx context.Context, req *dataflowpb.LaunchFlexTemplateRequest, opts ...gax.CallOption) (*dataflowpb.LaunchFlexTemplateResponse, error)

LaunchFlexTemplate launch a job with a FlexTemplate.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewFlexTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.LaunchFlexTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.LaunchFlexTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

JobIterator

type JobIterator struct {

	// Response is the raw response for the current page.
	// It must be cast to the RPC response type.
	// Calling Next() or InternalFetch() updates this value.
	Response interface{}

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataflowpb.Job, nextPageToken string, err error)
	// contains filtered or unexported fields
}

JobIterator manages a stream of *dataflowpb.Job.

func (*JobIterator) Next

func (it *JobIterator) Next() (*dataflowpb.Job, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*JobIterator) PageInfo

func (it *JobIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

JobMessageIterator

type JobMessageIterator struct {

	// Response is the raw response for the current page.
	// It must be cast to the RPC response type.
	// Calling Next() or InternalFetch() updates this value.
	Response interface{}

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataflowpb.JobMessage, nextPageToken string, err error)
	// contains filtered or unexported fields
}

JobMessageIterator manages a stream of *dataflowpb.JobMessage.

func (*JobMessageIterator) Next

func (it *JobMessageIterator) Next() (*dataflowpb.JobMessage, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*JobMessageIterator) PageInfo

func (it *JobMessageIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

JobsV1Beta3CallOptions

type JobsV1Beta3CallOptions struct {
	CreateJob          []gax.CallOption
	GetJob             []gax.CallOption
	UpdateJob          []gax.CallOption
	ListJobs           []gax.CallOption
	AggregatedListJobs []gax.CallOption
	CheckActiveJobs    []gax.CallOption
	SnapshotJob        []gax.CallOption
}

JobsV1Beta3CallOptions contains the retry settings for each method of JobsV1Beta3Client.

JobsV1Beta3Client

type JobsV1Beta3Client struct {

	// The call options for this service.
	CallOptions *JobsV1Beta3CallOptions
	// contains filtered or unexported fields
}

JobsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.

func NewJobsV1Beta3Client

func NewJobsV1Beta3Client(ctx context.Context, opts ...option.ClientOption) (*JobsV1Beta3Client, error)

NewJobsV1Beta3Client creates a new jobs v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*JobsV1Beta3Client) AggregatedListJobs

func (c *JobsV1Beta3Client) AggregatedListJobs(ctx context.Context, req *dataflowpb.ListJobsRequest, opts ...gax.CallOption) *JobIterator

AggregatedListJobs list the jobs of a project across all regions.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	"google.golang.org/api/iterator"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.ListJobsRequest{
		// TODO: Fill request struct fields.
	}
	it := c.AggregatedListJobs(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

func (*JobsV1Beta3Client) CheckActiveJobs

func (c *JobsV1Beta3Client) CheckActiveJobs(ctx context.Context, req *dataflowpb.CheckActiveJobsRequest, opts ...gax.CallOption) (*dataflowpb.CheckActiveJobsResponse, error)

CheckActiveJobs check for existence of active jobs in the given project across all regions.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.CheckActiveJobsRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.CheckActiveJobs(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*JobsV1Beta3Client) Close

func (c *JobsV1Beta3Client) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*JobsV1Beta3Client) Connection

func (c *JobsV1Beta3Client) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*JobsV1Beta3Client) CreateJob

func (c *JobsV1Beta3Client) CreateJob(ctx context.Context, req *dataflowpb.CreateJobRequest, opts ...gax.CallOption) (*dataflowpb.Job, error)

CreateJob creates a Cloud Dataflow job.

To create a job, we recommend using projects.locations.jobs.create with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). Using projects.jobs.create is not recommended, as your job will always start in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.CreateJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.CreateJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*JobsV1Beta3Client) GetJob

func (c *JobsV1Beta3Client) GetJob(ctx context.Context, req *dataflowpb.GetJobRequest, opts ...gax.CallOption) (*dataflowpb.Job, error)

GetJob gets the state of the specified Cloud Dataflow job.

To get the state of a job, we recommend using projects.locations.jobs.get with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). Using projects.jobs.get is not recommended, as you can only get the state of jobs that are running in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*JobsV1Beta3Client) ListJobs

func (c *JobsV1Beta3Client) ListJobs(ctx context.Context, req *dataflowpb.ListJobsRequest, opts ...gax.CallOption) *JobIterator

ListJobs list the jobs of a project.

To list the jobs of a project in a region, we recommend using projects.locations.jobs.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). To list the all jobs across all regions, use projects.jobs.aggregated. Using projects.jobs.list is not recommended, as you can only get the list of jobs that are running in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	"google.golang.org/api/iterator"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.ListJobsRequest{
		// TODO: Fill request struct fields.
	}
	it := c.ListJobs(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

func (*JobsV1Beta3Client) SnapshotJob

func (c *JobsV1Beta3Client) SnapshotJob(ctx context.Context, req *dataflowpb.SnapshotJobRequest, opts ...gax.CallOption) (*dataflowpb.Snapshot, error)

SnapshotJob snapshot the state of a streaming job.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.SnapshotJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.SnapshotJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*JobsV1Beta3Client) UpdateJob

func (c *JobsV1Beta3Client) UpdateJob(ctx context.Context, req *dataflowpb.UpdateJobRequest, opts ...gax.CallOption) (*dataflowpb.Job, error)

UpdateJob updates the state of an existing Cloud Dataflow job.

To update the state of an existing job, we recommend using projects.locations.jobs.update with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). Using projects.jobs.update is not recommended, as you can only update the state of jobs that are running in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewJobsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.UpdateJobRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.UpdateJob(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

MessagesV1Beta3CallOptions

type MessagesV1Beta3CallOptions struct {
	ListJobMessages []gax.CallOption
}

MessagesV1Beta3CallOptions contains the retry settings for each method of MessagesV1Beta3Client.

MessagesV1Beta3Client

type MessagesV1Beta3Client struct {

	// The call options for this service.
	CallOptions *MessagesV1Beta3CallOptions
	// contains filtered or unexported fields
}

MessagesV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.

func NewMessagesV1Beta3Client

func NewMessagesV1Beta3Client(ctx context.Context, opts ...option.ClientOption) (*MessagesV1Beta3Client, error)

NewMessagesV1Beta3Client creates a new messages v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMessagesV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*MessagesV1Beta3Client) Close

func (c *MessagesV1Beta3Client) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*MessagesV1Beta3Client) Connection

func (c *MessagesV1Beta3Client) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*MessagesV1Beta3Client) ListJobMessages

func (c *MessagesV1Beta3Client) ListJobMessages(ctx context.Context, req *dataflowpb.ListJobMessagesRequest, opts ...gax.CallOption) *JobMessageIterator

ListJobMessages request the job status.

To request the status of a job, we recommend using projects.locations.jobs.messages.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). Using projects.jobs.messages.list is not recommended, as you can only request the status of jobs that are running in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	"google.golang.org/api/iterator"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMessagesV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.ListJobMessagesRequest{
		// TODO: Fill request struct fields.
	}
	it := c.ListJobMessages(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

MetricsV1Beta3CallOptions

type MetricsV1Beta3CallOptions struct {
	GetJobMetrics            []gax.CallOption
	GetJobExecutionDetails   []gax.CallOption
	GetStageExecutionDetails []gax.CallOption
}

MetricsV1Beta3CallOptions contains the retry settings for each method of MetricsV1Beta3Client.

MetricsV1Beta3Client

type MetricsV1Beta3Client struct {

	// The call options for this service.
	CallOptions *MetricsV1Beta3CallOptions
	// contains filtered or unexported fields
}

MetricsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.

func NewMetricsV1Beta3Client

func NewMetricsV1Beta3Client(ctx context.Context, opts ...option.ClientOption) (*MetricsV1Beta3Client, error)

NewMetricsV1Beta3Client creates a new metrics v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMetricsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*MetricsV1Beta3Client) Close

func (c *MetricsV1Beta3Client) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*MetricsV1Beta3Client) Connection

func (c *MetricsV1Beta3Client) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*MetricsV1Beta3Client) GetJobExecutionDetails

func (c *MetricsV1Beta3Client) GetJobExecutionDetails(ctx context.Context, req *dataflowpb.GetJobExecutionDetailsRequest, opts ...gax.CallOption) *StageSummaryIterator

GetJobExecutionDetails request detailed information about the execution status of the job.

EXPERIMENTAL. This API is subject to change or removal without notice.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	"google.golang.org/api/iterator"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMetricsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetJobExecutionDetailsRequest{
		// TODO: Fill request struct fields.
	}
	it := c.GetJobExecutionDetails(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

func (*MetricsV1Beta3Client) GetJobMetrics

func (c *MetricsV1Beta3Client) GetJobMetrics(ctx context.Context, req *dataflowpb.GetJobMetricsRequest, opts ...gax.CallOption) (*dataflowpb.JobMetrics, error)

GetJobMetrics request the job status.

To request the status of a job, we recommend using projects.locations.jobs.getMetrics with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints (at https://cloud.google.com/dataflow/docs/concepts/regional-endpoints)). Using projects.jobs.getMetrics is not recommended, as you can only request the status of jobs that are running in us-central1.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMetricsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetJobMetricsRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetJobMetrics(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*MetricsV1Beta3Client) GetStageExecutionDetails

func (c *MetricsV1Beta3Client) GetStageExecutionDetails(ctx context.Context, req *dataflowpb.GetStageExecutionDetailsRequest, opts ...gax.CallOption) *WorkerDetailsIterator

GetStageExecutionDetails request detailed information about the execution status of a stage of the job.

EXPERIMENTAL. This API is subject to change or removal without notice.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	"google.golang.org/api/iterator"

	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewMetricsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetStageExecutionDetailsRequest{
		// TODO: Fill request struct fields.
	}
	it := c.GetStageExecutionDetails(ctx, req)
	for {
		resp, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		// TODO: Use resp.
		_ = resp
	}
}

SnapshotsV1Beta3CallOptions

type SnapshotsV1Beta3CallOptions struct {
	GetSnapshot    []gax.CallOption
	DeleteSnapshot []gax.CallOption
	ListSnapshots  []gax.CallOption
}

SnapshotsV1Beta3CallOptions contains the retry settings for each method of SnapshotsV1Beta3Client.

SnapshotsV1Beta3Client

type SnapshotsV1Beta3Client struct {

	// The call options for this service.
	CallOptions *SnapshotsV1Beta3CallOptions
	// contains filtered or unexported fields
}

SnapshotsV1Beta3Client is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

Provides methods to manage snapshots of Google Cloud Dataflow jobs.

func NewSnapshotsV1Beta3Client

func NewSnapshotsV1Beta3Client(ctx context.Context, opts ...option.ClientOption) (*SnapshotsV1Beta3Client, error)

NewSnapshotsV1Beta3Client creates a new snapshots v1 beta3 client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

Provides methods to manage snapshots of Google Cloud Dataflow jobs.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewSnapshotsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*SnapshotsV1Beta3Client) Close

func (c *SnapshotsV1Beta3Client) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*SnapshotsV1Beta3Client) Connection

func (c *SnapshotsV1Beta3Client) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*SnapshotsV1Beta3Client) DeleteSnapshot

func (c *SnapshotsV1Beta3Client) DeleteSnapshot(ctx context.Context, req *dataflowpb.DeleteSnapshotRequest, opts ...gax.CallOption) (*dataflowpb.DeleteSnapshotResponse, error)

DeleteSnapshot deletes a snapshot.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewSnapshotsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.DeleteSnapshotRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.DeleteSnapshot(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*SnapshotsV1Beta3Client) GetSnapshot

func (c *SnapshotsV1Beta3Client) GetSnapshot(ctx context.Context, req *dataflowpb.GetSnapshotRequest, opts ...gax.CallOption) (*dataflowpb.Snapshot, error)

GetSnapshot gets information about a snapshot.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewSnapshotsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetSnapshotRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetSnapshot(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*SnapshotsV1Beta3Client) ListSnapshots

func (c *SnapshotsV1Beta3Client) ListSnapshots(ctx context.Context, req *dataflowpb.ListSnapshotsRequest, opts ...gax.CallOption) (*dataflowpb.ListSnapshotsResponse, error)

ListSnapshots lists snapshots.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewSnapshotsV1Beta3Client(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.ListSnapshotsRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.ListSnapshots(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

StageSummaryIterator

type StageSummaryIterator struct {

	// Response is the raw response for the current page.
	// It must be cast to the RPC response type.
	// Calling Next() or InternalFetch() updates this value.
	Response interface{}

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataflowpb.StageSummary, nextPageToken string, err error)
	// contains filtered or unexported fields
}

StageSummaryIterator manages a stream of *dataflowpb.StageSummary.

func (*StageSummaryIterator) Next

func (it *StageSummaryIterator) Next() (*dataflowpb.StageSummary, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*StageSummaryIterator) PageInfo

func (it *StageSummaryIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

TemplatesCallOptions

type TemplatesCallOptions struct {
	CreateJobFromTemplate []gax.CallOption
	LaunchTemplate        []gax.CallOption
	GetTemplate           []gax.CallOption
}

TemplatesCallOptions contains the retry settings for each method of TemplatesClient.

TemplatesClient

type TemplatesClient struct {

	// The call options for this service.
	CallOptions *TemplatesCallOptions
	// contains filtered or unexported fields
}

TemplatesClient is a client for interacting with Dataflow API. Methods, except Close, may be called concurrently. However, fields must not be modified concurrently with method calls.

Provides a method to create Cloud Dataflow jobs from templates.

func NewTemplatesClient

func NewTemplatesClient(ctx context.Context, opts ...option.ClientOption) (*TemplatesClient, error)

NewTemplatesClient creates a new templates service client based on gRPC. The returned client must be Closed when it is done being used to clean up its underlying connections.

Provides a method to create Cloud Dataflow jobs from templates.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	// TODO: Use client.
	_ = c
}

func (*TemplatesClient) Close

func (c *TemplatesClient) Close() error

Close closes the connection to the API service. The user should invoke this when the client is no longer required.

func (*TemplatesClient) Connection

func (c *TemplatesClient) Connection() *grpc.ClientConn

Connection returns a connection to the API service.

Deprecated.

func (*TemplatesClient) CreateJobFromTemplate

func (c *TemplatesClient) CreateJobFromTemplate(ctx context.Context, req *dataflowpb.CreateJobFromTemplateRequest, opts ...gax.CallOption) (*dataflowpb.Job, error)

CreateJobFromTemplate creates a Cloud Dataflow job from a template.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.CreateJobFromTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.CreateJobFromTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*TemplatesClient) GetTemplate

func (c *TemplatesClient) GetTemplate(ctx context.Context, req *dataflowpb.GetTemplateRequest, opts ...gax.CallOption) (*dataflowpb.GetTemplateResponse, error)

GetTemplate get the template associated with a template.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.GetTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.GetTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

func (*TemplatesClient) LaunchTemplate

func (c *TemplatesClient) LaunchTemplate(ctx context.Context, req *dataflowpb.LaunchTemplateRequest, opts ...gax.CallOption) (*dataflowpb.LaunchTemplateResponse, error)

LaunchTemplate launch a template.

Example

package main

import (
	"context"

	dataflow "cloud.google.com/go/dataflow/apiv1beta3"
	dataflowpb "google.golang.org/genproto/googleapis/dataflow/v1beta3"
)

func main() {
	ctx := context.Background()
	c, err := dataflow.NewTemplatesClient(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	defer c.Close()

	req := &dataflowpb.LaunchTemplateRequest{
		// TODO: Fill request struct fields.
	}
	resp, err := c.LaunchTemplate(ctx, req)
	if err != nil {
		// TODO: Handle error.
	}
	// TODO: Use resp.
	_ = resp
}

WorkerDetailsIterator

type WorkerDetailsIterator struct {

	// Response is the raw response for the current page.
	// It must be cast to the RPC response type.
	// Calling Next() or InternalFetch() updates this value.
	Response interface{}

	// InternalFetch is for use by the Google Cloud Libraries only.
	// It is not part of the stable interface of this package.
	//
	// InternalFetch returns results from a single call to the underlying RPC.
	// The number of results is no greater than pageSize.
	// If there are no more results, nextPageToken is empty and err is nil.
	InternalFetch func(pageSize int, pageToken string) (results []*dataflowpb.WorkerDetails, nextPageToken string, err error)
	// contains filtered or unexported fields
}

WorkerDetailsIterator manages a stream of *dataflowpb.WorkerDetails.

func (*WorkerDetailsIterator) Next

func (it *WorkerDetailsIterator) Next() (*dataflowpb.WorkerDetails, error)

Next returns the next result. Its second return value is iterator.Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

func (*WorkerDetailsIterator) PageInfo

func (it *WorkerDetailsIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.