Cloud AI Platform v1beta1 API - Class PredictionService.PredictionServiceBase (1.0.0-beta03)

Version 1.0.0-beta03

• 1.0.0-beta09 (latest)
• 1.0.0-beta08

[BindServiceMethod(typeof(PredictionService), "BindService")]
public abstract class PredictionService.PredictionServiceBase

Reference documentation and code samples for the Cloud AI Platform v1beta1 API class PredictionService.PredictionServiceBase.

Base class for server-side implementations of PredictionService

Inheritance

object > PredictionService.PredictionServiceBase

Inherited Members

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ToString()

Namespace

Google.Cloud.AIPlatform.V1Beta1

Assembly

Google.Cloud.AIPlatform.V1Beta1.dll

Methods

ChatCompletions(ChatCompletionsRequest, IServerStreamWriter<HttpBody>, ServerCallContext)

public virtual Task ChatCompletions(ChatCompletionsRequest request, IServerStreamWriter<HttpBody> responseStream, ServerCallContext context)

Exposes an OpenAI-compatible endpoint for chat completions.

Parameters
Name	Description
request	ChatCompletionsRequest The request received from the client.
responseStream	IServerStreamWriterHttpBody Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

CountTokens(CountTokensRequest, ServerCallContext)

public virtual Task<CountTokensResponse> CountTokens(CountTokensRequest request, ServerCallContext context)

Perform a token counting.

Parameters
Name	Description
request	CountTokensRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskCountTokensResponse	The response to send back to the client (wrapped by a task).

DirectPredict(DirectPredictRequest, ServerCallContext)

public virtual Task<DirectPredictResponse> DirectPredict(DirectPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name	Description
request	DirectPredictRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskDirectPredictResponse	The response to send back to the client (wrapped by a task).

DirectRawPredict(DirectRawPredictRequest, ServerCallContext)

public virtual Task<DirectRawPredictResponse> DirectRawPredict(DirectRawPredictRequest request, ServerCallContext context)

Perform an unary online prediction request to a gRPC model server for custom containers.

Parameters
Name	Description
request	DirectRawPredictRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskDirectRawPredictResponse	The response to send back to the client (wrapped by a task).

Explain(ExplainRequest, ServerCallContext)

public virtual Task<ExplainResponse> Explain(ExplainRequest request, ServerCallContext context)

Perform an online explanation.

If [deployed_model_id][google.cloud.aiplatform.v1beta1.ExplainRequest.deployed_model_id] is specified, the corresponding DeployModel must have [explanation_spec][google.cloud.aiplatform.v1beta1.DeployedModel.explanation_spec] populated. If [deployed_model_id][google.cloud.aiplatform.v1beta1.ExplainRequest.deployed_model_id] is not specified, all DeployedModels must have [explanation_spec][google.cloud.aiplatform.v1beta1.DeployedModel.explanation_spec] populated.

Parameters
Name	Description
request	ExplainRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskExplainResponse	The response to send back to the client (wrapped by a task).

GenerateContent(GenerateContentRequest, ServerCallContext)

public virtual Task<GenerateContentResponse> GenerateContent(GenerateContentRequest request, ServerCallContext context)

Generate content with multimodal inputs.

Parameters
Name	Description
request	GenerateContentRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskGenerateContentResponse	The response to send back to the client (wrapped by a task).

Predict(PredictRequest, ServerCallContext)

public virtual Task<PredictResponse> Predict(PredictRequest request, ServerCallContext context)

Perform an online prediction.

Parameters
Name	Description
request	PredictRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskPredictResponse	The response to send back to the client (wrapped by a task).

RawPredict(RawPredictRequest, ServerCallContext)

public virtual Task<HttpBody> RawPredict(RawPredictRequest request, ServerCallContext context)

Perform an online prediction with an arbitrary HTTP payload.

The response includes the following HTTP headers:

• X-Vertex-AI-Endpoint-Id: ID of the [Endpoint][google.cloud.aiplatform.v1beta1.Endpoint] that served this prediction.

• X-Vertex-AI-Deployed-Model-Id: ID of the Endpoint's [DeployedModel][google.cloud.aiplatform.v1beta1.DeployedModel] that served this prediction.

Parameters
Name	Description
request	RawPredictRequest The request received from the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
TaskHttpBody	The response to send back to the client (wrapped by a task).

ServerStreamingPredict(StreamingPredictRequest, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task ServerStreamingPredict(StreamingPredictRequest request, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a server-side streaming online prediction request for Vertex LLM streaming.

Parameters
Name	Description
request	StreamingPredictRequest The request received from the client.
responseStream	IServerStreamWriterStreamingPredictResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest>, IServerStreamWriter<StreamDirectPredictResponse>, ServerCallContext)

public virtual Task StreamDirectPredict(IAsyncStreamReader<StreamDirectPredictRequest> requestStream, IServerStreamWriter<StreamDirectPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for Vertex first-party products and frameworks.

Parameters
Name	Description
requestStream	IAsyncStreamReaderStreamDirectPredictRequest Used for reading requests from the client.
responseStream	IServerStreamWriterStreamDirectPredictResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest>, IServerStreamWriter<StreamDirectRawPredictResponse>, ServerCallContext)

public virtual Task StreamDirectRawPredict(IAsyncStreamReader<StreamDirectRawPredictRequest> requestStream, IServerStreamWriter<StreamDirectRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request to a gRPC model server for custom containers.

Parameters
Name	Description
requestStream	IAsyncStreamReaderStreamDirectRawPredictRequest Used for reading requests from the client.
responseStream	IServerStreamWriterStreamDirectRawPredictResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamGenerateContent(GenerateContentRequest, IServerStreamWriter<GenerateContentResponse>, ServerCallContext)

public virtual Task StreamGenerateContent(GenerateContentRequest request, IServerStreamWriter<GenerateContentResponse> responseStream, ServerCallContext context)

Generate content with multimodal inputs with streaming support.

Parameters
Name	Description
request	GenerateContentRequest The request received from the client.
responseStream	IServerStreamWriterGenerateContentResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamRawPredict(StreamRawPredictRequest, IServerStreamWriter<HttpBody>, ServerCallContext)

public virtual Task StreamRawPredict(StreamRawPredictRequest request, IServerStreamWriter<HttpBody> responseStream, ServerCallContext context)

Perform a streaming online prediction with an arbitrary HTTP payload.

Parameters
Name	Description
request	StreamRawPredictRequest The request received from the client.
responseStream	IServerStreamWriterHttpBody Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamingPredict(IAsyncStreamReader<StreamingPredictRequest>, IServerStreamWriter<StreamingPredictResponse>, ServerCallContext)

public virtual Task StreamingPredict(IAsyncStreamReader<StreamingPredictRequest> requestStream, IServerStreamWriter<StreamingPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request for Vertex first-party products and frameworks.

Parameters
Name	Description
requestStream	IAsyncStreamReaderStreamingPredictRequest Used for reading requests from the client.
responseStream	IServerStreamWriterStreamingPredictResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.

StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest>, IServerStreamWriter<StreamingRawPredictResponse>, ServerCallContext)

public virtual Task StreamingRawPredict(IAsyncStreamReader<StreamingRawPredictRequest> requestStream, IServerStreamWriter<StreamingRawPredictResponse> responseStream, ServerCallContext context)

Perform a streaming online prediction request through gRPC.

Parameters
Name	Description
requestStream	IAsyncStreamReaderStreamingRawPredictRequest Used for reading requests from the client.
responseStream	IServerStreamWriterStreamingRawPredictResponse Used for sending responses back to the client.
context	ServerCallContext The context of the server-side call handler being invoked.

Returns
Type	Description
Task	A task indicating completion of the handler.