Try Gemini 1.5 models, the latest multimodal models in Vertex AI, and see what you can build with up to a 2M token context window. Try Gemini 1.5 models, the latest multimodal models in Vertex AI, and see what you can build with up to a 2M token context window.

Class CustomMetric (1.67.1)

CustomMetric(
    name: str,
    metric_function: typing.Callable[
        [typing.Dict[str, typing.Any]], typing.Dict[str, typing.Any]
    ],
)

The custom evaluation metric.

A fully-customized CustomMetric that can be used to evaluate a single model by defining a metric function for a computation-based metric. The CustomMetric is computed on the client-side using the user-defined metric function in SDK only, not by the Vertex Gen AI Evaluation Service.

Attributes: name: The name of the metric. metric_function: The user-defined evaluation function to compute a metric score. Must use the dataset row dictionary as the metric function input and return per-instance metric result as a dictionary output. The metric score must mapped to the name of the CustomMetric as key.

Methods

CustomMetric

CustomMetric(
    name: str,
    metric_function: typing.Callable[
        [typing.Dict[str, typing.Any]], typing.Dict[str, typing.Any]
    ],
)

Initializes the evaluation metric.