View analytics

This page describes how to view analytics for your recommendations and search apps.

Analytics give you insight into the usage trends, search and recommendation quality, and end-user engagement of your app. The Vertex AI Agent Builder console provides an interactive dashboard experience powered by Looker.

• For media recommendations apps: Public preview. You can see per-recommendation metrics based on user events. You can filter metrics by date range and device type.
• For generic search and media search apps: You can see per-search metrics and per-search-session metrics based on search logs and user events. You can filter metrics by date range, query, and device type.
• For healthcare search apps: Public preview. You can see per-search metrics based on search logs.

Metrics are refreshed about every six hours, so it can take several hours before you can use the Analytics page after you create your app.

Data sources for analytics

This section describes the data sources for analytics depending on app type.

You must complete user events integration to be able to see user event analytics.

Which user events you upload affect which analytics are generated.

Media recommendations apps require the following user events to view some metrics:

• View-item events. Required for click-through-rate.
• Media-complete events. Required for completion rate per play and completion rate per recommendation.
• Media-play events. Required for recommendation media play count, playthrough rate, completion rate per play, and average watch time per play.

Search apps require the following user events to view some metrics:

• Search events. Required for click-through-rate.
• View-item events. Required for click-through-rate.

The following information is also used for some analytics:

• Impressions. You can provide impressions (item views or clicks) that are attributable to Vertex AI Search by including the associated item in UserEvent.Documents when ingesting user events. This information is needed for attribution and to compute per-search metrics.
• Attribution tokens. Attribution tokens are unique IDs generated by Vertex AI Search and returned with each search request. Make sure to include that attribution token as UserEvent.attributionToken with any user events resulting from a search. This is needed to identify if a search is served by the API. Only user events with a Google-generated attribution token are used to compute metrics.
• User agent. Include UserInfo.userAgent with user events so that you can filter on user event metrics by device type.

Search logs

Search apps use search logs as data sources for some metrics.

Search logs are based on search requests. To allow Vertex AI Agent Builder to identify user sessions so that it can deliver per-session analytics and higher quality search results, Google strongly recommends that you set the optional field SearchRequest.UserPseudoId.

If there are not yet any search logs or user events, default values are shown for all metrics.

View media recommendations analytics

Note: Media recommendations analytics are in Public preview.

Use the following instructions to view analytics for media recommendations.

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Click the name of the app that you want to view analytics for.

3. Click Analytics.

4. To filter your metrics, specify any of the following filters and then click the Refresh button to apply them:

   • Date range. Select a preset date range or, if available, enter a custom date range.
   • Device type. Select a device type that queries occurred on.

View search analytics

Notes:

• For data stores in US and EU multi-regions, viewing search analytics is in Public preview.
• Because Vertex AI Agent Builder for healthcare search provides search services only in the US multi-region (`us`), analytics for healthcare search are in Public preview.
• Search analytics are GA for global data stores.
• By default, metrics from the widget don't appear on the Analytics page. To view widget metrics, contact your Google account team and ask to be added to the allowlist.

Use the following instructions to view analytics for search apps.

1. In the Google Cloud console, go to the Agent Builder page.

   Agent Builder

2. Click the name of the app that you want to view analytics for.

3. Click Analytics.

4. Click a tab to view that metric group:

   • Per Search. Metrics are grouped by searches. For healthcare search, this is the only available metric group.
   • Per Session. Metrics are grouped by search sessions.
   • Compare. Baseline metrics are shown alongside metrics from a comparison period. Select date ranges for the baseline metrics (an earlier time range) and comparison metrics (a later time range). These time ranges can't overlap.

5. To filter your metrics, specify the filters and then click the Refresh button to apply them.

   For media apps and for generic search apps, the following filters are available:

   • Date range. Select a preset date range or, if available, enter a custom date range.
   • Search query. Select a condition and search query value. You can add multiple search query filters. Available for Per Search and Compare metrics.
   • Device type. Select a device type that queries occurred on.

   For healthcare search apps, the following filters are available:

   • Date range. Select a preset date range or, if available, enter a custom date range. When you select a specific date for custom date range, that date is the date when the search took place in the UTC -08:00 time zone.
   • Search query. Select a condition and search query value. You can add multiple search query filters.
   • User location. Select the location of the hospital, clinic, or the healthcare service provider that was set in the search request. This filter is available only if it was set in the search request.
   • User role. Select the job category of the user who sent the search request. This filter is available only if it was set in the search request.
   • User setting. Select the healthcare setting set in the search request, such as inpatient, outpatient, or hybrid treatment. This filter is available only if it was set in the search request.

Metrics definitions

The following tables describe how metrics are defined.

Media recommendations metrics

Metric name	Metric definition	Notes
Recommendation count	Total number of recommendations
Recommendation play count	Number of times that recommended media items were played
Recommendation click count	Number of times that recommended media items were clicked
Click-through rate	Count of recommendation clicks / recommendation count	Recommendation clicks are view-item events that can be attributed to a recommendation
Recommendation media complete count	Number of times that recommended media items were played to completion
Playthrough rate per recommendation	Number of times that recommended media items were played / recommendation count
Completion rate per play	Number of times that recommended media items were played to completion / number of times that media items were played
Average watch time per play	Time in seconds that recommended media items were played / number of times that recommended media items were played
Completion rate per recommendation	Number of times that media items were played to completion / recommendation count
Average watch time per recommendation	Time in seconds that recommended media items were watched / recommendation count

Media and generic search metrics

Metric name	Metrics Definition	Notes
Search count	Count of search events	Based on search logs.
No result rate	Count of search events without results / Search count	Based on search logs.
Click-through rate (CTR) per search	Count of search clicks / Search count	Search clicks are view-item events that can be attributed to a previous search event. Based on user events.
Search session count	Count of search sessions	A search session is a user session containing at least one search event. A user session, also called a visit, is a continuous set of user events. When there are 30 minutes of inactivity, the session ends. Based on search logs.
Page view per search visit	Count of view-item events in search sessions / Search session count	This metric includes all page views in search visits regardless of whether they can be attributed to Vertex AI Search. Based on user events.
Bounce rate	Count of bounces in search sessions / Search session count	A search session bounce is defined as a session with a single search, where the user left after only making one search. Based on user events.

Healthcare search metrics

The following search metrics are available for healthcare search apps.

Metric name	Metrics Definition	Notes
Search count	Number of search events
No result count	Number of search events without results
No result rate per search	Number of search events without results per search
Patient count	Count of patient IDs for which searches took place	This is deduplicated per search user but not per patient. For example, if user 1 searches for patients A and B and user 2 searches for only patient B, then the count is three.
Customer search session count	Number of search user-defined sessions	You can send a session ID in your search. For more information, see session.
Search visitor count	Number of users who sent a search request	Users are counted based on the userInfo field.
Synthetic search session count	Number of Google-defined search sessions	A search session is a user session containing at least one search event. A user session, also called a visit, is a continuous set of user events. When there are 30 minutes of inactivity, the session ends.

Turn off analytics

You can't turn off analytics for an existing app, but when you create an app you can choose to create it with analytics turned off.

To create a search app that won't collect analytics data:

1. Follow the REST instructions in Create a search app, adding the following to the curl command:

      "disableAnalytics": true
   

   Example command

   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json" \
   -H "X-Goog-User-Project: my-project-123" \
   "https://discoveryengine.googleapis.com/v1/projects/my-project-123/locations/global/collections/default_collection/engines?engineId=my-app" \
   -d '{
     "displayName": "App without analytics",
     "dataStoreIds": ["my-data-store"],
     "solutionType": "SOLUTION_TYPE_SEARCH",
     "searchEngineConfig": {
        "searchTier": "SEARCH_TIER_ENTERPRISE",
        "searchAddOns": ["SEARCH_ADD_ON_LLM"]
      },
     "disableAnalytics": true
   }'
   
       

   This command creates an app that won't collect search analytics data.