Search Image Warehouse data

After you have created an image corpus and performed AnalyzeCorpus to generate embedding signals on corpus data, you can proceed to create an index and an index endpoint. After deploying the index to this index endpoint, you can then perform semantic search with criteria filter capabilities. The search result contains a list of images sorted by relevance score.

Search with input text

To search the assets in your corpus, populate the SearchIndexEndpointRequest with the intended search, in the following formats:

text_query: Text query to search with.
Optional criteria or exclusion_criteria: Filter criteria on user-provided annotations. It can be applied to texts, numbers, booleans, or datetime values.

In the following example, consider a corpus that contains natural scenery images. To retrieve all assets that are relevant to "sunset at beach" and tagged with the annotation "state": "California", send the following request (replace FIELD_NAME with state and replace FIELD_VALUE with California):

REST & CMD LINE

Before using any of the request data, make the following replacements:

REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the LOCATION_ID such as europe-west4-. See more about regionalized endpoints.
PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west4. See available regions.
INDEX_ENDPOINT_ID: The ID of your target index endpoint.

HTTP method and URL:

POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint

Request JSON body:

{
  "text_query": "sunset at beach",
  "criteria": [
    {
      "field": "state",
      "text_array": {
          "txt_values": [
            "California"
          ]
      }
    }
  ]
}

To send your request, choose one of these options:

curl

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login , or by using Cloud Shell, which automatically logs you into the gcloud CLI . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"

PowerShell

Note: The following command assumes that you have logged in to the gcloud CLI with your user account by running gcloud init or gcloud auth login . You can check the currently active account by running gcloud auth list.

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "searchResultItems": [
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1",
      "relevance": "0.99"
    },
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2",
      "relevance": "0.98"
    }
  ]
}

Search with input image

If you have an image of a sunset at a beach, you can also use it as a search query to retrieve similar images:

REST & CMD LINE

Before using any of the request data, make the following replacements:

REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the LOCATION_ID such as europe-west4-. See more about regionalized endpoints.
PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west4. See available regions.
INDEX_ENDPOINT_ID: The ID of your target index endpoint.

HTTP method and URL:

POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint

Request JSON body:

{
  "image_query": {
    "input_image": "IMAGE_BYTES"
  },
  "criteria": [
    {
      "field": "FIELD_NAME",
      "text_array": {
          "txt_values": [
            "FIELD_VALUE"
          ]
      }
    }
  ]
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "searchResultItems": [
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1",
      "relevance": 0.99
    },
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2",
      "relevance": 0.98
    }
  ]
}

Search with existing image

You can also use an existing image asset in the corpus to retrieve similar images. The first search result will be the same image asset with a 1.0 relevance score:

REST & CMD LINE

Before using any of the request data, make the following replacements:

REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the LOCATION_ID such as europe-west4-. See more about regionalized endpoints.
PROJECT_NUMBER: Your Google Cloud project number.
LOCATION_ID: The region where you are using Vertex AI Vision. For example: us-central1, europe-west4. See available regions.
INDEX_ENDPOINT_ID: The ID of your target index endpoint.
CORPUS_ID: The ID of your target corpus.
ASSET_ID: The ID of your target asset.

HTTP method and URL:

POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint

Request JSON body:

{
  "image_query": {
    "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID"
  }
}

To send your request, choose one of these options:

curl

Save the request body in a file named request.json, and execute the following command:

curl -X POST \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json; charset=utf-8" \
     -d @request.json \
     "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"

PowerShell

Save the request body in a file named request.json, and execute the following command:

$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }

Invoke-WebRequest `
    -Method POST `
    -Headers $headers `
    -ContentType: "application/json; charset=utf-8" `
    -InFile request.json `
    -Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content

You should receive a JSON response similar to the following:

{
  "searchResultItems": [
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1",
      "relevance": 1.0
    },
    {
      "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2",
      "relevance": 0.99
    }
  ]
}