After you have created an image corpus and performed
AnalyzeCorpus
to generate embedding signals on corpus data, you can proceed to create an index
and an index endpoint. After deploying the index to this index endpoint, you can then
perform semantic search with criteria filter capabilities. The search result
contains a list of images sorted by relevance score.
Search with input text
To search the assets
in your corpus
, populate the SearchIndexEndpointRequest
with the intended search, in the following formats:
text_query
: Text query to search with.- Optional
criteria
orexclusion_criteria
: Filter criteria on user-provided annotations. It can be applied to texts, numbers, booleans, or datetime values.
In the following example, consider a corpus that contains natural scenery images. To retrieve all assets that are relevant to "sunset at beach" and tagged with the annotation "state": "California"
, send the following request (replace FIELD_NAME with state
and replace FIELD_VALUE with California
):
REST & CMD LINE
Before using any of the request data, make the following replacements:
- REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the
LOCATION_ID
such aseurope-west4-
. See more about regionalized endpoints. - PROJECT_NUMBER: Your Google Cloud project number.
- LOCATION_ID: The region where you are using
Vertex AI Vision. For example:
us-central1
,europe-west4
. See available regions. - INDEX_ENDPOINT_ID: The ID of your target index endpoint.
HTTP method and URL:
POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint
Request JSON body:
{ "text_query": "sunset at beach", "criteria": [ { "field": "state", "text_array": { "txt_values": [ "California" ] } } ] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "searchResultItems": [ { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1", "relevance": "0.99" }, { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2", "relevance": "0.98" } ] }
Search with input image
If you have an image of a sunset at a beach, you can also use it as a search query to retrieve similar images:
REST & CMD LINE
Before using any of the request data, make the following replacements:
- REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the
LOCATION_ID
such aseurope-west4-
. See more about regionalized endpoints. - PROJECT_NUMBER: Your Google Cloud project number.
- LOCATION_ID: The region where you are using
Vertex AI Vision. For example:
us-central1
,europe-west4
. See available regions. - INDEX_ENDPOINT_ID: The ID of your target index endpoint.
HTTP method and URL:
POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint
Request JSON body:
{ "image_query": { "input_image": "IMAGE_BYTES" }, "criteria": [ { "field": "FIELD_NAME", "text_array": { "txt_values": [ "FIELD_VALUE" ] } } ] }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "searchResultItems": [ { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1", "relevance": 0.99 }, { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2", "relevance": 0.98 } ] }
Search with existing image
You can also use an existing image asset in the corpus to retrieve similar images. The first search result will be the same image asset with a 1.0 relevance score:
REST & CMD LINE
Before using any of the request data, make the following replacements:
- REGIONALIZED_ENDPOINT: Endpoint might include a prefix matching the
LOCATION_ID
such aseurope-west4-
. See more about regionalized endpoints. - PROJECT_NUMBER: Your Google Cloud project number.
- LOCATION_ID: The region where you are using
Vertex AI Vision. For example:
us-central1
,europe-west4
. See available regions. - INDEX_ENDPOINT_ID: The ID of your target index endpoint.
- CORPUS_ID: The ID of your target corpus.
- ASSET_ID: The ID of your target asset.
HTTP method and URL:
POST https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint
Request JSON body:
{ "image_query": { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID" } }
To send your request, choose one of these options:
curl
Save the request body in a file named request.json
,
and execute the following command:
curl -X POST \
-H "Authorization: Bearer $(gcloud auth print-access-token)" \
-H "Content-Type: application/json; charset=utf-8" \
-d @request.json \
"https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint"
PowerShell
Save the request body in a file named request.json
,
and execute the following command:
$cred = gcloud auth print-access-token
$headers = @{ "Authorization" = "Bearer $cred" }
Invoke-WebRequest `
-Method POST `
-Headers $headers `
-ContentType: "application/json; charset=utf-8" `
-InFile request.json `
-Uri "https://warehouse-visionai.googleapis.com/v1/projects/PROJECT_NUMBER/locations/LOCATION_ID/indexEndpoints/INDEX_ENDPOINT_ID:searchIndexEndpoint" | Select-Object -Expand Content
You should receive a JSON response similar to the following:
{ "searchResultItems": [ { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_1", "relevance": 1.0 }, { "asset": "projects/PROJECT_NUMBER/locations/LOCATION_ID/corpora/CORPUS_ID/assets/ASSET_ID_2", "relevance": 0.99 } ] }