Stay organized with collections
Save and categorize content based on your preferences.
This page gives a conceptual overview of improving vector query performance with the ScaNN Index.
ScaNN index uses tree-quantization based indexing. In Tree-quantization
techniques, indexes learn a search tree together with a quantization (or
hashing) function. When you run a query, the search tree is used to prune the
search space while quantization is used to compress the index size. This pruning
speeds up the scoring of the similarity (i.e., distance) between the query vector
and the database vectors.
To achieve both a high query-per-second rate (QPS)
and a high recall with your nearest-neighbor queries, you must partition
the tree of your ScaNN index in a way that is most appropriate to your data
and your queries.
A common characteristic of the current generation of high-dimensional embedding models is they can still retain much of the information at much lower dimensionality, for example 90% of the information can be retained with only 20% of the embedding's dimensions. To help speed up such datasets, AlloyDB ScaNN will automatically perform dimension reduction using Principal Component Analysis (see scann.enable_pca below) on the indexed vectors which further reduces cpu & memory usage for the vector search. Because dimension reduction still causes minor recall loss in the index, AlloyDB ScaNN compensates through an initial ranking step with a larger number of PCA'ed vector candidates from the index then re-ranking them by the original vectors (see scann.pre_reordering_num_neighbors).
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Hard to understand","hardToUnderstand","thumb-down"],["Incorrect information or sample code","incorrectInformationOrSampleCode","thumb-down"],["Missing the information/samples I need","missingTheInformationSamplesINeed","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2025-02-26 UTC."],[[["ScaNN index employs tree-quantization to accelerate vector similarity scoring by pruning the search space and compressing index size."],["Optimal tree partitioning in ScaNN is crucial for achieving high query-per-second rates and recall in nearest-neighbor queries."],["AlloyDB ScaNN automatically reduces dimensionality using Principal Component Analysis (PCA) to enhance speed and minimize resource consumption for high-dimensional embedding datasets."],["AlloyDB ScaNN compensates for recall loss from PCA by initially ranking a larger pool of PCA'ed vector candidates and subsequently re-ranking them using the original vectors."]]],[]]
