Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Docement about how knowhere works in milvus v2.0: TODO 

https://github.com/milvus-io/milvus/blob/master/docs/design_docs/index_design.md

https://github.com/milvus-io/milvus/blob/master/docs/design_docs/milvus_create_index_en.md

Motivation

Google recently released a new algorithm to find Approximate Nearest Neighbors called ScaNNThey show through benchmarks that it can perform significantly better than the existing solutions like Annoy, FAISS, hnsw. The goal is to add support for ScaNN algorithm to the list of Milvus supported indexes.


Public Interfaces





Design Details

Follwing public As part of third party library, we will have to add a new library for ScaNN. This directory here: https://github.com/google-research/google-research/tree/master/scann/scann/scann_ops contains APIs for build/search etc. Also, ScaNN uses Bazel as it's dependency management while we would need to use CMake.


Follwing interfaces would be updated in knowhere =>

...

Code Block
languagecpp
titleIndexType.cpp
namespace IndexParams {
	// SCANN params
	constexpr const char* num_leaves = "num_leaves";
	constexpr const char* num_leaves_to_search = "num_leaves_to_search";
	constexpr const char* dimensions_per_block = "dimensions_per_block";
	constexpr const char* scoring = "scoring";
	constexpr const char* quantize = "quantize";
	constexpr const char* reordering_num_neighbors = "reordering_num_neighbors";
}

...

As part of third party library, we will have to add a new library for ScaNN. This directory here: https://github.com/google-research/google-research/tree/master/scann/scann/scann_ops contains APIs for build/search etc. Also, ScaNN uses Bazel as it's dependency management while we would need to use CMake.


We will create new wrapper classes called IndexScann.h and IndexScann.cpp in knowhere. 

...


Compatibility, Deprecation, and Migration Plan

N/A as it's a net new index type to be released in newer versions.


Test Plan


Rejected Alternatives


References

scann