Current state: Under Discussion

ISSUE: #2771


Keywords: Vector Index

Released: TBD


We got requirement for supporting ScaNN index in Milvus. This proposal is about how to integrate a new index into knowhere and expose it to client. 

Document about how knowhere works in milvus v1.x: m1_chain.pdf

Docement about how knowhere works in milvus v2.0: TODO


Google recently released a new algorithm to find Approximate Nearest Neighbors called ScaNNThey show through benchmarks that it can perform significantly better than the existing solutions like Annoy, FAISS, hnsw. The goal is to add support for ScaNN algorithm to the list of Milvus supported indexes.

Public Interfaces

Follwing public interfaces would be updated in knowhere =>

namespace IndexEnum {
	extern const char* INDEX_SCANN;

namespace IndexEnum {
	const char* INDEX_SCANN = "SCANN";
namespace IndexParams {
	// SCANN params
	constexpr const char* num_leaves = "num_leaves";
	constexpr const char* num_leaves_to_search = "num_leaves_to_search";
	constexpr const char* dimensions_per_block = "dimensions_per_block";
	constexpr const char* scoring = "scoring";
	constexpr const char* quantize = "quantize";
	constexpr const char* reordering_num_neighbors = "reordering_num_neighbors";

Design Details

As part of third party library, we will have to add a new library for ScaNN. This directory here: contains APIs for build/search etc. Also, ScaNN uses Bazel as it's dependency management while we would need to use CMake.

We will create new wrapper classes called IndexScann.h and IndexScann.cpp in knowhere. 

class IndexScann : public VecIndex { 

    IndexScann() {
        index_type_ = IndexEnum::INDEX_SCANN;

    Serialize(const Config& config) override;

    Load(const BinarySet& index_binary) override;

    Train(const DatasetPtr& dataset_ptr, const Config& config) override;

    AddWithoutIds(const DatasetPtr&, const Config&) override;

    Query(const DatasetPtr& dataset_ptr, const Config& config, const faiss::BitsetView bitset) override;

    Count() override;

    Dim() override;

    UpdateIndexSize() override;


IndexScann.cpp would be the implementation of IndexScann.h.


if (type == IndexEnum::INDEX_SCANN) {
    return std::make_shared<knowhere::IndexSCANN>();

ConfAdapter.cpp/ConfAdapter.go: Add conf adapters for ScaNN params(described above)

Compatibility, Deprecation, and Migration Plan

Test Plan

Rejected Alternatives