...
ISSUE: #17754
PRs:
Keywords: IDMAP, knowherebrute force search, chunk
Released: Milvus-v2.2.0
Summary(required)
In this MEP, we put forward an IDMAP enhancement Enhancement proposal that let knowhere index type IDMAP accept external vector data instead of adding real vector data in.
This Enhanced IDMAP can be used for growing segment searching to improve code reuse and reduce code maintenance effort.
Motivation(required)
Generally no one will create IDMAP index type for sealed segment, because it does not bring any search performance improvement but consumes identical size of memory and disk.
The only reasonable use scenario for IDMAP is for growing segment. But creating Creating vector index is a resource consuming operation, because it involves all Milvus nodes in -- an index file is created by index node, saved by data node and loaded by query node, meanwhile proxy / rootcoord / indexcoord / datacoord / querycoord are also involved to coordinate all these operations.
So currently in Milvus, it uses chunk index (this functionality is disabled for some particular reason) and brute force search (most part of the API realization is copied from knowhere IDMAP) for growing segment searching.
- Advantages
- Cons
Public Interfaces(optional)
...