Page History

...

The advantage of this solution is obviousresource saving, except query node, no other nodes will be involved in, it can save a lot of resources.The ; while the shortcoming is code duplication. See following "Search Flow" chart, `FloatSearchBruteForce` and `BinarySearchBruteForce` are copied from knowhere::IDMAP and knowhere::BinaryIDMAP. More code duplicated, more effort on code maintenance. And when realize new feature on IDMAP/BinaryIDMAP in Knowhere, such as range search, same work need be re-done on Milvus segcore.

This is why enhanced IDMAP/BinaryIDMAP is proposed. For enhanced IDMAP/BinaryIDMAP, vector data is not really added in, but only set an external vector data pointer. User should guarantee that the memory is contiguous and safe.

In this way:

• no CPU time, memory and disk consumption when creating index
• resource saving, only query node is involved in
• no code duplication for growing segment search
• unified search result for sealed segment and growing segment

Public Interfaces(optional)

Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed.

Knowhere need add a new interface `AddExWithoutIds()` for IDMAP and BinaryIDMAP

    // set external data pointer instead really add data
    void
    AddExWithoutIds(const DatasetPtr&, const Config&);

Design Details(required)

Describe the new thing you want to do in appropriate detail. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences. Use judgment based on the scope of the change.

Compatibility, Deprecation, and Migration Plan(optional)

• What impact (if any) will there be on existing users?
• If we are changing behaviors how will we phase out the older behavior?
• If we need special migration tools, describe them here.
• When will we remove the existing behavior?This MEP will be transparent for users, and will not introduce any compatibility issue.

Test Plan(required)

Describe in a few sentences how the MEP will be tested. We are mostly interested in system tests (since unit tests are specific to implementation details). How will we know that the implementation works as expected? How will we know nothing broke?

Rejected Alternatives(optional)

Knowhere need add some unittests to test new interface `AddExWithoutIds()`.

No extra testcases need be added in Milvus because current growing segment search testcases can cover this change.

Search result and performance will be identical with beforeIf there are alternative ways of accomplishing the same thing, what were they? The purpose of this section is to motivate why the design is the way it is and not some other ways.

References(optional)

Briefly list all references