...
Released:
Summary
This article focuses on mainly introduces the importance of handoff and load balance segment features, as well as proposing a scenario for functions, and proposes a solution to handoff and load balance segments segment without interrupting queriesthe query.
Motivation
The query node of milvus currently has the following issues:
- In the load collection/partition process, query nodes not only load the sealed segments, but also watch dmChannels so that they can receive vectors inserted in real time, and if new vectors are continuously inserted after the load collection operation, the query nodes will create a large number of growing segments, which can only service query requests in a near-brutal manner. At the same time, data node may have flushed these vectors into persistent storage and created the corresponding vector index.
- Query nodes may experience unbalanced sealed segments during dynamic expansionIn the process of dynamic expansion and contraction of query nodes, there may be an imbalance in the number of sealed segments on different nodes. This results in a relatively slow query for high-load query node, which will becomes the final bottleneck for queries.
...