...
Result segments availability
By definition, the result segments shall be available altogether. Which means there shall be no intermediate state for loading.
To achieve this property, the segments shall be marked as "Loading" state and be invisible before the whole loading procedure completes.
Bulk Load with Delete
Constraint: The segments generated by Bulk Load shall not be affected by delete operations before the whole procedure is finished.
...
- Allow adding segments without removing one
- Bring target segments online atomically.
Bulk Load and Index Building
The current behavior of query cluster is that if there is an index built for the collection, the segments will not be loaded(as sealed segment) before the index is built.
This constraint shall remain in first implementation of Bulk Load:
Constraint: The bulk load procedure shall include the period of index building of the result segments
Bulk Load as a tool
The bulk load logic can be extracted into a tool to run outside of Milvus process. It shall be implemented in the next release.