Page History

...

1. Insert channel, handle following msg types:
   1. DDL msg (CreateCollection/DropCollection/CreatePartition/DropPartition)
   2. InsertMsg
   3. DeleteMsg
2. Search Query channel, handle following msg types:
   1. SearchMsg
   2. RetrieveMsg

...

QueryNode consumes messages from both Insert channel and Search Query channel.

To support delete, we will send DeleteMsg into Insert channel also.

...

Since Milvus's storage is an append-only, `delete` function is implemented using soft delete, setting a flag on entity to indicate this entity has been deleted.

...

1. Proxy receives a delete request,
   1. parse `expr` to get primary keys, then split DeleteMsg into insert channels by primary keys (done)
2. DataNode receives a delete request from the insert channel, save it in buffer, and write it into the delta channelwatches insert channel
   1. receive DeleteMsg and persist delete data
      1. when DataNode start up
      2. load , load all segments info from Minio into memory (done)load all segments' deleted ids from Minio into in-memory structure delBuf
      3. update datanode flowgraph
         1. when DDNode receive DeleteMsg, save it into FlowGraphMsg structure, and send FlowGraphMsg to next node InsertBufferNode (done)
         2. when InsertBufferNode receive FlowGraphMsg, process InsertMsg in it, then wrapper DeleteMsg into another FlowGraphMsg and sent it to next node DeleteNode (done)
      add
      1. 1. when DeleteNode
      , when
      1. 1. receive FlowGraphMsg, process DeleteMsg in it, save deleted ids and timestamps into
      sync.Map
      1. 1. delBuf (done)
      2. update DeleteNode to handle flushMsg
         1. add another flushChan for DeleteNode (
      doing
      1. 1. done)
         2. when DeleteNode
      receive
      1. 1. receives flushMsg, save all data in delBuf into
      Minio
      1. 1. MinIO 
            1. deleted ids and timestamps are saved into two separated Minio file with name "/by-dev/deltalog/collectionID/partitionID/segmentID/xxx"
3. DataNode notifies IndexNode to building indexes
4. 1. 1. 1. 1. (done)
   2.  DataNode forwards ttMsg and DeleteMsg to delta channel (doing)
   finish

Proposal delete operation serving search(sealed+growing)

...