Page History

...

Proposal delete operation persistent

1. DataNode subscribe the insert channel
2. Proxy receives a delete request, parse `expr` to get primary keys, then split into insert channels by primary keys (done)
3. DataNode receives a delete request from the insert channel, save it in buffer, and write it into the delta channel
   1. when DataNode start up
      1. load all segments info from Minio into memory (done)
      2. load all segments' deleted ids from Minio into in-memory structure delBuf
   2. save deleted id into structure map<segID, list(id, ts)>
   DataNode receives a flush request, write out the deletions saved above
   1. when DDNode receive DeleteMsg, save it into FlowGraphMsg structure, and send FlowGraphMsg to next node (done)
   2. when InsertBufferNode receive FlowGraphMsg, process InsertMsg in it, then wrapper DeleteMsg into another FlowGraphMsg and sent it to next node (done)
   3. add DeleteNode, when receive FlowGraphMsg, process DeleteMsg in it, save deleted ids and timestamps into sync.Map delBuf (done)
   4. add another flushChan for DeleteNode (doing)
   5. when DeleteNode receive flushMsg, save all data in delBuf into Minio
      
      1. deleted ids and timestamps are saved into two
   6. deleted ids are saved into a
      1. separated Minio file with name "/by-dev/deltalog/collectionID/partitionID/segmentID/xxx"
4. DataNode notifies IndexNode to building indexes
5. finish

...