Page History

Current state:

ISSUE: #6383

PRs:

Keywords: delete

Released:

...

Process delete operation in system recovery

Unaffected

SegmentFilter

..SegmentFilter provide a method that can used to get the segment id which a PK possible existed in. Implement by segments statistics and bloomfilter.

DeltaLog

...

Inverted delta logs

...

Bitset

DeltaLog is the persistent file, recording the primary keys deleted and the delete timestamp. Each DeltaLog only belongs to a segment.

InvertedDeltaLog

InvertedDeltaLog provides a method that can be used to get deletion that meets the timestamp condition fastly...

Test Plan

Testcase1

Search a deleted entity, except not in the resultset

client.insert()
client.search()
client.delete()
client.search()

Rejected Alternatives[WIP]

1. Without any channel changes, the delete requests will be forward into the insert channels by Proxy
   1. Since the balance policy of QueryNode, some QueryNodes will only serve the sealed segments. They should discard all insert data from this channel. It cause a lot of performance waste.
2. Add a delta channel for all shards, and all the delete requests will be forward into this delta channel by Proxy
   1. DataNode should subscribe insert channel and delta channel, each time to consume data need to align timetick between these two channel, complicatedly
3. Add a delta channel for each shards, and the delete requests will be forward into this channel and the origin insert channel concurrently. DataNode subscribe the insert channel only, and the QueryNode(serving sealed segments) subscribe the delta channel
   1. It's difficult to sync the insert channel and delta channel

References