Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The RocksDB checkpoint used to feed replicas during full synchronization was
purged with two hard-coded thresholds in
Storage::TryPurgeCheckpoint:more than 30 seconds;
are still fetching its files.
Both values were fixed constants, which cannot be tuned per deployment: a slow or
far-away replica may legitimately need to fetch checkpoint files for longer than
the idle window, while an operator on a disk-constrained node may want to reclaim
the checkpoint (and the SST files it keeps alive) much earlier than 24 hours.
This change turns those thresholds into two runtime-configurable options while
keeping the current values as the defaults, so the existing behavior is unchanged
for everyone who does not set them:
checkpoint-alive-time-secscheckpoint-idle-time-secsThis removes the long-standing
TODO(shooterit): support to config the alive time of checkpoint, originally introduced when the checkpoint-based full replicationwas added (#205).
Changes
checkpoint_alive_time_secsandcheckpoint_idle_time_secstoConfigand register them as
IntFields (range[1, INT_MAX]) so they can be set viakvrocks.conf,CONFIG SET/GETand surviveCONFIG REWRITE.Storage::TryPurgeCheckpointinstead of the hard-coded30and24 * 60 * 60, and drop the TODO comment.SetCheckpointCreateTimeSecsaccessor (symmetric with the existing accesstime setter) to make the purge policy directly testable.
kvrocks.confnext topurge-backup-on-fullsync.Note: the shared-checkpoint reuse window in
GetFullReplDataInfo(derived fromrocksdb.wal-ttl-seconds) is a separate concern about whether a replica may reusean existing checkpoint, not about when the checkpoint is purged; it is left
untouched here.
Side effects / Compatibility
No storage-format, replication-protocol or column-family changes. The new options
are backward compatible: when unset, the behavior is byte-for-byte identical to the
previous hard-coded 30s / 24h policy.
AI Usage Disclosure