Issue 4804: Fix txn compaction recovery leaving entry logs unreclaimed#4803
Open
void-ptr974 wants to merge 1 commit into
Open
Issue 4804: Fix txn compaction recovery leaving entry logs unreclaimed#4803void-ptr974 wants to merge 1 commit into
void-ptr974 wants to merge 1 commit into
Conversation
68bf7a2 to
91c4a33
Compare
91c4a33 to
d3dd87d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4804
Problem
DefaultEntryLogger.incompleteCompactionLogs()rebuilds txn compaction logs from*.compactedfiles after a restart. Entry log file names are hex-based, but this recovery path parsed the destination log id from the hex file name and then rebuilt the final.logfile name from the numeric id directly.For example,
1f4.log.78.compactedhas destination log id1f4, which is decimal500. Recovery should make1f4.logavailable, but the old path targeted500.log.Impact
For affected log ids,
makeAvailable()creates the recovered log under the decimal file name. Later scan/recovery resolves log id500through the normal hex file name and looks for1f4.log.That mismatch prevents txn compaction recovery from scanning and finalizing the recovered log. The
.compactedmarker and old source entry log remain on disk, so disk space for that compaction is not reclaimed. Ledger entry contents are not changed.Fix
Build the recovered final
.logfile name withlogId2HexString(compactionLogId).Coverage
1f4.logfor log id500and does not create500.log.500.logwhile the.compactedmarker remains, fixed recovery still creates and scans1f4.logand leaves the unrelated stale file untouched.Testing
mvn -pl bookkeeper-server -Dtest=DefaultEntryLogTest#testRecoverCompactedLogUsesHexFinalLogFileName+testRecoverCompactedLogWithStaleDecimalLogFileFromOldRecovery+testNewCompactionLogKeepsHexFinalLogFileName testmvn -pl bookkeeper-server -Dtest=DefaultEntryLogTest test