Repository navigation
Use older valid carrier copies during recovery and re-import packs after a prune - #24
Merged
Merged
Conversation
…ter a prune - Carrier history reads stopped at the first corrupt or missing copy of an object, even when an older carrier commit still held a valid copy, so recovery failed with a content-id mismatch. recovery_history and restore_historical_object now try each copy newest to oldest within the existing commit budget and use the first whose content hash matches. Bytes read from rejected copies count toward the byte budget. - After a user deleted a remote-tracking ref and ran gc --prune, every later fetch of that ref failed, because the import trusted the recorded pack list and skipped generations it believed were already imported. When the incremental connectivity check fails, the helper now re-imports every pack of the authenticated chain and checks the full graph. The pin is validated first and written only after that check passes, so it never moves backwards.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes C4 and C7 from the implementation review. Both were reproduced on
mainbefore the fix.C4: carrier history stopped at the first corrupt copy
Reproduced. A storage-level edit overwrote chunk
00000000of the HEAD manifest and of its pack in a new carrier commit, while older commits still held valid copies.recovery_historyfailed with "historical carrier object content id mismatch".restore_historical_objecthad the same flaw, because its first candidate is the corrupt HEAD copy.Fix.
copy_commitslists the commits that wrote each object (git rev-list <newest> --max-count=N -- <object dir>) inside the existing commit window.read_copy_atreads one copy and reports whether it validates: present, well formed, the right size, and a matching content hash.restore_historical_objecttry copies newest to oldest and use the first that validates.Test:
carrier_history_reads_skip_corrupt_copies_for_an_older_valid_one.C7: fetch failed for good after a ref delete and prune
Reproduced through the helper. Push
mainandfeature, fetch,update-ref -d refs/remotes/private/feature,gc --prune=now, then fetch. The second fetch failed with "fetched ref refs/heads/feature does not resolve to a complete Git object graph", and kept failing on every retry.Root cause.
import_packs_lockedskipped every generation up toimported_generationand trusted the recorded pack list, so the incremental connectivity check could never pass again.Fix. The import loop moves into
import_chain_packs. When the incremental check fails, the helper clears the imported set, re-imports every pack of the authenticated chain, and checks the full graph with an empty frontier.validate_pinned_headstill runs first, and state is written only after a check succeeds, so the pin never moves backwards.Tests.
native_git_fetch_reimports_packs_after_local_objects_are_pruned.missing_verified_frontier_tip_fails_closedasserted the old permanent failure. It becomesmissing_verified_frontier_tip_forces_full_reverification, which asserts that the fetch succeeds and the bogus frontier is replaced by the real tip.docs/spec.md§9 and the recovery discovery paragraph state both new rules.Verification
cargo fmt --check,cargo clippy --all-targets -- -D warnings,cargo test --all-targets(151 passed, with currentmainmerged in).Trade-offs
read_copy_attreats any non-io failure while listing chunks as an invalid copy. The worst case is a report of "unavailable" instead of a hard error. Unverified bytes are never accepted.restore_historical_objectstill has no byte budget, as before. It holds at most one 32 MiB chunk in memory at a time.