Skip to content

Use older valid carrier copies during recovery and re-import packs after a prune - #24

Merged
hmirin merged 3 commits into
mainfrom
fix/history-robustness
Oct 29, 2026
Merged

hmirin merged 3 commits into
mainfrom
fix/history-robustness

Conversation

@hmirin

@hmirin hmirin commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Fixes C4 and C7 from the implementation review. Both were reproduced on main before the fix.

C4: carrier history stopped at the first corrupt copy

Reproduced. A storage-level edit overwrote chunk 00000000 of the HEAD manifest and of its pack in a new carrier commit, while older commits still held valid copies. recovery_history failed with "historical carrier object content id mismatch". restore_historical_object had the same flaw, because its first candidate is the corrupt HEAD copy.

Fix.

  • copy_commits lists the commits that wrote each object (git rev-list <newest> --max-count=N -- <object dir>) inside the existing commit window.
  • read_copy_at reads one copy and reports whether it validates: present, well formed, the right size, and a matching content hash.
  • Recovery reads and restore_historical_object try copies newest to oldest and use the first that validates.
  • Commit budgets are unchanged. Bytes read from rejected copies count toward the byte budget.
  • Accepted copies are still checked by content id here and by signatures downstream.

Test: carrier_history_reads_skip_corrupt_copies_for_an_older_valid_one.

C7: fetch failed for good after a ref delete and prune

Reproduced through the helper. Push main and feature, fetch, update-ref -d refs/remotes/private/feature, gc --prune=now, then fetch. The second fetch failed with "fetched ref refs/heads/feature does not resolve to a complete Git object graph", and kept failing on every retry.

Root cause. import_packs_locked skipped every generation up to imported_generation and trusted the recorded pack list, so the incremental connectivity check could never pass again.

Fix. The import loop moves into import_chain_packs. When the incremental check fails, the helper clears the imported set, re-imports every pack of the authenticated chain, and checks the full graph with an empty frontier. validate_pinned_head still runs first, and state is written only after a check succeeds, so the pin never moves backwards.

Tests.

  • New: native_git_fetch_reimports_packs_after_local_objects_are_pruned.
  • missing_verified_frontier_tip_fails_closed asserted the old permanent failure. It becomes missing_verified_frontier_tip_forces_full_reverification, which asserts that the fetch succeeds and the bogus frontier is replaced by the real tip.
  • docs/spec.md §9 and the recovery discovery paragraph state both new rules.

Verification

cargo fmt --check, cargo clippy --all-targets -- -D warnings, cargo test --all-targets (151 passed, with current main merged in).

Trade-offs

  • Storage that publishes a signed ref advance without its pack now costs a full chain re-import on each fetch attempt before failing. It still fails closed, and storage could already deny availability.
  • read_copy_at treats any non-io failure while listing chunks as an invalid copy. The worst case is a report of "unavailable" instead of a hard error. Unverified bytes are never accepted.
  • restore_historical_object still has no byte budget, as before. It holds at most one 32 MiB chunk in memory at a time.

…ter a prune

- Carrier history reads stopped at the first corrupt or missing copy of an
  object, even when an older carrier commit still held a valid copy, so
  recovery failed with a content-id mismatch. recovery_history and
  restore_historical_object now try each copy newest to oldest within the
  existing commit budget and use the first whose content hash matches. Bytes
  read from rejected copies count toward the byte budget.
- After a user deleted a remote-tracking ref and ran gc --prune, every later
  fetch of that ref failed, because the import trusted the recorded pack list
  and skipped generations it believed were already imported. When the
  incremental connectivity check fails, the helper now re-imports every pack
  of the authenticated chain and checks the full graph. The pin is validated
  first and written only after that check passes, so it never moves backwards.
@hmirin
hmirin merged commit 691ade2 into main Oct 29, 2026
1 check passed
@hmirin
hmirin deleted the fix/history-robustness branch October 29, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant