Skip to content

Refetch mmap blocks when recreating a missing backing file - #2257

Merged
martindurant merged 2 commits into
fsspec:masterfrom
adenzhou1350:codex/fix-mmap-missing-backing-file
Oct 14, 2026
Merged

martindurant merged 2 commits into
fsspec:masterfrom
adenzhou1350:codex/fix-mmap-missing-backing-file

Conversation

@adenzhou1350

Copy link
Copy Markdown
Contributor

A named MMapCache recreates its backing file when it no longer exists, but previously retained the old set of cached blocks. Restoring a pickled cache after deleting that file therefore returned zero-filled bytes instead of fetching the original data. The same condition is reachable when reopening an HTTP reader with the saved location/blocks cache options.

Clear the block set in place whenever a new backing file is created. Existing files keep their cached blocks; keeping the same set object preserves shared bookkeeping. Read bounds and prefetch behavior are unchanged.

Validation on current master (94afb3f6):

  • Four new regression/control cases: 3 fail and 1 passes before, all pass after; includes a real loopback HTTP reader.
  • Seven core/local/memory/cache test files, Linux Python 3.12: 637 passed, 39 skipped (optional providers and Windows-specific cases).
  • Three additional real Windows Python 3.12 mmap/pickle controls: 1/3 passes before, 3/3 after.
  • Repository-pinned Ruff 0.14.3 check/format and diff check pass.

This is not full CI or cloud-provider qualification. AI-assisted implementation and tests; results were checked against the unmodified source. It does not address serialization of an HTTP-bound fetcher itself.

Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
@martindurant
martindurant merged commit dae2ac9 into fsspec:master Oct 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants