Skip to content

Respect path boundaries in reference filesystem find - #2252

Open
fhgffy wants to merge 1 commit into
fsspec:masterfrom
fhgffy:fix/reference-find-path-boundaries
Open

fhgffy wants to merge 1 commit into
fsspec:masterfrom
fhgffy:fix/reference-find-path-boundaries

Conversation

@fhgffy

@fhgffy fhgffy commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

ReferenceFileSystem.find() matches raw string prefixes, so find("dataset") includes dataset-other/file, and find("dataset/file") includes dataset/file-extra. A missing path such as data can also return entries from dataset. Both the name list and the detailed result are affected.

This also gives FSMap phantom keys: a mapper rooted at dataset exposes -other/file from its sibling, and its length is inflated. The directory-aware find(..., withdirs=True) path already keeps the correct boundary.

Match either the exact path or a descendant beginning with path + "/". Normalize the input before testing for the root so reference:// continues to list all references. The optimized reference scan remains in place.

Regression tests cover directories, trailing slashes, protocol-qualified paths, exact filenames, missing prefixes, root inputs, both detail modes, and mapper keys/length/values.

Validation on Windows / Python 3.14.3:

  • Before the fix: 13 failed / 4 passed in the new regression selection.
  • After the fix: all 17 new cases passed.
  • Reference, mapping and base specification modules: 359 passed / 121 skipped / 1 xfailed.
  • Rebased onto current master 26ebc63: all 17 new cases passed again; neither changed file differs between the original and current bases.
  • Ruff 0.14.3 check and format check, and git diff --check, passed.

The complete suite was not run. The related selection includes local FTP fixtures and an expected-failure external FTP test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant