Skip to content

Avoid recursion when listing deeply nested archives - #2255

Open
RRiiiccckkk wants to merge 1 commit into
fsspec:masterfrom
RRiiiccckkk:fix/archive-deep-directory-listing
Open

RRiiiccckkk wants to merge 1 commit into
fsspec:masterfrom
RRiiiccckkk:fix/archive-deep-directory-listing

Conversation

@RRiiiccckkk

Copy link
Copy Markdown

Issue for this PR

Closes #2254

Type of change

  • Bug fix

What does this PR do?

While checking archive listings, I found that a valid ZIP with 1,010 directory levels could be read by zipfile, but even listing its root through fsspec raised RecursionError. Collect parent directories iteratively, stopping at parents already visited. The regression test lists and reads real ZIP, TAR, compressed TAR and 7-Zip archives.

How did you verify your code works?

On macOS arm64 with Python 3.12.9:

python -m pytest fsspec/implementations/tests/test_archive.py -k deep_directory_listing --tb=short -q

Before the fix: all six cases failed with RecursionError.

python -m pytest fsspec/implementations/tests/test_archive.py fsspec/implementations/tests/test_zip.py fsspec/implementations/tests/test_tar.py fsspec/implementations/tests/test_libarchive.py -q

After the fix: 260 passed in 8.10s, including the FTP/libarchive test. Python printed a resource_tracker warning about six semaphore objects at shutdown after the FTP test; pytest exited successfully.

pre-commit run --files fsspec/archive.py fsspec/implementations/tests/test_archive.py docs/source/changelog.rst
git diff --check

Both passed. The reproduction now returns ['d'] and reads the original member bytes. Temporary files were placed in the ignored .cache/contribution directory using TMPDIR.

Screenshots / recordings

Not applicable; this changes filesystem operations with no UI.

Checklist

  • Ran the relevant tests locally.
  • Added regression coverage that fails before the fix.
  • Updated the changelog.
  • Included no unrelated changes.

Copilot AI balanced review requested due to automatic review settings October 9, 2026 19:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Listing a deeply nested archive raises RecursionError

2 participants