Skip to content

Close owned synchronous HTTP download files - #2259

Merged
martindurant merged 4 commits into
fsspec:masterfrom
adenzhou1350:codex/fix-sync-http-download-close
Oct 20, 2026
Merged

martindurant merged 4 commits into
fsspec:masterfrom
adenzhou1350:codex/fix-sync-http-download-close

Conversation

@adenzhou1350

@adenzhou1350 adenzhou1350 commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

The experimental synchronous HTTP get_file() opens local path destinations without closing them explicitly. In particular, an exception from a progress callback retains the open file in its traceback until the exception is discarded. Successful cleanup should not depend on garbage collection either.

Use a context manager for owned files and nullcontext for caller-provided streams, matching the ownership convention already used by put_file() and the async HTTP implementation. Preserve the original callback exception and leave caller-owned streams open.

Four parametrized regressions use the real Requests session and repository loopback HTTP server: owned path / caller stream, each with successful or failing progress. They check closure, exact downloaded bytes and exception propagation while retaining the traceback. On unchanged master, both owned-file cases fail and both caller-owned controls pass. After the fix all four pass.

Validation on Linux Python 3.12 against master 94afb3f6: 686 passed, 16 optional dependency skips, 1 existing xfail across synchronous/async HTTP, spec, utils and core tests. Two warnings concern an asyncio event-loop deprecation and a threaded-process fork. Repository-pinned Ruff 0.14.3 check/format and diff check pass.

This is scoped Requests/loopback evidence, not Pyodide/XHR or full CI qualification. AI-assisted implementation and tests, independently reproduced before and after. Merged #2066 and #1191 address generic/async ownership, not this synchronous implementation; open #2132 does not touch this file.

Conflict refresh (2026-10-20)

Merged current master 941e6f9 and resolved the changelog conflict, retaining both the upstream entries and this PR entry. The introduced Python source/test patch is unchanged.

On the merged source, native Windows / Python 3.14 validation of the complete synchronous HTTP test module: 60 passed. Repository-pinned Ruff 0.14.3 lint and formatting checks, plus git diff --check, pass.

This is scoped current-source local validation; older-head CI/results remain historical. Hosted CI for the new head still needs to complete.

Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
@adenzhou1350

Copy link
Copy Markdown
Contributor Author

The latest Windows job failed only six GitHub-backend tests, each reporting 403 Client Error: rate limit exceeded from the GitHub API. The job reports 2499 passed, 315 skipped and 10 xfailed; the other ten check runs, including lint, passed. Could a maintainer retry this failed job once the API limit resets? Contributor permissions do not allow me to rerun upstream workflows. Log: https://github.com/fsspec/filesystem_spec/actions/runs/39013529844/job/117340840128

Signed-off-by: Xucheng Zhou <aden1350@outlook.com>
@adenzhou1350

Copy link
Copy Markdown
Contributor Author

Merged current master 941e6f9 and resolved the changelog conflict, retaining both the upstream entries and this PR entry. The introduced Python source/test patch is unchanged.

On the merged source, native Windows / Python 3.14 validation of the complete synchronous HTTP test module: 60 passed. Repository-pinned Ruff 0.14.3 lint and formatting checks, plus git diff --check, pass.

This is scoped current-source local validation; older-head CI/results remain historical. Hosted CI for the new head still needs to complete.

@martindurant
martindurant merged commit f7b0302 into fsspec:master Oct 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants