Repository navigation
Reject readonly buffers before consuming buffered file data - #2260
Conversation
martindurant
left a comment
There was a problem hiding this comment.
Is .readonly guaranteed to exist?
| """ | ||
| out = memoryview(b).cast("B") | ||
| if out.readonly: | ||
| self.read(0) |
There was a problem hiding this comment.
| self.read(0) |
|
Peer check on I independently reran the 50 added regressions on Windows/CPython 3.14: the base plus identical tests gives 36 failures/14 passes; this head gives 50 passes. Removing only That line preserves fsspec compatibility, rather than enforcing a universal stdlib rule: closed |
AbstractBufferedFile.readinto()andreadinto1()currently advance the file position before raisingTypeErrorfor a read-only target buffer. Retrying with a writable buffer then skips bytes. This affectsbytes, read-only memoryviews, and read-only NumPy arrays; standard file objects reject them without moving the position.Reject read-only buffers before consuming file data, retaining the existing closed-file error through a zero-length read on that error path. Writable-buffer behavior is unchanged. Add regressions for both methods, three cache types, initial/nonzero positions, read-only NumPy arrays, empty buffers, and closed files, plus a changelog entry.
Validation on Windows with Python 3.12.14, NumPy 2.5.3 and aiohttp 3.14.4:
pytest fsspec/tests/test_spec.py fsspec/implementations/tests/test_http.py -q: 378 passed, 124 skipped, 1 xfailed. The unchanged base gives 328 passed with the same skips and xfail..npycheckpoint probe changes from 36 failures / 51 normal controls to 87 passes, including retry bytes, writable typed buffers, andnp.loadcontrols. This overlaps formal coverage and is not an additional suite count.The Windows test run emits an existing event-loop deprecation and an HTTP-fixture thread teardown warning; the latter was independently reproduced in the unchanged baseline fixture without calling
readinto. Docker-dependent backends, downstream projects, other platforms and performance were not tested locally.