You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: deprecate nonpositive batch sizes before 5.0 - #62
Sizes below 1 have been accepted since batcher was introduced. Rejecting them immediately would break existing 4.x callers, including batcher([], 0) and sizes computed from a small input. This revision follows the maintainer's compatibility plan: warn in 4.x and reserve ValueError for 5.0.
batcher emits one DeprecationWarning for a size below 1 when iteration begins, before consuming the source. Its existing result is preserved: one batch containing all items, or no batches for an empty source.
Without an interval, abatcher emits the same warning and preserves existing behavior, including the assertion for size 0 and the single final batch for a negative size.
With an interval, abatcher accepts None, 0 and negative sizes as no size limit, without warning. Interval deadlines and cancellation behavior remain covered by the existing tests, now also exercised with 0 and -1.
Warnings point to the caller. Callers who turn deprecations into errors fail before any source items are consumed or scheduled. Positive sizes retain lazy, bounded consumption and issue no warning.
Both docstrings and the changelog explain the staged deprecation. This PR does not implement the future 5.0 exception.
Validation on Python 3.13.5, using the existing development environment over SSH on Linux:
Full tests and doctests: 405 passed, 100% statement and branch coverage.
Batching module: 51 passed, covering empty/nonempty sources, warning timing/category/location, warnings promoted to errors, positive sizes, and interval-only behavior and deadlines.
Ruff lint and format, Mypy, basedpyright, Pyrefly and Codespell passed.
Sphinx HTML build with warnings treated as errors passed.
Other interpreter versions are left to CI. This revision was prepared and validated with Codex.
Thank you for the pull request. The tests that check nothing is consumed are a nice touch.
The catch is that sizes below 1 have been accepted since batcher was added in 3.3.0, and they're easy to reach by accident: batcher(rows, len(rows) // workers) gets a size of 0 whenever there are fewer rows than workers, and batcher([], 0) returns [] on 4.1.4. With this change those jobs start raising ValueError, and since many people install python-utils without an upper bound, they would get that from a minor release. abatcher(source, batch_size=0, interval=5) is a similar case: it batches by interval only today, and the interval already keeps the buffer bounded.
I wouldn't oppose rejecting these sizes, but I'd rather do it in two steps: a DeprecationWarning for sizes below 1 in 4.x that keeps the current result, and the ValueError in 5.0. For abatcher with an interval, a size below 1 could simply keep meaning "no size limit". Would you be up for changing the PR that way?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sizes below 1 have been accepted since
batcherwas introduced. Rejecting them immediately would break existing 4.x callers, includingbatcher([], 0)and sizes computed from a small input. This revision follows the maintainer's compatibility plan: warn in 4.x and reserveValueErrorfor 5.0.batcheremits oneDeprecationWarningfor a size below 1 when iteration begins, before consuming the source. Its existing result is preserved: one batch containing all items, or no batches for an empty source.abatcheremits the same warning and preserves existing behavior, including the assertion for size 0 and the single final batch for a negative size.abatcheracceptsNone, 0 and negative sizes as no size limit, without warning. Interval deadlines and cancellation behavior remain covered by the existing tests, now also exercised with 0 and -1.Both docstrings and the changelog explain the staged deprecation. This PR does not implement the future 5.0 exception.
Validation on Python 3.13.5, using the existing development environment over SSH on Linux:
Other interpreter versions are left to CI. This revision was prepared and validated with Codex.