gh-149557: Make itertools.pairwise safe under re-entry and concurrency #150589
Open
eendebakpt wants to merge 1 commit into
Open
gh-149557: Make itertools.pairwise safe under re-entry and concurrency #150589eendebakpt wants to merge 1 commit into
eendebakpt wants to merge 1 commit into
Conversation
…concurrency and re-entry Wrap pairwise.__next__ in a critical section so concurrent callers from different threads serialize cleanly in the free-threaded build, and add a re-entrancy guard so a same-thread re-entrant call (e.g. when the input iterator's __next__ calls back into pairwise.__next__) raises RuntimeError instead of crashing or producing garbled output. Matches the long-standing behavior of itertools.tee and of generators, and fixes the use-after-free crash reported in pythongh-149557. With re-entry blocked, po->old is safe as a borrowed reference for the duration of __next__, so the defensive incref/decref and re-read guard added by pythongh-109788 are no longer needed and have been removed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
eendebakpt
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
__next__calls back intopairwise.__next__) raisesRuntimeErrorinstead of crashing or producing garbled output. Matches the behavior ofitertools.teeand of generators, and fixes the use-after-free crash reported in Single-threaded re-entrancy via a recursive generator causes an access violation (segfault) inpairwise_next#149557. (instead of raisingRuntimeErrorwe could consider raisingValueError)pairwise.__next__in a critical section so concurrent callers from different threads serialize cleanly in the free-threaded build.Also addresses #123471