test(assignor): fix flaky PyPy CI leg (disable Hypothesis deadline)#732
Closed
wbarnha wants to merge 1 commit into
Closed
test(assignor): fix flaky PyPy CI leg (disable Hypothesis deadline)#732wbarnha wants to merge 1 commit into
wbarnha wants to merge 1 commit into
Conversation
The copartitioned-assignor property tests run on PyPy since the PyPy skips were removed. They carried a fixed 4000ms Hypothesis deadline, but PyPy's JIT warmup makes a single example's timing vary by over a second, so runs intermittently exceed it and Hypothesis reports DeadlineExceeded / FlakyFailure -- failing the (non-required) PyPy leg and turning master CI red even though the assignment produced is valid. These tests assert assignment correctness, not performance, so set deadline=None (as the DeadlineExceeded message itself recommends). The property assertions are unchanged and the job timeout still guards against a real hang. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HHPL4VFWQRQPpjR1gXSKyL
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #732 +/- ##
=======================================
Coverage 95.11% 95.11%
=======================================
Files 104 104
Lines 11169 11169
Branches 1206 1206
=======================================
Hits 10623 10623
Misses 452 452
Partials 94 94 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Master CI intermittently goes red on the
Python pypy3.11leg. The failure is:in
tests/meticulous/assignor/test_copartitioned_assignor.py::test_remove_clients(and its siblings).Root cause
Those copartitioned-assignor property tests now run on PyPy (the PyPy
skipifguards were removed when full PyPy support landed). They carried a fixed 4000 ms Hypothesis deadline. PyPy's JIT warmup makes a single example's timing vary by more than a second, so an example intermittently exceeds 4000 ms on its first run but not on retry — which Hypothesis reports asDeadlineExceeded→FlakyFailure. The assignment produced is perfectly valid; only the timing is nondeterministic. This flakes the (non-required) PyPy leg and turns the master run red.Fix
Set
TEST_DEADLINE = None, disabling the Hypothesis deadline for these four property tests — exactly what theDeadlineExceededmessage recommends. These tests assert assignment correctness (validity, stickiness, termination), not performance, so the deadline adds only flakiness. The property assertions are unchanged, and the CI job's own timeout still guards against a genuine hang.Verification
pytest tests/meticulous/assignor/test_copartitioned_assignor.py→ 4 passed (run takes ~5.7 s, confirming examples legitimately approach/exceed the old 4 s deadline). flake8/black clean.🤖 Generated with Claude Code
Generated by Claude Code