Skip to content

test: deflake test_actor_create_update_delete against retried create conflict - #971

Draft
vdusek wants to merge 5 commits into
masterfrom
worktree-fix-flaky-1
Draft

test: deflake test_actor_create_update_delete against retried create conflict#971
vdusek wants to merge 5 commits into
masterfrom
worktree-fix-flaky-1

Conversation

@vdusek

@vdusek vdusek commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

The integration test test_actor_create_update_delete intermittently failed with ConflictError: Some other Actor already has this name, e.g. in this CI run (on an unrelated Renovate PR).

Root cause: the HTTP client retries every request on transient 5xx and network errors (max_retries=8). A POST /v2/actors create can commit server-side on one attempt yet still be retried; the retry then hits a 409 on the unique name it just created. The failing traceback shows attempt=2, and the 61^8 random resource name rules out cross-worker or leftover collisions.

Create calls in the integration tests now go through conflict-recovering helpers in tests/_utils.py. On the API error that signals a lost name or version number, the helper fetches what the first attempt committed and the test continues. Anything else propagates, and so does a matching error whose resource cannot be found afterwards - a genuine create failure still fails the test rather than being masked.

This covers every create of a resource with a uniqueness constraint: Actors (actor-name-not-unique), tasks (actor-task-name-not-unique), schedules (schedule-name-not-unique), Actor versions (version-already-exists) and env vars (env-var-already-exists). Webhooks have no name uniqueness, and the dataset/KVS/queue creates use get_or_create, so neither needs a helper.

The recovery branches only run when a retried create already committed, which never happens in a normal integration run. tests/unit/test_create_helpers.py therefore drives each helper against a mocked API, so a wrong error type or lookup route is caught there instead of resurfacing as the flake the helpers exist to prevent.

This is a test-side fix. The underlying client behaviour - retrying non-idempotent requests - is tracked in #990; besides these spurious conflicts it can also silently duplicate run starts and webhook creations for callers.

✍️ Drafted by Claude Code

@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Jul 21, 2026
@vdusek vdusek self-assigned this Jul 21, 2026
@github-actions github-actions Bot added this to the 145th sprint - Tooling team milestone Jul 21, 2026
@github-actions github-actions Bot added the tested Temporary label used only programatically for some analytics. label Jul 21, 2026
@codecov

codecov Bot commented Jul 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.71%. Comparing base (297a53d) to head (5ef747a).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #971      +/-   ##
==========================================
+ Coverage   94.64%   94.71%   +0.06%     
==========================================
  Files          58       58              
  Lines        5248     5259      +11     
==========================================
+ Hits         4967     4981      +14     
+ Misses        281      278       -3     
Flag Coverage Δ
integration 92.14% <ø> (-0.41%) ⬇️
unit 84.92% <ø> (+1.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek
vdusek requested a review from Pijukatel July 21, 2026 08:13
@vdusek
vdusek marked this pull request as ready for review July 21, 2026 08:14
@vdusek
vdusek marked this pull request as draft August 3, 2026 11:22
@vdusek

vdusek commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

converting to draft as I'm not sure whether these test utils are worth it, let me @Pijukatel know WDYT

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants