Skip to content

fix(code-index): stop reading a held projection slot as work - #3077

Open
devin-ai-integration[bot] wants to merge 2 commits into
masterfrom
devin/1791129523-master-flakes-3041
Open

devin-ai-integration[bot] wants to merge 2 commits into
masterfrom
devin/1791129523-master-flakes-3041

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • text_projection_needs_work reported any contended slot lock as work in progress. Another probe, or a no-op advance over ready owners, holds that lock too. When the two collided, the published text projection stamped a phantom worker continuation, and freshness stayed Verifying / Pending at the serving swap. This is the reconcile flake in test: four intermittent master failures (reachability, reconcile, git_watch, dashboard) #3041.
  • Fixes the git-watch linked-worktree debounce flake, which was a test-clock race.

Fixes #3041

Motivation

#3041 lists four intermittent master failures. Status of each:

  • Reconcile freshness, (Some(Verifying), Pending): failed 59 of 3,000 runs under local stress. I instrumented every pending-wake writer. There were two late wake sources:
    1. The test's request_complete_generation demand sometimes landed after the first pass had claimed its wake. That queues a real follow-up pass, so Verifying is the correct reading. Fixed in the test by holding admission until the demand is posted.
    2. Every remaining failure was the published projection task's note_worker_continuation, fired because the slot try_lock returned WouldBlock while the slot was idle. This is a production bug, fixed here.
  • Git-watch debounce (tests.rs:643): the test ran on start_paused time. The paused clock auto-advanced past the test's deadlines while a live notify wake was still in flight. The watcher's own copies of the marker writes could also produce a second drain.
  • Reachability: PR fix(code-index): serve a complete seat that lands after the read misses #3044 already addresses this. With fix(code-index): serve a complete seat that lands after the read misses #3044 and this branch, the test passed all 80 stress runs. The issue reported 5 failures in 40 runs on the earlier master.
  • Dashboard ChunkLoadError: that message is the error RouteChunkBoundary.dom.test.tsx deliberately rejects with. It appears in the output whenever that suite runs, so it does not identify the failing test. The suite passed 15 of 15 local runs. No change here.

Changes

  • code_index_scheduler/serving.rs: CodeTextProjectionStateV1 keeps the slot's idle state in an atomic (slot_idle), written on every slot transition through replace_slot. The probe reads that atomic and never takes the lock, so it still never queues behind an advance's slice.
  • tests/serving.rs: adds a_held_idle_projection_slot_is_not_text_projection_work. It holds an idle slot's lock and asserts no work is reported. It fails before the fix (serving.rs:908) and passes after.
  • tests/reconcile.rs: ready_wait_ends_only_after_the_graph_tail_seats_the_generation holds admission until the complete-generation demand is posted.
  • git_watch/tests.rs, state.rs, git_watch.rs: the linked-worktree test now runs on real time. It waits on a test-only operation_held signal (debounce observed InFlight) and asserts the release with >= 1 drain instead of an exact count.

Test plan

  • bash scripts/require-exact-test.sh cargo test -p tracedecay-code-index-runtime --lib <test> -- --exact passes for each of the three tests above, after the rebase on origin/master
  • Stress results:
    • reconcile ready-wait: 0 failures in 3,000 runs (59 before)
    • git-watch linked hold: 0 failures in 2,000 runs
    • reachability: 0 failures in 80 runs
  • cargo test -p tracedecay-code-index-runtime --lib -- code_index_scheduler::tests::serving code_index_scheduler::tests::reconcile git_watch: 280 passed
  • cargo fmt --all -- --check and cargo clippy -p tracedecay-code-index-runtime --all-targets -- -D warnings are clean
  • python3 scripts/linux-test-partitions.py check and node scripts/lint-commit-range.mjs --repository . origin/master HEAD pass
  • Full workspace cargo nextest run: left to CI

Checklist

  • CHANGELOG.md updated (not updated; release-please generates it from commits)
  • No secrets, credentials, or .env files included
  • Breaking changes documented (none)

Link to Devin session: https://app.devin.ai/sessions/e8f223a3ec5c476885a5d8903bafa4c2
Open in Devin Desktop: https://app.devin.ai/desktop/session/e8f223a3ec5c476885a5d8903bafa4c2?variant=devin
Requested by: @ScriptedAlchemy


Devin Review

ScriptedAlchemy and others added 2 commits October 4, 2026 15:58
The paused clock auto-advanced past the test's deadlines while a live
notify wake was in flight, and the watcher's own copies of the marker
writes could add a second drain. Wait on the debounce observing the
operation instead, and assert the release rather than an exact count.

Refs #3041

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
text_projection_needs_work treated any contended slot lock as work in
progress. Another probe or a no-op advance over ready owners holds the
lock too, so the published projection stamped a phantom continuation
and freshness stayed Verifying at the serving swap. Mirror the slot's
idle state in an atomic written on every transition and read that.

The graph-tail ready test also holds admission until the
complete-generation demand is posted, so the demand joins the first
pass instead of queueing a real follow-up.

Fixes #3041

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@changeset-bot

changeset-bot Bot commented Oct 4, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 1aa654c

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: four intermittent master failures (reachability, reconcile, git_watch, dashboard)

1 participant