Repository navigation
fix(file-search): claim a file write that lands on an owner the backfill already queued - #9666
Merged
Merged
Conversation
…ill already queued Backfill reconciliation wake-ups enter the owner queue as if just dispatched so they rotate in behind owners woken for a write. A write that landed while such a row was still queued only advanced its updated_at through the file trigger's upsert, so it kept the reconciliation stamp and waited behind every older-stamped owner. The same held for an owner that was served, left queued with work, and then woken again. Claims now take owners woken since their last dispatch first: never dispatched, or updated_at later than last_dispatched_at, which every real wake-up (file write, released claim, reaped claim, artifact-ready requeue) advances and a reconciliation wake-up never does; it no longer touches an owner already queued. Within each group the dispatch-age rotation is unchanged. Queue timestamps the comparison reads come from the transaction clock, the same clock the trigger writes.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Contributor
[Medium impact] The PR appears safe to merge; no new actionable issues were found. SummaryThe PR gives file writes priority over backfill wake-ups, including writes to owners already queued.
Reviews (2) · Last reviewed commit: "fix(file-search): rotate a woken owner w..." · Reviewed by Greptile |
This comment has been minimized.
This comment has been minimized.
…ch age An owner already holding both of its outstanding claims can claim nothing, but a write between passes kept marking it woken, so with little global capacity left it took every owner turn and the free slot went unused. Woken owners now go first only while they have a free slot; a full owner keeps the dispatch-age rotation it had before. Also gives the due-reconcile work-check case the same one-minute margin as its not-yet-due counterpart: exactly at the interval it compared a database timestamp with the application clock and flaked on skew.
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
updated_at(the file trigger'sON CONFLICTupsert), so it kept the reconciliation stamp and waited behind every older-stamped owner — during a long walk, several passes. The same held for an owner that was served, left queued with work, and then woken again.updated_at > last_dispatched_at(wokenSinceDispatch). Every real wake-up advancesupdated_at(file write, released claim, reaped claim, artifact-ready requeue); a reconciliation wake-up never does — it now usesON CONFLICT DO NOTHING, so it leaves an already queued owner untouched. Within each group the dispatch-age rotation is unchanged, so fairness is unchanged.artifact-readywrite them with the transaction'snow(), as the trigger does. The remaining window is a write whose transaction began before the claim that stamped its owner; it waits one rotation by dispatch age (documented onwokenSinceDispatch).file_search_dispatch_schedule_idxand sorts the queue instead (hundreds to low thousands of narrow rows, once a minute, row locks still only on returned rows).Type of Change
Testing
dispatcher.integration.ts(real PostgreSQL, writes through the realworkspace_file_search_mark_pendingtrigger):serves a file write to %s ahead of a backfill page of reconciliation wake-ups— now table-driven; the new "owner the page left queued" case is red on the old code and without thewokenSinceDispatchorderingkeeps a queued reconciliation wake-up behind woken work when the backfill passes it again— red on the old code, without the ordering, and on a mutant whose reconciliation upsert bumpsupdated_atleaveSlotshelper replaces three copies of the capacity setupowners.integration18/18,chunks.integration49/49 (1 opt-in skip),index-plan.integration9/9,content-rendered.integration31/31,lib/workspace-files/searchunit tests 178/178dispatch work check > reports work for a backfill reconcile that is duecompares a DB-clockcompleted_atagainst a JSDateexactly at the reconcile threshold and can flake by milliseconds; untouched hereChecklist
test-auditauthoring gate)