Repository navigation
fix(file-search): fill dispatch capacity in selection order and keep reaped claims ahead of the rotation - #9671
Merged
Conversation
…before the backfill The claim cut ranked every selected owner's candidates by revision age, so with less capacity than candidates a woken owner selected first could lose every slot to an older backlog selected after it, and the claim's stamp then filed it behind the rotation. Claims now fill capacity owner by owner in the pass's selection order (each still up to its free slots). The backfill page ran before the reaper in the same transaction, so an owner it queued as a reconciliation wake-up absorbed the reaper's wake-up (both writes at the transaction time) and the released claim waited behind the rotation. The reaper now runs first; the backfill's wake-ups leave queued owners untouched. Tests: a two-slot pass with a woken owner against an older backlog, and a reaped claim whose owner the backfill queues in the same pass; the capacity filler's files are deleted so the backfill never re-queues it.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Contributor
[Medium impact] The PR appears safe to merge; no new actionable issue was found. SummaryThe dispatcher fills available slots in owner selection order and keeps recovered claims ahead of ordinary queued work.
The previous thread was unnumbered, so it has no entry in
|
Reaping before the backfill held the owner's queue row while the backfill took FOR SHARE on its unindexed files; a concurrent save holds its file while its trigger waits on that queue row, so the two could deadlock. The pass runs backfill then reap again, and a wake that conflicts with a row this transaction's backfill just stamped as a reconciliation clears the stamp, so a reaped claim stays ahead of the rotation.
Collaborator
Author
Collaborator
Author
|
@cubic-dev-ai review this PR |
Contributor
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #9655 and #9666, from a final review of the release branch.
lockFileSearchOwnerssorts by key for lock order, so the selection order is restored from the candidates before claiming.last_dispatched_at = now()) absorbed the reaper's wake-up (updated_at = now(), soupdated_at > last_dispatched_atwas false) and the released claim waited behind the rotation. A wake that conflicts with a row stamped by this same transaction now clears the stamp, exactly as if the wake had queued the owner first; a stamp from an earlier transaction is always older thannow(), so nothing else changes. The phase order stays backfill → reap, which keeps the pass's lock order file → queue row, the same order a file save's trigger takes.wokenSinceDispatchnow lists the artifact-ready requeue and states the write-overlapping-a-claim window accurately; README describes the free-slot condition and the fill order; one stale test comment.Not changed (noted for later): owner selection no longer walks
file_search_dispatch_schedule_idxsince #9666 (a top-N sort over the queue each pass); a partial index on woken rows plus a two-statement select would restore index order but needs a migration and buys nothing at current queue sizes. Usingclock_timestamp()in the file trigger would close the write-overlapping-a-claim window; also a migration.Type of Change
Testing
dispatcher.integration.ts(real PostgreSQL):fills the free capacity in selection order, so a woken owner is not outbid by an older backlog— red on the previous code and with the revision-age cut restoredkeeps a reaped claim woken when the backfill queues its owner in the same pass— red on the previous code and with the stamp reset removedleaveSlotsnow deletes its filler owner's files so the backfill never re-queues it (removes a tie-break dependence on owner ids)owners.integration18/18,chunks.integration49/49 (1 opt-in skip),index-plan.integration9/9,content-rendered.integration31/31,lib/workspace-files/searchunit tests 178/178Checklist
test-auditauthoring gate)