Skip to content

fix(file-search): fill dispatch capacity in selection order and keep reaped claims ahead of the rotation - #9671

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/file-search-claim-selection-order
Nov 4, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/file-search-claim-selection-order

Conversation

@waleedlatif1

@waleedlatif1 waleedlatif1 commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #9655 and #9666, from a final review of the release branch.

  • Fill capacity in selection order. The claim cut ranked every selected owner's candidates by revision age, so with less free capacity than candidates (any loaded pass) a woken owner selected first could lose every slot to an older backlog selected after it. The claim then stamped it, filing it behind the rotation. Claims now fill capacity owner by owner in the pass's selection order, each still up to its free slots; ties within an owner keep revision age. lockFileSearchOwners sorts by key for lock order, so the selection order is restored from the candidates before claiming.
  • Wake a reaped owner the backfill just queued. Both phases run in one transaction, backfill first, so an owner the page queued as a reconciliation wake-up (stamped last_dispatched_at = now()) absorbed the reaper's wake-up (updated_at = now(), so updated_at > last_dispatched_at was false) and the released claim waited behind the rotation. A wake that conflicts with a row stamped by this same transaction now clears the stamp, exactly as if the wake had queued the owner first; a stamp from an earlier transaction is always older than now(), so nothing else changes. The phase order stays backfill → reap, which keeps the pass's lock order file → queue row, the same order a file save's trigger takes.
  • TSDoc on wokenSinceDispatch now lists the artifact-ready requeue and states the write-overlapping-a-claim window accurately; README describes the free-slot condition and the fill order; one stale test comment.

Not changed (noted for later): owner selection no longer walks file_search_dispatch_schedule_idx since #9666 (a top-N sort over the queue each pass); a partial index on woken rows plus a two-statement select would restore index order but needs a migration and buys nothing at current queue sizes. Using clock_timestamp() in the file trigger would close the write-overlapping-a-claim window; also a migration.

Type of Change

  • Bug fix

Testing

  • dispatcher.integration.ts (real PostgreSQL):
    • fills the free capacity in selection order, so a woken owner is not outbid by an older backlog — red on the previous code and with the revision-age cut restored
    • keeps a reaped claim woken when the backfill queues its owner in the same pass — red on the previous code and with the stamp reset removed
    • leaveSlots now deletes its filler owner's files so the backfill never re-queues it (removes a tie-break dependence on owner ids)
  • Suite 42/42; owners.integration 18/18, chunks.integration 49/49 (1 opt-in skip), index-plan.integration 9/9, content-rendered.integration 31/31, lib/workspace-files/search unit tests 178/178
  • lint and type-check (pre-push hooks); remaining audits in CI

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…before the backfill

The claim cut ranked every selected owner's candidates by revision age, so with less capacity
than candidates a woken owner selected first could lose every slot to an older backlog selected
after it, and the claim's stamp then filed it behind the rotation. Claims now fill capacity owner
by owner in the pass's selection order (each still up to its free slots).

The backfill page ran before the reaper in the same transaction, so an owner it queued as a
reconciliation wake-up absorbed the reaper's wake-up (both writes at the transaction time) and
the released claim waited behind the rotation. The reaper now runs first; the backfill's wake-ups
leave queued owners untouched.

Tests: a two-slot pass with a woken owner against an older backlog, and a reaped claim whose
owner the backfill queues in the same pass; the capacity filler's files are deleted so the
backfill never re-queues it.
@vercel

vercel Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 4, 2026 10:47am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issue was found.

Summary

The dispatcher fills available slots in owner selection order and keeps recovered claims ahead of ordinary queued work.

  • File-search claims follow owner selection order.
  • Reaped claims stay near the front when backfill runs too.

The previous thread was unnumbered, so it has no entry in previousFindings. Its reported lock cycle is addressed. Tests were inspected, not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Backfill missing revisions] --> B[Queue owners for reconciliation]
  B --> C[Release expired claims]
  C --> D[Clear same-pass stamp for recovered owners]
  D --> E[Select owners with free slots]
  E --> F[Acquire owner locks]
  F --> G[Restore selection order]
  G --> H[Fill available slots owner by owner]
Loading

Reviews (2) · Last reviewed commit: "fix(file-search): keep the dispatch pass..." · Reviewed by Greptile

Reaping before the backfill held the owner's queue row while the backfill took FOR SHARE on its
unindexed files; a concurrent save holds its file while its trigger waits on that queue row, so the
two could deadlock. The pass runs backfill then reap again, and a wake that conflicts with a row this
transaction's backfill just stamped as a reconciliation clears the stamp, so a reaped claim stays
ahead of the rotation.
@waleedlatif1 waleedlatif1 changed the title fix(file-search): fill dispatch capacity in selection order and reap before the backfill fix(file-search): fill dispatch capacity in selection order and keep reaped claims ahead of the rotation Nov 4, 2026
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit f279d9e into staging Nov 4, 2026
80 of 81 checks passed

This branch was previously deployed

1 inactive deployment
Preview — 55167644 Deployed Nov 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant