fix(index): finalize full FTS rebuild maintenance (#1630) - #141
fix(index): finalize full FTS rebuild maintenance (#1630)#141DevNexsler wants to merge 1 commit into
Conversation
Left open by the 2026-08-27 reconciliation — the premise does not hold in productionI reproduced this ticket's defect and it is real: the A read-only But this PR cannot fix it, because the path it hardens has never executed.
The full-rebuild fallback has fired zero times in five weeks. The 440 orphans are not its What they actually areOrphan directory mtimes, by UTC day: Against ~103 incremental merges on 08-23 and ~102 on 08-24, that is roughly two index Why the check is still red
VerdictLeft open. Per the reconciliation brief, "if you cannot make the suite go red on a defect you To be fair to the change on its own terms: making a full rebuild finalize index maintenance is Also blocking a clean merge: this branch is based on What would change my mindA completed index run that leaves the orphan count high. I am about to get one: the sweep A second opinion (GPT-5.6 Sol, asked to refute) returned REFUTED against my first, |
…tion Every defect below was reproduced from production state or from a control run here before its fix was read, and every red/green was executed on both sides by this pass rather than taken from a PR body. Where a PR's red was only an ImportError on a new symbol (#1655, #1657), a behavioural test that runs unchanged on both sides was written so the red proves wrong behaviour, not an absent name. Landed (PR -> ticket): #154 -> #1625 serve queued source work at bounded checkpoints during a long sweep, and return 503 stale_tail once a request waits past its SLO #153 -> #1654 tier tests by location before name; match the _live.py suffix #152 -> #1663 stop the unit tier reaching the real detached indexer launcher #151 -> #1659 stop the staging gate racing for fixed host ports #150 -> #1661 keep Lance stale-read recovery in the store, not in mcp_server #149 -> #1662 keep the upstream body at every provider rejection site #148 -> #1657 keep the upstream body on a permanent HTTP rejection #147 -> #1658 match provider-sim's embedding character cap to the real route #146 -> #1650 let the telemetry seam see qwen's discarded first answer #145 -> #1655 bound embed inputs by the route's character cap, not tokens alone #144 -> #1656 recover Lance reads while a peer writer keeps moving the table #143 -> #1649 bound the nested index write lock join for deadlock, not scheduling #142 -> #1629 observe the enrichment route contract on every answer Overlaps resolved rather than merged blind: #149 stacks on #148, so one merge lands both; #150 depends on #144's table-identity recovery and would be a downgrade alone, so they land together; #142 and #146 are two fixes to one root cause (the in-request JSON validation retry) and compose cleanly rather than competing; #145 and #147 are the client and simulator halves of the same character cap. One change is this pass's own, written test-first: #1662 left providers/llm/litellm_llm.py and providers/llm/openrouter_llm.py on the bare raise_for_status, and litellm_llm is the enrichment route that was observably discarding 400 bodies in production on 2026-08-26 and 2026-08-27. Gate on the reconcile branch, all five tiers: static PASS | unit 1864 passed | integration 90 passed staging-e2e 41 passed (tool coverage 21/21) | live 123 passed, 0 failed Baseline at origin/main run the same way: unit 1723 passed, integration 90 passed. No new failures. Because lancedb_store.py changed and the venv resolves lance 4.0.0 while the production image resolves lance 10.0.0, the deterministic suite was re-run inside the candidate image: tests/test_store.py 149 passed on lance 10. PROTECTED SURFACE: #154 changes the /health contract (new 503 stale_tail, new INDEX_PENDING_MAX_AGE_S in docker-compose.yml). The container healthcheck raises on 503 and the post-deploy verify requires status=ok, so a stale tail can mark the container unhealthy and can trip an auto-rollback. Status composition was checked: stale_tail sits between index_failed and degraded, and disk_full still overwrites last, so #0910's ordering holds. Left open with written reasons: #141 (#1630 — hardens a path that has executed zero times in five weeks; the 440 orphan _indices were reclaimed by the existing incremental finalizer during this pass, confirming it) and #79 (#0584 — a month stale, and its label alias map matches none of the 512 unstable doc_types measured in production). Report: Maint-Manager/.briefs/reconcile/20260827/RAG-in-a-Box.md
1 participant
Ticket #1630
Ensure all full FTS rebuild fallbacks finish Lance restore-point, version-prune, and orphan-index cleanup lifecycle.
Validation: make gate code tiers passed (static 2014 collected; unit 1712 passed; integration 90 passed; staging-e2e 39 passed, 21/21 MCP tools covered/traced). Live preflight blocked externally: worktree config_test.yaml missing, production indexer active, Comm-Data-Store Postgres hostname unreachable.
Production follow-up: after merge/deploy, run 1160-lance-orphan-indices.sh once daily for three consecutive days; all runs must exit 0 before ticket moves to done.
Maint ticket: #1630 — tickets/review/1630-regression-1160-was-marked-done-on-2026-08-18-but-.md
On merge: verify acceptance boxes against production (not tests), tick them, then move ticket to tickets/done/ and set status: done.
Merged PR != closed ticket; unmerged PR != shipped fix.