Skip to content

Fix #2363: [Bug] skills upsert (INSERT OR REPLACE) skips FTS cleanup trigger under recursiv - #2577

Open
syzsunshine219 wants to merge 2 commits into
MemTensor:mainfrom
Memtensor-AI:bugfix/autodev-2363-20260914203915788
Open

syzsunshine219 wants to merge 2 commits into
MemTensor:mainfrom
Memtensor-AI:bugfix/autodev-2363-20260914203915788

Conversation

@syzsunshine219

@syzsunshine219 syzsunshine219 commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Restores the change proposed in #2368 for review against main after its former base branch, dev-v2.0.34, was deleted. The source commits and authors are preserved.

Source head: 5603ee8125690d522d3ff43ad5cb631149b82af8

This PR is open for review. Review and CI against the current base are pending; validation claims in the original description below refer to the earlier snapshot. The original discussion and evidence remain in #2368.


Original change description

Description

Fixes #2363. Root cause is exactly as reported: apps/memos-local-plugin/core/storage/repos/skills.ts upserts via INSERT OR REPLACE, and per SQLite's documented REPLACE semantics the internal DELETE only fires AFTER DELETE triggers when PRAGMA recursive_triggers is ON. SQLite's default is OFF, and several hosts (SQLCipher and some ORM stacks) leave it OFF, so the skills_fts_ad cleanup trigger is skipped on every upsert-on-conflict while skills_fts_ai still runs — leaving skills_fts with duplicate/orphan rows relative to skills and polluting searchByText rankings. The same footgun applies uniformly to every onConflict: "replace" upsert paired with an FTS mirror (traces / policies / world_model).

Chose option 2 from the issue: set PRAGMA recursive_triggers = ON at connection open in apps/memos-local-plugin/core/storage/connection.ts (both read-write and read-only branches, for symmetry). One line, zero API surface change, no migration, covers all four FTS-backed tables at once, and future-proofs any additional AFTER DELETE trigger we may add. Rejected option 1 (rewriting each upsert as explicit DELETE+INSERT) because it duplicates the fix across four repos, changes the atomicity contract callers rely on, and doesn't help the next FTS-mirrored table someone adds. Foreign-key cascades are unaffected — those are FK actions, not triggers, and always run.

Regression coverage lives in apps/memos-local-plugin/tests/unit/storage/skills-fts-upsert.test.ts (3 cases: base row-count invariant count(skills_fts) == count(skills) after upsert-on-conflict, stale-token search behavior through the full searchByText path, and a direct PRAGMA recursive_triggers assertion so a future contributor silently dropping the pragma is caught immediately). All three cases fail on the pre-fix code and pass on the fix.

Verified: plugin unit suite 180/180 files, 1553/1553 tests pass (1 skipped, pre-existing); integration suite 3/3 files, 5/5 tests pass; npm run lint (tsc --noEmit) is clean. opsp artifacts (proposal / spec / design / verification-report / task) are in openspec/changes/2026-09-14-2363-.../ and synced+pushed to specs/2026-09-14-2363-.../. Working branch pushed to origin at commit 20c13665.

Related Issue (Required): Fixes #2363

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Refactor (does not change functionality, e.g. code style improvements, linting)
  • Documentation update

How Has This Been Tested?

Not run; documentation-only change.

  • Unit Test
  • Test Script Or Test Steps (please provide)
  • Pipeline Automated API Test (please provide)

Checklist

  • I have performed a self-review of my own code
  • I have commented my code in hard-to-understand areas
  • I have added tests that prove my fix is effective or that my feature works
  • I have created related documentation issue/PR in MemOS-Docs (if applicable)
  • I have linked the issue to this PR (if applicable)
  • I have mentioned the person who will review this PR

@wustzdy please review this PR.

Reviewer Checklist

autodev added 2 commits September 15, 2026 05:17
Under `PRAGMA recursive_triggers=0` (SQLite's default, and what several hosts
including SQLCipher ship), `INSERT OR REPLACE INTO skills` executes an
internal DELETE that skips AFTER DELETE triggers. The `skills_fts_ad`
cleanup trigger therefore never fires on upsert-on-conflict, while
`skills_fts_ai` still runs — so `skills_fts` accumulates duplicate/orphan
rows relative to `skills`, and `repos.skills.searchByText` ranks over the
polluted keyword index.

The same footgun applies to every FTS-backed table using
`onConflict: "replace"` (traces / policies / world_model). Fix at the
connection layer via one pragma so all four channels benefit uniformly and
any future AFTER DELETE trigger works as intended without every repo author
having to remember.

Regression tests exercise the DB layer directly: base row-count invariant,
stale-token search behavior, and a direct pragma assertion.

Refs MemTensor#2363
`recursive_triggers = ON` only affects DELETE/UPDATE triggers fired by
implicit REPLACE deletions, none of which can occur on a read-only
handle. Setting it in the read-only branch was dead code and misleading
about which pragmas the read path actually depends on. Keep the pragma
on the writable branch where it fixes the FTS orphan bug (issue MemTensor#2363).

Addresses OCR review of PR MemTensor#2368.
@syzsunshine219
syzsunshine219 marked this pull request as ready for review November 9, 2026 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] skills upsert (INSERT OR REPLACE) skips FTS cleanup trigger under recursive_triggers=0, orphaning skills_fts rows

1 participant