feat: memory versions, expected_version and superseded_by by mthines · Pull Request #745 · mthines/lorekit

mthines · 2026-11-09T06:35:02Z

Memories get corrected: a runbook from January is fixed in March. Today a reader can't tell. Writing to the same scope and key overwrites in place and only updated_at changes, and supersedes (#736) sits on the newer record, so a read of the old record shows nothing.

What changes

version on every record, hosted and local. It starts at 1 and goes up only when the content changes: value, content_type, attributes, tags, or a supersedes link being set or changed.
- A BEFORE trigger (migration 00120) enforces this on every write path and overrides any version a statement sets.
- TTL, counters, embeddings, archive/restore, owner moves, identical rewrites, and a link cleared by on delete set null don't bump it.
- In the local two-tier store, a write continues from the higher of the two tiers' versions, so a version never goes down.
expected_version (optional) on MCP memory.write, REST create/update and lorekit write --expected-version N.
- A mismatch writes nothing and returns version_conflict with the current version: REST 409, an MCP isError result, CLI exit 1.
- 0 means "must not exist yet".
- An archived or expired record counts as absent (version 0), the same as reads. A guarded write that revives an expired record with no TTL named clears its old expiry, so the write is visible.
- Hosted, the check is atomic inside memory_write.
- Omitting expected_version keeps today's behaviour.
superseded_by, computed on read and never stored: the ids of live records that name this one in supersedes and that the caller can see.
- One query per page, under the same tenant filter as the read.
- Returned on MCP read/list/search and REST get/read/list/search.
- List and search defaults are unchanged: superseded records still appear, marked.
REST PATCH now treats an expired record as not found (404), like GET /memories/{id}. The shared NOT_EXPIRED filter keeps the two in step.
Migrations: 00120 adds the version >= 1 check as not valid. 00121 validates it in its own transaction, so the scan doesn't run under the add column lock.

Limits and out of scope

The local CLI store computes superseded_by on show/read only, not on list or search, which would walk the whole store on the hook's hot path.
migrate to remote doesn't carry version; the server starts the counter at 1.
A plain write (no expected_version) to an expired key still keeps its past expiry, as on main. Changing that affects TTL behaviour for every writer, so it's left for a separate change.

Docs: docs/mcp-tools.md, docs/cli.md, use-cases.mdx, the OpenAPI spec (409s, PATCH 404s), the tool catalog and llms.txt (regenerated). bootstrap.sql, the edge mirrors and surfaces.generated.mjs are regenerated.

Verification

migrations.test.sql §120 passes on a throwaway PG16 cluster with every migration through 00121.
Edge deno test: 73 passed. deno check stays within the baseline.
pnpm nx test cli: green except the known loopback failures, which fail the same way on main (see docs/sandbox.md). pnpm nx test schemas: 343 of 343.
The generator --check modes and the llms drift check are clean.

Follows from the "version rule" point raised on the blog post (#744).

https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

…version Migration 00120 adds memories.version (NOT NULL DEFAULT 1, so existing rows read 1 without a backfill). A BEFORE INSERT OR UPDATE trigger owns it: an insert starts at 1, and an update gets old + 1 exactly when value, content_type, attributes, tags or a non-null supersedes changed, so every write path obeys the rule and no statement can set the number itself. TTL, counters, embeddings, archive/restore, owner moves, identical rewrites and a link cleared by its target's deletion leave it alone. memory_write gains p_expected_version and returns version. A stale value raises LK003 version_conflict naming the stored version; 0 means the key must not exist. The live row is locked FOR UPDATE for the check and every do-update re-states it, so a concurrent writer cannot slip between check and write. lorekit_memory_list returns version too. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

memory.write (MCP, REST POST /memories, mcp-core) accepts expected_version, validated by the shared record-extensions parser, and returns version. The RPC's LK003 refusal becomes a VersionConflictError: an isError result on MCP, a 409 with code version_conflict and current_version on REST. PATCH /memories/:id takes expected_version as a predicate on the update itself and answers 409 when the row it could have patched has moved on. Reads return version, and superseded_by: the ids of the live records the caller can see whose supersedes names this one, fetched with ONE query per page under the read's own tenant filter (MCP read/refs/list/search; REST get/read/list/search). Omitted when empty; list and search filtering is unchanged. lorekit_memory_list returns version for REST list. Catalog, OpenAPI and llms.txt (template + regenerated) document it. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

…eded_by The local store appends `version` to the frontmatter: 1 on a first write or a revived archived key, +1 on a content change (value, content_type, merged attributes, tags, a new supersedes link) under the hosted trigger's rule, and kept otherwise. `expected_version` is checked against the live record (0 when absent) before anything is written; a mismatch returns code version_conflict with the current version. read/readMany attach superseded_by across both tiers. `lorekit write --expected-version N` forwards the check to either store, exits 1 with the stored version on a conflict, and reports the new version; show and list --json carry version. A remote write error object no longer prints as [object Object]. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

docs/mcp-tools.md (the expected_version parameter, a Versions section, and version / superseded_by on every read return shape), docs/cli.md (versions and --expected-version on write), the CI state-record use case (guard overlapping jobs with --expected-version), and docs/key-files.md (superseded-by.ts and migration 00120). Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

github-actions · 2026-11-09T06:51:53Z

The dashboard preview for this PR — redeployed on each push that changes the web app.

Project	Deployment	Actions	Updated
lorekit	Ready	Preview	Nov 9, 2026 9:20am UTC

_{Preview always points at this PR's latest commit; Deployment is the immutable build of e158219. · Comment /web-preview to force a redeploy. · Workflow logs}

mthines · 2026-11-09T08:46:53Z

✅ No issues found

Since 8cf7aba: PATCH /memories/:id now 404s on an expired memory via a shared NOT_EXPIRED filter (new tests fail without it), and the catalog/llms.txt describe expired-as-version-0 and revive-clears-TTL. No new findings.

Review details

Gate		Details
Description vs. code	✅	Description now covers 00121, revive-clears-TTL, the two-tier version rule, and PATCH 404s.
Prior review feedback	✅	Earlier review comments are resolved.
Documentation	✅	Catalog, llms.txt, mirrors, mcp-tools.md, and the OpenAPI PATCH text document the change.
Self-review signals	✅	No debug logs, leftover TODOs, or unreviewed stubs.
Code review	✅	The multi-lens review found no blocking issues.

Run	Details
Mode	full · 152 lines in delta · tier deep · depth checkout
Progress	open review threads 2 → 3 → 0 → 0 across the last 4 reviews
Checked	13 of 52 changed files read · no possible issues found
Reach	47 changed exports · 54 dependent files · 0 checked · 54 not checked · 8 changed files imported by 24 files, not traced `RECORD_FIELD_COLUMNS` — 2 consumer files · 0 checked `RECORD_FIELD_COLUMNS` — 1 consumer file · 0 checked `Database` — 16 consumer files · 0 checked `MEMORY_SELECT` — 3 consumer files · 0 checked `normalizeEntry` — 3 consumer files · 0 checked +42 more `packages/cli/src/store/local.mjs` — imported by 8 files `packages/cli/src/store/remote.mjs` — imported by 7 files `packages/cli/src/store/format.mjs` — imported by 5 files `packages/cli/src/shared/completions.mjs` — imported by 4 files `supabase/functions/_shared/schemas/tool-catalog.ts` — imported by 3 files +3 more
Found	produced 0 → posted inline 0 · cleared 0 · carried forward 0 · deferred 0 · below-bar 0
Lenses	migration-locking-and-rewrites (repo lens) · migration-query-performance (repo lens) · migration-safety-and-compat (repo lens) · tenant-isolation (repo lens) · optimize-approach (auto) · tdd (auto)
Standards	ran · 6 docs · 0 findings on the delta
Integrations	none
CI	still pending: `Web — Vercel preview (changed) / Deploy Vercel preview`, `Web — Storybook interaction + visual tests (changed)`, `Web — Playwright e2e (local build + local Supabase)`, `Typecheck, Test & Lint (affected)` · 7 passed
Memories	76 indexed · 0 used

^{Nothing to report — measurability (skipped), severity, 0 files skipped.}

^{pr-reviewer · commit e158219 · full review · how these findings are produced · updated Nov 9, 2026 9:20am UTC}

Generated by Claude Code

mthines

Generated by Claude Code

…esolves TwoTierStore.write compared expected_version against the target tier's copy only, while TwoTierStore.read falls through to home. With a project tier active and the key only in home at version 2, --expected-version 2 was refused with current 0. The check now uses the two-tier read's answer, and a project copy written over a home-only record continues its version (it shadows that record, so a reader never sees a reset to 1). Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

Every read hides a row whose expires_at has passed, but the version check treated it as existing, so expected_version 0 was refused for a key a reader saw as missing. memory_write's gate, its do-update guards and its conflict report now read an expired row as version 0; the local store uses isLive for the check. The expired row is still rewritten in place (same id, created_at, seen_count), so its version continues counting rather than restarting at 1 — restarting would need the bump trigger to judge liveness on every update. Regenerates the BYOD bootstrap; adds §120 AC-2k..n. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

Adding the CHECK directly scans every row under the ADD COLUMN's ACCESS EXCLUSIVE lock. It is now added NOT VALID (catalog-only) and validated in a separate statement, which needs only SHARE UPDATE EXCLUSIVE. Validation is safe: every existing row reads version 1 from the default. Regenerates the BYOD bootstrap; §120 AC-1r asserts the constraint ends validated. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

mthines

Generated by Claude Code

A migration file runs in one transaction, so the VALIDATE in 00120 scanned while the ACCESS EXCLUSIVE lock from `add column` was still held. 00120 now adds the CHECK NOT VALID only; 00121 validates it in its own transaction under SHARE UPDATE EXCLUSIVE. Corrects the 00120 comment, regenerates the BYOD bootstrap, and points §120 AC-1r at 00121. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

…rd clears its past TTL A write whose expected_version check passed because the stored row was expired updated that row in place but, naming no TTL, kept the past expires_at, so the record stayed hidden and the nightly purge deleted it. memory_write (00120) and the local store now clear expires_at in that case (what a fresh record with no TTL gets); an explicit TTL is still honoured, and a write without expected_version is unchanged. Adds §120 AC-2o..q (without p_clear_ttl) and a local test. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

…ackwards With an expired project copy at v2 and a live home copy at v5, a write with expected_version 5 continued from the project copy and landed at v3. The local store now continues from the max of this tier's stored copy and the record the two-tier read resolved, comparing content against what the reader saw. Adds a test reproducing v5 -> v6. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

… and a revive clears its TTL The expected_version and ttl_days descriptions now state that an archived or expired memory counts as absent (version 0), and that an expected_version write reviving an expired row with no TTL named clears the past expiry. docs/mcp-tools.md names every TTL field. Edge mirror, surfaces and llms.txt regenerated. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

…ke GET PATCH's target() filtered archived rows only, so it could edit a row GET /:id 404s on, and its expected_version 409 lookup reported an expired row's version where memory_write counts it as 0. The target now carries the same unexpired predicate as get.ts (NOT_EXPIRED, one shared constant for both handlers), so an expired row is a 404 for the update and the conflict lookup alike. OpenAPI description says so; edge test covers 404/409/200. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa

claude added 4 commits November 9, 2026 06:12

mthines commented Nov 9, 2026

View reviewed changes

Comment thread packages/cli/src/store/local.mjs Resolved

Comment thread supabase/migrations/00120_memory_version.sql Resolved

mthines mentioned this pull request Nov 9, 2026

docs(blog): add the version rule to the agent-memory post #744

Merged

claude added 3 commits November 9, 2026 08:51

mthines commented Nov 9, 2026

View reviewed changes

Comment thread supabase/migrations/00120_memory_version.sql Resolved

Comment thread packages/cli/src/store/local.mjs Resolved

Comment thread supabase/migrations/00120_memory_version.sql Resolved

claude added 5 commits November 9, 2026 09:05

mthines marked this pull request as ready for review November 9, 2026 09:24

mthines merged commit 8095a1d into main Nov 9, 2026
19 checks passed

lorekitbot Bot mentioned this pull request Nov 9, 2026

chore: release main #746

Merged

Navigation Menu

Repository navigation

feat: memory versions, expected_version and superseded_by - #745

feat: memory versions, expected_version and superseded_by#745
mthines merged 12 commits into
mainfrom
feat/memory-versioning

mthines commented Nov 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 9, 2026 •

edited

Loading

Uh oh!

mthines commented Nov 9, 2026 •

edited

Loading

Uh oh!

mthines left a comment

Uh oh!

Uh oh!

Uh oh!

mthines left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mthines commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mthines commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ No issues found

Uh oh!

mthines left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mthines left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mthines commented Nov 9, 2026 •

edited

Loading

github-actions Bot commented Nov 9, 2026 •

edited

Loading

mthines commented Nov 9, 2026 •

edited

Loading