From 3462426586987b3b61ba7f663837458762bd283a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 06:59:00 +0000 Subject: [PATCH 01/17] blog: add post on an open agent-memory format A tour of how agent tools shape memory today (instruction files, the Anthropic memory tool, Copilot Memory, Letta blocks, Mem0, LangGraph's store, MCP and temporal knowledge graphs), what they agree on, why LoreKit's record looks the way it does, and how the same record carries a CI job's versioned JSON state. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../content/blog/open-agent-memory-format.mdx | 199 ++++++++++++++++++ packages/web/src/lib/blog/sections.ts | 6 + 2 files changed, 205 insertions(+) create mode 100644 packages/web/src/content/blog/open-agent-memory-format.mdx diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx new file mode 100644 index 000000000..49bc2d7f2 --- /dev/null +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -0,0 +1,199 @@ +--- +title: "Every agent tool is inventing its own memory format. What would an open one look like?" +description: "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool has its own shape for remembering things, and none of them can read the others. A curious tour of the formats out there, what they quietly agree on, why LoreKit's record looks the way it does, and how the same record carries a CI job's JSON state without a second schema." +date: "2026-11-07" +author: "Mads Thines" +readingMinutes: 11 +tags: ["agent-memory", "open-format", "standards", "ci", "lorekit"] +order: 5 +keywords: ["agent memory format", "open memory standard", "memory interoperability", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "CI state", "JSON state records", "lorekit"] +--- + +## Count the memory formats on your machine + +Try a small exercise. List every place your coding tools keep something they "remember". + +On my laptop the list goes something like this: an `AGENTS.md` and a `CLAUDE.md` in most repos, `.cursor/rules/*.mdc` in a few, a `/memories` directory an API agent writes to, the facts Copilot keeps for each repo on GitHub's side, a JSONL knowledge graph left over from trying the MCP reference memory server, and a `.lorekit/` folder. Six or seven shapes. **No two of them can read each other.** + +Each one is reasonable on its own. Taken together, they amount to an industry reinventing the same small idea over and over: *a note an agent left for its future self, addressed somehow, true for a while.* That shape keeps getting redefined from scratch, and I think it's worth asking what a shared version would need. + +I don't have a finished answer. What I have is a tour of the formats, a few patterns they quietly share, the reasoning behind the shape LoreKit settled on, and one test I think any open format should pass: can it hold memory that **no model wrote**? + +## A tour of the shapes + +Here are the families I've found, each in one line. These are all moving targets, so treat the details as a snapshot from late 2026. + +| Family | Example | The unit of memory | Addressed by | Time | +| --- | --- | --- | --- | --- | +| Instruction files | `AGENTS.md`, `CLAUDE.md`, Cursor `.mdc` rules | A whole Markdown file | File path (and `globs` for Cursor) | None. Lives until someone edits it | +| Agent-managed files | Anthropic's memory tool | A file under `/memories` | Path, chosen by the model | None | +| Hosted repo facts | GitHub Copilot Memory | A fact with code citations | One repository | Deleted after 28 days unused | +| Context blocks | Letta core memory | A labeled block with a character `limit` | `label` on an agent | None. Always in context | +| Extracted facts | Mem0 | A fact the service extracts from messages | `user_id` / `agent_id` / `run_id` / `app_id` | Optional expiration date | +| Namespaced store | LangGraph `BaseStore` | A JSON `value` dict | A `namespace` tuple + `key` | Optional, depends on the backend | +| Knowledge graph | MCP reference memory server | Entities, relations, observations | Entity name | None | +| Temporal graph | Graphiti / Zep | A fact on a graph edge | Entities + edge | Two timelines: when it was true, and when it was recorded | + +A few of these deserve a closer look, because each one made a choice the others didn't. + +**Instruction files are memory with a human in the loop.** `AGENTS.md` (now under the Linux Foundation's Agentic AI Foundation, next to MCP) is the nearest thing to a standard here, and the reason is simple: it's a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It has no idea of recurrence, no expiry, and no address finer than "this directory". + +**Anthropic's memory tool goes all in on files.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where that actually lives. No embeddings and no schema. That makes it very portable and very unstructured. Two agents writing to the same directory have no agreement about what a file *means*. + +**Copilot Memory validates against the code.** Every fact carries citations, meaning links to code locations, and Copilot checks them against the current branch before using the fact. Facts are scoped to one repository and go away after 28 days without use. I find that combination of *provenance you can check* and *forgetting by default* the most interesting design in the list. It's also locked inside one product. + +**Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the system prompt on every step. The format admits that memory isn't free: it costs tokens every turn. + +**LangGraph is nearly a key-value store.** It's `put(namespace, key, value)`, with the namespace as a tuple like `("memories", user_id)`. Of everything here, it's the closest relative of LoreKit's shape. The difference is that the namespace is a programming-language value, not a grammar you can write down in a URL or a filename. + +**The graphs model the world, not the work.** The MCP reference server stores entities, relations and observations in a single `memory.jsonl` file. Graphiti puts four timestamps on each edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?" Both are built for facts *about things*: people, companies, how one fact supersedes another. + +## What they quietly agree on + +Once you strip the branding, almost every format answers the same four questions. They just answer them differently: + +1. **Where is this true?** A path, a repo, a user ID, a namespace tuple, an entity. +2. **What is its identity?** Something that decides whether a second write is an *update* or a *duplicate*. +3. **What does it say?** Almost always plain text, or JSON when a program is the reader. +4. **How long is it true?** Often skipped. When it isn't, the answers range from "28 days unused" to "two full timelines". + +They disagree on **who writes**: a person (instruction files), the model directly (memory tool, Letta, LoreKit), or a service that extracts facts from a conversation (Mem0, Graphiti). That's a real fork. Extraction gives you memory without asking the agent for anything, and costs you a model call on every write plus a store you can't fully predict. + +They also disagree on the **body**: free text, a capped block, a JSON dict, or a triple. That one matters less than it seems, as the CI section below shows. + +## Why LoreKit's record looks the way it does + +I've written up [every column of the data model](/blog/agent-memory-data-model) before, so I won't repeat it here. What I find more interesting now is which of the four questions LoreKit answered *on purpose*, and what each answer borrows from or rejects in the formats above. + +### "Where" is a grammar, not a value + +```text +global +project::{name} +repo::{owner}/{repo} +branch::{owner}/{repo}::{branch} +``` + +LangGraph's tuple and LoreKit's scope do the same job. The difference is that a scope is a **string you can write down anywhere**: in a URL, a filename, a CLI flag, a YAML key, a log line. It has a fixed set of levels, a read order (most specific first), and a separator (`::`) chosen so it can't collide with the `/` in a repo path or the `:` in a branch name. A tuple only means something inside one language's runtime. A grammar means the same thing to a Deno edge function, a Node CLI and a `grep`. + +It's also coarser than a file path and finer than "one repo", which is where coding lessons actually live. "This branch's spike found X" and "this whole org uses Y" are both easy to say. + +### "Identity" is `scope` + `key`, and a rewrite is a signal + +Writing the same `scope` + `key` again updates the row and increments `seen_count`. Compare that with an append-only log of facts: if you re-learn the same thing five times, an append log holds five near-duplicates, while LoreKit holds one row that knows it was re-learned five times. Recurrence is the one salience signal you get for free from a deterministic store. + +Copilot gets *trust* from citations. LoreKit gets a weaker version of the same thing from provenance: `origin_repo`, `origin_branch`, `origin_commit` and `origin_pr` record where a lesson was learned. It doesn't re-check the code before use. If I were starting a standard from scratch, a citation field that readers *may* verify is the first thing I'd take from Copilot. + +### "What it says" is plain text, and advisory + +The `value` is text, at most 64 KiB, and the read path labels it *"considerations, not rules."* That's a deliberate line between LoreKit and instruction files. `AGENTS.md` holds what a human has decided. Lore holds what an agent has *noticed*, and it should be easy to ignore when it's wrong. Keeping both in one file is how instruction files grow into an unreviewed pile of half-truths. + +### "How long" is a TTL that resets on write + +`ttl_days` sets `expires_at`, and a re-write restarts the countdown. So a lesson that keeps getting re-learned stays alive, and one nobody touches anymore fades out. It's the same intuition as Copilot's "28 days unused", reached independently, which is either a nice convergence or a sign the idea is obvious. Either way, it's a hint that **forgetting by default** belongs in any shared format. + +What LoreKit doesn't do is Graphiti's two timelines. A coding lesson like "the integration tests need Postgres up first" is rarely a historical fact you'd ask about at a point in time. It's either still useful or it isn't. I'm not certain that holds forever, and it's one of the open questions below. + +### "What kind" is a small, closed vocabulary + +`kind` is one of `lesson`, `bus` or `signal`, and `host` names the loop that wrote the record. The list is closed on purpose: adding a kind is a schema change, because each kind is read on a different schedule. A `lesson` is read every run. A `bus` event is a transient outcome, read only when deciding what to promote. A `signal` is a durable per-repo filter. None of the other formats separates these, and without that separation a store fills up with records that crowd each other out of the context window. + +### On disk, it's a file you can `cat` + +Offline, a record is one Markdown file per `scope` + `key`, with frontmatter that mirrors the hosted row column for column. Here's one, trimmed (the real file also lists the empty provenance fields as `null`): + +```markdown +--- +scope: "repo::acme/checkout" +key: "tests-need-local-postgres" +tags: ["ci","gotcha"] +created: "2026-10-02T09:14:00.000Z" +updated: "2026-11-04T16:41:12.000Z" +origin_commit: "0f4a1c9" +expires_at: "2026-12-04T16:41:12.000Z" +seen_count: 4 +kind: "lesson" +host: null +--- +Integration tests need Postgres up first: docker compose up -d db. +``` + +Every scalar in the frontmatter is JSON-encoded. That's a strict subset of YAML, so the file is still valid YAML, still greppable, and parses with no YAML library at all. The parser ignores fields it doesn't recognise and treats missing ones as absent, which is how a file written before `seen_count` existed still reads fine today. That forward tolerance matters more for an open format than any single field does. + +It's also close enough to a memory-tool file or an `AGENTS.md` that the formats could meet halfway: frontmatter for the address and lifecycle, a Markdown body for the note. + +## The test: memory no model wrote + +Here's the part I find most convincing, and it came from somewhere I didn't plan for. + +CI jobs need memory too. A test workflow wants to know which tests flaked last time. A deploy job wants the last SHA it shipped. A review bot wants to know what it already commented on. None of that involves a model, and the usual answers are `actions/cache` or an artifact bucket nobody else can read. + +LoreKit handles this with the **same record**. The only difference is the body: a versioned JSON envelope instead of prose. + +```json +{ + "v": 1, + "updated_by_run": "https://github.com/acme/checkout/actions/runs/123456789", + "commit": "0f4a1c9", + "data": { + "flaky": ["src/queue.test.ts::retries on 429"], + "consecutive_green": 3 + } +} +``` + +It's written with the CLI, piped straight from `jq`: + +```bash +jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ + '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \ +| npx @lorekit/cli write \ + --scope "repo::${GITHUB_REPOSITORY}" --key 'ci-state::flaky-tests' \ + --tags 'ci::test-state' --kind bus --host ci \ + --ttl-days 7 --remote --json +``` + +Every concept from the lesson record maps across without a new field: + +- **Scope** is the same. Trunk state lives at `repo::`, and per-PR state at `branch::`, so it goes away with the branch. +- **Identity** is the same, but the discipline is stricter: *one key per fact, overwritten in place, never one key per run.* Because same `scope` + `key` is an update, a job that runs a thousand times still holds one row. Keying on the run ID would turn a memory store into a slow artifact bucket. +- **Kind** keeps it out of the way. `--kind bus` keeps the record out of the lessons an agent sees at session start, so a JSON blob never competes with a lesson for context. +- **TTL changes meaning, nicely.** The countdown restarts on every write, so a 7-day TTL on a daily job doesn't measure how *old* the record is. It measures how long the job has been *silent*. The state lives as long as the job keeps running, and disappears a week after someone deletes the workflow. A stale flaky-test list is worse than none, so letting it expire is the safe default. +- **Version the body, not the schema.** `v` is the reader's job. A reader that sees a version it doesn't know falls back to its first-run path instead of guessing. + +The payoff is the reason to do this instead of `actions/cache`: **the agent and the job read the same store.** CI writes `ci-state::flaky-tests`. An agent asked to "fix the flaky tests" reads that exact key in the same repo scope and starts from the real list. Its findings go back as a *lesson* next to it. The state record says what's true right now, and the lesson says what we learned about it. Two kinds of record in one shape, with no import step between them. + +That's why I think it's a good test for an open format. If the shape only fits prose written by a model, it's a chat feature. If it can also hold a JSON record a shell script wrote, with the same address, identity and lifecycle, it starts to look like infrastructure. (The [full CI recipe](/docs/use-cases) covers tokens, guards, and why the store must never be on a build's critical path.) + +## What an open format might need + +Here's my current guess at the smallest core, written down so people can argue with it: + +- **An address grammar.** A string with defined levels and a read order. Not a path, and not a language-specific tuple. +- **An identity rule.** Address + key is unique, and a rewrite is an update that's *counted*. +- **A body.** Text by default. JSON allowed when the reader is a program, versioned inside the body. +- **A lifecycle.** Created, updated, an optional expiry that resets on write, and soft deletion. +- **Provenance.** Where the record was learned, and ideally *citations a reader may verify*. +- **A kind.** Enough to tell a lesson from an event from a filter, so readers can budget for each. +- **Forward tolerance.** Readers ignore unknown fields and treat missing ones as absent. + +Everything else should stay free to vary: storage engine, ranking, whether you add embeddings on top, whether extraction or the agent does the writing. OpenTelemetry worked because it standardised the *shape of a span*, not anyone's backend. I spend my days at [Dash0](https://www.dash0.com) watching what that one agreement made possible. + +And the questions I genuinely don't know the answers to: + +- **Is the four-level scope set right?** Monorepos want something between `repo` and `branch`. Orgs want something above `project`. +- **Do coding lessons need two timelines?** Graphiti's argument is strong for facts about the world. Is there a coding case where "what was true in March" matters? +- **Should citations be in the core?** Copilot's check-before-use is the best guard against stale memory I've seen. It also ties a record to a code host. +- **Who should write?** The agent, explicitly, or an extractor watching the conversation? A format can probably allow both, but the trust model is very different. + +The schema, the scope validator and the on-disk format are all open in the repo. If you're building one of the formats in the table, I'd love to hear where this shape breaks for you. A standard nobody argued about isn't one. + +## See the files for yourself + +The quickest way to decide whether this shape is any good is to look at the files it writes: + +```bash +npx @lorekit/cli install +``` + +It creates a folder of plain Markdown records you can open, grep and diff. Then tell me what's missing. diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index be2e0f48b..a5ee31435 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -44,6 +44,12 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ summary: "Everyone reached for vector databases. But most of what a coding agent needs to remember is small, specific, and recurring — a scoped note, not a semantic blob. Here's the deterministic, no-AI data model LoreKit runs on, the rationale for every column, and an invitation to converge on a shared shape.", }, + { + id: 'open-agent-memory-format', + label: 'Every agent tool is inventing its own memory format. What would an open one look like?', + summary: + "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool has its own shape for remembering things, and none of them can read the others. A curious tour of the formats out there, what they quietly agree on, why LoreKit's record looks the way it does, and how the same record carries a CI job's JSON state without a second schema.", + }, ]; /** URL slugs / MDX filename stems, in listing order (newest first). */ From 53de73dcb9e0e5891bf8b583f5000a0d344ec039 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:08:33 +0000 Subject: [PATCH 02/17] blog: correct and strengthen the open memory format post Fix claims that did not match the code: TTL is opt-in and only a write carrying a TTL resets it (LoreKit is permanent by default), rewrites drop unknown frontmatter fields, kind is null unless inferred, and the :: rationale (git forbids ':' in branch names). Add the existing portability proposals (Portable Memory, Open Memory Protocol, COGX), LangGraph's read-refreshed TTL, the GitHub-shaped repo grammar, and a section answering the objections a skeptic would raise. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../content/blog/open-agent-memory-format.mdx | 188 +++++++++++------- packages/web/src/lib/blog/sections.ts | 4 +- 2 files changed, 119 insertions(+), 73 deletions(-) diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index 49bc2d7f2..decdee9d2 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -1,69 +1,81 @@ --- -title: "Every agent tool is inventing its own memory format. What would an open one look like?" -description: "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool has its own shape for remembering things, and none of them can read the others. A curious tour of the formats out there, what they quietly agree on, why LoreKit's record looks the way it does, and how the same record carries a CI job's JSON state without a second schema." +title: "Every agent tool invents its own memory format. What would an open one look like?" +description: "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool stores memory in its own shape, and none can read the others. A tour of the formats, what they quietly agree on, the choices behind LoreKit's record (including the ones I'm unsure of), how the same record carries a CI job's JSON state, and what a skeptic would say about all of it." date: "2026-11-07" author: "Mads Thines" -readingMinutes: 11 +readingMinutes: 12 tags: ["agent-memory", "open-format", "standards", "ci", "lorekit"] order: 5 -keywords: ["agent memory format", "open memory standard", "memory interoperability", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "CI state", "JSON state records", "lorekit"] +keywords: ["agent memory format", "open memory standard", "memory interoperability", "memory portability", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "Portable Memory", "Open Memory Protocol", "CI state", "JSON state records", "lorekit"] --- ## Count the memory formats on your machine -Try a small exercise. List every place your coding tools keep something they "remember". +Here's a small exercise. List every place your coding tools keep something they "remember". -On my laptop the list goes something like this: an `AGENTS.md` and a `CLAUDE.md` in most repos, `.cursor/rules/*.mdc` in a few, a `/memories` directory an API agent writes to, the facts Copilot keeps for each repo on GitHub's side, a JSONL knowledge graph left over from trying the MCP reference memory server, and a `.lorekit/` folder. Six or seven shapes. **No two of them can read each other.** +A typical setup today looks something like this. There's an `AGENTS.md` or `CLAUDE.md` in most repos, plus `.cursor/rules/*.mdc` in a few. An API agent writes files to a `/memories` directory. Copilot keeps facts about each repo on GitHub's side. A memory MCP server keeps a JSONL knowledge graph. That's five or six shapes, and **none of them can read the others**. -Each one is reasonable on its own. Taken together, they amount to an industry reinventing the same small idea over and over: *a note an agent left for its future self, addressed somehow, true for a while.* That shape keeps getting redefined from scratch, and I think it's worth asking what a shared version would need. +Each one is reasonable on its own. Together, they're an industry rebuilding the same small idea again and again: *a note an agent left for its future self, stored at some address, true for a while.* -I don't have a finished answer. What I have is a tour of the formats, a few patterns they quietly share, the reasoning behind the shape LoreKit settled on, and one test I think any open format should pass: can it hold memory that **no model wrote**? +I build one of these formats, so take this with that in mind. Here's what this post covers: + +- a tour of the formats out there +- what they quietly agree on +- why LoreKit's record looks the way it does, including the parts I'm not sure about +- one test I think any open format should pass: can it hold memory that **no model wrote**? +- what a skeptic would say about all of this ## A tour of the shapes -Here are the families I've found, each in one line. These are all moving targets, so treat the details as a snapshot from late 2026. +Here are the families I've found. They all change quickly, so treat the details as a snapshot from late 2026. -| Family | Example | The unit of memory | Addressed by | Time | +| Family | Example | The unit of memory | Addressed by | Forgetting | | --- | --- | --- | --- | --- | -| Instruction files | `AGENTS.md`, `CLAUDE.md`, Cursor `.mdc` rules | A whole Markdown file | File path (and `globs` for Cursor) | None. Lives until someone edits it | -| Agent-managed files | Anthropic's memory tool | A file under `/memories` | Path, chosen by the model | None | +| Instruction files | `AGENTS.md`, `CLAUDE.md`, Cursor `.mdc` rules | A whole Markdown file | File path (plus `globs` for Cursor) | Never. A person edits it | +| Agent-managed files | Anthropic's memory tool | A file under `/memories` | A path the model picks | Never, unless the model deletes it | | Hosted repo facts | GitHub Copilot Memory | A fact with code citations | One repository | Deleted after 28 days unused | -| Context blocks | Letta core memory | A labeled block with a character `limit` | `label` on an agent | None. Always in context | +| Context blocks | Letta core memory | A labeled block with a character `limit` | A `label` on an agent | Never. Always in context | | Extracted facts | Mem0 | A fact the service extracts from messages | `user_id` / `agent_id` / `run_id` / `app_id` | Optional expiration date | -| Namespaced store | LangGraph `BaseStore` | A JSON `value` dict | A `namespace` tuple + `key` | Optional, depends on the backend | -| Knowledge graph | MCP reference memory server | Entities, relations, observations | Entity name | None | -| Temporal graph | Graphiti / Zep | A fact on a graph edge | Entities + edge | Two timelines: when it was true, and when it was recorded | +| Namespaced store | LangGraph `BaseStore` | A JSON `value` dict | A `namespace` tuple + `key` | Optional TTL, reset when the item is read (by default) | +| Knowledge graph | MCP reference memory server | Entities, relations, observations | Entity name | Never | +| Temporal graph | Graphiti / Zep | A fact on a graph edge | Entities + edge | Facts are invalidated rather than deleted, on two timelines | + +Several of these made a choice the others didn't, and those choices are worth looking at. + +**Instruction files keep a person in the loop.** `AGENTS.md` now sits under the Linux Foundation's Agentic AI Foundation, next to MCP. It's the closest thing to a standard here because it's simply a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It doesn't track how often something recurs, nothing in it expires, and it can't target anything finer than a directory. -A few of these deserve a closer look, because each one made a choice the others didn't. +**Anthropic's memory tool goes all in on files.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where the files actually live. There are no embeddings and no schema. That makes it very portable and completely unstructured: two agents writing to the same directory have no shared idea of what a file *means*. -**Instruction files are memory with a human in the loop.** `AGENTS.md` (now under the Linux Foundation's Agentic AI Foundation, next to MCP) is the nearest thing to a standard here, and the reason is simple: it's a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It has no idea of recurrence, no expiry, and no address finer than "this directory". +**Copilot Memory checks itself against the code.** Every fact carries citations, which are links to specific code locations. Before using a fact, Copilot checks those citations against the current branch. Facts belong to one repository and disappear after 28 days without use. To me, that pairing of *provenance you can check* and *forgetting by default* is the most interesting design in this list. It's also locked inside one product. -**Anthropic's memory tool goes all in on files.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where that actually lives. No embeddings and no schema. That makes it very portable and very unstructured. Two agents writing to the same directory have no agreement about what a file *means*. +**Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the system prompt on every step. The format is honest about the cost: memory isn't free, because you pay for it in tokens every turn. -**Copilot Memory validates against the code.** Every fact carries citations, meaning links to code locations, and Copilot checks them against the current branch before using the fact. Facts are scoped to one repository and go away after 28 days without use. I find that combination of *provenance you can check* and *forgetting by default* the most interesting design in the list. It's also locked inside one product. +**LangGraph is nearly a key-value store.** Its interface is `put(namespace, key, value)`, where the namespace is a tuple such as `("memories", user_id)`. Its TTL can reset whenever an item is read. Of everything here, it's the closest relative of LoreKit's shape. -**Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the system prompt on every step. The format admits that memory isn't free: it costs tokens every turn. +**The graphs model the world, not the work.** The MCP reference server keeps entities, relations and observations in a single `memory.jsonl` file. Graphiti stores four timestamps on every edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?". Both are built for facts *about things*: people, companies, and how a newer fact replaces an older one. -**LangGraph is nearly a key-value store.** It's `put(namespace, key, value)`, with the namespace as a tuple like `("memories", user_id)`. Of everything here, it's the closest relative of LoreKit's shape. The difference is that the namespace is a programming-language value, not a grammar you can write down in a URL or a filename. +### And yes, people are already proposing standards -**The graphs model the world, not the work.** The MCP reference server stores entities, relations and observations in a single `memory.jsonl` file. Graphiti puts four timestamps on each edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?" Both are built for facts *about things*: people, companies, how one fact supersedes another. +I'm not the first to notice this. MacPaw Research has proposed **Portable Memory**, a `.mem` bundle with byte-reproducible serialization, deletion tombstones and optional signing. Its own authors say it's not yet widely adopted. The **Open Memory Protocol** is an April 2026 draft for moving conversations and the memories derived from them between platforms, as a ZIP file with a manifest. Cognee offers a **COGX** exchange format with importers from Mem0, Letta and Zep. + +These are worth reading. They have one thing in common: they solve **export**. They answer "how do I move my memory from vendor A to vendor B?" That's a real problem, mostly for personal assistants. The problem I care about is slightly different: **live addressing**. When a CI job, an editor agent and a review bot all work in the same repo at the same moment, how do they write to, and read from, the *same* place? A bundle format doesn't answer that, and an address format doesn't replace a bundle. The two could fit together. ## What they quietly agree on -Once you strip the branding, almost every format answers the same four questions. They just answer them differently: +Strip away the branding and almost every format answers the same four questions, just in different ways: 1. **Where is this true?** A path, a repo, a user ID, a namespace tuple, an entity. 2. **What is its identity?** Something that decides whether a second write is an *update* or a *duplicate*. 3. **What does it say?** Almost always plain text, or JSON when a program is the reader. -4. **How long is it true?** Often skipped. When it isn't, the answers range from "28 days unused" to "two full timelines". +4. **When does it stop being true?** Often skipped. When it isn't, the answers range from "28 days unused" to "two full timelines". -They disagree on **who writes**: a person (instruction files), the model directly (memory tool, Letta, LoreKit), or a service that extracts facts from a conversation (Mem0, Graphiti). That's a real fork. Extraction gives you memory without asking the agent for anything, and costs you a model call on every write plus a store you can't fully predict. +They also split on a fifth question: **who writes?** In instruction files it's a person. In the memory tool, Letta and LoreKit, the model writes directly. In Mem0 and Graphiti, a service extracts facts from the conversation. That's a real fork. Extraction gives you memory without asking the agent to do anything. In exchange, it costs a model call on every write and produces a store whose contents you can't fully predict. -They also disagree on the **body**: free text, a capped block, a JSON dict, or a triple. That one matters less than it seems, as the CI section below shows. +They disagree on the **body** too: free text, a capped block, a JSON dict, or a triple. That matters less than it looks, as the CI section below shows. ## Why LoreKit's record looks the way it does -I've written up [every column of the data model](/blog/agent-memory-data-model) before, so I won't repeat it here. What I find more interesting now is which of the four questions LoreKit answered *on purpose*, and what each answer borrows from or rejects in the formats above. +I've written up [every column of the data model](/blog/agent-memory-data-model) before, so I won't repeat that here. What's more useful is to go through the four questions and show where LoreKit borrowed from the formats above, where it went another way, and where I'm not sure it got it right. ### "Where" is a grammar, not a value @@ -74,33 +86,41 @@ repo::{owner}/{repo} branch::{owner}/{repo}::{branch} ``` -LangGraph's tuple and LoreKit's scope do the same job. The difference is that a scope is a **string you can write down anywhere**: in a URL, a filename, a CLI flag, a YAML key, a log line. It has a fixed set of levels, a read order (most specific first), and a separator (`::`) chosen so it can't collide with the `/` in a repo path or the `:` in a branch name. A tuple only means something inside one language's runtime. A grammar means the same thing to a Deno edge function, a Node CLI and a `grep`. +LangGraph's tuple and LoreKit's scope do the same job. The difference is that a scope is a **string you can write down anywhere**: a URL, a filename, a CLI flag, a YAML key, a log line. It has fixed levels and a fixed read order, with the most specific level read first. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name, so `::` can't be confused with either. A tuple only means something inside one language's runtime. A grammar means the same thing to a Deno edge function, a Node CLI and `grep`. + +It's coarser than a file path and finer than "one repo", which is where coding lessons tend to live. "This branch's spike found X" and "this whole project uses Y" are both easy to express. -It's also coarser than a file path and finer than "one repo", which is where coding lessons actually live. "This branch's spike found X" and "this whole org uses Y" are both easy to say. +It's also clearly shaped by GitHub. `repo::` accepts exactly `owner/repo`, so a GitLab subgroup path like `group/subgroup/repo` doesn't fit today. A standard couldn't keep that. -### "Identity" is `scope` + `key`, and a rewrite is a signal +### "Identity" is `scope` + `key`, and a rewrite is counted -Writing the same `scope` + `key` again updates the row and increments `seen_count`. Compare that with an append-only log of facts: if you re-learn the same thing five times, an append log holds five near-duplicates, while LoreKit holds one row that knows it was re-learned five times. Recurrence is the one salience signal you get for free from a deterministic store. +Writing the same `scope` + `key` again updates the existing row and increments `seen_count`. Compare that with an append-only log: if you re-learn something five times, the log holds five near-duplicates, while LoreKit holds one row that knows it was learned five times. A deterministic store gets this salience signal for free, without any model. -Copilot gets *trust* from citations. LoreKit gets a weaker version of the same thing from provenance: `origin_repo`, `origin_branch`, `origin_commit` and `origin_pr` record where a lesson was learned. It doesn't re-check the code before use. If I were starting a standard from scratch, a citation field that readers *may* verify is the first thing I'd take from Copilot. +To be precise about what it measures: `seen_count` tells you how often something came up. It doesn't tell you whether it's true. A wrong lesson that's re-learned five times is still wrong, just louder. That's why the next two choices exist. -### "What it says" is plain text, and advisory +Copilot gets *trust* from citations. LoreKit gets a weaker version of that from provenance: `origin_repo`, `origin_branch`, `origin_commit` and `origin_pr` record where a lesson was learned, but nothing re-checks the code before the lesson is used. If I were starting a standard from scratch, a citation field that readers *may* verify is the first thing I'd borrow from Copilot. -The `value` is text, at most 64 KiB, and the read path labels it *"considerations, not rules."* That's a deliberate line between LoreKit and instruction files. `AGENTS.md` holds what a human has decided. Lore holds what an agent has *noticed*, and it should be easy to ignore when it's wrong. Keeping both in one file is how instruction files grow into an unreviewed pile of half-truths. +### "What it says" is plain text, and only advisory -### "How long" is a TTL that resets on write +The `value` is text, capped at 64 KiB. When LoreKit puts lessons into an agent's context, it labels them *"considerations, not rules."* That's a deliberate line between lore and instruction files. `AGENTS.md` holds what a person has decided. Lore holds what an agent has *noticed*, and it should be easy to ignore when it's wrong. Mix the two in one file and instruction files turn into an unreviewed pile of half-truths. -`ttl_days` sets `expires_at`, and a re-write restarts the countdown. So a lesson that keeps getting re-learned stays alive, and one nobody touches anymore fades out. It's the same intuition as Copilot's "28 days unused", reached independently, which is either a nice convergence or a sign the idea is obvious. Either way, it's a hint that **forgetting by default** belongs in any shared format. +### "When it stops being true" is opt-in, and that's open to debate -What LoreKit doesn't do is Graphiti's two timelines. A coding lesson like "the integration tests need Postgres up first" is rarely a historical fact you'd ask about at a point in time. It's either still useful or it isn't. I'm not certain that holds forever, and it's one of the open questions below. +A write can carry `ttl_days`, which sets `expires_at`. Every later write that also carries a TTL restarts the countdown. A write *without* one leaves the existing expiry alone. A lesson written without any TTL never expires, unless a repo or user config sets a default for its scope. -### "What kind" is a small, closed vocabulary +So LoreKit is **permanent by default, with forgetting as an opt-in**. Copilot is the opposite: it forgets by default, after 28 days unused. LangGraph sits in between, with an optional TTL that resets when the item is *read*. LoreKit deliberately never lets a read change a record, so only a write can extend a lesson's life. Old lessons are handled by grooming instead: a review pass that suggests merges, expiries and archives before anything changes. -`kind` is one of `lesson`, `bus` or `signal`, and `host` names the loop that wrote the record. The list is closed on purpose: adding a kind is a schema change, because each kind is read on a different schedule. A `lesson` is read every run. A `bus` event is a transient outcome, read only when deciding what to promote. A `signal` is a durable per-repo filter. None of the other formats separates these, and without that separation a store fills up with records that crowd each other out of the context window. +I go back and forth on which default is right. Forgetting by default keeps a store honest without anyone tending it. Permanent by default means you don't lose the hard-won lesson about the bug that only shows up once a quarter. A shared format should probably let both exist and make the choice visible on each record, rather than settle it once for everyone. + +What LoreKit doesn't do at all is keep Graphiti's two timelines. A coding lesson like "the integration tests need Postgres up first" is rarely a historical fact you'd want to look up as of a past date. It's either still useful or it isn't. I'm not sure that holds forever, and it's one of the open questions at the end. + +### "What kind" is a short, closed list + +`kind` is one of `lesson`, `bus` or `signal`, and `host` names the loop that wrote the record. Each kind is read on a different schedule. A `lesson` is read every run. A `bus` record is a transient outcome, read only when deciding what to promote to a permanent rule. A `signal` is a durable filter for one repo. The list is closed on purpose, so adding a kind means changing the schema. None of the other formats makes this split, and without it, a store fills up with records that push each other out of the context window. ### On disk, it's a file you can `cat` -Offline, a record is one Markdown file per `scope` + `key`, with frontmatter that mirrors the hosted row column for column. Here's one, trimmed (the real file also lists the empty provenance fields as `null`): +In the offline store, each record is one Markdown file per `scope` + `key`, and its frontmatter mirrors the columns of the hosted database row. Here's one, trimmed (the real file also lists the empty provenance fields as `null`): ```markdown --- @@ -112,23 +132,25 @@ updated: "2026-11-04T16:41:12.000Z" origin_commit: "0f4a1c9" expires_at: "2026-12-04T16:41:12.000Z" seen_count: 4 -kind: "lesson" +kind: null host: null --- Integration tests need Postgres up first: docker compose up -d db. ``` -Every scalar in the frontmatter is JSON-encoded. That's a strict subset of YAML, so the file is still valid YAML, still greppable, and parses with no YAML library at all. The parser ignores fields it doesn't recognise and treats missing ones as absent, which is how a file written before `seen_count` existed still reads fine today. That forward tolerance matters more for an open format than any single field does. +Every value in the frontmatter is JSON-encoded. JSON is a strict subset of YAML, so the file is still valid YAML, still easy to grep, and LoreKit reads it with a few lines of code and no YAML library. -It's also close enough to a memory-tool file or an `AGENTS.md` that the formats could meet halfway: frontmatter for the address and lifecycle, a Markdown body for the note. +When it reads a file, LoreKit is tolerant: a field missing from an old file counts as absent, which is why files written before `seen_count` existed still load. But it isn't yet *lossless*. Rewriting a record keeps only the fields LoreKit knows about, so a field some other tool added would be dropped. That's exactly the kind of gap a shared format has to close: **preserve what you don't understand**. + +The shape is also close enough to a memory-tool file or an `AGENTS.md` that these formats could meet halfway. The frontmatter would carry the address and the lifecycle, and the Markdown body would carry the note. ## The test: memory no model wrote -Here's the part I find most convincing, and it came from somewhere I didn't plan for. +This part came from somewhere I didn't plan, and it's the part I find most convincing. -CI jobs need memory too. A test workflow wants to know which tests flaked last time. A deploy job wants the last SHA it shipped. A review bot wants to know what it already commented on. None of that involves a model, and the usual answers are `actions/cache` or an artifact bucket nobody else can read. +CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. A review bot wants to know what it already commented on. None of that involves a model. The usual answers are `actions/cache`, or an artifact bucket nobody else reads. -LoreKit handles this with the **same record**. The only difference is the body: a versioned JSON envelope instead of prose. +LoreKit stores this in the **same record**. The only difference is the body: a versioned JSON envelope instead of prose. ```json { @@ -142,9 +164,11 @@ LoreKit handles this with the **same record**. The only difference is the body: } ``` -It's written with the CLI, piped straight from `jq`: +The job writes it with the CLI, piped straight from `jq`: ```bash +RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" + jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \ | npx @lorekit/cli write \ @@ -153,47 +177,69 @@ jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ --ttl-days 7 --remote --json ``` -Every concept from the lesson record maps across without a new field: +Every concept from the lesson record carries over, with no new fields: + +- **Scope works the same way.** State for the main branch lives at `repo::`. State for a single PR lives at `branch::`, so it goes away when the branch does. +- **Identity works the same way, with a stricter rule: one key per fact, overwritten in place, never one key per run.** Because the same `scope` + `key` is an update, a job that runs a thousand times still holds one row. A key per run ID would turn a memory store into a slow artifact bucket. +- **`kind` keeps the record out of an agent's way.** A record with `kind: bus` is left out of the lessons an agent is shown at the start of a session, so a JSON blob never competes with real lessons for context. +- **The TTL becomes a liveness check.** The job passes `--ttl-days 7` on every run, so every run restarts the countdown. The expiry then measures how long the job has been *silent*, not how old the record is. The state lasts as long as the job keeps running, and disappears a week after someone deletes the workflow. A stale flaky-test list is worse than none, so this is the right failure mode. +- **The body carries its own version.** Checking `v` is the reader's job. A reader that sees a version it doesn't know falls back to its first-run behaviour instead of guessing. + +The payoff, and the reason to do this instead of `actions/cache`, is that **the agent and the job read the same store**. CI writes `ci-state::flaky-tests`. An agent asked to "fix the flaky tests" reads that exact key in the same repo scope and starts from the real list. The agent's findings go back as a *lesson* right next to it. The state record says what's true right now, and the lesson says what was learned about it. These are two kinds of record in one shape, with no import step between them. + +That's why I think it's a good test for an open format. A shape that only fits prose written by a model is a chat feature. A shape that also holds JSON written by a shell script, with the same address, identity and lifecycle, is infrastructure. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never sit on a build's critical path.) + +## What a skeptic would say + +These are the objections I'd raise if someone else had written this post, with my best answer to each. + +**"This is just your product's schema dressed up as a standard."** That's fair, and it's why I've tried to separate the core from LoreKit's extras below. LoreKit's job is to show that a shape like this holds up in daily use. It's not a claim that the shape should win. If an existing proposal adopted a scope grammar and the four core fields, I'd happily move LoreKit onto it. + +**"MCP is already the standard."** MCP standardises the *call*, not the *record*. Two memory servers that both speak MCP can expose completely different tools, with different arguments and different meanings: the reference server's `create_entities`, LoreKit's `memory.write`, and dozens of others. The agent can call any of them, but no two of them agree on what a memory is. + +**"Just use files in git."** For one person and one tool, I mostly agree, and LoreKit's offline store *is* files. What plain files don't give you is agreement on what a file *means*: what it applies to, whether it repeats an earlier lesson, and when it stops being true. Frontmatter is where that agreement would live. + +**"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and a shared store widens who gets to write those things. LoreKit's partial defences are: + +- lore is labeled advisory +- every record carries provenance, so you can see where it came from +- tokens can be write-only or read-only, so a CI job's write-only token can't read the team's lore back out -- **Scope** is the same. Trunk state lives at `repo::`, and per-PR state at `branch::`, so it goes away with the branch. -- **Identity** is the same, but the discipline is stricter: *one key per fact, overwritten in place, never one key per run.* Because same `scope` + `key` is an update, a job that runs a thousand times still holds one row. Keying on the run ID would turn a memory store into a slow artifact bucket. -- **Kind** keeps it out of the way. `--kind bus` keeps the record out of the lessons an agent sees at session start, so a JSON blob never competes with a lesson for context. -- **TTL changes meaning, nicely.** The countdown restarts on every write, so a 7-day TTL on a daily job doesn't measure how *old* the record is. It measures how long the job has been *silent*. The state lives as long as the job keeps running, and disappears a week after someone deletes the workflow. A stale flaky-test list is worse than none, so letting it expire is the safe default. -- **Version the body, not the schema.** `v` is the reader's job. A reader that sees a version it doesn't know falls back to its first-run path instead of guessing. +None of that makes poisoning impossible. A standard would need a trust story, and citations plus signing (which Portable Memory already proposes) are the obvious starting points. -The payoff is the reason to do this instead of `actions/cache`: **the agent and the job read the same store.** CI writes `ci-state::flaky-tests`. An agent asked to "fix the flaky tests" reads that exact key in the same repo scope and starts from the real list. Its findings go back as a *lesson* next to it. The state record says what's true right now, and the lesson says what we learned about it. Two kinds of record in one shape, with no import step between them. +**"Without embeddings, you'll miss everything that's phrased differently."** Sometimes, yes. The [data-model post](/blog/agent-memory-data-model) makes the case for structure first. The point here is narrower: a shared format shouldn't *require* vectors. An index is something a reader can build on top of the records, not something stored in them. -That's why I think it's a good test for an open format. If the shape only fits prose written by a model, it's a chat feature. If it can also hold a JSON record a shell script wrote, with the same address, identity and lifecycle, it starts to look like infrastructure. (The [full CI recipe](/docs/use-cases) covers tokens, guards, and why the store must never be on a build's critical path.) +**"Another standard won't fix fragmentation."** That's the xkcd 927 objection: try to unify everything and you end up with one more competing standard. It's the strongest one. The honest answer is that I'd rather add an **address and lifecycle** to formats that already exist than create a new container. That's a smaller ask, and a scope string fits in a frontmatter field, a JSON key, or an `.omp` memory record equally well. ## What an open format might need -Here's my current guess at the smallest core, written down so people can argue with it: +Here's my current guess at the smallest useful core, written down so people can argue with it: -- **An address grammar.** A string with defined levels and a read order. Not a path, and not a language-specific tuple. -- **An identity rule.** Address + key is unique, and a rewrite is an update that's *counted*. -- **A body.** Text by default. JSON allowed when the reader is a program, versioned inside the body. -- **A lifecycle.** Created, updated, an optional expiry that resets on write, and soft deletion. -- **Provenance.** Where the record was learned, and ideally *citations a reader may verify*. +- **An address grammar.** A string with defined levels and a defined read order. Not a file path, and not a tuple tied to one language. It should handle more than GitHub's `owner/repo`. +- **An identity rule.** Address plus key is unique, and a rewrite updates the record and is *counted*. +- **A body.** Text by default. JSON is allowed when the reader is a program, with the version stamped inside the body. +- **A lifecycle.** Created and updated times, an optional expiry with an explicit rule for what extends it, and soft deletion. +- **Provenance.** Where a record was learned, ideally with citations a reader *may* verify. - **A kind.** Enough to tell a lesson from an event from a filter, so readers can budget for each. -- **Forward tolerance.** Readers ignore unknown fields and treat missing ones as absent. +- **Lossless round-trips.** Readers keep fields they don't understand when they rewrite a record. LoreKit doesn't do this yet. -Everything else should stay free to vary: storage engine, ranking, whether you add embeddings on top, whether extraction or the agent does the writing. OpenTelemetry worked because it standardised the *shape of a span*, not anyone's backend. I spend my days at [Dash0](https://www.dash0.com) watching what that one agreement made possible. +Everything else should stay free to vary: the storage engine, the ranking, whether you add embeddings on top, and whether an extractor or the agent does the writing. OpenTelemetry succeeded because it standardised the data model and the wire format, not anyone's backend. I work with it every day at [Dash0](https://www.dash0.com), and I've seen how much one shared shape makes possible. -And the questions I genuinely don't know the answers to: +And the questions I honestly can't answer yet: -- **Is the four-level scope set right?** Monorepos want something between `repo` and `branch`. Orgs want something above `project`. +- **Are four scope levels right?** Monorepos want a level between `repo` and `branch`. Organisations want one above `project`. +- **Should forgetting be the default?** Copilot says yes, LoreKit says no, and LangGraph extends a record's life whenever it's read. Which of those should a shared format assume? - **Do coding lessons need two timelines?** Graphiti's argument is strong for facts about the world. Is there a coding case where "what was true in March" matters? -- **Should citations be in the core?** Copilot's check-before-use is the best guard against stale memory I've seen. It also ties a record to a code host. -- **Who should write?** The agent, explicitly, or an extractor watching the conversation? A format can probably allow both, but the trust model is very different. +- **Should citations be part of the core?** Copilot's check-before-use is the best defence against stale memory I've seen, but it ties a record to a code host. -The schema, the scope validator and the on-disk format are all open in the repo. If you're building one of the formats in the table, I'd love to hear where this shape breaks for you. A standard nobody argued about isn't one. +The schema, the scope validator and the on-disk format are all open in the repo. If you build one of the formats in the table above, I'd especially like to hear where this shape breaks for you. ## See the files for yourself -The quickest way to decide whether this shape is any good is to look at the files it writes: +The quickest way to judge whether this shape is any good is to look at the files it writes: ```bash npx @lorekit/cli install ``` -It creates a folder of plain Markdown records you can open, grep and diff. Then tell me what's missing. +That gives you a folder of plain Markdown records you can open, grep and diff. Then tell me what's missing. diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index a5ee31435..a043a4b17 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -46,9 +46,9 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ }, { id: 'open-agent-memory-format', - label: 'Every agent tool is inventing its own memory format. What would an open one look like?', + label: 'Every agent tool invents its own memory format. What would an open one look like?', summary: - "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool has its own shape for remembering things, and none of them can read the others. A curious tour of the formats out there, what they quietly agree on, why LoreKit's record looks the way it does, and how the same record carries a CI job's JSON state without a second schema.", + "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool stores memory in its own shape, and none can read the others. A tour of the formats, what they quietly agree on, the choices behind LoreKit's record (including the ones I'm unsure of), how the same record carries a CI job's JSON state, and what a skeptic would say about all of it.", }, ]; From 6994010f6d82aacaceb79a9d6bf4c06c26c83773 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:20:25 +0000 Subject: [PATCH 03/17] blog: add figures to the open memory format post Four SVGs in the blog's figure style: a map of memory formats by who writes them and how structured they are, export vs live addressing, how Copilot, LangGraph and LoreKit decide when a memory expires, and one record envelope carrying a prose lesson and CI JSON state. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../web/public/blog/export-vs-address.svg | 46 +++++++++++++ .../web/public/blog/forgetting-defaults.svg | 62 +++++++++++++++++ .../web/public/blog/memory-formats-map.svg | 66 +++++++++++++++++++ .../web/public/blog/one-record-two-bodies.svg | 50 ++++++++++++++ .../content/blog/open-agent-memory-format.mdx | 24 +++++++ 5 files changed, 248 insertions(+) create mode 100644 packages/web/public/blog/export-vs-address.svg create mode 100644 packages/web/public/blog/forgetting-defaults.svg create mode 100644 packages/web/public/blog/memory-formats-map.svg create mode 100644 packages/web/public/blog/one-record-two-bodies.svg diff --git a/packages/web/public/blog/export-vs-address.svg b/packages/web/public/blog/export-vs-address.svg new file mode 100644 index 000000000..92646ec2f --- /dev/null +++ b/packages/web/public/blog/export-vs-address.svg @@ -0,0 +1,46 @@ + + + + + + + + + EXPORT · what the proposals solve + + memory in vendor A + + + a bundle + .mem · .omp.zip · COGX + + + memory in vendor B + one owner, moving house + + + + + LIVE ADDRESS · what's still missing + + CI job + + editor + + review bot + + + + + repo::acme/checkout + write · read · same meaning + many tools, one place, one moment + + + Complementary, not competing — a scope string + fits inside a bundle's record as easily as in a file's frontmatter. + diff --git a/packages/web/public/blog/forgetting-defaults.svg b/packages/web/public/blog/forgetting-defaults.svg new file mode 100644 index 000000000..4fd410367 --- /dev/null +++ b/packages/web/public/blog/forgetting-defaults.svg @@ -0,0 +1,62 @@ + + + + + + + + + write + read + expires + time → + + + Copilot Memory + forgets by default + + + + + + 28 days unused + + + LangGraph store + read resets the TTL + + + + + + + one TTL after the last read + + + LoreKit + no ttl_days (default) + + + + + kept — grooming proposes clean-up + + + LoreKit + with ttl_days + + + + + + + read: no change + a write with a TTL restarts it + + + Three tools, three answers to "what keeps a memory alive?" — use, reads, or explicit writes. + diff --git a/packages/web/public/blog/memory-formats-map.svg b/packages/web/public/blog/memory-formats-map.svg new file mode 100644 index 000000000..2e44cb3a5 --- /dev/null +++ b/packages/web/public/blog/memory-formats-map.svg @@ -0,0 +1,66 @@ + + + + + + + + WHO WRITES × HOW STRUCTURED + + + A PERSON + THE AGENT + AN EXTRACTOR + + + + + + + + MORE STRUCTURE + + + plain files + labeled blocks + facts + metadata + typed records + & graphs + + + + + + + AGENTS.md · CLAUDE.md + + Cursor rules + + + + memory-tool files + + Letta blocks + + Copilot Memory + + MCP knowledge graph + + LangGraph store + + LoreKit record + + + + Mem0 facts + + Graphiti edges + + + Rough placement. Neither axis decides whether two tools can share memory — + that comes down to whether an address means the same thing to every reader. + diff --git a/packages/web/public/blog/one-record-two-bodies.svg b/packages/web/public/blog/one-record-two-bodies.svg new file mode 100644 index 000000000..c2ee69b83 --- /dev/null +++ b/packages/web/public/blog/one-record-two-bodies.svg @@ -0,0 +1,50 @@ + + + + + + + + + + THE SAME ENVELOPE + scope · key · kind · tags · ttl_days · origin_* + + + + + + + a lesson + written by an agent + + Integration tests need Postgres + up first: docker compose up -d db. + kind: lesson (or unset) + read as advisory + shown at session start + lifetime kept unless a TTL is set + + + + CI state + written by a script + + {"v":1, "updated_by_run":…, + "commit":…, "data":{"flaky":[…]}} + kind: bus · host: ci + read as parsed, version-checked + shown only when looked up + lifetime 7-day TTL = liveness + + + + + + repo::acme/checkout + One address, one store — the agent and the CI job both read it. + diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index decdee9d2..e7992db36 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -42,6 +42,12 @@ Here are the families I've found. They all change quickly, so treat the details Several of these made a choice the others didn't, and those choices are worth looking at. +
+ A grid placing agent memory formats by who writes them and how structured they are. A person writes AGENTS.md, CLAUDE.md and Cursor rules. The agent writes memory-tool files, Letta blocks, Copilot Memory, the MCP knowledge graph, LangGraph store items and LoreKit records, with LoreKit highlighted. An extractor writes Mem0 facts and Graphiti edges. Rows run from plain files at the bottom to typed records and graphs at the top. +
The same idea in eight shapes, placed roughly by who writes each one and how much structure it has. Neither axis decides whether two tools can share memory. What decides it is whether an address means the same thing to every reader.
+
+ + **Instruction files keep a person in the loop.** `AGENTS.md` now sits under the Linux Foundation's Agentic AI Foundation, next to MCP. It's the closest thing to a standard here because it's simply a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It doesn't track how often something recurs, nothing in it expires, and it can't target anything finer than a directory. **Anthropic's memory tool goes all in on files.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where the files actually live. There are no embeddings and no schema. That makes it very portable and completely unstructured: two agents writing to the same directory have no shared idea of what a file *means*. @@ -60,6 +66,12 @@ I'm not the first to notice this. MacPaw Research has proposed **Portable Memory These are worth reading. They have one thing in common: they solve **export**. They answer "how do I move my memory from vendor A to vendor B?" That's a real problem, mostly for personal assistants. The problem I care about is slightly different: **live addressing**. When a CI job, an editor agent and a review bot all work in the same repo at the same moment, how do they write to, and read from, the *same* place? A bundle format doesn't answer that, and an address format doesn't replace a bundle. The two could fit together. +
+ Two problems side by side. Left, export: memory in vendor A is packed into a bundle such as a .mem file, an .omp.zip or COGX, and unpacked in vendor B. Right, live addressing: a CI job, an editor agent and a review bot all read and write one address, repo::acme/checkout, at the same time. +
Export moves one owner's memory between vendors. Live addressing lets many tools share one place at the same time. Existing proposals cover the first, and a scope string could sit inside their records to cover the second.
+
+ + ## What they quietly agree on Strip away the branding and almost every format answers the same four questions, just in different ways: @@ -110,6 +122,12 @@ A write can carry `ttl_days`, which sets `expires_at`. Every later write that al So LoreKit is **permanent by default, with forgetting as an opt-in**. Copilot is the opposite: it forgets by default, after 28 days unused. LangGraph sits in between, with an optional TTL that resets when the item is *read*. LoreKit deliberately never lets a read change a record, so only a write can extend a lesson's life. Old lessons are handled by grooming instead: a review pass that suggests merges, expiries and archives before anything changes. +
+ Four timelines comparing expiry. Copilot Memory deletes a fact 28 days after its last use. LangGraph's optional TTL is reset by each read. LoreKit without ttl_days keeps the record indefinitely. LoreKit with ttl_days ignores reads and restarts the countdown only when a write carries a TTL. +
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or explicitly writing it again (LoreKit). Only LoreKit keeps memories forever by default.
+
+ + I go back and forth on which default is right. Forgetting by default keeps a store honest without anyone tending it. Permanent by default means you don't lose the hard-won lesson about the bug that only shows up once a quarter. A shared format should probably let both exist and make the choice visible on each record, rather than settle it once for everyone. What LoreKit doesn't do at all is keep Graphiti's two timelines. A coding lesson like "the integration tests need Postgres up first" is rarely a historical fact you'd want to look up as of a past date. It's either still useful or it isn't. I'm not sure that holds forever, and it's one of the open questions at the end. @@ -185,6 +203,12 @@ Every concept from the lesson record carries over, with no new fields: - **The TTL becomes a liveness check.** The job passes `--ttl-days 7` on every run, so every run restarts the countdown. The expiry then measures how long the job has been *silent*, not how old the record is. The state lasts as long as the job keeps running, and disappears a week after someone deletes the workflow. A stale flaky-test list is worse than none, so this is the right failure mode. - **The body carries its own version.** Checking `v` is the reader's job. A reader that sees a version it doesn't know falls back to its first-run behaviour instead of guessing. +
+ One record envelope with scope, key, kind, tags, ttl_days and origin fields, carrying two bodies. Left, a lesson written by an agent: prose, kind lesson or unset, read as advisory at session start. Right, CI state written by a script: versioned JSON, kind bus, parsed when looked up, with a 7-day TTL as a liveness check. Both live at repo::acme/checkout. +
One envelope, two bodies. The prose lesson and the JSON state share an address, an identity and a lifecycle. Only how they're read differs.
+
+ + The payoff, and the reason to do this instead of `actions/cache`, is that **the agent and the job read the same store**. CI writes `ci-state::flaky-tests`. An agent asked to "fix the flaky tests" reads that exact key in the same repo scope and starts from the real list. The agent's findings go back as a *lesson* right next to it. The state record says what's true right now, and the lesson says what was learned about it. These are two kinds of record in one shape, with no import step between them. That's why I think it's a good test for an open format. A shape that only fits prose written by a model is a chat feature. A shape that also holds JSON written by a shell script, with the same address, identity and lifecycle, is infrastructure. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never sit on a build's critical path.) From 5e2db1a3345085cab557dc98d37ebdba37729ba8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:20:25 +0000 Subject: [PATCH 04/17] skills: add write-blog-post for researching, reviewing and illustrating posts Captures the workflow used for the open memory format post: check LoreKit claims against code, outside claims against sources, run a skeptic's review and plain-language rewrite, draw SVG figures in the blog style (with a render script to inspect them), register the post, and compile it through the real MDX pipeline. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .claude/skills/write-blog-post/SKILL.md | 198 ++++++++++++++++++ .../references/figure-style.md | 83 ++++++++ .../write-blog-post/scripts/render-svg.cjs | 16 ++ 3 files changed, 297 insertions(+) create mode 100644 .claude/skills/write-blog-post/SKILL.md create mode 100644 .claude/skills/write-blog-post/references/figure-style.md create mode 100644 .claude/skills/write-blog-post/scripts/render-svg.cjs diff --git a/.claude/skills/write-blog-post/SKILL.md b/.claude/skills/write-blog-post/SKILL.md new file mode 100644 index 000000000..39377624a --- /dev/null +++ b/.claude/skills/write-blog-post/SKILL.md @@ -0,0 +1,198 @@ +--- +name: write-blog-post +description: > + Writes, critically reviews, illustrates, and ships a post for the LoreKit + blog (packages/web/src/content/blog) end to end: researches the topic, + checks every LoreKit claim against the code and every outside claim against + sources, drafts in the house voice, runs a skeptic's review and rewrites for + plain, strong language, draws SVG figures in the blog's figure style, + registers the post, and verifies it compiles. Triggers on "write a blog + post", "new blog post", "draft a post about", "review this blog post", + "add graphics to the post", "/write-blog-post". +argument-hint: ' [--review-only] [--figures-only]' +license: MIT +metadata: + author: mthines + version: '1.0.0' + workflow_type: applied + tags: + - blog + - content + - mdx + - svg + - lorekit +--- + +# Write a blog post + +A LoreKit blog post is a public claim about how the product works and how it +compares to other tools. Readers include people who build those tools. One +wrong claim about the code, or about a competitor, costs more credibility than +the post earns. So this skill is mostly about **checking**, and only partly +about writing. + +Run the phases in order. `--review-only` starts at Phase 4 on an existing post. +`--figures-only` runs Phase 5 alone. + +## Files you will touch + +| Path | What it is | +| --- | --- | +| `packages/web/src/content/blog/.mdx` | The post. The slug is the URL and the filename | +| `packages/web/src/lib/blog/sections.ts` | The registry. One entry per post: `id` = slug, `label` = title, `summary` = description, character for character | +| `packages/web/public/blog/.svg` | Figures, referenced as `/blog/.svg` | + +`sections.spec.ts` fails when the registry and the frontmatter disagree, and +when the `order` values don't match the registry order. + +## Phase 1 — Orient + +1. Read **every** existing post's frontmatter and at least two full posts. + Learn the voice (see [Voice](#voice)) and, above all, **what has already + been said**. A new post must not repeat an earlier one. Link to it instead. +2. Read `sections.ts` to find the next `order` (the highest existing value + 1). +3. Settle the angle in one sentence before writing anything. If it overlaps + an existing post, find the part that's new. + +## Phase 2 — Research + +**LoreKit claims: check the code, not the docs or your memory.** For every +behaviour the post states, open the implementation and note the file. Claims +that drifted in earlier posts, and are worth checking every time: + +- **TTL / expiry.** Read the latest `memory_write` in `supabase/migrations/` + and `packages/cli/src/store/local.mjs`. Memories are permanent by default. + Only a write that carries a TTL (or a config default) resets the countdown. + A read never extends anything. +- **Scope grammar.** `packages/mcp-core/src/scope/scope.ts` defines exactly + what is accepted. For example, `repo::` takes `owner/repo` only. +- **On-disk format.** `packages/cli/src/store/format.mjs`: which fields + exist, and what survives a rewrite. Unknown fields are dropped. +- **What the session-start digest includes.** `packages/cli/src/core/lessons.mjs` + (`isGeneralLesson`). +- **Limits.** `MAX_VALUE_BYTES` in `packages/schemas/src/domain/memory.ts`, + the memory cap and the rate limit in `docs/limits.md`. +- **Inferred fields.** `kind`/`host` are inferred only from `loop::` tags. + Show realistic values in examples, not what you assume a field holds. + +**Outside claims: search, cite, and look for competitors.** Use WebSearch or +WebFetch for every statement about another tool. Keep the source URLs; the +summary you give the user must list them. Always search explicitly for +**existing proposals or standards** in the post's area. Leaving out a +competing effort is the first thing a critic will point to. + +**Never invent first-person experience** for the author ("on my laptop…", +"I tried X last week"). Write it as a general or hypothetical case unless the +user has told you it's true. + +## Phase 3 — Draft + +Frontmatter (every field required, and the same order as other posts): + +```yaml +--- +title: "…" # must equal sections.ts `label` +description: "…" # must equal sections.ts `summary` +date: "YYYY-MM-DD" # a future date keeps it a draft: hidden in prod, visible on previews +author: "Mads Thines" +readingMinutes: 10 # about 220 words a minute, rounded +tags: ["…"] +order: 5 # next integer, matching the registry position +keywords: ["…"] +--- +``` + +Then add the registry entry at the end of `BLOG_SECTIONS`. Python's `repr()` +picks quote styles that break TypeScript; edit the entry by hand, or check for +the trailing comma afterwards. + +Structure that works for this blog: + +- Open with something concrete the reader recognises, not a thesis. +- Use `##` sections with plain, specific headings (they become the table of contents). +- Every fenced code block gets a language tag (`bash`, `json`, `ts`, `text`, + `markdown`, `yaml`). Shiki highlights only tagged blocks. +- Use internal links (`/blog/`, `/docs/`) rather than repeating material. +- Always write the concrete MCP endpoint, never a `` placeholder. +- End with something the reader can do, usually `npx @lorekit/cli install`. + +## Phase 4 — Critical review and rewrite + +Do this as a separate pass, as if someone else wrote the draft. Produce a +findings list for the user, then fix the post. + +1. **Compare to the code.** Re-check each LoreKit claim against Phase 2. Look + for overstatement ("by default", "always", "never", "lossless"): those are + the words that turn out to be false. +2. **Compare to the industry.** Is every tool described fairly and currently? + Is any competing proposal missing? Does the post claim something is + original when someone else already did it? +3. **Write the skeptic's objections.** List the five or six things a sharp + critic would say: conflict of interest, "X already solves this", security, + scale, "yet another standard". Answer the strong ones in the post, in a + section or inline. Concede where the critic is right. A post that admits + its weak points is more convincing than one that hides them. +4. **Rewrite for plain, strong language.** + - One idea per sentence. Short sentences. Active voice. + - Concrete nouns and real examples instead of abstractions. + - Cut hedges ("quite", "arguably", "it seems") and empty intensifiers. + - Turn run-on lists in prose into bullets. + - Bold at most one phrase per section: the thing to remember. + - Say "I'm not sure" plainly where it's true. Don't invent certainty. +5. Update `description` and the registry `summary` together if the angle moved. + +## Phase 5 — Figures + +Each post usually has 2–4 figures, one per idea that is easier to see than to +read: a comparison, a timeline, a hierarchy, a data shape. Don't draw a figure +that only repeats a table. + +Follow [references/figure-style.md](./references/figure-style.md) exactly: +the palette, fonts, background, `role="img"` with a full `aria-label`, and the +`
` markup with `alt`, `width`, `height` and a `
` that +states the takeaway, not just the subject. + +Then **look at every figure** before you embed it: + +```bash +node .claude/skills/write-blog-post/scripts/render-svg.cjs "$SCRATCH" packages/web/public/blog/.svg +``` + +Read the PNG and check for text overflowing a box, labels colliding with +marks, and glyphs rendering in the wrong colour. Fix, re-render, and look +again. Validate the XML with `xmllint --noout`. + +## Phase 6 — Verify + +```bash +cd packages/web +npx vitest run src/lib/blog # registry drift, order, dates, render +``` + +The render spec compiles only one fixed post. Compile the new one through the +real pipeline too, with a throwaway copy that you delete afterwards: + +```bash +sed 's|self-healing-agents.mdx|.mdx|' src/lib/blog/mdx-render.spec.ts > src/lib/blog/tmp-post.spec.ts +npx vitest run src/lib/blog/tmp-post.spec.ts; rm src/lib/blog/tmp-post.spec.ts +``` + +Install dependencies first in a fresh sandbox (`pnpm install --frozen-lockfile`). + +## Phase 7 — Ship + +- Commit the post, the registry entry and the figures together. The blog is + not part of `llms.txt`, so the PR description should say in one line that + no other docs needed changes. +- Follow the repo's PR workflow (root `CLAUDE.md`) only if the user asks for a PR. +- Report to the user: the post's URL path, its date (and whether that makes it + a draft), the review findings you fixed, anything you couldn't verify, and + the source links. + +## Voice + +- First person singular, as the author: curious and direct, and open about + what's uncertain. +- Name the problem in the reader's terms before naming LoreKit. +- Invite argument at the end. The blog's posts treat design as an open question. +- No marketing superlatives. Let a concrete example make the point. diff --git a/.claude/skills/write-blog-post/references/figure-style.md b/.claude/skills/write-blog-post/references/figure-style.md new file mode 100644 index 000000000..9dffab770 --- /dev/null +++ b/.claude/skills/write-blog-post/references/figure-style.md @@ -0,0 +1,83 @@ +# Blog figure style + +Every figure in `packages/web/public/blog/` is a hand-written, self-contained +SVG drawn in one style. The site is dark-only, so figures are dark too. + +## Skeleton + +```xml + + + + + + + + + +``` + +- Width 640–720. Pick the height to fit the content plus a caption band. +- Always paint the full background ``. + +## Palette + +| Role | Colour | +| --- | --- | +| Background | `#0d0e11` | +| Card / panel | `#13151a`, stroke `#2a2d38` | +| Box inside a card | `#1a1d24`, stroke `#2a2d38` | +| Faint guide lines | `#1d2028` | +| Primary text | `#e7ebf2` | +| Secondary text | `#c5ccd6`, `#aeb6c2` | +| Muted labels (`.lbl`) | `#8892a4`, dimmest `#556072` | +| **LoreKit / the highlighted thing** | amber `#f5a623`; a highlighted box is fill `#2a2010` with an amber stroke | +| The contrasting alternative | indigo `#7c8cf8`, light `#aeb9ff` | +| Good / positive | green `#34d399`, light `#7ce6bf` | +| Bad / expiry / removal | red `#f87171` | + +Use amber for one thing per figure: the point the reader should notice. + +## Type + +- `.lbl` (monospace, muted, often uppercase) for section labels, axes, and captions. +- `.k` (monospace) for identifiers: field names, scopes, keys, file names. +- Sans for prose labels inside boxes, at 11–13px. Nothing below 9px. + +## Common patterns + +- **Side-by-side comparison:** two columns at x≈80 and x≈400, a dashed + divider at x=360, a coloured `.lbl` heading per column, and a two-line + centred caption under a rule at the bottom (see + `structure-vs-similarity.svg`, `export-vs-address.svg`). +- **Record / card:** one big card, `.lbl` zone headings, field names left in + `.k`, explanations right-aligned in `.lbl` (see `memory-record.svg`). +- **Timeline:** label column on the left, track lines from x=200, filled dots + for writes, hollow dots for reads, a tinted bar for a window (see + `forgetting-defaults.svg`). + +## Gotchas + +- **Don't use symbol glyphs (✕, ✓, emoji) for meaning.** They can fall back to + a font that ignores `fill`. Draw them with `` (for example, two + crossing strokes with `stroke-linecap="round"`). +- Monospace text runs about 0.6 × font-size per character. Measure before + putting it in a box, and render to check. +- Escape `&` as `&` and `>` as `>` in text. + +## Embedding + +```mdx +
+ What the figure shows, in one or two sentences. +
The takeaway, not just the subject. One to three sentences.
+
+``` + +`width` and `height` must match the `viewBox` so the layout doesn't jump while +the figure loads. Put the figure right after the paragraph it illustrates, +with a blank line on both sides. diff --git a/.claude/skills/write-blog-post/scripts/render-svg.cjs b/.claude/skills/write-blog-post/scripts/render-svg.cjs new file mode 100644 index 000000000..ca96276ea --- /dev/null +++ b/.claude/skills/write-blog-post/scripts/render-svg.cjs @@ -0,0 +1,16 @@ +// Render blog SVG figures to PNG so you can LOOK at them before shipping. +// Usage: node .claude/skills/write-blog-post/scripts/render-svg.cjs ... +// Uses the preinstalled Playwright Chromium; the viewport is taken from each viewBox. +const { chromium } = require('playwright'); +const fs = require('fs'); const path = require('path'); +(async () => { + const b = await chromium.launch(); const p = await b.newPage({ deviceScaleFactor: 1 }); + for (const f of process.argv.slice(3)) { + const svg = fs.readFileSync(f, 'utf8'); + const vb = svg.match(/viewBox="0 0 (\d+) (\d+)"/); + await p.setViewportSize({ width: +vb[1], height: +vb[2] }); + await p.setContent(`${svg}`); + await p.screenshot({ path: path.join(process.argv[2], path.basename(f, '.svg') + '.png') }); + } + await b.close(); +})(); From 18581032099c5aac5702c7aa667550845b23c056 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:26:33 +0000 Subject: [PATCH 05/17] blog: reframe the memory format post as a call for a shared spec Retitle to 'Agent memory needs its OpenTelemetry moment'. Lead with the OpenTracing/OpenCensus to OpenTelemetry history and its four layers, present LoreKit as one open-source proposal, and add a section listing seven gaps in LoreKit's own format (stable id, open attributes, content type, verifiable anchors, explicit lifecycle, supersession, host-neutral address and version) with a sketch record. Add an OTel-parallel figure and stop highlighting LoreKit in the format map. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../web/public/blog/memory-formats-map.svg | 8 +- packages/web/public/blog/otel-parallel.svg | 49 ++++ .../content/blog/open-agent-memory-format.mdx | 271 ++++++++---------- packages/web/src/lib/blog/sections.ts | 4 +- 4 files changed, 177 insertions(+), 155 deletions(-) create mode 100644 packages/web/public/blog/otel-parallel.svg diff --git a/packages/web/public/blog/memory-formats-map.svg b/packages/web/public/blog/memory-formats-map.svg index 2e44cb3a5..7c75839bf 100644 --- a/packages/web/public/blog/memory-formats-map.svg +++ b/packages/web/public/blog/memory-formats-map.svg @@ -1,4 +1,4 @@ - + Two columns compare observability, which OpenTelemetry settled, with agent memory, which is still open, across four layers: data model, semantic conventions, protocol, and implementations. For memory, the first three layers are drawn dashed because nobody has agreed on them yet. +
The same four layers. Observability agreed on the first three and kept competing on the fourth. Agent memory hasn't agreed on any of them yet.
+
+ +Agent memory is at the stage observability was in before 2019. There are many formats, a few early proposals and no agreement. The timing is good for two reasons. The formats are still young enough to change. And the agent ecosystem already has a neutral home for shared standards: MCP and `AGENTS.md` both now sit under the Linux Foundation's Agentic AI Foundation. ## A tour of the shapes -Here are the families I've found. They all change quickly, so treat the details as a snapshot from late 2026. +To agree on a shape, it helps to see the ones we already have. These change quickly, so treat the details as a snapshot from late 2026. | Family | Example | The unit of memory | Addressed by | Forgetting | | --- | --- | --- | --- | --- | @@ -39,40 +60,34 @@ Here are the families I've found. They all change quickly, so treat the details | Namespaced store | LangGraph `BaseStore` | A JSON `value` dict | A `namespace` tuple + `key` | Optional TTL, reset when the item is read (by default) | | Knowledge graph | MCP reference memory server | Entities, relations, observations | Entity name | Never | | Temporal graph | Graphiti / Zep | A fact on a graph edge | Entities + edge | Facts are invalidated rather than deleted, on two timelines | - -Several of these made a choice the others didn't, and those choices are worth looking at. +| Addressed record | LoreKit | A record with a scope, key and lifecycle | A scope string + `key` | Never, unless a TTL is set |
- A grid placing agent memory formats by who writes them and how structured they are. A person writes AGENTS.md, CLAUDE.md and Cursor rules. The agent writes memory-tool files, Letta blocks, Copilot Memory, the MCP knowledge graph, LangGraph store items and LoreKit records, with LoreKit highlighted. An extractor writes Mem0 facts and Graphiti edges. Rows run from plain files at the bottom to typed records and graphs at the top. -
The same idea in eight shapes, placed roughly by who writes each one and how much structure it has. Neither axis decides whether two tools can share memory. What decides it is whether an address means the same thing to every reader.
+ A grid placing agent memory formats by who writes them and how structured they are. A person writes AGENTS.md, CLAUDE.md and Cursor rules. The agent writes memory-tool files, Letta blocks, Copilot Memory, the MCP knowledge graph, LangGraph store items and LoreKit records. An extractor writes Mem0 facts and Graphiti edges. Rows run from plain files at the bottom to typed records and graphs at the top. +
The same idea in ten shapes, placed roughly by who writes each one and how much structure it has. Neither axis decides whether two tools can share memory. What decides it is whether an address means the same thing to every reader.
+Each of these got something right that a shared spec should keep: -**Instruction files keep a person in the loop.** `AGENTS.md` now sits under the Linux Foundation's Agentic AI Foundation, next to MCP. It's the closest thing to a standard here because it's simply a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It doesn't track how often something recurs, nothing in it expires, and it can't target anything finer than a directory. +- **Instruction files keep a person in the loop.** `AGENTS.md` is the closest thing to a standard here because it's simply a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It can't tell you how often something recurs, nothing in it expires, and it can't target anything finer than a directory. +- **Anthropic's memory tool proves plain files go a long way.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where the files live. It's very portable. The cost is that two agents writing to the same directory have no shared idea of what a file *means*. +- **Copilot Memory checks itself against the code.** Every fact carries citations, which are links to specific code locations, and Copilot checks them against the current branch before using the fact. Facts disappear after 28 days without use. Of everything here, that pairing of *provenance you can check* and *forgetting by default* is the design I'd most like a shared spec to have. Today it's locked inside one product. +- **Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the prompt on every step. The format is honest that memory costs tokens every turn. +- **LangGraph shows how small the core can be.** It's `put(namespace, key, value)`, where the namespace is a tuple such as `("memories", user_id)`. Its optional TTL resets whenever an item is read. +- **The graphs model the world, and they model time.** The MCP reference server keeps entities, relations and observations in one `memory.jsonl` file. Graphiti stores four timestamps on every edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?" and keep a record of what replaced what. -**Anthropic's memory tool goes all in on files.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where the files actually live. There are no embeddings and no schema. That makes it very portable and completely unstructured: two agents writing to the same directory have no shared idea of what a file *means*. +### The proposals that already exist -**Copilot Memory checks itself against the code.** Every fact carries citations, which are links to specific code locations. Before using a fact, Copilot checks those citations against the current branch. Facts belong to one repository and disappear after 28 days without use. To me, that pairing of *provenance you can check* and *forgetting by default* is the most interesting design in this list. It's also locked inside one product. +Several people have already started on this. MacPaw Research has proposed **Portable Memory**, a `.mem` bundle with byte-reproducible serialization, deletion tombstones and optional signing. Its own authors say it's not yet widely adopted. The **Open Memory Protocol** is an April 2026 draft for moving conversations, and the memories derived from them, between platforms as a ZIP file with a manifest. Cognee offers a **COGX** exchange format with importers from Mem0, Letta and Zep. -**Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the system prompt on every step. The format is honest about the cost: memory isn't free, because you pay for it in tokens every turn. - -**LangGraph is nearly a key-value store.** Its interface is `put(namespace, key, value)`, where the namespace is a tuple such as `("memories", user_id)`. Its TTL can reset whenever an item is read. Of everything here, it's the closest relative of LoreKit's shape. - -**The graphs model the world, not the work.** The MCP reference server keeps entities, relations and observations in a single `memory.jsonl` file. Graphiti stores four timestamps on every edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?". Both are built for facts *about things*: people, companies, and how a newer fact replaces an older one. - -### And yes, people are already proposing standards - -I'm not the first to notice this. MacPaw Research has proposed **Portable Memory**, a `.mem` bundle with byte-reproducible serialization, deletion tombstones and optional signing. Its own authors say it's not yet widely adopted. The **Open Memory Protocol** is an April 2026 draft for moving conversations and the memories derived from them between platforms, as a ZIP file with a manifest. Cognee offers a **COGX** exchange format with importers from Mem0, Letta and Zep. - -These are worth reading. They have one thing in common: they solve **export**. They answer "how do I move my memory from vendor A to vendor B?" That's a real problem, mostly for personal assistants. The problem I care about is slightly different: **live addressing**. When a CI job, an editor agent and a review bot all work in the same repo at the same moment, how do they write to, and read from, the *same* place? A bundle format doesn't answer that, and an address format doesn't replace a bundle. The two could fit together. +These are starting points, not competitors. They mostly solve **export**: "how do I move my memory from vendor A to vendor B?" That matters most for personal assistants. Coding tools have a second problem that's still open: **live addressing**. When a CI job, an editor agent and a review bot work in the same repo at the same time, how do they write to and read from the *same* place, and agree on what it means? A complete spec needs both, the way OpenTelemetry needed both a data model and a protocol.
Two problems side by side. Left, export: memory in vendor A is packed into a bundle such as a .mem file, an .omp.zip or COGX, and unpacked in vendor B. Right, live addressing: a CI job, an editor agent and a review bot all read and write one address, repo::acme/checkout, at the same time. -
Export moves one owner's memory between vendors. Live addressing lets many tools share one place at the same time. Existing proposals cover the first, and a scope string could sit inside their records to cover the second.
+
Export moves one owner's memory between vendors. Live addressing lets many tools share one place at the same time. The existing proposals focus on the first. A shared address could sit inside their records to cover the second.
- -## What they quietly agree on +## What every format already agrees on Strip away the branding and almost every format answers the same four questions, just in different ways: @@ -81,109 +96,80 @@ Strip away the branding and almost every format answers the same four questions, 3. **What does it say?** Almost always plain text, or JSON when a program is the reader. 4. **When does it stop being true?** Often skipped. When it isn't, the answers range from "28 days unused" to "two full timelines". -They also split on a fifth question: **who writes?** In instruction files it's a person. In the memory tool, Letta and LoreKit, the model writes directly. In Mem0 and Graphiti, a service extracts facts from the conversation. That's a real fork. Extraction gives you memory without asking the agent to do anything. In exchange, it costs a model call on every write and produces a store whose contents you can't fully predict. +That's the part I find hopeful. These four questions are the beginning of a data model, and nobody disagrees that they matter. The tools only disagree on the answers. OpenTelemetry started from a similar place: everyone already had spans, they just shaped them differently. -They disagree on the **body** too: free text, a capped block, a JSON dict, or a triple. That matters less than it looks, as the CI section below shows. - -## Why LoreKit's record looks the way it does - -I've written up [every column of the data model](/blog/agent-memory-data-model) before, so I won't repeat that here. What's more useful is to go through the four questions and show where LoreKit borrowed from the formats above, where it went another way, and where I'm not sure it got it right. - -### "Where" is a grammar, not a value - -```text -global -project::{name} -repo::{owner}/{repo} -branch::{owner}/{repo}::{branch} -``` +There's one real fork. **Who writes the memory?** In instruction files it's a person. In the memory tool, Letta and LoreKit, the model writes directly. In Mem0 and Graphiti, a service extracts facts from the conversation. A shared format doesn't need to pick one. It needs to *record* which one, so a reader knows how much to trust what it reads. -LangGraph's tuple and LoreKit's scope do the same job. The difference is that a scope is a **string you can write down anywhere**: a URL, a filename, a CLI flag, a YAML key, a log line. It has fixed levels and a fixed read order, with the most specific level read first. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name, so `::` can't be confused with either. A tuple only means something inside one language's runtime. A grammar means the same thing to a Deno edge function, a Node CLI and `grep`. +## One proposal: LoreKit's record -It's coarser than a file path and finer than "one repo", which is where coding lessons tend to live. "This branch's spike found X" and "this whole project uses Y" are both easy to express. +Here is how LoreKit answers the four questions, offered as one data point. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. -It's also clearly shaped by GitHub. `repo::` accepts exactly `owner/repo`, so a GitLab subgroup path like `group/subgroup/repo` doesn't fit today. A standard couldn't keep that. - -### "Identity" is `scope` + `key`, and a rewrite is counted - -Writing the same `scope` + `key` again updates the existing row and increments `seen_count`. Compare that with an append-only log: if you re-learn something five times, the log holds five near-duplicates, while LoreKit holds one row that knows it was learned five times. A deterministic store gets this salience signal for free, without any model. - -To be precise about what it measures: `seen_count` tells you how often something came up. It doesn't tell you whether it's true. A wrong lesson that's re-learned five times is still wrong, just louder. That's why the next two choices exist. - -Copilot gets *trust* from citations. LoreKit gets a weaker version of that from provenance: `origin_repo`, `origin_branch`, `origin_commit` and `origin_pr` record where a lesson was learned, but nothing re-checks the code before the lesson is used. If I were starting a standard from scratch, a citation field that readers *may* verify is the first thing I'd borrow from Copilot. - -### "What it says" is plain text, and only advisory - -The `value` is text, capped at 64 KiB. When LoreKit puts lessons into an agent's context, it labels them *"considerations, not rules."* That's a deliberate line between lore and instruction files. `AGENTS.md` holds what a person has decided. Lore holds what an agent has *noticed*, and it should be easy to ignore when it's wrong. Mix the two in one file and instruction files turn into an unreviewed pile of half-truths. - -### "When it stops being true" is opt-in, and that's open to debate - -A write can carry `ttl_days`, which sets `expires_at`. Every later write that also carries a TTL restarts the countdown. A write *without* one leaves the existing expiry alone. A lesson written without any TTL never expires, unless a repo or user config sets a default for its scope. - -So LoreKit is **permanent by default, with forgetting as an opt-in**. Copilot is the opposite: it forgets by default, after 28 days unused. LangGraph sits in between, with an optional TTL that resets when the item is *read*. LoreKit deliberately never lets a read change a record, so only a write can extend a lesson's life. Old lessons are handled by grooming instead: a review pass that suggests merges, expiries and archives before anything changes. +- **Where: a scope string.** One of `global`, `project::{name}`, `repo::{owner}/{repo}` or `branch::{owner}/{repo}::{branch}`, read from most specific to least. Unlike LangGraph's tuple, it's a string with a grammar, so it means the same thing in a URL, a filename, a CLI flag or a log line. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name. +- **Identity: `scope` + `key`, and a rewrite is counted.** Writing the same `scope` + `key` again updates the record and increments `seen_count`. One record that knows it was learned five times is more useful than five near-duplicates. But `seen_count` measures how often something came up, not whether it's true. +- **Body: plain text, read as advice.** The value is text up to 64 KiB, and LoreKit labels lessons *"considerations, not rules"* when it shows them to an agent. A rule belongs in `AGENTS.md`, decided by a person. A lesson is something an agent *noticed*, and it should be easy to ignore when it's wrong. +- **Lifecycle: permanent unless a TTL is set.** A write can carry `ttl_days`. Each later write that also carries one restarts the countdown, and a read never extends anything. Copilot and LangGraph made different choices here, and the figure below shows all three. +- **Kind: a short, closed list.** `lesson` (read every run), `bus` (a transient outcome, read only when deciding what to promote) and `signal` (a durable filter for one repo). This keeps records that serve different purposes from crowding each other out of an agent's context. +- **On disk: one Markdown file per record.** The frontmatter mirrors the hosted record's columns, with every value JSON-encoded so the file stays valid YAML and easy to grep.
Four timelines comparing expiry. Copilot Memory deletes a fact 28 days after its last use. LangGraph's optional TTL is reset by each read. LoreKit without ttl_days keeps the record indefinitely. LoreKit with ttl_days ignores reads and restarts the countdown only when a write carries a TTL. -
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or explicitly writing it again (LoreKit). Only LoreKit keeps memories forever by default.
+
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or writing it again (LoreKit). None is obviously right, which is why a shared format should record the rule on each record instead of choosing one for everyone.
+## Where LoreKit's format falls short -I go back and forth on which default is right. Forgetting by default keeps a store honest without anyone tending it. Permanent by default means you don't lose the hard-won lesson about the bug that only shows up once a quarter. A shared format should probably let both exist and make the choice visible on each record, rather than settle it once for everyone. - -What LoreKit doesn't do at all is keep Graphiti's two timelines. A coding lesson like "the integration tests need Postgres up first" is rarely a historical fact you'd want to look up as of a past date. It's either still useful or it isn't. I'm not sure that holds forever, and it's one of the open questions at the end. - -### "What kind" is a short, closed list +If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. -`kind` is one of `lesson`, `bus` or `signal`, and `host` names the loop that wrote the record. Each kind is read on a different schedule. A `lesson` is read every run. A `bus` record is a transient outcome, read only when deciding what to promote to a permanent rule. A `signal` is a durable filter for one repo. The list is closed on purpose, so adding a kind means changing the schema. None of the other formats makes this split, and without it, a store fills up with records that push each other out of the context window. +1. **No stable ID that's separate from the address.** A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. +2. **No place for other tools' data, and rewrites lose it.** LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. +3. **The body doesn't say what it is.** LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. +4. **Provenance says where a lesson came from, not what it depends on.** `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. +5. **The expiry rule is implicit.** Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. +6. **Corrections leave no trail.** When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. +7. **The address grammar assumes GitHub, and nothing says which version a file uses.** `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. -### On disk, it's a file you can `cat` +Here's a sketch of what a record could look like with those changes. It's an illustration, not a spec: -In the offline store, each record is one Markdown file per `scope` + `key`, and its frontmatter mirrors the columns of the hosted database row. Here's one, trimmed (the real file also lists the empty provenance fields as `null`): - -```markdown +```yaml --- -scope: "repo::acme/checkout" +spec: "agent-memory/0.1" # format version (7) +id: "01JC7Z3K9V2Q8M4T6R1W5X0YAB" # stable, independent of the address (1) +scope: "repo::github.com/acme/checkout" # host-qualified address (7) key: "tests-need-local-postgres" -tags: ["ci","gotcha"] -created: "2026-10-02T09:14:00.000Z" -updated: "2026-11-04T16:41:12.000Z" -origin_commit: "0f4a1c9" -expires_at: "2026-12-04T16:41:12.000Z" +kind: "lesson" +content_type: "text/markdown" # (3) +written_by: { type: "agent", name: "claude-code" } +created: "2026-10-02T09:14:00Z" +updated: "2026-11-04T16:41:12Z" seen_count: 4 -kind: null -host: null +lifecycle: { expires_at: "2026-12-04T16:41:12Z", extended_by: "write", last_used: "2026-11-06T08:02:51Z" } # (5) +anchors: # (4) + - { path: "docker-compose.yml", lines: "12-20", commit: "0f4a1c9", sha256: "9b1e…" } +supersedes: null # (6) +attributes: # open, namespaced, kept on rewrite (2) + vcs.pr: 482 + ci.run_url: "https://github.com/acme/checkout/actions/runs/123456789" --- Integration tests need Postgres up first: docker compose up -d db. ``` -Every value in the frontmatter is JSON-encoded. JSON is a strict subset of YAML, so the file is still valid YAML, still easy to grep, and LoreKit reads it with a few lines of code and no YAML library. - -When it reads a file, LoreKit is tolerant: a field missing from an old file counts as absent, which is why files written before `seen_count` existed still load. But it isn't yet *lossless*. Rewriting a record keeps only the fields LoreKit knows about, so a field some other tool added would be dropped. That's exactly the kind of gap a shared format has to close: **preserve what you don't understand**. - -The shape is also close enough to a memory-tool file or an `AGENTS.md` that these formats could meet halfway. The frontmatter would carry the address and the lifecycle, and the Markdown body would carry the note. +None of this is new. Every item is borrowed from a format in the table or from OpenTelemetry. That's the point: the pieces already exist, spread across tools that can't read each other. -## The test: memory no model wrote +## A test any shared format should pass -This part came from somewhere I didn't plan, and it's the part I find most convincing. +One test has convinced me the shape is general enough: can it hold memory that **no model wrote**? -CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. A review bot wants to know what it already commented on. None of that involves a model. The usual answers are `actions/cache`, or an artifact bucket nobody else reads. - -LoreKit stores this in the **same record**. The only difference is the body: a versioned JSON envelope instead of prose. +CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. Usually that goes into `actions/cache` or an artifact bucket nobody else reads. LoreKit stores it in the same record as a lesson, with a versioned JSON body: ```json { "v": 1, "updated_by_run": "https://github.com/acme/checkout/actions/runs/123456789", "commit": "0f4a1c9", - "data": { - "flaky": ["src/queue.test.ts::retries on 429"], - "consecutive_green": 3 - } + "data": { "flaky": ["src/queue.test.ts::retries on 429"], "consecutive_green": 3 } } ``` -The job writes it with the CLI, piped straight from `jq`: - ```bash RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" @@ -195,75 +181,62 @@ jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ --ttl-days 7 --remote --json ``` -Every concept from the lesson record carries over, with no new fields: - -- **Scope works the same way.** State for the main branch lives at `repo::`. State for a single PR lives at `branch::`, so it goes away when the branch does. -- **Identity works the same way, with a stricter rule: one key per fact, overwritten in place, never one key per run.** Because the same `scope` + `key` is an update, a job that runs a thousand times still holds one row. A key per run ID would turn a memory store into a slow artifact bucket. -- **`kind` keeps the record out of an agent's way.** A record with `kind: bus` is left out of the lessons an agent is shown at the start of a session, so a JSON blob never competes with real lessons for context. -- **The TTL becomes a liveness check.** The job passes `--ttl-days 7` on every run, so every run restarts the countdown. The expiry then measures how long the job has been *silent*, not how old the record is. The state lasts as long as the job keeps running, and disappears a week after someone deletes the workflow. A stale flaky-test list is worse than none, so this is the right failure mode. -- **The body carries its own version.** Checking `v` is the reader's job. A reader that sees a version it doesn't know falls back to its first-run behaviour instead of guessing. +The same address, identity and lifecycle carry over. The job writes one key per fact and overwrites it in place, never one key per run. Because the job passes `--ttl-days 7` on every run, the expiry measures how long the job has been *silent*: the state lasts while the job runs and disappears a week after the workflow is deleted. And an agent asked to "fix the flaky tests" reads the same key and starts from the real list.
One record envelope with scope, key, kind, tags, ttl_days and origin fields, carrying two bodies. Left, a lesson written by an agent: prose, kind lesson or unset, read as advisory at session start. Right, CI state written by a script: versioned JSON, kind bus, parsed when looked up, with a 7-day TTL as a liveness check. Both live at repo::acme/checkout. -
One envelope, two bodies. The prose lesson and the JSON state share an address, an identity and a lifecycle. Only how they're read differs.
+
One envelope, two bodies. A prose lesson and a JSON state record share an address, an identity and a lifecycle. Only how they're read differs.
+This is also where gaps 2 and 3 above hurt most. The JSON is only recognisable through conventions (`kind: bus`, a `v` inside the body). With `content_type` and `attributes`, any tool could read it. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) -The payoff, and the reason to do this instead of `actions/cache`, is that **the agent and the job read the same store**. CI writes `ci-state::flaky-tests`. An agent asked to "fix the flaky tests" reads that exact key in the same repo scope and starts from the real list. The agent's findings go back as a *lesson* right next to it. The state record says what's true right now, and the lesson says what was learned about it. These are two kinds of record in one shape, with no import step between them. - -That's why I think it's a good test for an open format. A shape that only fits prose written by a model is a chat feature. A shape that also holds JSON written by a shell script, with the same address, identity and lifecycle, is infrastructure. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never sit on a build's critical path.) +## What a shared spec could look like -## What a skeptic would say +If agent memory followed OpenTelemetry's path, the spec would have the same four layers: -These are the objections I'd raise if someone else had written this post, with my best answer to each. +- **A data model.** One record: an address, a stable ID, a body with a content type, a lifecycle, provenance with optional anchors, a kind, and an open attribute map. Small enough that a shell script can write a valid record. +- **Conventions.** Shared names for address levels (`global`, `project`, `repo`, `branch`, and whatever monorepos need), for kinds, and for attribute namespaces (`vcs.*`, `ci.*`, `agent.*`). OpenTelemetry's semantic conventions are the model: one registry, versioned, open to anyone. +- **A contract.** A small set of memory operations (write, read, list, search, archive) with fixed inputs and outputs, exposed over MCP. Every memory server today speaks MCP but exposes different tools, so agents can call any of them and none of them mean the same thing. +- **Conformance tests.** Round-trip a record and check nothing was lost. Check that unknown attributes survive a rewrite. Check that expiry rules behave as stated. -**"This is just your product's schema dressed up as a standard."** That's fair, and it's why I've tried to separate the core from LoreKit's extras below. LoreKit's job is to show that a shape like this holds up in daily use. It's not a claim that the shape should win. If an existing proposal adopted a scope grammar and the four core fields, I'd happily move LoreKit onto it. +What stays *out* of the spec matters as much: storage, ranking, embeddings, whether a model or an extractor writes, the user interface. Those are where tools should compete, the way observability vendors compete on everything after the data arrives. -**"MCP is already the standard."** MCP standardises the *call*, not the *record*. Two memory servers that both speak MCP can expose completely different tools, with different arguments and different meanings: the reference server's `create_entities`, LoreKit's `memory.write`, and dozens of others. The agent can call any of them, but no two of them agree on what a memory is. +The spec should live somewhere neutral, not in LoreKit's repo or any single vendor's. The Agentic AI Foundation already looks after MCP and `AGENTS.md`, which makes it one natural option, though that's for the people involved to decide. -**"Just use files in git."** For one person and one tool, I mostly agree, and LoreKit's offline store *is* files. What plain files don't give you is agreement on what a file *means*: what it applies to, whether it repeats an earlier lesson, and when it stops being true. Frontmatter is where that agreement would live. - -**"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and a shared store widens who gets to write those things. LoreKit's partial defences are: +## What a skeptic would say -- lore is labeled advisory -- every record carries provenance, so you can see where it came from -- tokens can be write-only or read-only, so a CI job's write-only token can't read the team's lore back out +These are the objections I'd raise if someone else had written this post. -None of that makes poisoning impossible. A standard would need a trust story, and citations plus signing (which Portable Memory already proposes) are the obvious starting points. +**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post lists LoreKit's gaps, and why the sketch above changes LoreKit's own format in seven places. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. -**"Without embeddings, you'll miss everything that's phrased differently."** Sometimes, yes. The [data-model post](/blog/agent-memory-data-model) makes the case for structure first. The point here is narrower: a shared format shouldn't *require* vectors. An index is something a reader can build on top of the records, not something stored in them. +**"MCP already is the standard."** MCP standardises the *call*, not the *record*. It's like having a shared network protocol without a shared data model. OpenTelemetry needed OTLP *and* the data model. Memory needs both too. -**"Another standard won't fix fragmentation."** That's the xkcd 927 objection: try to unify everything and you end up with one more competing standard. It's the strongest one. The honest answer is that I'd rather add an **address and lifecycle** to formats that already exist than create a new container. That's a smaller ask, and a scope string fits in a frontmatter field, a JSON key, or an `.omp` memory record equally well. +**"Just use files in git."** For one person and one tool, files work well. The memory tool and LoreKit's offline store are both files. What plain files lack is agreement on what a file *means*: what it applies to, whether it repeats an earlier lesson, and when it stops being true. Frontmatter is a good place for that agreement to live. -## What an open format might need +**"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and shared memory widens who gets to write those things. A spec should make the risk visible rather than hide it: record who wrote each memory and how, let readers check anchors, and allow signatures, which Portable Memory already proposes. No format makes poisoning impossible. -Here's my current guess at the smallest useful core, written down so people can argue with it: +**"Another standard won't fix fragmentation."** That's the xkcd 927 problem: an attempt to unify everything adds one more competing standard. It's the strongest objection. My answer is the OpenTelemetry one. Agreement happened there when the existing projects *merged*, not when someone added a fifth. The useful work here is the same: take what Portable Memory, the Open Memory Protocol, Copilot, Letta, LangGraph and LoreKit each got right, and agree on one shape for the parts they share. -- **An address grammar.** A string with defined levels and a defined read order. Not a file path, and not a tuple tied to one language. It should handle more than GitHub's `owner/repo`. -- **An identity rule.** Address plus key is unique, and a rewrite updates the record and is *counted*. -- **A body.** Text by default. JSON is allowed when the reader is a program, with the version stamped inside the body. -- **A lifecycle.** Created and updated times, an optional expiry with an explicit rule for what extends it, and soft deletion. -- **Provenance.** Where a record was learned, ideally with citations a reader *may* verify. -- **A kind.** Enough to tell a lesson from an event from a filter, so readers can budget for each. -- **Lossless round-trips.** Readers keep fields they don't understand when they rewrite a record. LoreKit doesn't do this yet. +**"It's too early. Memory designs are still changing."** Maybe. But OpenTelemetry's data model has kept evolving since 2019, through versioning and conventions. Agreeing on a small core now doesn't freeze anything. It just stops every new tool from inventing its own core from scratch. -Everything else should stay free to vary: the storage engine, the ranking, whether you add embeddings on top, and whether an extractor or the agent does the writing. OpenTelemetry succeeded because it standardised the data model and the wire format, not anyone's backend. I work with it every day at [Dash0](https://www.dash0.com), and I've seen how much one shared shape makes possible. +## Open questions -And the questions I honestly can't answer yet: +I don't know the answers to these, and I'd rather work them out in public: -- **Are four scope levels right?** Monorepos want a level between `repo` and `branch`. Organisations want one above `project`. -- **Should forgetting be the default?** Copilot says yes, LoreKit says no, and LangGraph extends a record's life whenever it's read. Which of those should a shared format assume? -- **Do coding lessons need two timelines?** Graphiti's argument is strong for facts about the world. Is there a coding case where "what was true in March" matters? -- **Should citations be part of the core?** Copilot's check-before-use is the best defence against stale memory I've seen, but it ties a record to a code host. +- **Which address levels?** Monorepos want a level between `repo` and `branch`, and organisations want one above `project`. Is a fixed list right, or should levels be conventions too? +- **Should forgetting be the default?** Copilot forgets by default, LoreKit doesn't, and LangGraph extends life on read. A per-record rule avoids choosing, but tools still need a default. +- **Do coding lessons need two timelines?** Graphiti's case is strong for facts about the world. Is there a coding case where "what was true in March" matters? +- **Are anchors part of the core?** They're the best protection against stale memory, but they tie a record to a code host. +- **Who should coordinate?** A working group across tool builders, a foundation, or a draft spec that people send changes to? -The schema, the scope validator and the on-disk format are all open in the repo. If you build one of the formats in the table above, I'd especially like to hear where this shape breaks for you. +If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are in the repo as one concrete input to that conversation. -## See the files for yourself +## Look at a real record -The quickest way to judge whether this shape is any good is to look at the files it writes: +The quickest way to judge any of this is to look at real files. LoreKit's offline store writes plain Markdown records you can open, grep and diff: ```bash npx @lorekit/cli install ``` -That gives you a folder of plain Markdown records you can open, grep and diff. Then tell me what's missing. +Then compare them with the files your other tools write, and tell me what a shared format would need that none of them have yet. diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index a043a4b17..fd99488b8 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -46,9 +46,9 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ }, { id: 'open-agent-memory-format', - label: 'Every agent tool invents its own memory format. What would an open one look like?', + label: "Agent memory needs its OpenTelemetry moment", summary: - "Files, blocks, namespaces, knowledge graphs, extracted facts: every agent tool stores memory in its own shape, and none can read the others. A tour of the formats, what they quietly agree on, the choices behind LoreKit's record (including the ones I'm unsure of), how the same record carries a CI job's JSON state, and what a skeptic would say about all of it.", + "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, what they already agree on, LoreKit's open-source record offered as one proposal with its gaps listed, and what a shared spec could look like.", }, ]; From 4fec8e2e6045854cbf683dba4e086f86ea9bd7ab Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 07:32:00 +0000 Subject: [PATCH 06/17] blog: split the memory format post in two Keep 'Agent memory needs its OpenTelemetry moment' (~8 min) focused on the formats, the existing proposals and what a shared spec could look like. Move LoreKit's record, its seven gaps, the sketch record and the CI test into 'Seven gaps in LoreKit's memory format' (~7 min), dated 2026-11-14 so it stays a draft in production until then. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../blog/lorekit-memory-format-gaps.mdx | 122 ++++++++++++++++++ .../content/blog/open-agent-memory-format.mdx | 117 ++--------------- packages/web/src/lib/blog/sections.ts | 8 +- 3 files changed, 142 insertions(+), 105 deletions(-) create mode 100644 packages/web/src/content/blog/lorekit-memory-format-gaps.mdx diff --git a/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx b/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx new file mode 100644 index 000000000..0074b2326 --- /dev/null +++ b/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx @@ -0,0 +1,122 @@ +--- +title: "Seven gaps in LoreKit's memory format" +description: "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it has seven gaps, each tied to a failure you can point to. Here they are, with a sketch of a better record and the CI test any shared format should pass." +date: "2026-11-14" +author: "Mads Thines" +readingMinutes: 7 +tags: ["agent-memory", "open-format", "data-model", "ci", "lorekit"] +order: 6 +keywords: ["agent memory format", "lorekit", "memory record", "stable id", "attributes", "content type", "code anchors", "memory expiry", "supersedes", "scope grammar", "CI state", "JSON state records", "open memory standard"] +--- + +## Why write about our own gaps + +In [the previous post](/blog/open-agent-memory-format) I argued that agent memory needs what observability got from OpenTelemetry: one shared shape that every tool can read, so tools compete on what they do with memory instead of on how it's stored. + +If I'm asking other tool builders to put their formats on the table, LoreKit's should go first, gaps included. This post describes LoreKit's record as it is today, then lists seven places where it falls short compared with the other formats and with OpenTelemetry. None of these gaps is hypothetical. Each one is a failure I can point to in the code. + +## LoreKit's record today + +Every memory format answers the same four questions: where a memory is true, what its identity is, what it says, and when it stops being true. Here is how LoreKit answers them. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. + +- **Where: a scope string.** One of `global`, `project::{name}`, `repo::{owner}/{repo}` or `branch::{owner}/{repo}::{branch}`, read from most specific to least. Unlike LangGraph's tuple, it's a string with a grammar, so it means the same thing in a URL, a filename, a CLI flag or a log line. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name. +- **Identity: `scope` + `key`, and a rewrite is counted.** Writing the same `scope` + `key` again updates the record and increments `seen_count`. One record that knows it was learned five times is more useful than five near-duplicates. But `seen_count` measures how often something came up, not whether it's true. +- **Body: plain text, read as advice.** The value is text up to 64 KiB, and LoreKit labels lessons *"considerations, not rules"* when it shows them to an agent. A rule belongs in `AGENTS.md`, decided by a person. A lesson is something an agent *noticed*, and it should be easy to ignore when it's wrong. +- **Lifecycle: permanent unless a TTL is set.** A write can carry `ttl_days`. Each later write that also carries one restarts the countdown, and a read never extends anything. Copilot and LangGraph made different choices here, and the figure below shows all three. +- **Kind: a short, closed list.** `lesson` (read every run), `bus` (a transient outcome, read only when deciding what to promote) and `signal` (a durable filter for one repo). This keeps records that serve different purposes from crowding each other out of an agent's context. +- **On disk: one Markdown file per record.** The frontmatter mirrors the hosted record's columns, with every value JSON-encoded so the file stays valid YAML and easy to grep. + +
+ Four timelines comparing expiry. Copilot Memory deletes a fact 28 days after its last use. LangGraph's optional TTL is reset by each read. LoreKit without ttl_days keeps the record indefinitely. LoreKit with ttl_days ignores reads and restarts the countdown only when a write carries a TTL. +
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or writing it again (LoreKit). None is obviously right, which is why a shared format should record the rule on each record instead of choosing one for everyone.
+
+ +## Where LoreKit's format falls short + +If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. + +1. **No stable ID that's separate from the address.** A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. +2. **No place for other tools' data, and rewrites lose it.** LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. +3. **The body doesn't say what it is.** LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. +4. **Provenance says where a lesson came from, not what it depends on.** `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. +5. **The expiry rule is implicit.** Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. +6. **Corrections leave no trail.** When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. +7. **The address grammar assumes GitHub, and nothing says which version a file uses.** `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. + +Here's a sketch of what a record could look like with those changes. It's an illustration, not a spec: + +```yaml +--- +spec: "agent-memory/0.1" # format version (7) +id: "01JC7Z3K9V2Q8M4T6R1W5X0YAB" # stable, independent of the address (1) +scope: "repo::github.com/acme/checkout" # host-qualified address (7) +key: "tests-need-local-postgres" +kind: "lesson" +content_type: "text/markdown" # (3) +written_by: { type: "agent", name: "claude-code" } +created: "2026-10-02T09:14:00Z" +updated: "2026-11-04T16:41:12Z" +seen_count: 4 +lifecycle: { expires_at: "2026-12-04T16:41:12Z", extended_by: "write", last_used: "2026-11-06T08:02:51Z" } # (5) +anchors: # (4) + - { path: "docker-compose.yml", lines: "12-20", commit: "0f4a1c9", sha256: "9b1e…" } +supersedes: null # (6) +attributes: # open, namespaced, kept on rewrite (2) + vcs.pr: 482 + ci.run_url: "https://github.com/acme/checkout/actions/runs/123456789" +--- +Integration tests need Postgres up first: docker compose up -d db. +``` + +None of this is new. Every item is borrowed from a format in the table or from OpenTelemetry. That's the point: the pieces already exist, spread across tools that can't read each other. + +## A test any shared format should pass + +One test has convinced me the shape is general enough: can it hold memory that **no model wrote**? + +CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. Usually that goes into `actions/cache` or an artifact bucket nobody else reads. LoreKit stores it in the same record as a lesson, with a versioned JSON body: + +```json +{ + "v": 1, + "updated_by_run": "https://github.com/acme/checkout/actions/runs/123456789", + "commit": "0f4a1c9", + "data": { "flaky": ["src/queue.test.ts::retries on 429"], "consecutive_green": 3 } +} +``` + +```bash +RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" + +jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ + '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \ +| npx @lorekit/cli write \ + --scope "repo::${GITHUB_REPOSITORY}" --key 'ci-state::flaky-tests' \ + --tags 'ci::test-state' --kind bus --host ci \ + --ttl-days 7 --remote --json +``` + +The same address, identity and lifecycle carry over. The job writes one key per fact and overwrites it in place, never one key per run. Because the job passes `--ttl-days 7` on every run, the expiry measures how long the job has been *silent*: the state lasts while the job runs and disappears a week after the workflow is deleted. And an agent asked to "fix the flaky tests" reads the same key and starts from the real list. + +
+ One record envelope with scope, key, kind, tags, ttl_days and origin fields, carrying two bodies. Left, a lesson written by an agent: prose, kind lesson or unset, read as advisory at session start. Right, CI state written by a script: versioned JSON, kind bus, parsed when looked up, with a 7-day TTL as a liveness check. Both live at repo::acme/checkout. +
One envelope, two bodies. A prose lesson and a JSON state record share an address, an identity and a lifecycle. Only how they're read differs.
+
+ +This is also where gaps 2 and 3 above hurt most. The JSON is only recognisable through conventions (`kind: bus`, a `v` inside the body). With `content_type` and `attributes`, any tool could read it. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) + +## Where to start + +Four of the seven changes only add fields: a stable ID (1), an open attribute map (2), a content type (3) and a `supersedes` link (6). They break nothing, so they're where I'd start. Anchors (4) and an explicit expiry rule (5) need more design. Changing the address grammar (7) changes LoreKit's canonical scope format, so it deserves its own proposal and discussion. + +If a shared spec emerges and makes different choices, LoreKit should follow it. These gaps are worth closing either way, and writing them down is how I'd like the conversation to start. If you see an eighth, tell me. + +## Look at a real record + +LoreKit's offline store writes plain Markdown records you can open, grep and diff: + +```bash +npx @lorekit/cli install +``` + +Then compare one with the sketch above, and tell me what's still missing. diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index 210dc18b2..1784c052f 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -1,14 +1,15 @@ --- title: "Agent memory needs its OpenTelemetry moment" -description: "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, what they already agree on, LoreKit's open-source record offered as one proposal with its gaps listed, and what a shared spec could look like." +description: "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, what they agree on, and what a shared spec for agent memory could look like." date: "2026-11-07" author: "Mads Thines" -readingMinutes: 16 +readingMinutes: 8 tags: ["agent-memory", "open-format", "standards", "opentelemetry", "lorekit"] order: 5 keywords: ["agent memory format", "open memory standard", "memory interoperability", "OpenTelemetry for agent memory", "memory specification", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "Portable Memory", "Open Memory Protocol", "CI state", "lorekit"] --- + ## Count the memory formats on your machine Here's a small exercise. List every place your coding tools keep something they "remember". @@ -24,7 +25,7 @@ That's five or six shapes, and **none of them can read the others**. Each one is reasonable on its own. Together, they're an industry rebuilding the same small idea again and again: *a note an agent left for its future self, stored at some address, true for a while.* A lesson your CI agent learns stays in whatever wrote it. Your editor agent can't read it. Neither can the next tool you try. -I build one of these tools, LoreKit, which is open source. So I'll be clear about what this post is and isn't. It isn't a pitch that LoreKit has solved memory. LoreKit's format is one proposal, and later in the post I'll list where it falls short. The point of the post is that **this is the moment to agree on a shared shape**, and that we've watched another field get this right. +I build one of these tools, LoreKit, which is open source. So I'll be clear about what this post is and isn't. It isn't a pitch that LoreKit has solved memory. LoreKit's format is one proposal among several, and it has gaps I'll write about separately. The point of the post is that **this is the moment to agree on a shared shape**, and that we've watched another field get this right. ## We've seen this before @@ -69,12 +70,12 @@ To agree on a shape, it helps to see the ones we already have. These change quic Each of these got something right that a shared spec should keep: -- **Instruction files keep a person in the loop.** `AGENTS.md` is the closest thing to a standard here because it's simply a Markdown file at a known path. But it holds *rules a person curates*, not lessons an agent writes as it goes. It can't tell you how often something recurs, nothing in it expires, and it can't target anything finer than a directory. -- **Anthropic's memory tool proves plain files go a long way.** The model gets six commands (`view`, `create`, `str_replace`, `insert`, `delete`, `rename`) under a `/memories` prefix, and your application decides where the files live. It's very portable. The cost is that two agents writing to the same directory have no shared idea of what a file *means*. -- **Copilot Memory checks itself against the code.** Every fact carries citations, which are links to specific code locations, and Copilot checks them against the current branch before using the fact. Facts disappear after 28 days without use. Of everything here, that pairing of *provenance you can check* and *forgetting by default* is the design I'd most like a shared spec to have. Today it's locked inside one product. -- **Letta treats memory as context budget.** A block has a `label`, a `value` and a character `limit`, and it's compiled into the prompt on every step. The format is honest that memory costs tokens every turn. -- **LangGraph shows how small the core can be.** It's `put(namespace, key, value)`, where the namespace is a tuple such as `("memories", user_id)`. Its optional TTL resets whenever an item is read. -- **The graphs model the world, and they model time.** The MCP reference server keeps entities, relations and observations in one `memory.jsonl` file. Graphiti stores four timestamps on every edge (`valid_at`, `invalid_at`, `created_at`, `expired_at`), so it can answer "what was true in March?" and keep a record of what replaced what. +- **`AGENTS.md`** shows that a plain file at a known path can become a standard. But it holds rules a person curates, not lessons an agent learns. +- **Anthropic's memory tool** shows how far plain files go. It also shows the cost: two agents sharing a directory have no shared idea of what a file *means*. +- **Copilot Memory** pairs *citations you can check* against the code with *forgetting by default*. It's the design I'd most like a shared spec to have, and today it's locked inside one product. +- **Letta** is honest that memory costs context tokens on every turn. +- **LangGraph** shows how small the core can be: `put(namespace, key, value)`. +- **Graphiti** models time properly, with separate timelines for when a fact was true and when it was recorded. ### The proposals that already exist @@ -100,96 +101,6 @@ That's the part I find hopeful. These four questions are the beginning of a data There's one real fork. **Who writes the memory?** In instruction files it's a person. In the memory tool, Letta and LoreKit, the model writes directly. In Mem0 and Graphiti, a service extracts facts from the conversation. A shared format doesn't need to pick one. It needs to *record* which one, so a reader knows how much to trust what it reads. -## One proposal: LoreKit's record - -Here is how LoreKit answers the four questions, offered as one data point. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. - -- **Where: a scope string.** One of `global`, `project::{name}`, `repo::{owner}/{repo}` or `branch::{owner}/{repo}::{branch}`, read from most specific to least. Unlike LangGraph's tuple, it's a string with a grammar, so it means the same thing in a URL, a filename, a CLI flag or a log line. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name. -- **Identity: `scope` + `key`, and a rewrite is counted.** Writing the same `scope` + `key` again updates the record and increments `seen_count`. One record that knows it was learned five times is more useful than five near-duplicates. But `seen_count` measures how often something came up, not whether it's true. -- **Body: plain text, read as advice.** The value is text up to 64 KiB, and LoreKit labels lessons *"considerations, not rules"* when it shows them to an agent. A rule belongs in `AGENTS.md`, decided by a person. A lesson is something an agent *noticed*, and it should be easy to ignore when it's wrong. -- **Lifecycle: permanent unless a TTL is set.** A write can carry `ttl_days`. Each later write that also carries one restarts the countdown, and a read never extends anything. Copilot and LangGraph made different choices here, and the figure below shows all three. -- **Kind: a short, closed list.** `lesson` (read every run), `bus` (a transient outcome, read only when deciding what to promote) and `signal` (a durable filter for one repo). This keeps records that serve different purposes from crowding each other out of an agent's context. -- **On disk: one Markdown file per record.** The frontmatter mirrors the hosted record's columns, with every value JSON-encoded so the file stays valid YAML and easy to grep. - -
- Four timelines comparing expiry. Copilot Memory deletes a fact 28 days after its last use. LangGraph's optional TTL is reset by each read. LoreKit without ttl_days keeps the record indefinitely. LoreKit with ttl_days ignores reads and restarts the countdown only when a write carries a TTL. -
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or writing it again (LoreKit). None is obviously right, which is why a shared format should record the rule on each record instead of choosing one for everyone.
-
- -## Where LoreKit's format falls short - -If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. - -1. **No stable ID that's separate from the address.** A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. -2. **No place for other tools' data, and rewrites lose it.** LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. -3. **The body doesn't say what it is.** LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. -4. **Provenance says where a lesson came from, not what it depends on.** `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. -5. **The expiry rule is implicit.** Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. -6. **Corrections leave no trail.** When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. -7. **The address grammar assumes GitHub, and nothing says which version a file uses.** `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. - -Here's a sketch of what a record could look like with those changes. It's an illustration, not a spec: - -```yaml ---- -spec: "agent-memory/0.1" # format version (7) -id: "01JC7Z3K9V2Q8M4T6R1W5X0YAB" # stable, independent of the address (1) -scope: "repo::github.com/acme/checkout" # host-qualified address (7) -key: "tests-need-local-postgres" -kind: "lesson" -content_type: "text/markdown" # (3) -written_by: { type: "agent", name: "claude-code" } -created: "2026-10-02T09:14:00Z" -updated: "2026-11-04T16:41:12Z" -seen_count: 4 -lifecycle: { expires_at: "2026-12-04T16:41:12Z", extended_by: "write", last_used: "2026-11-06T08:02:51Z" } # (5) -anchors: # (4) - - { path: "docker-compose.yml", lines: "12-20", commit: "0f4a1c9", sha256: "9b1e…" } -supersedes: null # (6) -attributes: # open, namespaced, kept on rewrite (2) - vcs.pr: 482 - ci.run_url: "https://github.com/acme/checkout/actions/runs/123456789" ---- -Integration tests need Postgres up first: docker compose up -d db. -``` - -None of this is new. Every item is borrowed from a format in the table or from OpenTelemetry. That's the point: the pieces already exist, spread across tools that can't read each other. - -## A test any shared format should pass - -One test has convinced me the shape is general enough: can it hold memory that **no model wrote**? - -CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. Usually that goes into `actions/cache` or an artifact bucket nobody else reads. LoreKit stores it in the same record as a lesson, with a versioned JSON body: - -```json -{ - "v": 1, - "updated_by_run": "https://github.com/acme/checkout/actions/runs/123456789", - "commit": "0f4a1c9", - "data": { "flaky": ["src/queue.test.ts::retries on 429"], "consecutive_green": 3 } -} -``` - -```bash -RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" - -jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ - '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \ -| npx @lorekit/cli write \ - --scope "repo::${GITHUB_REPOSITORY}" --key 'ci-state::flaky-tests' \ - --tags 'ci::test-state' --kind bus --host ci \ - --ttl-days 7 --remote --json -``` - -The same address, identity and lifecycle carry over. The job writes one key per fact and overwrites it in place, never one key per run. Because the job passes `--ttl-days 7` on every run, the expiry measures how long the job has been *silent*: the state lasts while the job runs and disappears a week after the workflow is deleted. And an agent asked to "fix the flaky tests" reads the same key and starts from the real list. - -
- One record envelope with scope, key, kind, tags, ttl_days and origin fields, carrying two bodies. Left, a lesson written by an agent: prose, kind lesson or unset, read as advisory at session start. Right, CI state written by a script: versioned JSON, kind bus, parsed when looked up, with a 7-day TTL as a liveness check. Both live at repo::acme/checkout. -
One envelope, two bodies. A prose lesson and a JSON state record share an address, an identity and a lifecycle. Only how they're read differs.
-
- -This is also where gaps 2 and 3 above hurt most. The JSON is only recognisable through conventions (`kind: bus`, a `v` inside the body). With `content_type` and `attributes`, any tool could read it. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) - ## What a shared spec could look like If agent memory followed OpenTelemetry's path, the spec would have the same four layers: @@ -207,15 +118,13 @@ The spec should live somewhere neutral, not in LoreKit's repo or any single vend These are the objections I'd raise if someone else had written this post. -**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post lists LoreKit's gaps, and why the sketch above changes LoreKit's own format in seven places. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. +**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post spends more time on other tools than on LoreKit, and why I'm writing up LoreKit's own gaps separately. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. **"MCP already is the standard."** MCP standardises the *call*, not the *record*. It's like having a shared network protocol without a shared data model. OpenTelemetry needed OTLP *and* the data model. Memory needs both too. -**"Just use files in git."** For one person and one tool, files work well. The memory tool and LoreKit's offline store are both files. What plain files lack is agreement on what a file *means*: what it applies to, whether it repeats an earlier lesson, and when it stops being true. Frontmatter is a good place for that agreement to live. - **"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and shared memory widens who gets to write those things. A spec should make the risk visible rather than hide it: record who wrote each memory and how, let readers check anchors, and allow signatures, which Portable Memory already proposes. No format makes poisoning impossible. -**"Another standard won't fix fragmentation."** That's the xkcd 927 problem: an attempt to unify everything adds one more competing standard. It's the strongest objection. My answer is the OpenTelemetry one. Agreement happened there when the existing projects *merged*, not when someone added a fifth. The useful work here is the same: take what Portable Memory, the Open Memory Protocol, Copilot, Letta, LangGraph and LoreKit each got right, and agree on one shape for the parts they share. +**"Another standard won't fix fragmentation."** That's the xkcd 927 problem: an attempt to unify everything adds one more competing standard. It's the strongest objection. My answer is the OpenTelemetry one. Agreement happened there when the existing projects *merged*, not when someone added a fifth. The useful work here is the same: take what the existing proposals and tools each got right, and agree on one shape for the parts they share. **"It's too early. Memory designs are still changing."** Maybe. But OpenTelemetry's data model has kept evolving since 2019, through versioning and conventions. Agreeing on a small core now doesn't freeze anything. It just stops every new tool from inventing its own core from scratch. @@ -229,7 +138,7 @@ I don't know the answers to these, and I'd rather work them out in public: - **Are anchors part of the core?** They're the best protection against stale memory, but they tie a record to a code host. - **Who should coordinate?** A working group across tool builders, a foundation, or a draft spec that people send changes to? -If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are in the repo as one concrete input to that conversation. +If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are MIT-licensed in the repo as one concrete input, and a follow-up post goes through where that format falls short. ## Look at a real record diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index fd99488b8..be9b5c3f7 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -48,7 +48,13 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ id: 'open-agent-memory-format', label: "Agent memory needs its OpenTelemetry moment", summary: - "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, what they already agree on, LoreKit's open-source record offered as one proposal with its gaps listed, and what a shared spec could look like.", + "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, what they agree on, and what a shared spec for agent memory could look like.", + }, + { + id: 'lorekit-memory-format-gaps', + label: "Seven gaps in LoreKit's memory format", + summary: + "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it has seven gaps, each tied to a failure you can point to. Here they are, with a sketch of a better record and the CI test any shared format should pass.", }, ]; From 5593bda6ab36f5b04ed563a5f4d5b4779793d9b6 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 7 Nov 2026 10:08:07 +0000 Subject: [PATCH 07/17] blog: mark the four additive memory-format gaps as closed The 'Seven gaps' post now describes the stable id, attributes, content_type and supersedes as shipped (#736), adds a What changed section with the new CLI flags, and keeps anchors, an explicit expiry rule and the address grammar as the open gaps. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../blog/lorekit-memory-format-gaps.mdx | 64 ++++++++++++++----- packages/web/src/lib/blog/sections.ts | 2 +- 2 files changed, 48 insertions(+), 18 deletions(-) diff --git a/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx b/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx index 0074b2326..6bcd575aa 100644 --- a/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx +++ b/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx @@ -1,9 +1,9 @@ --- title: "Seven gaps in LoreKit's memory format" -description: "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it has seven gaps, each tied to a failure you can point to. Here they are, with a sketch of a better record and the CI test any shared format should pass." +description: "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it had seven gaps, each tied to a failure you can point to. Four are now closed: a stable id, open attributes, a content type and a supersedes link. Here is what changed, what is still open, and the CI test any shared format should pass." date: "2026-11-14" author: "Mads Thines" -readingMinutes: 7 +readingMinutes: 10 tags: ["agent-memory", "open-format", "data-model", "ci", "lorekit"] order: 6 keywords: ["agent memory format", "lorekit", "memory record", "stable id", "attributes", "content type", "code anchors", "memory expiry", "supersedes", "scope grammar", "CI state", "JSON state records", "open memory standard"] @@ -13,11 +13,13 @@ keywords: ["agent memory format", "lorekit", "memory record", "stable id", "attr In [the previous post](/blog/open-agent-memory-format) I argued that agent memory needs what observability got from OpenTelemetry: one shared shape that every tool can read, so tools compete on what they do with memory instead of on how it's stored. -If I'm asking other tool builders to put their formats on the table, LoreKit's should go first, gaps included. This post describes LoreKit's record as it is today, then lists seven places where it falls short compared with the other formats and with OpenTelemetry. None of these gaps is hypothetical. Each one is a failure I can point to in the code. +If I'm asking other tool builders to put their formats on the table, LoreKit's should go first, gaps included. This post describes LoreKit's record, then lists seven places where it fell short compared with the other formats and with OpenTelemetry. None of these gaps was hypothetical. Each one was a failure I could point to in the code. + +Four of the seven only needed new fields, so LoreKit has now added them. The other three need more design, and they're still open. I've kept all seven in the list so you can see both what changed and what didn't. ## LoreKit's record today -Every memory format answers the same four questions: where a memory is true, what its identity is, what it says, and when it stops being true. Here is how LoreKit answers them. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. +Every memory format answers the same four questions: where a memory is true, what its identity is, what it says, and when it stops being true. Here is how LoreKit answers them, before the changes described below. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. - **Where: a scope string.** One of `global`, `project::{name}`, `repo::{owner}/{repo}` or `branch::{owner}/{repo}::{branch}`, read from most specific to least. Unlike LangGraph's tuple, it's a string with a grammar, so it means the same thing in a URL, a filename, a CLI flag or a log line. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name. - **Identity: `scope` + `key`, and a rewrite is counted.** Writing the same `scope` + `key` again updates the record and increments `seen_count`. One record that knows it was learned five times is more useful than five near-duplicates. But `seen_count` measures how often something came up, not whether it's true. @@ -31,17 +33,17 @@ Every memory format answers the same four questions: where a memory is true, wha
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or writing it again (LoreKit). None is obviously right, which is why a shared format should record the rule on each record instead of choosing one for everyone.
-## Where LoreKit's format falls short +## Where LoreKit's format fell short -If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. +If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. Gaps 1, 2, 3 and 6 are now closed; [What changed](#what-changed) shows how. -1. **No stable ID that's separate from the address.** A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. -2. **No place for other tools' data, and rewrites lose it.** LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. -3. **The body doesn't say what it is.** LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. -4. **Provenance says where a lesson came from, not what it depends on.** `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. -5. **The expiry rule is implicit.** Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. -6. **Corrections leave no trail.** When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. -7. **The address grammar assumes GitHub, and nothing says which version a file uses.** `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. +1. **No stable ID that's separate from the address.** *(Closed.)* A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. +2. **No place for other tools' data, and rewrites lose it.** *(Closed.)* LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. +3. **The body doesn't say what it is.** *(Closed.)* LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. +4. **Provenance says where a lesson came from, not what it depends on.** *(Open.)* `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. +5. **The expiry rule is implicit.** *(Open.)* Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. +6. **Corrections leave no trail.** *(Closed.)* When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. +7. **The address grammar assumes GitHub, and nothing says which version a file uses.** *(Open.)* `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. Here's a sketch of what a record could look like with those changes. It's an illustration, not a spec: @@ -70,6 +72,28 @@ Integration tests need Postgres up first: docker compose up -d db. None of this is new. Every item is borrowed from a format in the table or from OpenTelemetry. That's the point: the pieces already exist, spread across tools that can't read each other. +## What changed + +The four additive fields are now part of LoreKit's record, on the hosted store, the offline store, the MCP tools, the REST API and the CLI. A write that doesn't use them behaves exactly as before. + +- **A stable `id` (gap 1).** `memory.read` returns the record's `id`, and the offline store now gives every record one too. The id survives every update, and `cited` accepts it as well as `scope::key`, so a citation keeps pointing at the right lesson after its key is renamed. Moving a lesson to another scope still writes a new record, so the id doesn't follow it there. That case is what `supersedes` is for. +- **An open `attributes` map (gap 2).** Namespaced keys such as `ci.run_url` or `vcs.pr`, with string, number or boolean values, up to 32 per record. A write merges its attributes into what's stored, and a key set to `null` is removed. The offline store also stopped dropping fields it doesn't know: a field another tool adds to a record's frontmatter now survives a rewrite. +- **A `content_type` (gap 3).** `text/markdown` by default, or `application/json`. A JSON record whose value doesn't parse is rejected, so a reader that trusts the type never meets a body it can't read. +- **A `supersedes` link (gap 6).** The id of the record this one replaces. It has to name a record you can read and that has the same owner: personal to personal, or within the same org. Any other id fails exactly like one that doesn't exist, so the field can't be used to discover other people's records. + +Here's the CI record from the next section, written with the new fields: + +```bash +npx @lorekit/cli write \ + --scope "repo::acme/checkout" --key 'ci-state::flaky-tests' \ + --kind bus --host ci --ttl-days 7 \ + --content-type application/json \ + --attributes '{"ci.run_url":"https://github.com/acme/checkout/actions/runs/123456789"}' \ + --value '{"v":1,"data":{"flaky":["src/queue.test.ts::retries on 429"]}}' +``` + +All four are optional and additive, so a record written before this change reads exactly as it did. The MCP tool reference in [`llms.txt`](/llms.txt) and the [docs](/docs) list the exact limits. + ## A test any shared format should pass One test has convinced me the shape is general enough: can it hold memory that **no model wrote**? @@ -93,6 +117,8 @@ jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ | npx @lorekit/cli write \ --scope "repo::${GITHUB_REPOSITORY}" --key 'ci-state::flaky-tests' \ --tags 'ci::test-state' --kind bus --host ci \ + --content-type application/json \ + --attributes "{\"ci.run_url\":\"$RUN_URL\"}" \ --ttl-days 7 --remote --json ``` @@ -103,11 +129,15 @@ The same address, identity and lifecycle carry over. The job writes one key per
One envelope, two bodies. A prose lesson and a JSON state record share an address, an identity and a lifecycle. Only how they're read differs.
-This is also where gaps 2 and 3 above hurt most. The JSON is only recognisable through conventions (`kind: bus`, a `v` inside the body). With `content_type` and `attributes`, any tool could read it. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) +This is also where gaps 2 and 3 hurt most. Until recently, the JSON was only recognisable through conventions (`kind: bus`, a `v` inside the body). Now the record says `application/json`, and the run URL sits in `attributes`, where a tool can find it without parsing the body. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) + +## What's still open -## Where to start +Four of the seven changes only added fields, so they came first, and they broke nothing. Three remain: -Four of the seven changes only add fields: a stable ID (1), an open attribute map (2), a content type (3) and a `supersedes` link (6). They break nothing, so they're where I'd start. Anchors (4) and an explicit expiry rule (5) need more design. Changing the address grammar (7) changes LoreKit's canonical scope format, so it deserves its own proposal and discussion. +- **Anchors (4)** need a decision on what a reader does when the code under an anchor has changed: flag the lesson, hide it, or ask. +- **An explicit expiry rule (5)** needs the three tools' rules expressed as values of one field before it's worth adding. +- **The address grammar (7)** is LoreKit's canonical scope format. Changing it deserves its own proposal and discussion. If a shared spec emerges and makes different choices, LoreKit should follow it. These gaps are worth closing either way, and writing them down is how I'd like the conversation to start. If you see an eighth, tell me. @@ -119,4 +149,4 @@ LoreKit's offline store writes plain Markdown records you can open, grep and dif npx @lorekit/cli install ``` -Then compare one with the sketch above, and tell me what's still missing. +Write one with `--attributes` or `--content-type`, open the file, and compare it with the sketch above. Then tell me what's still missing. diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index be9b5c3f7..dc1427986 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -54,7 +54,7 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ id: 'lorekit-memory-format-gaps', label: "Seven gaps in LoreKit's memory format", summary: - "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it has seven gaps, each tied to a failure you can point to. Here they are, with a sketch of a better record and the CI test any shared format should pass.", + "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it had seven gaps, each tied to a failure you can point to. Four are now closed: a stable id, open attributes, a content type and a supersedes link. Here is what changed, what is still open, and the CI test any shared format should pass.", }, ]; From 4fba45a40f911dc7474b549e4fdcf99808cbd546 Mon Sep 17 00:00:00 2001 From: mthines Date: Sun, 8 Nov 2026 06:36:57 +0100 Subject: [PATCH 08/17] chore: trigger Vercel preview deployment From 251277c9b67507150c75fb42664187a0a6877199 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 07:01:04 +0000 Subject: [PATCH 09/17] blog: show one illustrative record in the shared-spec section Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../content/blog/open-agent-memory-format.mdx | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index 1784c052f..eb56465e7 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -110,6 +110,25 @@ If agent memory followed OpenTelemetry's path, the spec would have the same four - **A contract.** A small set of memory operations (write, read, list, search, archive) with fixed inputs and outputs, exposed over MCP. Every memory server today speaks MCP but exposes different tools, so agents can call any of them and none of them mean the same thing. - **Conformance tests.** Round-trip a record and check nothing was lost. Check that unknown attributes survive a rewrite. Check that expiry rules behave as stated. +Here's roughly what one record could look like on the wire. It's an illustration of the data model, not a proposal for exact field names: + +```json +{ + "spec": "agent-memory/0.1", + "id": "8f0c2a6e-4b1d-4c7a-9e2f-3a5b6c7d8e9f", + "address": { "level": "repo", "value": "github.com/acme/checkout" }, + "key": "tests-need-local-postgres", + "kind": "lesson", + "written_by": { "type": "agent", "name": "claude-code" }, + "content_type": "text/markdown", + "body": "Integration tests need Postgres up first: docker compose up -d db.", + "lifecycle": { "expires_at": null, "extended_by": "write" }, + "attributes": { "vcs.pr": 482, "ci.run_url": "https://ci.example/runs/123" } +} +``` + +Each line answers one of the four questions or the fork above: `address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` records who wrote it. Everything a tool needs beyond that goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. [The follow-up post](/blog/lorekit-memory-format-gaps) holds LoreKit's own record up against this shape. + What stays *out* of the spec matters as much: storage, ranking, embeddings, whether a model or an extractor writes, the user interface. Those are where tools should compete, the way observability vendors compete on everything after the data arrives. The spec should live somewhere neutral, not in LoreKit's repo or any single vendor's. The Agentic AI Foundation already looks after MCP and `AGENTS.md`, which makes it one natural option, though that's for the people involved to decide. From e66ce3b6aa1bc3f64448da6724c6f3b33537ef1f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 07:15:20 +0000 Subject: [PATCH 10/17] blog: drop the 'Seven gaps' post and every reference to it Removes the post, its registry entry and its two figures, and rewords the three sentences in the OpenTelemetry post that pointed to it. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../references/figure-style.md | 3 +- .../web/public/blog/forgetting-defaults.svg | 62 ------- .../web/public/blog/one-record-two-bodies.svg | 50 ------ .../blog/lorekit-memory-format-gaps.mdx | 152 ------------------ .../content/blog/open-agent-memory-format.mdx | 6 +- packages/web/src/lib/blog/sections.ts | 6 - 6 files changed, 4 insertions(+), 275 deletions(-) delete mode 100644 packages/web/public/blog/forgetting-defaults.svg delete mode 100644 packages/web/public/blog/one-record-two-bodies.svg delete mode 100644 packages/web/src/content/blog/lorekit-memory-format-gaps.mdx diff --git a/.claude/skills/write-blog-post/references/figure-style.md b/.claude/skills/write-blog-post/references/figure-style.md index 9dffab770..1da94db79 100644 --- a/.claude/skills/write-blog-post/references/figure-style.md +++ b/.claude/skills/write-blog-post/references/figure-style.md @@ -57,8 +57,7 @@ Use amber for one thing per figure: the point the reader should notice. - **Record / card:** one big card, `.lbl` zone headings, field names left in `.k`, explanations right-aligned in `.lbl` (see `memory-record.svg`). - **Timeline:** label column on the left, track lines from x=200, filled dots - for writes, hollow dots for reads, a tinted bar for a window (see - `forgetting-defaults.svg`). + for writes, hollow dots for reads, a tinted bar for a window. ## Gotchas diff --git a/packages/web/public/blog/forgetting-defaults.svg b/packages/web/public/blog/forgetting-defaults.svg deleted file mode 100644 index 4fd410367..000000000 --- a/packages/web/public/blog/forgetting-defaults.svg +++ /dev/null @@ -1,62 +0,0 @@ - - - - - - - - - write - read - expires - time → - - - Copilot Memory - forgets by default - - - - - - 28 days unused - - - LangGraph store - read resets the TTL - - - - - - - one TTL after the last read - - - LoreKit - no ttl_days (default) - - - - - kept — grooming proposes clean-up - - - LoreKit - with ttl_days - - - - - - - read: no change - a write with a TTL restarts it - - - Three tools, three answers to "what keeps a memory alive?" — use, reads, or explicit writes. - diff --git a/packages/web/public/blog/one-record-two-bodies.svg b/packages/web/public/blog/one-record-two-bodies.svg deleted file mode 100644 index c2ee69b83..000000000 --- a/packages/web/public/blog/one-record-two-bodies.svg +++ /dev/null @@ -1,50 +0,0 @@ - - - - - - - - - - THE SAME ENVELOPE - scope · key · kind · tags · ttl_days · origin_* - - - - - - - a lesson - written by an agent - - Integration tests need Postgres - up first: docker compose up -d db. - kind: lesson (or unset) - read as advisory - shown at session start - lifetime kept unless a TTL is set - - - - CI state - written by a script - - {"v":1, "updated_by_run":…, - "commit":…, "data":{"flaky":[…]}} - kind: bus · host: ci - read as parsed, version-checked - shown only when looked up - lifetime 7-day TTL = liveness - - - - - - repo::acme/checkout - One address, one store — the agent and the CI job both read it. - diff --git a/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx b/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx deleted file mode 100644 index 6bcd575aa..000000000 --- a/packages/web/src/content/blog/lorekit-memory-format-gaps.mdx +++ /dev/null @@ -1,152 +0,0 @@ ---- -title: "Seven gaps in LoreKit's memory format" -description: "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it had seven gaps, each tied to a failure you can point to. Four are now closed: a stable id, open attributes, a content type and a supersedes link. Here is what changed, what is still open, and the CI test any shared format should pass." -date: "2026-11-14" -author: "Mads Thines" -readingMinutes: 10 -tags: ["agent-memory", "open-format", "data-model", "ci", "lorekit"] -order: 6 -keywords: ["agent memory format", "lorekit", "memory record", "stable id", "attributes", "content type", "code anchors", "memory expiry", "supersedes", "scope grammar", "CI state", "JSON state records", "open memory standard"] ---- - -## Why write about our own gaps - -In [the previous post](/blog/open-agent-memory-format) I argued that agent memory needs what observability got from OpenTelemetry: one shared shape that every tool can read, so tools compete on what they do with memory instead of on how it's stored. - -If I'm asking other tool builders to put their formats on the table, LoreKit's should go first, gaps included. This post describes LoreKit's record, then lists seven places where it fell short compared with the other formats and with OpenTelemetry. None of these gaps was hypothetical. Each one was a failure I could point to in the code. - -Four of the seven only needed new fields, so LoreKit has now added them. The other three need more design, and they're still open. I've kept all seven in the list so you can see both what changed and what didn't. - -## LoreKit's record today - -Every memory format answers the same four questions: where a memory is true, what its identity is, what it says, and when it stops being true. Here is how LoreKit answers them, before the changes described below. The full rationale for each column is in [an earlier post](/blog/agent-memory-data-model). Everything here, including the schema and the scope validator, is MIT-licensed in the repo. - -- **Where: a scope string.** One of `global`, `project::{name}`, `repo::{owner}/{repo}` or `branch::{owner}/{repo}::{branch}`, read from most specific to least. Unlike LangGraph's tuple, it's a string with a grammar, so it means the same thing in a URL, a filename, a CLI flag or a log line. The separator is `::` because `/` already appears inside repo and branch names, and git doesn't allow `:` in a branch name. -- **Identity: `scope` + `key`, and a rewrite is counted.** Writing the same `scope` + `key` again updates the record and increments `seen_count`. One record that knows it was learned five times is more useful than five near-duplicates. But `seen_count` measures how often something came up, not whether it's true. -- **Body: plain text, read as advice.** The value is text up to 64 KiB, and LoreKit labels lessons *"considerations, not rules"* when it shows them to an agent. A rule belongs in `AGENTS.md`, decided by a person. A lesson is something an agent *noticed*, and it should be easy to ignore when it's wrong. -- **Lifecycle: permanent unless a TTL is set.** A write can carry `ttl_days`. Each later write that also carries one restarts the countdown, and a read never extends anything. Copilot and LangGraph made different choices here, and the figure below shows all three. -- **Kind: a short, closed list.** `lesson` (read every run), `bus` (a transient outcome, read only when deciding what to promote) and `signal` (a durable filter for one repo). This keeps records that serve different purposes from crowding each other out of an agent's context. -- **On disk: one Markdown file per record.** The frontmatter mirrors the hosted record's columns, with every value JSON-encoded so the file stays valid YAML and easy to grep. - -
- Four timelines comparing expiry. Copilot Memory deletes a fact 28 days after its last use. LangGraph's optional TTL is reset by each read. LoreKit without ttl_days keeps the record indefinitely. LoreKit with ttl_days ignores reads and restarts the countdown only when a write carries a TTL. -
Three tools, three answers to "what keeps a memory alive?": using it (Copilot), reading it (LangGraph), or writing it again (LoreKit). None is obviously right, which is why a shared format should record the rule on each record instead of choosing one for everyone.
-
- -## Where LoreKit's format fell short - -If I hold LoreKit's record up against the other formats and against what OpenTelemetry got right, I find seven gaps. Each one maps to a failure I can point to. I'd make these changes whether or not a shared spec ever happens, and I'd want a shared spec to include them. Gaps 1, 2, 3 and 6 are now closed; [What changed](#what-changed) shows how. - -1. **No stable ID that's separate from the address.** *(Closed.)* A record's identity *is* its `scope` + `key`. So moving a lesson from a branch to the repo, or renaming its key, turns it into a different record. Everything that pointed at it, like the citations agents write when a lesson helped them, now points at nothing. OpenTelemetry gives every span an ID that doesn't depend on its name. Memory records need the same: a stable `id`, plus the address as a field that can change. -2. **No place for other tools' data, and rewrites lose it.** *(Closed.)* LoreKit has a fixed list of fields. When it rewrites a local file, any field another tool added is dropped. OpenTelemetry's answer is an open map of **attributes**, with shared names agreed through conventions. A memory record should have the same: an `attributes` map, namespaced keys like `ci.run_url` or `vcs.commit`, and a rule that a reader keeps keys it doesn't understand. -3. **The body doesn't say what it is.** *(Closed.)* LoreKit stores CI state as JSON inside the text value, and a reader can only tell from conventions like `kind: bus`. A `content_type` field (`text/markdown`, `application/json`) would let any reader know whether to show the body or parse it. -4. **Provenance says where a lesson came from, not what it depends on.** *(Open.)* `origin_commit` records the commit a lesson was written on. It doesn't say which code the lesson is *about*. Copilot shows the better design: **anchors** (a file path, a line range, a commit and a content hash) that a reader *may* check. If the code under an anchor has changed, the lesson is flagged as possibly stale instead of being used blindly. That's the strongest protection against stale memory I've seen. -5. **The expiry rule is implicit.** *(Open.)* Permanent-unless-a-TTL-is-set is a LoreKit default you have to know about. A record should state its rule: what extends its life (a write, a read, or use), and when it was last used. That makes the Copilot, LangGraph and LoreKit approaches different values of the same field, not three incompatible systems. -6. **Corrections leave no trail.** *(Closed.)* When a lesson turns out to be wrong, LoreKit's options are to overwrite it or archive it. A `supersedes` link from the new record to the old one would keep the history. You'd get the useful part of Graphiti's timelines without needing a graph. -7. **The address grammar assumes GitHub, and nothing says which version a file uses.** *(Open.)* `repo::` accepts exactly `owner/repo`, so a GitLab path like `group/subgroup/repo` or a self-hosted server doesn't fit, and monorepos have no level between `repo` and `branch`. Records also carry no format version, so a future reader can't tell which rules a file was written under. - -Here's a sketch of what a record could look like with those changes. It's an illustration, not a spec: - -```yaml ---- -spec: "agent-memory/0.1" # format version (7) -id: "01JC7Z3K9V2Q8M4T6R1W5X0YAB" # stable, independent of the address (1) -scope: "repo::github.com/acme/checkout" # host-qualified address (7) -key: "tests-need-local-postgres" -kind: "lesson" -content_type: "text/markdown" # (3) -written_by: { type: "agent", name: "claude-code" } -created: "2026-10-02T09:14:00Z" -updated: "2026-11-04T16:41:12Z" -seen_count: 4 -lifecycle: { expires_at: "2026-12-04T16:41:12Z", extended_by: "write", last_used: "2026-11-06T08:02:51Z" } # (5) -anchors: # (4) - - { path: "docker-compose.yml", lines: "12-20", commit: "0f4a1c9", sha256: "9b1e…" } -supersedes: null # (6) -attributes: # open, namespaced, kept on rewrite (2) - vcs.pr: 482 - ci.run_url: "https://github.com/acme/checkout/actions/runs/123456789" ---- -Integration tests need Postgres up first: docker compose up -d db. -``` - -None of this is new. Every item is borrowed from a format in the table or from OpenTelemetry. That's the point: the pieces already exist, spread across tools that can't read each other. - -## What changed - -The four additive fields are now part of LoreKit's record, on the hosted store, the offline store, the MCP tools, the REST API and the CLI. A write that doesn't use them behaves exactly as before. - -- **A stable `id` (gap 1).** `memory.read` returns the record's `id`, and the offline store now gives every record one too. The id survives every update, and `cited` accepts it as well as `scope::key`, so a citation keeps pointing at the right lesson after its key is renamed. Moving a lesson to another scope still writes a new record, so the id doesn't follow it there. That case is what `supersedes` is for. -- **An open `attributes` map (gap 2).** Namespaced keys such as `ci.run_url` or `vcs.pr`, with string, number or boolean values, up to 32 per record. A write merges its attributes into what's stored, and a key set to `null` is removed. The offline store also stopped dropping fields it doesn't know: a field another tool adds to a record's frontmatter now survives a rewrite. -- **A `content_type` (gap 3).** `text/markdown` by default, or `application/json`. A JSON record whose value doesn't parse is rejected, so a reader that trusts the type never meets a body it can't read. -- **A `supersedes` link (gap 6).** The id of the record this one replaces. It has to name a record you can read and that has the same owner: personal to personal, or within the same org. Any other id fails exactly like one that doesn't exist, so the field can't be used to discover other people's records. - -Here's the CI record from the next section, written with the new fields: - -```bash -npx @lorekit/cli write \ - --scope "repo::acme/checkout" --key 'ci-state::flaky-tests' \ - --kind bus --host ci --ttl-days 7 \ - --content-type application/json \ - --attributes '{"ci.run_url":"https://github.com/acme/checkout/actions/runs/123456789"}' \ - --value '{"v":1,"data":{"flaky":["src/queue.test.ts::retries on 429"]}}' -``` - -All four are optional and additive, so a record written before this change reads exactly as it did. The MCP tool reference in [`llms.txt`](/llms.txt) and the [docs](/docs) list the exact limits. - -## A test any shared format should pass - -One test has convinced me the shape is general enough: can it hold memory that **no model wrote**? - -CI jobs need memory too. A test workflow wants to know which tests were flaky last time. A deploy job wants the last SHA it shipped. Usually that goes into `actions/cache` or an artifact bucket nobody else reads. LoreKit stores it in the same record as a lesson, with a versioned JSON body: - -```json -{ - "v": 1, - "updated_by_run": "https://github.com/acme/checkout/actions/runs/123456789", - "commit": "0f4a1c9", - "data": { "flaky": ["src/queue.test.ts::retries on 429"], "consecutive_green": 3 } -} -``` - -```bash -RUN_URL="${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" - -jq -nc --arg run "$RUN_URL" --arg sha "$GITHUB_SHA" --argjson data "$NEW_DATA" \ - '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \ -| npx @lorekit/cli write \ - --scope "repo::${GITHUB_REPOSITORY}" --key 'ci-state::flaky-tests' \ - --tags 'ci::test-state' --kind bus --host ci \ - --content-type application/json \ - --attributes "{\"ci.run_url\":\"$RUN_URL\"}" \ - --ttl-days 7 --remote --json -``` - -The same address, identity and lifecycle carry over. The job writes one key per fact and overwrites it in place, never one key per run. Because the job passes `--ttl-days 7` on every run, the expiry measures how long the job has been *silent*: the state lasts while the job runs and disappears a week after the workflow is deleted. And an agent asked to "fix the flaky tests" reads the same key and starts from the real list. - -
- One record envelope with scope, key, kind, tags, ttl_days and origin fields, carrying two bodies. Left, a lesson written by an agent: prose, kind lesson or unset, read as advisory at session start. Right, CI state written by a script: versioned JSON, kind bus, parsed when looked up, with a 7-day TTL as a liveness check. Both live at repo::acme/checkout. -
One envelope, two bodies. A prose lesson and a JSON state record share an address, an identity and a lifecycle. Only how they're read differs.
-
- -This is also where gaps 2 and 3 hurt most. Until recently, the JSON was only recognisable through conventions (`kind: bus`, a `v` inside the body). Now the record says `application/json`, and the run URL sits in `attributes`, where a tool can find it without parsing the body. (The [full CI recipe](/docs/use-cases) covers tokens, guardrails, and why the store must never block a build.) - -## What's still open - -Four of the seven changes only added fields, so they came first, and they broke nothing. Three remain: - -- **Anchors (4)** need a decision on what a reader does when the code under an anchor has changed: flag the lesson, hide it, or ask. -- **An explicit expiry rule (5)** needs the three tools' rules expressed as values of one field before it's worth adding. -- **The address grammar (7)** is LoreKit's canonical scope format. Changing it deserves its own proposal and discussion. - -If a shared spec emerges and makes different choices, LoreKit should follow it. These gaps are worth closing either way, and writing them down is how I'd like the conversation to start. If you see an eighth, tell me. - -## Look at a real record - -LoreKit's offline store writes plain Markdown records you can open, grep and diff: - -```bash -npx @lorekit/cli install -``` - -Write one with `--attributes` or `--content-type`, open the file, and compare it with the sketch above. Then tell me what's still missing. diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index eb56465e7..13d548d01 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -127,7 +127,7 @@ Here's roughly what one record could look like on the wire. It's an illustration } ``` -Each line answers one of the four questions or the fork above: `address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` records who wrote it. Everything a tool needs beyond that goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. [The follow-up post](/blog/lorekit-memory-format-gaps) holds LoreKit's own record up against this shape. +Each line answers one of the four questions or the fork above: `address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` records who wrote it. Everything a tool needs beyond that goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. What stays *out* of the spec matters as much: storage, ranking, embeddings, whether a model or an extractor writes, the user interface. Those are where tools should compete, the way observability vendors compete on everything after the data arrives. @@ -137,7 +137,7 @@ The spec should live somewhere neutral, not in LoreKit's repo or any single vend These are the objections I'd raise if someone else had written this post. -**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post spends more time on other tools than on LoreKit, and why I'm writing up LoreKit's own gaps separately. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. +**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post spends more time on other tools than on LoreKit, and why LoreKit is one row in the table rather than the template. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. **"MCP already is the standard."** MCP standardises the *call*, not the *record*. It's like having a shared network protocol without a shared data model. OpenTelemetry needed OTLP *and* the data model. Memory needs both too. @@ -157,7 +157,7 @@ I don't know the answers to these, and I'd rather work them out in public: - **Are anchors part of the core?** They're the best protection against stale memory, but they tie a record to a code host. - **Who should coordinate?** A working group across tool builders, a foundation, or a draft spec that people send changes to? -If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are MIT-licensed in the repo as one concrete input, and a follow-up post goes through where that format falls short. +If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are MIT-licensed in the repo as one concrete input. ## Look at a real record diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index dc1427986..1fd4e530e 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -50,12 +50,6 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ summary: "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, what they agree on, and what a shared spec for agent memory could look like.", }, - { - id: 'lorekit-memory-format-gaps', - label: "Seven gaps in LoreKit's memory format", - summary: - "LoreKit's memory record is one open-source proposal for a shared agent-memory format, and it isn't finished. Held up against Copilot Memory, Graphiti, OpenTelemetry and the rest, it had seven gaps, each tied to a failure you can point to. Four are now closed: a stable id, open attributes, a content type and a supersedes link. Here is what changed, what is still open, and the CI test any shared format should pass.", - }, ]; /** URL slugs / MDX filename stems, in listing order (newest first). */ From 378ef979a11e6aa1023b083b93b694a38c258399 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 07:15:28 +0000 Subject: [PATCH 11/17] blog: drop the last pointer to a separate gaps post Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- packages/web/src/content/blog/open-agent-memory-format.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index 13d548d01..8284a6868 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -25,7 +25,7 @@ That's five or six shapes, and **none of them can read the others**. Each one is reasonable on its own. Together, they're an industry rebuilding the same small idea again and again: *a note an agent left for its future self, stored at some address, true for a while.* A lesson your CI agent learns stays in whatever wrote it. Your editor agent can't read it. Neither can the next tool you try. -I build one of these tools, LoreKit, which is open source. So I'll be clear about what this post is and isn't. It isn't a pitch that LoreKit has solved memory. LoreKit's format is one proposal among several, and it has gaps I'll write about separately. The point of the post is that **this is the moment to agree on a shared shape**, and that we've watched another field get this right. +I build one of these tools, LoreKit, which is open source. So I'll be clear about what this post is and isn't. It isn't a pitch that LoreKit has solved memory. LoreKit's format is one proposal among several, and it has gaps of its own. The point of the post is that **this is the moment to agree on a shared shape**, and that we've watched another field get this right. ## We've seen this before From 8e091b6d38178050895c6646d65968f414bf7ad0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 08:05:39 +0000 Subject: [PATCH 12/17] blog: tighten the OpenTelemetry post and add the everyone-builds-their-own angle Cuts about a third: folds the four-questions section into the spec section, shortens the tour, drops the weakest objection and the map figure that repeated the table. Fixes the shape counts, the OpenTelemetry figure caption, the four-layers mismatch, an overstated MCP claim, and an undefined use of 'anchors'. Adds an early paragraph on how many people are building their own memory layer, and why that needs an agreement rather than another tool. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../web/public/blog/memory-formats-map.svg | 66 ---------- .../content/blog/open-agent-memory-format.mdx | 115 +++++++----------- packages/web/src/lib/blog/sections.ts | 2 +- 3 files changed, 42 insertions(+), 141 deletions(-) delete mode 100644 packages/web/public/blog/memory-formats-map.svg diff --git a/packages/web/public/blog/memory-formats-map.svg b/packages/web/public/blog/memory-formats-map.svg deleted file mode 100644 index 7c75839bf..000000000 --- a/packages/web/public/blog/memory-formats-map.svg +++ /dev/null @@ -1,66 +0,0 @@ - - - - - - - - WHO WRITES × HOW STRUCTURED - - - A PERSON - THE AGENT - AN EXTRACTOR - - - - - - - - MORE STRUCTURE - - - plain files - labeled blocks - facts + metadata - typed records - & graphs - - - - - - - AGENTS.md · CLAUDE.md - - Cursor rules - - - - memory-tool files - - Letta blocks - - Copilot Memory - - MCP knowledge graph - - LangGraph store - - LoreKit record - - - - Mem0 facts - - Graphiti edges - - - Rough placement. Neither axis decides whether two tools can share memory — - that comes down to whether an address means the same thing to every reader. - diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index 8284a6868..c6d355660 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -1,55 +1,55 @@ --- title: "Agent memory needs its OpenTelemetry moment" -description: "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, what they agree on, and what a shared spec for agent memory could look like." +description: "Every agent tool stores memory in its own shape, and every week someone builds another one. None of them can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, and what a shared spec could look like." date: "2026-11-07" author: "Mads Thines" -readingMinutes: 8 +readingMinutes: 7 tags: ["agent-memory", "open-format", "standards", "opentelemetry", "lorekit"] order: 5 -keywords: ["agent memory format", "open memory standard", "memory interoperability", "OpenTelemetry for agent memory", "memory specification", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "Portable Memory", "Open Memory Protocol", "CI state", "lorekit"] +keywords: ["agent memory format", "open memory standard", "memory interoperability", "OpenTelemetry for agent memory", "memory specification", "AGENTS.md", "Copilot Memory", "Letta memory blocks", "Mem0", "LangGraph store", "MCP memory server", "Graphiti", "Portable Memory", "Open Memory Protocol", "lorekit"] --- ## Count the memory formats on your machine -Here's a small exercise. List every place your coding tools keep something they "remember". - -A typical setup today looks like this: +List every place your coding tools keep something they "remember". A typical setup today: - an `AGENTS.md` or `CLAUDE.md` in most repos, and `.cursor/rules/*.mdc` in a few -- an API agent writing files to a `/memories` directory +- an agent writing files to a `/memories` directory through Anthropic's memory tool - Copilot keeping facts about each repo on GitHub's side - a memory MCP server keeping a JSONL knowledge graph -That's five or six shapes, and **none of them can read the others**. +That's four different shapes, and **none of them can read the others**. A lesson your CI agent learns stays in whatever wrote it. Your editor agent can't read it, and neither can the next tool you try. + +Each format is reasonable on its own. Together, they rebuild the same small idea again and again: *a note an agent left for its future self, stored at some address, true for a while.* -Each one is reasonable on its own. Together, they're an industry rebuilding the same small idea again and again: *a note an agent left for its future self, stored at some address, true for a while.* A lesson your CI agent learns stays in whatever wrote it. Your editor agent can't read it. Neither can the next tool you try. +And the list keeps growing. Browse r/ClaudeAI, r/LocalLLaMA or r/cursor and you'll find a steady stream of "I built a memory layer for my agent" posts, each with its own storage, its own schema and its own MCP tools. That's good evidence the problem is real: people keep hitting it badly enough to build something. But every new solution adds one more format that nothing else can read. **The problem isn't a missing tool. It's a missing agreement**, and that only comes from the community working on it together. -I build one of these tools, LoreKit, which is open source. So I'll be clear about what this post is and isn't. It isn't a pitch that LoreKit has solved memory. LoreKit's format is one proposal among several, and it has gaps of its own. The point of the post is that **this is the moment to agree on a shared shape**, and that we've watched another field get this right. +I build one of these tools, LoreKit, which is open source. This isn't a pitch that LoreKit has solved memory. Its format is one proposal among several, with gaps of its own. The point is that **this is the moment to agree on a shared shape**, and that another field has already shown how. ## We've seen this before -Ten years ago, observability looked a lot like agent memory does now. Every monitoring vendor shipped its own agent, its own instrumentation library and its own data format. If you instrumented your code for one vendor, switching meant doing it again. Two open projects tried to fix this from different directions: OpenTracing (a common API) and OpenCensus (Google's instrumentation libraries and exporters). For a while they competed too. +Before 2019, every observability vendor shipped its own agent, its own instrumentation library and its own data format. Switching vendors meant instrumenting your code again. Two open projects, OpenTracing and OpenCensus, tried to fix that and competed with each other too. -In 2019 they merged into **OpenTelemetry**. The important part wasn't the code. It was what the vendors agreed to stop competing on. OpenTelemetry standardised four layers: +In 2019 they merged into **OpenTelemetry**. What mattered was what the vendors agreed to stop competing on: 1. **A data model**: what a span, a metric and a log record are. -2. **Semantic conventions**: shared names like `service.name`, `http.*` and `db.*`, so the same thing is called the same thing everywhere. +2. **Semantic conventions**: shared names like `service.name` and `http.*`. 3. **A protocol**: OTLP, one way to send the data. -4. **Reference implementations**: SDKs and the Collector, so nobody had to start from zero. +4. **Reference implementations**: shared SDKs and the Collector. -Vendors kept competing, but on what they *do* with the data: storage, querying, analysis, user experience. Users got to instrument once. I work with OpenTelemetry every day at [Dash0](https://www.dash0.com), and that one agreement is why a whole ecosystem of tools can read each other's data today. +Vendors kept competing on what they *do* with the data: storage, querying, analysis. Users instrument once. I work with OpenTelemetry every day at [Dash0](https://www.dash0.com), and that agreement is why a whole ecosystem of tools can read each other's data.
Two columns compare observability, which OpenTelemetry settled, with agent memory, which is still open, across four layers: data model, semantic conventions, protocol, and implementations. For memory, the first three layers are drawn dashed because nobody has agreed on them yet. -
The same four layers. Observability agreed on the first three and kept competing on the fourth. Agent memory hasn't agreed on any of them yet.
+
The same four layers. Observability agreed on them and competes on what happens after the data arrives. Agent memory has plenty of implementations, but nothing agreed underneath them.
-Agent memory is at the stage observability was in before 2019. There are many formats, a few early proposals and no agreement. The timing is good for two reasons. The formats are still young enough to change. And the agent ecosystem already has a neutral home for shared standards: MCP and `AGENTS.md` both now sit under the Linux Foundation's Agentic AI Foundation. +Agent memory is where observability was before 2019. The timing is good: the formats are young enough to change, and there's already a neutral home for agent standards. MCP and `AGENTS.md` both sit under the Linux Foundation's Agentic AI Foundation. ## A tour of the shapes -To agree on a shape, it helps to see the ones we already have. These change quickly, so treat the details as a snapshot from late 2026. +These change quickly, so treat the details as a snapshot from late 2026. | Family | Example | The unit of memory | Addressed by | Forgetting | | --- | --- | --- | --- | --- | @@ -63,54 +63,26 @@ To agree on a shape, it helps to see the ones we already have. These change quic | Temporal graph | Graphiti / Zep | A fact on a graph edge | Entities + edge | Facts are invalidated rather than deleted, on two timelines | | Addressed record | LoreKit | A record with a scope, key and lifecycle | A scope string + `key` | Never, unless a TTL is set | -
- A grid placing agent memory formats by who writes them and how structured they are. A person writes AGENTS.md, CLAUDE.md and Cursor rules. The agent writes memory-tool files, Letta blocks, Copilot Memory, the MCP knowledge graph, LangGraph store items and LoreKit records. An extractor writes Mem0 facts and Graphiti edges. Rows run from plain files at the bottom to typed records and graphs at the top. -
The same idea in ten shapes, placed roughly by who writes each one and how much structure it has. Neither axis decides whether two tools can share memory. What decides it is whether an address means the same thing to every reader.
-
- -Each of these got something right that a shared spec should keep: - -- **`AGENTS.md`** shows that a plain file at a known path can become a standard. But it holds rules a person curates, not lessons an agent learns. -- **Anthropic's memory tool** shows how far plain files go. It also shows the cost: two agents sharing a directory have no shared idea of what a file *means*. -- **Copilot Memory** pairs *citations you can check* against the code with *forgetting by default*. It's the design I'd most like a shared spec to have, and today it's locked inside one product. -- **Letta** is honest that memory costs context tokens on every turn. -- **LangGraph** shows how small the core can be: `put(namespace, key, value)`. -- **Graphiti** models time properly, with separate timelines for when a fact was true and when it was recorded. +A few of them got something right that a shared spec should keep. **Copilot Memory** pairs citations you can check against the code with forgetting by default, though it's locked inside one product. **LangGraph** shows how small the core can be: `put(namespace, key, value)`. **Graphiti** models time properly, with separate timelines for when a fact was true and when it was recorded. And **`AGENTS.md`** shows that a plain file at a known path can become a standard. ### The proposals that already exist -Several people have already started on this. MacPaw Research has proposed **Portable Memory**, a `.mem` bundle with byte-reproducible serialization, deletion tombstones and optional signing. Its own authors say it's not yet widely adopted. The **Open Memory Protocol** is an April 2026 draft for moving conversations, and the memories derived from them, between platforms as a ZIP file with a manifest. Cognee offers a **COGX** exchange format with importers from Mem0, Letta and Zep. +Several people have started on this. MacPaw Research's **Portable Memory** is a `.mem` bundle with byte-reproducible serialization, deletion tombstones and optional signing; its authors say it isn't widely adopted yet. The **Open Memory Protocol** is an April 2026 draft for moving conversations and the memories derived from them between platforms as a ZIP with a manifest. Cognee's **COGX** format imports from Mem0, Letta and Zep. -These are starting points, not competitors. They mostly solve **export**: "how do I move my memory from vendor A to vendor B?" That matters most for personal assistants. Coding tools have a second problem that's still open: **live addressing**. When a CI job, an editor agent and a review bot work in the same repo at the same time, how do they write to and read from the *same* place, and agree on what it means? A complete spec needs both, the way OpenTelemetry needed both a data model and a protocol. +These mostly solve **export**: moving one person's memory from vendor A to vendor B. Coding tools also have a second, open problem: **live addressing**. When a CI job, an editor agent and a review bot work in the same repo at once, how do they read and write the *same* place and agree on what it means? A complete spec needs both.
Two problems side by side. Left, export: memory in vendor A is packed into a bundle such as a .mem file, an .omp.zip or COGX, and unpacked in vendor B. Right, live addressing: a CI job, an editor agent and a review bot all read and write one address, repo::acme/checkout, at the same time.
Export moves one owner's memory between vendors. Live addressing lets many tools share one place at the same time. The existing proposals focus on the first. A shared address could sit inside their records to cover the second.
-## What every format already agrees on - -Strip away the branding and almost every format answers the same four questions, just in different ways: - -1. **Where is this true?** A path, a repo, a user ID, a namespace tuple, an entity. -2. **What is its identity?** Something that decides whether a second write is an *update* or a *duplicate*. -3. **What does it say?** Almost always plain text, or JSON when a program is the reader. -4. **When does it stop being true?** Often skipped. When it isn't, the answers range from "28 days unused" to "two full timelines". - -That's the part I find hopeful. These four questions are the beginning of a data model, and nobody disagrees that they matter. The tools only disagree on the answers. OpenTelemetry started from a similar place: everyone already had spans, they just shaped them differently. - -There's one real fork. **Who writes the memory?** In instruction files it's a person. In the memory tool, Letta and LoreKit, the model writes directly. In Mem0 and Graphiti, a service extracts facts from the conversation. A shared format doesn't need to pick one. It needs to *record* which one, so a reader knows how much to trust what it reads. - ## What a shared spec could look like -If agent memory followed OpenTelemetry's path, the spec would have the same four layers: +Strip away the branding and every format answers the same four questions differently: **where** a memory is true, **what its identity is** (does a second write update it or duplicate it?), **what it says**, and **when it stops being true**. Nobody disagrees that those matter. The tools only disagree on the answers, which is where OpenTelemetry started too. -- **A data model.** One record: an address, a stable ID, a body with a content type, a lifecycle, provenance with optional anchors, a kind, and an open attribute map. Small enough that a shell script can write a valid record. -- **Conventions.** Shared names for address levels (`global`, `project`, `repo`, `branch`, and whatever monorepos need), for kinds, and for attribute namespaces (`vcs.*`, `ci.*`, `agent.*`). OpenTelemetry's semantic conventions are the model: one registry, versioned, open to anyone. -- **A contract.** A small set of memory operations (write, read, list, search, archive) with fixed inputs and outputs, exposed over MCP. Every memory server today speaks MCP but exposes different tools, so agents can call any of them and none of them mean the same thing. -- **Conformance tests.** Round-trip a record and check nothing was lost. Check that unknown attributes survive a rewrite. Check that expiry rules behave as stated. +There's one real fork: **who writes the memory**. In instruction files it's a person. In the memory tool, Letta and LoreKit, it's the model. In Mem0 and Graphiti, a service extracts facts from the conversation. A spec doesn't need to pick one. It needs to *record* which, so a reader knows how much to trust what it reads. -Here's roughly what one record could look like on the wire. It's an illustration of the data model, not a proposal for exact field names: +Here's roughly what one record could look like. It illustrates the data model; the field names aren't a proposal: ```json { @@ -127,44 +99,39 @@ Here's roughly what one record could look like on the wire. It's an illustration } ``` -Each line answers one of the four questions or the fork above: `address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` records who wrote it. Everything a tool needs beyond that goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. - -What stays *out* of the spec matters as much: storage, ranking, embeddings, whether a model or an extractor writes, the user interface. Those are where tools should compete, the way observability vendors compete on everything after the data arrives. +`address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` answers the fork. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. -The spec should live somewhere neutral, not in LoreKit's repo or any single vendor's. The Agentic AI Foundation already looks after MCP and `AGENTS.md`, which makes it one natural option, though that's for the people involved to decide. +Around that record, a spec would mirror OpenTelemetry's layers: -## What a skeptic would say +- **Conventions**: shared names for address levels (`global`, `project`, `repo`, `branch`, and whatever monorepos need), for kinds, and for attribute namespaces (`vcs.*`, `ci.*`, `agent.*`), in one versioned registry. +- **A contract**: a few operations (write, read, list, search, archive) with fixed inputs and outputs, exposed over MCP. Many memory servers now offer an MCP interface, but each exposes different tools with different meanings. +- **Conformance tests** in place of a reference implementation: round-trip a record and check nothing was lost, check that unknown attributes survive a rewrite, check that expiry behaves as stated. -These are the objections I'd raise if someone else had written this post. +What stays *out* matters as much: storage, ranking, embeddings, the user interface. That's where tools should compete. And the spec should live somewhere neutral, not in LoreKit's repo or any single vendor's. -**"You build a memory tool, so of course you want a standard shaped like it."** That's a fair suspicion. It's why this post spends more time on other tools than on LoreKit, and why LoreKit is one row in the table rather than the template. If a shared spec ends up looking different, LoreKit should adopt it anyway. Being the tool that wrote the spec matters less than being one of many tools that can read each other's memory. +## What a skeptic would say -**"MCP already is the standard."** MCP standardises the *call*, not the *record*. It's like having a shared network protocol without a shared data model. OpenTelemetry needed OTLP *and* the data model. Memory needs both too. +**"You build a memory tool, so of course you want a standard shaped like it."** Fair. It's why LoreKit is one row in the table rather than the template. If a shared spec looks different, LoreKit should adopt it anyway. -**"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and shared memory widens who gets to write those things. A spec should make the risk visible rather than hide it: record who wrote each memory and how, let readers check anchors, and allow signatures, which Portable Memory already proposes. No format makes poisoning impossible. +**"MCP already is the standard."** MCP standardises the *call*, not the *record*. OpenTelemetry needed a protocol *and* a data model. Memory needs both too. -**"Another standard won't fix fragmentation."** That's the xkcd 927 problem: an attempt to unify everything adds one more competing standard. It's the strongest objection. My answer is the OpenTelemetry one. Agreement happened there when the existing projects *merged*, not when someone added a fifth. The useful work here is the same: take what the existing proposals and tools each got right, and agree on one shape for the parts they share. +**"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and shared memory widens who gets to write it. A spec should make that visible: record who wrote each memory, allow signatures (Portable Memory already does), and let a record point at the code it depends on so a reader can check whether that code changed. No format makes poisoning impossible. -**"It's too early. Memory designs are still changing."** Maybe. But OpenTelemetry's data model has kept evolving since 2019, through versioning and conventions. Agreeing on a small core now doesn't freeze anything. It just stops every new tool from inventing its own core from scratch. +**"Another standard won't fix fragmentation."** That's [xkcd 927](https://xkcd.com/927/), and it's the strongest objection. My answer is OpenTelemetry's: agreement came when existing projects *merged*, not when someone added another one. The work here is the same: take what the existing proposals and tools each got right, and agree on the parts they share. ## Open questions I don't know the answers to these, and I'd rather work them out in public: -- **Which address levels?** Monorepos want a level between `repo` and `branch`, and organisations want one above `project`. Is a fixed list right, or should levels be conventions too? -- **Should forgetting be the default?** Copilot forgets by default, LoreKit doesn't, and LangGraph extends life on read. A per-record rule avoids choosing, but tools still need a default. -- **Do coding lessons need two timelines?** Graphiti's case is strong for facts about the world. Is there a coding case where "what was true in March" matters? -- **Are anchors part of the core?** They're the best protection against stale memory, but they tie a record to a code host. -- **Who should coordinate?** A working group across tool builders, a foundation, or a draft spec that people send changes to? - -If you build any of the tools in the table, or you've written one of the existing proposals, I'd like to compare notes. Disagreement is the useful part. LoreKit's schema, scope validator and on-disk format are MIT-licensed in the repo as one concrete input. - -## Look at a real record +- **Which address levels?** Monorepos want a level between `repo` and `branch`. Should levels be a fixed list or conventions? +- **Should forgetting be the default?** Copilot forgets by default, LoreKit doesn't, and LangGraph extends life on read. +- **Should a record point at the code it depends on?** It's the best protection against stale memory, but it ties a record to a code host. +- **Who should coordinate?** A working group, a foundation, or a draft spec people send changes to? -The quickest way to judge any of this is to look at real files. LoreKit's offline store writes plain Markdown records you can open, grep and diff: +If you build any of the tools in the table, or wrote one of the proposals, I'd like to compare notes. LoreKit's schema and on-disk format are MIT-licensed as one concrete input. To see real records, LoreKit's offline store writes plain Markdown files you can open, grep and diff: ```bash npx @lorekit/cli install ``` -Then compare them with the files your other tools write, and tell me what a shared format would need that none of them have yet. +Compare them with what your other tools write, and tell me what a shared format would need that none of them have yet. diff --git a/packages/web/src/lib/blog/sections.ts b/packages/web/src/lib/blog/sections.ts index 1fd4e530e..6b2b1938b 100644 --- a/packages/web/src/lib/blog/sections.ts +++ b/packages/web/src/lib/blog/sections.ts @@ -48,7 +48,7 @@ export const BLOG_SECTIONS: readonly BlogSection[] = [ id: 'open-agent-memory-format', label: "Agent memory needs its OpenTelemetry moment", summary: - "Every agent tool stores memory in its own shape, and none can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, what they agree on, and what a shared spec for agent memory could look like.", + "Every agent tool stores memory in its own shape, and every week someone builds another one. None of them can read the others. Observability had the same problem until OpenTelemetry gave everyone one data model to share. A tour of today's memory formats, the proposals that already exist, and what a shared spec could look like.", }, ]; From 79bb46d0bd8717e8b523b964dff27658e8c483af Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 08:06:23 +0000 Subject: [PATCH 13/17] blog: show the example record as a Markdown file with a real memory body Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- .../content/blog/open-agent-memory-format.mdx | 37 +++++++++++-------- 1 file changed, 21 insertions(+), 16 deletions(-) diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx index c6d355660..643bb3bfd 100644 --- a/packages/web/src/content/blog/open-agent-memory-format.mdx +++ b/packages/web/src/content/blog/open-agent-memory-format.mdx @@ -82,24 +82,29 @@ Strip away the branding and every format answers the same four questions differe There's one real fork: **who writes the memory**. In instruction files it's a person. In the memory tool, Letta and LoreKit, it's the model. In Mem0 and Graphiti, a service extracts facts from the conversation. A spec doesn't need to pick one. It needs to *record* which, so a reader knows how much to trust what it reads. -Here's roughly what one record could look like. It illustrates the data model; the field names aren't a proposal: - -```json -{ - "spec": "agent-memory/0.1", - "id": "8f0c2a6e-4b1d-4c7a-9e2f-3a5b6c7d8e9f", - "address": { "level": "repo", "value": "github.com/acme/checkout" }, - "key": "tests-need-local-postgres", - "kind": "lesson", - "written_by": { "type": "agent", "name": "claude-code" }, - "content_type": "text/markdown", - "body": "Integration tests need Postgres up first: docker compose up -d db.", - "lifecycle": { "expires_at": null, "extended_by": "write" }, - "attributes": { "vcs.pr": 482, "ci.run_url": "https://ci.example/runs/123" } -} +Here's roughly what one record could look like as a file. It illustrates the data model; the field names aren't a proposal: + +```markdown +--- +spec: agent-memory/0.1 +id: 8f0c2a6e-4b1d-4c7a-9e2f-3a5b6c7d8e9f +address: { level: repo, value: github.com/acme/checkout } +key: tests-need-local-postgres +kind: lesson +written_by: { type: agent, name: claude-code } +content_type: text/markdown +lifecycle: { expires_at: null, extended_by: write } +attributes: { vcs.pr: 482, ci.run_url: "https://ci.example/runs/123" } +--- +Integration tests fail with `ECONNREFUSED 127.0.0.1:5432` unless Postgres +is already running. Start it before `pnpm test`: + + docker compose up -d db + +The test setup does not start it for you. ``` -`address` and `key` say where it's true, `id` is its identity, `content_type` and `body` say what it says, `lifecycle` says when it stops, and `written_by` answers the fork. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record could just as well be a Markdown file with frontmatter. +The frontmatter is the envelope: `address` and `key` say where it's true, `id` is its identity, `lifecycle` says when it stops, and `written_by` answers the fork. The Markdown below it is the memory itself, and `content_type` tells a reader how to treat it. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record serialises to JSON just as well, for a tool that wants to send it over the wire. Around that record, a spec would mirror OpenTelemetry's layers: From 57a14840e45f05f23d38985e4a6c6b44fe8e2e37 Mon Sep 17 00:00:00 2001 From: mthines Date: Sun, 8 Nov 2026 09:24:26 +0100 Subject: [PATCH 14/17] chore: empty commit to trigger Vercel preview From 1e366ff7c7f0205cdeb762017b6249a1e0c088b8 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 8 Nov 2026 08:35:08 +0000 Subject: [PATCH 15/17] fix(web): highlight markdown fences in MDX The blog's example record is a markdown fence, and MDX_SHIKI_LANGS only loads the grammars content declares, so rehype-shiki.spec failed. Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa --- packages/web/src/lib/mdx/rehype-shiki.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/web/src/lib/mdx/rehype-shiki.ts b/packages/web/src/lib/mdx/rehype-shiki.ts index 7b6ee64b6..a346bdfcf 100644 --- a/packages/web/src/lib/mdx/rehype-shiki.ts +++ b/packages/web/src/lib/mdx/rehype-shiki.ts @@ -49,7 +49,7 @@ export const SHIKI_THEME = 'github-dark-default'; * specs catch an addition that forgets to (`docs/mdx-render.spec.ts`, * `blog/mdx-render.spec.ts`). */ -export const MDX_SHIKI_LANGS = ['bash', 'json', 'jsonc', 'yaml', 'javascript', 'js'] as const; +export const MDX_SHIKI_LANGS = ['bash', 'json', 'jsonc', 'yaml', 'javascript', 'js', 'markdown'] as const; /** * Strip Shiki's inline `background-color` from the `
` so the block keeps the

From e912ae8919b294f0d3c3237f6631b6fb298f698a Mon Sep 17 00:00:00 2001
From: Claude 
Date: Mon, 9 Nov 2026 06:02:52 +0000
Subject: [PATCH 16/17] docs(blog): add the version rule to the agent-memory
 post

A reader pointed out that traces are append-only but memories get
corrected, so a shared record needs a version rule or every tool invents
its own. The post now asks a fifth question (which version is current),
adds version and supersedes to the example record and the conformance
tests, and answers the objection in the skeptic section.

Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa
---
 .../web/src/content/blog/open-agent-memory-format.mdx  | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx
index 643bb3bfd..01919ed65 100644
--- a/packages/web/src/content/blog/open-agent-memory-format.mdx
+++ b/packages/web/src/content/blog/open-agent-memory-format.mdx
@@ -78,7 +78,7 @@ These mostly solve **export**: moving one person's memory from vendor A to vendo
 
 ## What a shared spec could look like
 
-Strip away the branding and every format answers the same four questions differently: **where** a memory is true, **what its identity is** (does a second write update it or duplicate it?), **what it says**, and **when it stops being true**. Nobody disagrees that those matter. The tools only disagree on the answers, which is where OpenTelemetry started too.
+Strip away the branding and every format answers the same five questions differently: **where** a memory is true, **what its identity is** (does a second write update it or duplicate it?), **what it says**, **when it stops being true**, and **which version is current**. Nobody disagrees that those matter. The tools only disagree on the answers, which is where OpenTelemetry started too.
 
 There's one real fork: **who writes the memory**. In instruction files it's a person. In the memory tool, Letta and LoreKit, it's the model. In Mem0 and Graphiti, a service extracts facts from the conversation. A spec doesn't need to pick one. It needs to *record* which, so a reader knows how much to trust what it reads.
 
@@ -91,6 +91,8 @@ id: 8f0c2a6e-4b1d-4c7a-9e2f-3a5b6c7d8e9f
 address: { level: repo, value: github.com/acme/checkout }
 key: tests-need-local-postgres
 kind: lesson
+version: 3
+supersedes: 41b9e0c2-7d3a-4f15-8c6e-0a9b8c7d6e5f
 written_by: { type: agent, name: claude-code }
 content_type: text/markdown
 lifecycle: { expires_at: null, extended_by: write }
@@ -104,13 +106,13 @@ is already running. Start it before `pnpm test`:
 The test setup does not start it for you.
 ```
 
-The frontmatter is the envelope: `address` and `key` say where it's true, `id` is its identity, `lifecycle` says when it stops, and `written_by` answers the fork. The Markdown below it is the memory itself, and `content_type` tells a reader how to treat it. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record serialises to JSON just as well, for a tool that wants to send it over the wire.
+The frontmatter is the envelope: `address` and `key` say where it's true, `id` is its identity, `lifecycle` says when it stops, and `written_by` answers the fork. `version` counts the corrections to this record, and `supersedes` names an older record this one replaces, so a reader of that older record can be told it's out of date. The Markdown below it is the memory itself, and `content_type` tells a reader how to treat it. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record serialises to JSON just as well, for a tool that wants to send it over the wire.
 
 Around that record, a spec would mirror OpenTelemetry's layers:
 
 - **Conventions**: shared names for address levels (`global`, `project`, `repo`, `branch`, and whatever monorepos need), for kinds, and for attribute namespaces (`vcs.*`, `ci.*`, `agent.*`), in one versioned registry.
 - **A contract**: a few operations (write, read, list, search, archive) with fixed inputs and outputs, exposed over MCP. Many memory servers now offer an MCP interface, but each exposes different tools with different meanings.
-- **Conformance tests** in place of a reference implementation: round-trip a record and check nothing was lost, check that unknown attributes survive a rewrite, check that expiry behaves as stated.
+- **Conformance tests** in place of a reference implementation: round-trip a record and check nothing was lost, check that unknown attributes survive a rewrite, check that expiry behaves as stated, check that a superseded record reads as superseded.
 
 What stays *out* matters as much: storage, ranking, embeddings, the user interface. That's where tools should compete. And the spec should live somewhere neutral, not in LoreKit's repo or any single vendor's.
 
@@ -120,6 +122,8 @@ What stays *out* matters as much: storage, ranking, embeddings, the user interfa
 
 **"MCP already is the standard."** MCP standardises the *call*, not the *record*. OpenTelemetry needed a protocol *and* a data model. Memory needs both too.
 
+**"Traces are append-only. Memories get corrected."** This is where the OpenTelemetry analogy is weakest. A span is written once and never changes. A runbook written in January gets corrected in March, and if nothing records that, both copies look right to the next agent. So the version rule is part of the standard, not a detail for each tool: a counter that goes up on every correction, and a link from the new record to the one it replaces. If every tool invents its own rule here, the lock-in stays, however well the formats line up.
+
 **"Shared memory is a prompt-injection channel."** Yes. Anything an agent reads can steer it, and shared memory widens who gets to write it. A spec should make that visible: record who wrote each memory, allow signatures (Portable Memory already does), and let a record point at the code it depends on so a reader can check whether that code changed. No format makes poisoning impossible.
 
 **"Another standard won't fix fragmentation."** That's [xkcd 927](https://xkcd.com/927/), and it's the strongest objection. My answer is OpenTelemetry's: agreement came when existing projects *merged*, not when someone added another one. The work here is the same: take what the existing proposals and tools each got right, and agree on the parts they share.

From 079cc42a84fa8f55d1f1dc45e6e11a4f427abfe2 Mon Sep 17 00:00:00 2001
From: Claude 
Date: Mon, 9 Nov 2026 08:43:08 +0000
Subject: [PATCH 17/17] docs(blog): say when a correction bumps version and
 when it supersedes

Claude-Session: https://claude.ai/code/session_012YX3RPBKN75heHAMkL2Toa
---
 packages/web/src/content/blog/open-agent-memory-format.mdx | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/packages/web/src/content/blog/open-agent-memory-format.mdx b/packages/web/src/content/blog/open-agent-memory-format.mdx
index 01919ed65..51d353ee9 100644
--- a/packages/web/src/content/blog/open-agent-memory-format.mdx
+++ b/packages/web/src/content/blog/open-agent-memory-format.mdx
@@ -106,7 +106,7 @@ is already running. Start it before `pnpm test`:
 The test setup does not start it for you.
 ```
 
-The frontmatter is the envelope: `address` and `key` say where it's true, `id` is its identity, `lifecycle` says when it stops, and `written_by` answers the fork. `version` counts the corrections to this record, and `supersedes` names an older record this one replaces, so a reader of that older record can be told it's out of date. The Markdown below it is the memory itself, and `content_type` tells a reader how to treat it. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record serialises to JSON just as well, for a tool that wants to send it over the wire.
+The frontmatter is the envelope: `address` and `key` say where it's true, `id` is its identity, `lifecycle` says when it stops, and `written_by` answers the fork. `version` counts the corrections to this record, and `supersedes` names an older record this one replaces, so a reader of that older record can be told it's out of date. A fix to the same claim bumps `version`. A different claim that replaces it is a new record, starting again at version 1, that `supersedes` the old one. The Markdown below it is the memory itself, and `content_type` tells a reader how to treat it. Anything else a tool needs goes in `attributes`, under a namespace other tools can ignore. The same record serialises to JSON just as well, for a tool that wants to send it over the wire.
 
 Around that record, a spec would mirror OpenTelemetry's layers: