feat(skill-memory): per-skill cross-session recall + historian auto-extraction

ARCHITECTURE.md

@@ -20,7 +20,7 @@ Magic Context is an `@opencode-ai/plugin` (entry `src/index.ts`) that rewrites t
 - **Adapters** (`src/plugin/`): hook wrappers, tool registry, RPC handlers, dream-timer lifecycle, per-session hook construction.
 - **Runtime** (`src/hooks/magic-context/`): the transform pipeline, postprocess phase, event/command handlers, system-prompt injection, compartment runners, decay rendering, strip-and-replay, nudges, m[0]/m[1] injection, and the shadow-transform sender (a flag-gated dev lane that mirrors transform passes to the Rust subc module).
 - **Feature services** (`src/features/magic-context/`): storage, scheduler, tagger, memory, dreamer, sidekick, git-commit + message FTS indexes, unified search, overflow detection, session-project mapping and backfill, migrations, clone-state copy helpers (`src/features/magic-context/storage-clone.ts`), project identity resolution (resolves `git:<sha>` or fallback `dir:<md5-12>` identifiers, caching directory fallbacks, and utilizing a cooldown period for transient git errors).
-- **Tools** (`src/tools/`): `ctx_reduce`, `ctx_expand`, `ctx_note`, `ctx_memory`, `ctx_search`. Gating: when `memory.enabled` is false, `ctx_memory` is not registered (on Pi, it registers but refuses queries for memory-off projects), and all memory-related guidance is stripped from the system prompt.
+- **Tools** (`src/tools/`): `ctx_reduce`, `ctx_expand`, `ctx_note`, `ctx_memory`, `ctx_search`, `ctx_skill_note`, `ctx_skill_recall`. Gating: when `memory.enabled` is false, `ctx_memory` is not registered (on Pi, it registers but refuses queries for memory-off projects), and all memory-related guidance is stripped from the system prompt; the `ctx_skill_*` tools stay registered (skill-memory is an independent store).
 - **Config + shared** (`src/config/`, `src/shared/`): Zod config (deep-merge raw JSONC before validation; invalid leaves fall back to defaults with warnings, never disable the plugin). **Project-tier trust boundaries** (`src/config/project-security.ts`) strip unsafe fields from untrusted repository configs before merge — preventing escalation via `sqlite` PRAGMAs, hidden-agent reprogramming (`prompt`, `tools`), `embedding` destinations, or developer-only shadow transform settings. Project-level compaction thresholds are raise-only to prevent cloned repos from forcing extra historian cost. Also includes logger, data paths, SQLite selector, harness id, RPC transport, conflict detector, tag-transcript primitive (shared with Pi), provider-id translation map (`src/shared/harness-provider-map.ts`), and exit-abort listener coordinator (`src/shared/exit-abort-registry.ts`).
 - **TUI** (`src/tui/`): sidebar + `/ctx-status` / `/ctx-recomp` dialogs, RPC-backed; shipped as raw TS via the `./tui` export (not bundled into `dist/index.js`).
 - **CLI** (`packages/cli/`, separate `@cortexkit/magic-context` package): `npx` setup / doctor / migrate wizard, including local embedding runtime checks for missing or broken `onnxruntime-node` native bindings.
@@ -107,6 +107,23 @@ The long-history pipeline. Tiered compartments + deterministic decay renderer (r
 
 `protected-tail-boundary.ts` decides, per pass, which prefix of the raw tail is eligible for the historian and which suffix stays protected — from true-raw token sizes (not user-turn counts), so sparse-user-turn sessions can't deadlock the historian (#132). Boundary anchors at `lastCompartmentEnd + 1`; token target `N` capped at `0.40 × usable` (ABS_CAP 96k); a live-prompt floor keeps it from crossing the newest meaningful user message on routine (<80%) passes. An **emergency drain catch-up latch** (usage-driven) bypasses the per-run token cap when pressure spikes ≥85%, draining continuously until usage falls back below the safe threshold. **Open tool arcs** (a tool invocation with no result in the window) only hold the boundary back when **recent** (≥ the size-walk start = the live window); a stale/interrupted open arc older than that is compactable — otherwise one dead `running` tool call at the eligible-head edge would freeze the historian indefinitely. The trigger/runner share a content-stable range fingerprint for cross-view staleness validation. For manual wrapup passes, `resolveWrapupProtectedTailBoundary` counts raw messages (including tools) rather than user turns to determine the keep watermark (default 20), protecting the newest tail segment while using the same tool-arc fencing and user-boundary snapping logic. On verbatim-tail profiles (where `fold_is_only_reclaim` is true, e.g. `claude-code-anthropic`), folding is the only reclaim path while the newest message is forwarded in full. Keep the newest message and its completed tool arc in the tail and out of the fold so that the live turn does not become a durable compartment boundary.
 
+**Skill-memory flow (per-skill cross-session recall):** transparent augmentation of opencode's built-in `skill` tool — when a loaded skill declares `skill-memory: { enabled: true }` in its frontmatter, accumulated gotchas/discoveries surface in a `<skill-memory>` block appended to the skill tool's RESULT on every load. Three opencode hooks plus two agent-callable tools implement the loop.
+1. **Definition (`tool.definition` in `src/hooks/magic-context/skill-tool-definition.ts`)** — augments the `skill` tool's JSON Schema with an optional `intent` string parameter. Effect-Schema strips unknown keys (`onExcessProperty: "ignore"`) before the skill tool runs, so `intent` never reaches the skill itself; the before-hook captures it pre-validation. Idempotent (re-adds are guarded).
+2. **Before (`tool.execute.before` in `src/hooks/magic-context/hook-handlers.ts` — `createToolExecuteBeforeHook`)** — stashes the raw `intent` by `callID` in a bounded closure-state `Map<string, {intent, ts}>` (60s TTL sweep + 256-entry cap + full clear on session delete, so unpaired before-hooks never leak). The stash is the only place `intent` is observable; it's deleted in the after-hook's `finally`.
+3. **After (`tool.execute.after` in `src/hooks/magic-context/hook-handlers.ts` — `createToolExecuteAfterHook`)** — runs only for `input.tool === "skill"`. Parses the `Base directory for this skill: file:///...` line from `output.output` via `parseSkillProvenance()` (`fileURLToPath`-based, cross-platform) to recover the resolved `SKILL.md` path + tier (project/global) + `skill_source`. Then re-reads `SKILL.md` from disk (opencode's skill loader strips the `skill-memory:` block from the model-facing output, so the frontmatter is unreadable from `output.output`). Populates a session-scoped `SkillLoadRegistry` keyed by `${sessionId}:${skillId}` (NOT persisted, cleaned in `onSessionDeleted`). When frontmatter has `enabled: true` and notes exist, delegates to `recallSkillMemoryBlock` (feature layer) and appends the block to `output.output` BEFORE the Channel-1 ctx_reduce nudge runs.
+4. **Cache safety (keystone).** The append lands in the skill tool RESULT = conversation tail, NOT the cached m[0]/m[1] prefix. This is why the feature cannot regress the prompt-cache hit rate. Channel-1 already appends to tool output strings the same way (precedent in `maybeInjectChannel1Nudge`) — this is proven production behavior.
+5. **Write-back (`ctx_skill_note`)** — `kind` is a hard gate rejecting `'general'` at the tool level; duplicates dedup on `normalized_hash` and bump `hit_count` (`computeNormalizedHash` from `memory/normalize-hash.ts`). Resolves `(skill_id, tier, project_identity, resolved_path)` from the session-scoped `SkillLoadRegistry` (so the agent must load the skill first — actionable error otherwise). Inserts into the `skill_memory` table (migration v50). The injected block footer reinforces: "After using this skill, call `ctx_skill_note` — record only gotchas, novel discoveries, or error→fix; skip routine successes."
+6. **Explicit recall (`ctx_skill_recall`)** — companion tool to the transparent path; reuses `recallSkillMemoryBlock` so P2 embeddings upgrade both at once. Registry-first resolution (exact, free, no disk I/O when the skill was loaded this session) with a cold-start disk fallback that walks opencode's real `discoverSkills()` order (project dirs first — they shadow global — then global external + config dirs).
+7. **Multi-rung recall cascade (P2)** — `recallSkillMemoryBlock` is a four-rung selector: no intent → flat recency×hit (`mode="no-intent"`); intent + model-matched embeddings → cosine blend across `intent_embedding` + `delta_embedding` with tunable `ranking_relevance`/`ranking_recency`/`ranking_hit` weights (`mode="full"`); intent + no model match → FTS5 over the content-linked `skill_memory_fts` vtable (`mode="fts5-fallback"`); intent tokenized to empty → flat fallback (`mode="flat-fts"`). Embeddings live on `skill_memory` (BLOBs scanned via `Float32Array` cosine, mirroring the memories-embed path); a programmatic no-LLM `reembedStaleSkillNotes` keeps them fresh during the `distill-skill-memory` task. Read-side `recall_count` (migration v51) is bumped on every surfaced note — distinct from write-side `hit_count` so the recency term cannot be poisoned by which notes are queried most.
+8. **Global-tier unification (P3a)** — global-tier notes collapse to `project_identity = '*'` (collision-merge at migration v52). The `partitionKey(tier, projectIdentity)` helper is the single chokepoint for every global write/recall/reembed/stats call site. A historian-extracted lesson learned in repoA is recallable from repoB without per-repo replication; the originating repo is preserved in `origin_project` (`source_type='historian'` for P3b writes, `'agent'` for tool writes).
+9. **Historian auto-extraction (P3b)** — the historian prompt emits an optional `<skill_observations>` block when the chunk shows the agent USING a skill (surfaced via the `TC: skill(<name>)` marker emitted by `extractToolCallSummaries` in `read-session-formatting.ts`) and learning a UNIVERSAL, reusable lesson. Both OpenCode (`compartment-runner-incremental.ts`) and Pi (`pi-historian-runner.ts`) runners promote the validated observations post-commit via the shared `promoteSkillObservations` helper, gated by `promotionActive && !discardedLast`. Writes go to the global `'*'` partition with `source_type='historian'` and `resolved_path=''` sentinel; hash-dedup bumps `hit_count` on collisions.
+10. **Dreamer distill (`distill-skill-memory` task — opt-in, NOT a default)** — `DREAMER_TASKS` enum carries it (line 25 of `src/config/schema/magic-context.ts`); `DEFAULT_DREAMER_TASKS` does NOT (mirroring the `maintain-docs` precedent). The task prompt lives in `src/features/magic-context/dreamer/task-prompts.ts` and runs the merge/prune/promote maintenance cycle documented in CONFIGURATION.md.
+
+**Git-commit indexing:**
+- `src/features/magic-context/git-commits/indexer.ts` reads HEAD-only non-merge commits via `git log` (NUL-byte-free format separator `\x1f`), bounded by `experimental.git_commit_indexing.{since_days, max_commits}`.
+- Embeddings are generated through the same embedding provider chain as memories.
+- Indexing fires from the dream-timer startup tick and periodic interval; manual `/ctx-dream` does NOT trigger commit indexing.
+
 ## Memory, search & embeddings
 
 - **Memories** (`memory/storage-memory.ts`): project-scoped durable knowledge in the 5-category taxonomy (PROJECT_RULES / ARCHITECTURE / CONSTRAINTS / CONFIG_VALUES / NAMING), with FTS + vector side tables. `ctx_memory` exposes write/archive/update/merge/list; `list` is dreamer-only; primary agents may only mutate their own project's memories (workspace-shared categories aside). Writes are idempotent, and an **embedding hash guard** (`saveEmbeddingIfHashMatches`) ensures vectors are only saved if the memory content hasn't changed during the provider call.
@@ -143,7 +160,7 @@ Background maintenance (V2: per-task cron scheduling). A process-wide 15-min tim
 
 ## Storage & migrations
 
-`storage-db.ts` creates the schema and runs versioned migrations (`migrations.ts`, currently v1–v50). `LATEST_SUPPORTED_VERSION` is a schema fence — it MUST be bumped with every new migration (a unit test asserts it equals the highest migration), and a stale value makes the DB refuse to open after the migration applies. Schema helpers `ensureColumn()` + `healAllNullColumns()` (defined in `storage-schema-helpers.ts` to prevent cycles between `storage-db` and `migrations`) backfill upgraded DBs even if a migration row is lost. New session-scoped tables must be added to `clearSession()`. A bulletproof `MAGIC_CONTEXT_TEST_DATA_DIR` guard keeps the test suite off the live DB (running `bun test` once migrated a live DB and fail-closed running binaries). SQLite binds must use SPREAD positional args, never the array form (`bun:sqlite` binds a lone array positionally; `node:sqlite` reads it as named params and throws). For branch forks in Pi, copy durable session state (compartments, tags, pending operations, and session metadata) to the new session via `copySessionStateForClone()` in `src/features/magic-context/storage-clone.ts`. Run this copy inside an immediate SQLite transaction, filtering and mapping message ordinals and tag composite keys to the copied branch entries, and clearing cached cache bytes to trigger fresh rematerialization.
+`storage-db.ts` creates the schema and runs versioned migrations (`migrations.ts`, currently v1–v53). `LATEST_SUPPORTED_VERSION` is a schema fence — it MUST be bumped with every new migration (a unit test asserts it equals the highest migration), and a stale value makes the DB refuse to open after the migration applies. Schema helpers `ensureColumn()` + `healAllNullColumns()` (defined in `storage-schema-helpers.ts` to prevent cycles between `storage-db` and `migrations`) backfill upgraded DBs even if a migration row is lost. New session-scoped tables must be added to `clearSession()`. A bulletproof `MAGIC_CONTEXT_TEST_DATA_DIR` guard keeps the test suite off the live DB (running `bun test` once migrated a live DB and fail-closed running binaries). SQLite binds must use SPREAD positional args, never the array form (`bun:sqlite` binds a lone array positionally; `node:sqlite` reads it as named params and throws). For branch forks in Pi, copy durable session state (compartments, tags, pending operations, and session metadata) to the new session via `copySessionStateForClone()` in `src/features/magic-context/storage-clone.ts`. Run this copy inside an immediate SQLite transaction, filtering and mapping message ordinals and tag composite keys to the copied branch entries, and clearing cached cache bytes to trigger fresh rematerialization.
 
 ## Session modes
 

CONFIGURATION.md

@@ -374,6 +374,7 @@ To disable the dreamer entirely, set `dreamer.disable: true`. To disable a singl
 | `refresh-primers` | `0 3 * * *` | Re-investigate stale primers against current code and refresh their answers. |
 | `evaluate-smart-notes` | `0 3 * * *` | Surface smart notes whose `ctx_note` conditions have come true. |
 | `review-user-memories` | `0 3 * * *` | Promote recurring behavioral observations into the `<user-profile>` block (privacy-sensitive). |
+| `distill-skill-memory` | `""` (off) | **Opt-in** — add a schedule to enable. Merge near-duplicate skill notes, prune stale low-hit notes, promote recurring gotchas to `pinned=1`, enforce per-skill note caps. Requires the `skill_memory` table (migration v50) — auto-created on upgrade. |
 
 ### Retrospective privacy
 
@@ -612,6 +613,39 @@ Tier boundaries are hardcoded to keep behavior predictable and prevent cache-bus
 
 **When to enable.** Turn it on if you run very long, edit-heavy sessions and want to reclaim more context without losing the agent's record of what it did. The default stays off while cache stability is being validated in the wild. Requires a restart to take effect.
 
+## Skill-Memory (per-skill frontmatter)
+
+Skill-memory is the "motor memory" for skills — per-skill, cross-session recall of gotchas, discoveries, fixes, and workflow steps. The plugin transparently augments opencode's built-in `skill` tool: when a skill declares `skill-memory: { enabled: true }` in its YAML frontmatter, accumulated notes for that skill surface in a `<skill-memory>` block appended to the skill tool's RESULT on every load. Agents write back via `ctx_skill_note`; explicit recall (without re-loading) is `ctx_skill_recall`.
+
+Unlike every other setting in this file, **skill-memory is configured per-skill in each `SKILL.md`'s frontmatter, not in `magic-context.jsonc`**. Absent or malformed block = inert. A bad config in one skill cannot break other skills.
+
+```yaml
+---
+name: test-driven-development
+description: ...
+skill-memory:
+  enabled: true                # required: true to activate
+  max_tokens: 1500             # default 1500 — token budget for unpinned notes
+  max_pinned_tokens: 4000      # default 4000 — separate cap for pinned notes
+  dedup_threshold: 0.92        # default 0.92 — P2 cosine near-dedup threshold
+---
+```
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `enabled` | `boolean` | (required `true`) | Master switch per skill. When absent or `false`, the transparent after-hook skips this skill entirely and `ctx_skill_recall` returns "skill-memory is not enabled for '<skill>'". |
+| `max_tokens` | `number` | `1500` | Hard cap on tokens for unpinned notes in the injected block. Greedy fill by composite score (P1: recency × hit_count). |
+| `max_pinned_tokens` | `number` | `4000` | Separate cap for pinned notes. Pinned notes are always included first; on cap overflow, least-used pinned notes are truncated in ascending `hit_count` order with an "N pinned notes omitted" marker. |
+| `dedup_threshold` | `number` | `0.92` | P2 cosine near-dedup threshold. P1 ships without embeddings, so this is reserved for the P2 rollout. Tune per-skill in the `0.85`–`0.95` range. |
+
+**Cache safety.** The injected block lands in the tool RESULT = conversation tail, never the cached m[0]/m[1] prefix. This is the same pattern as Channel-1 (`maybeInjectChannel1Nudge`) and is why skill-memory cannot regress the prompt-cache hit rate.
+
+**Write-back (`ctx_skill_note`).** The injected block's footer prompts: *"After using this skill, call `ctx_skill_note` — record only gotchas, novel discoveries, or error→fix; skip routine successes."* The `kind` parameter is a hard gate: `kind: "general"` is rejected at the tool level — general observations belong in `ctx_memory` with an appropriate category.
+
+**Dreamer integration.** Add `"distill-skill-memory"` to your `dreamer.tasks` list to opt in to overnight maintenance (merges near-duplicates, prunes stale zero-hit notes older than 30 days, promotes recurring gotchas to pinned). It is **not** a default task — the feature is opt-in like `maintain-docs`.
+
+**P1 vs P2.** P1 (shipped) is flat recall (recency × hit_count, no embeddings). P2 (planned) adds intent-aware ranking via the project's existing embedding provider. The per-skill `dedup_threshold` field is reserved for P2 cosine near-dedup and has no effect on P1.
+
 ## Commands
 
 | Command | Description |