Skip to content
12 changes: 12 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
## 2026-06-18 — lite-unified-surrealql-lance-v1 (one store + one query surface, feature-gated; CONJECTURE, test-don't-commit)

**Status:** CONJECTURE / design — feature-gated test path, NOT a default-build change. **Plan file:** `.claude/plans/lite-unified-surrealql-lance-v1.md`.
**Owns:** the "lite unified" bet — collapse the two query engines (datafusion + SurrealQL) + two stores (lance + rocksdb) to **ONE store (lance-KV) + ONE primary query surface (SurrealQL/AR-API)**, datafusion feature-gated (`datafusion-analytical`), rocksdb dropped; the DO-arm `ExecTarget::SurrealQl` becomes the primary exec path.
- **Win** (graph/AR/CRUD/cognitive/vector): Cypher→SurrealQL is a better lowering than Cypher→datafusion-SQL (surreal is natively graph); drops the rocksdb C++ build + makes datafusion optional. **Downgrade** (heavy analytical SQL): datafusion's strength → kept feature-gated, not deleted.
- **Falsifier (truth-architect):** lance-graph `datafusion_planner` test queries → can SurrealQL express each? Covered → drop datafusion for that path; gaps → keep `datafusion-analytical`. Measure footprint (proxy: lance-graph ≈889 crates, surreal-all ≈1148, SurrealQL-engine marginal ~260, rocksdb separate C++).
- **Blockers (OQ-LU-1/2/3):** surreal kv-lance not yet feature-wired (`surrealdb/core/src/kvs/lance/` module implemented, no `kv-lance` feature); polyglot→SurrealQL lowering doesn't exist (today polyglot→datafusion); SPARQL/Gremlin lowering cleanliness unknown.
**Gate before any promotion:** convergence + cross-domain (mechanism-vs-rhyme) + truth-architect (query-shape coverage). Do NOT touch the default build until green.
**Repos:** lance-graph (+ surrealdb fork for kv-lance). Surfaced from the footprint discussion (drop datafusion+rocksdb) on branch `claude/soa-write-deinterlace-inc2`.

---

## 2026-06-18 — mailbox-belief-update-and-substrate-test-v1 ("what did I learn" = NARS-revision delta + two-axis test; 5+3-ratified; slots S2.5b)

**Status:** CONJECTURE / design — 5+3 COMPLETE. **Plan file:** `.claude/plans/mailbox-belief-update-and-substrate-test-v1.md`. Parent: `bindspace-singleton-to-mailbox-soa-v1` §11 + `E-SOA-CYCLE-OWNERSHIP`.
Expand Down
2 changes: 2 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,8 @@

## Current Contract Inventory (lance-graph-contract)

> **2026-06-18 — ADDED (D-DO-ARM-1, the OGAR DO arm)**: `lance_graph_contract::action::{ActionState, StateGuard, ActionDef, ClassActions, actions_for, effective_actions, ActionInvocation}` — the Perdurant DO arm completing the OGAR IR (the action-axis sibling of `codegen_manifest`'s `MethodSig`/THINK). Both the 4-agent `sale_order` AR→DO probe (runtime-archaeologist) AND the merged cross-repo PR survey (ruff/OGAR/lance-graph/openproject/tesseract) agreed this was the ONE missing wire: the THINK arm (`classid → ClassView`, `has_function → MethodSig`) is converged + merged; the DO-arm `ActionInvocation`/`ActionDef` type was ABSENT. **`ActionDef`** (static, `const`-constructible, all `&'static`/`Copy`): `predicate` (= harvested `has_function` method), `object_class` (classid), `exec` (`ExecTarget` incl `SurrealQl`), `guard` (`StateGuard` = KausalSpec field==value), `required_role` (RBAC), `overrides` (OGAR `classid→ClassView` inheritance). **`ClassActions`+`actions_for`** (zero-fallback) mirror `ClassMethods`/`methods_for`. **`effective_actions(parent, child)`** = OGAR inheritance on the action axis (child overrides parent by predicate). **`ActionInvocation`** (dynamic, `Copy`): lifecycle `ActionState{Pending→Committed|Failed|Cancelled}` (sticky terminals), S2.5 `cycle` stamp, idempotency/trace keys, HLC `emitted_at_millis`. **`ActionInvocation::commit(def, actor, impact, now)`** is the gated egress — RBAC FIRST (`auth::ActorContext` must hold `required_role` or be admin → else `Failed`), THEN MUL impact (`mul::GateDecision`: `Flow→Committed`+stamped, `Hold→`Pending/escalate, `Block→Cancelled`). This IS "commit to the external consumer (odoo/openproject/woa/tesseract) after the cycle decides sound." Dispatched via `UnifiedStep`/`ExecTarget`, NOT a per-crate endpoint. Additive, zero-dep. +5 tests green. Consumer reference: `docs/OGAR_CONSUMER_API.md`. Branch `claude/soa-write-deinterlace-inc2`.

> **2026-06-18 — ADDED (D-UNICHARSET-KEYSTONE, classid → ClassView → adapter wiring)**: `lance_graph_contract::unicharset_adapter::{UniCharSetStore, UniCharCall, UniCharOut, DispatchError, invoke_unicharset}` — steps 2–3 of `PROBE-OGAR-ADAPTER-UNICHARSET`, the keystone composing the proven `UniCharSet` adapter through the OGAR Core's three movable parts. `invoke_unicharset(registry, store, classid, call)`: (1) **ClassView composition gate** — `codegen_manifest::methods_for(registry, classid)` must list the call's method (the harvested `has_function` manifest), else `MethodNotComposed` (zero-fallback: an unconfigured classid composes nothing); (2) **content-store tier** — `UniCharSetStore::unicharset(classid)`, a consumer-provided trait (dependency-inverted like `ClassView`/`PlannerContract`; the adapter holds NO state — `I-VSA-IDENTITIES`); (3) **adapter leaf** — routes to `UniCharSet::{id_to_unichar, unichar_to_id}`. DO-in (`UniCharCall`) / DO-out (`UniCharOut`, zero-copy borrow). **Byte-parity inherited** from `UniCharSet` (112/112); the keystone proves the dispatch path is faithful (the `NULL`→space edge survives it), the gate works, and there is **no Core gap** (the doctrine's iron guard holds — the variable-length bijection rides the content tier cleanly). NOT routed through the heavy `OrchestrationBridge` (cross-subsystem router); this is the adapter-invocation primitive a `UnifiedStep` calls. Additive, zero-dep. +5 tests; clippy `--all-targets -D warnings` + fmt clean. Completes the core-first doctrine END-TO-END for the unicharset leaf (`E-CPP-KEYSTONE-1`).

> **2026-06-17 — ADDED (D-UNICHAR-1, SECOND byte-parity adapter)**: `lance_graph_contract::unichar::{utf8_step, utf8_to_utf32}` — the Tesseract `UNICHAR` UTF-8 codec that `UNICHARSET` sits on top of (`ccutil/unichar.cpp`). `utf8_step(lead) -> u8` is a `const fn` transcription of Tesseract's 256-entry lead-byte table (1/2/3/4 for legal leads, 0 for continuation bytes `0x80..=0xBF` + `0xF8..`); `utf8_to_utf32(bytes) -> Option<Vec<i32>>` mirrors `UNICHAR::UTF8ToUTF32` (lead-byte validation only, `None` on an illegal lead). **The second adapter through the transcode pipeline, byte-parity proven**: `examples/unichar_dump.rs` vs a libtesseract `UNICHAR` oracle is **268/268 identical** (256 EXHAUSTIVE `utf8_step` lead-byte values + 12 `utf8_to_utf32` corpus rows). Faithful-transcode note (the point of the exercise): Tesseract maps `0xC0`/`0xC1` to step 2 and decodes the overlong NUL `C0 80` to `[0]`; `core::str::from_utf8` REJECTS both, so a native-UTF-8 shortcut would silently diverge — mirroring the exact table is mandatory (`from_utf8_rejects_what_tesseract_accepts` test pins it). Additive, zero-dep, pure text (no leptonica). +8 tests + the `unichar_dump` example; 653 contract lib green; clippy `--all-targets -D warnings` clean. Sibling of D-UNICHARSET-1, same `PROBE-OGAR-ADAPTER-UNICHARSET` falsifier family (E-CPP-PARITY-2).
Expand Down
81 changes: 81 additions & 0 deletions .claude/plans/lite-unified-surrealql-lance-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# lite-unified-surrealql-lance-v1 — one store + one query surface, behind a feature gate

> **Status:** CONJECTURE / design. **Test via feature gate; do NOT commit the
> stack change.** Needs a convergence + cross-domain + truth-architect probe
> (mechanism-vs-rhyme + the query-shape measurement) before any promotion.
> **Date:** 2026-06-18. **Parent threads:** the DO-arm (`ExecTarget::SurrealQl`,
> `lance-graph-contract::action`), `docs/STACK_SCAFFOLD.md`, the
> "cold TS + kanban stay Lance-native" ruling.

## Epiphany (less is more)

Today there are **two query engines over the same lance storage** (lance-graph's
*datafusion* planner + surreal's *SurrealQL*) and **two storage engines**
(lance vs rocksdb). The "lite unified" bet collapses both: **ONE store (lance-KV)
+ ONE primary query surface (SurrealQL via the AR-API adapter)**, datafusion
**feature-gated**, rocksdb **dropped**. Cypher/SQL/neo4j lower to SurrealQL —
which is *natively* graph (`->edge->`), a better target than Cypher→datafusion-SQL.

## The bet, as a feature gate (default-OFF)

A `lite-unified` feature that, when ON:
1. **Storage = surreal kv-lance** (one store; drop rocksdb). *Blocked on:* surreal
kv-lance is implemented as a module but not yet feature-wired
(`surrealdb/core/src/kvs/lance/`, the `.claude/lance-backend` integration).
2. **Query/exec = SurrealQL** via the AR-API adapter. The polyglot parser
(Cypher/GQL/Gremlin/SPARQL/neo4j) lowers to **SurrealQL** (or the DO-arm
`ActionInvocation`) instead of datafusion SQL. *Missing today:* the
polyglot→SurrealQL lowering (today it's polyglot→datafusion).
3. **datafusion = `optional`, OFF** on this path. Kept behind a separate
`datafusion-analytical` feature for the workloads that genuinely need
vectorized/analytical SQL (joins, aggregations) — SurrealQL's weak spot.
4. The DO-arm `ExecTarget::SurrealQl` becomes the **primary** exec path, not one
of four.

## What stays regardless (NOT datafusion)

lance vector search, CAM-PQ / bgz17 codec stack, the cognitive substrate
(BindSpace→MailboxSoA, the write contract, the SPO/AriGraph tissue). These are
orthogonal to the query-engine choice.

## Where it's a win vs a downgrade (the honest split)

- **Win (the bulk):** graph traversal, AR CRUD, cognitive/SPO, vector search —
SurrealQL-on-lance fits, and Cypher→SurrealQL graph is a *better* lowering.
Footprint: drop the rocksdb C++ build outright; make datafusion (a large Rust
dep) optional.
- **Downgrade:** heavy analytical SQL (multi-way joins, aggregations, columnar
scan) — datafusion's strength, SurrealQL's weakness. Hence datafusion stays
feature-gated, not deleted.

## Falsifier (truth-architect — measure before promoting)

Take lance-graph's `datafusion_planner` test queries (the Cypher→SQL cases) and
check **SurrealQL can express each**. Covered → drop datafusion for that path;
analytical gaps → keep `datafusion-analytical` for those only. Also measure the
real footprint delta (`cargo tree --no-default-features` + release `cargo bloat`)
once kv-lance is feature-wired — the proxy is lance-graph ≈ 889 crates, surreal
(all backends) ≈ 1148; the marginal SurrealQL-engine cost is ~260 crates, rocksdb
is a separate C++ build.

## Increments (all behind `lite-unified`, none committed to the default path)

1. **Probe (no code):** convergence + cross-domain (mechanism-vs-rhyme) +
truth-architect (the datafusion_planner query-shape coverage check). Gate.
2. **Wire surreal kv-lance** as a feature (finish the `.claude/lance-backend`
integration; add the `kv-lance` feature + lance dep + `mod lance` in `kvs/mod.rs`).
3. **Polyglot→SurrealQL lowering** — the missing front-end leg (parallel to the
existing polyglot→datafusion).
4. **`datafusion` → `optional`** + a `datafusion-analytical` feature; default the
common path to SurrealQL-on-lance under `lite-unified`.
5. **Measure** footprint + query-shape coverage; promote CONJECTURE→FINDING or
correct.

## Blockers / open questions

- **OQ-LU-1:** surreal kv-lance feature-wiring (the integration TODOs).
- **OQ-LU-2:** does SurrealQL cover the lance-graph datafusion_planner query
shapes the live workloads actually use? (the falsifier).
- **OQ-LU-3:** is the polyglot→SurrealQL lowering cleaner than polyglot→datafusion
for the non-graph dialects (SPARQL/Gremlin)?
- Do NOT touch the default build until the probe is green.
55 changes: 55 additions & 0 deletions crates/cognitive-shader-driver/src/backing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ use lance_graph_contract::cognitive_shader::{ColumnWindow, MetaFilter};
use crate::bindspace::BindSpace;
#[cfg(feature = "mailbox-thoughtspace")]
use crate::mailbox_soa::MailboxSoA;
use crate::mailbox_soa::{WriteCell, WriteOutcome};

/// Read-only substrate the dispatch hot path sweeps.
///
Expand Down Expand Up @@ -256,6 +257,60 @@ impl BackingStoreWrite<'_> {
BackingStoreWrite::Mailbox(mb) => mb.set_sigma(row, s),
}
}

/// Cycle-aware row write (S2.5 deinterlacing) — routes a [`WriteCell`]
/// through the per-mailbox cycle gate.
///
/// - **Mailbox arm:** delegates to [`MailboxSoA::write_row`], which gates the
/// write (wrap-aware) against the mailbox's `current_cycle`: a stale batch
/// never overwrites a row the owner advanced past.
/// - **Singleton arm:** **cycle-blind BY CONSTRUCTION** (CATCH-CRITICAL,
/// baton-handoff). `BindSpace` owns no `current_cycle`, so it cannot gate;
/// it applies the cell's present fields via the per-field setters and
/// returns [`WriteOutcome::Accepted`] unconditionally. The cycle gate is a
/// Mailbox-only guarantee until W7 deletes `BindSpace`. `topic`/`angle` are
/// Mailbox-only on the write shim; the legacy singleton path does not carry
/// them (it has no dense-plane setter on this surface).
#[inline]
pub(crate) fn write_row(
&mut self,
row: usize,
cycle: u32,
cell: &WriteCell<'_>,
) -> WriteOutcome {
#[cfg(feature = "mailbox-thoughtspace")]
if let BackingStoreWrite::Mailbox(mb) = self {
return mb.write_row(row, cycle, cell);
}
// Singleton arm: cycle-blind by construction. The `cycle` argument is
// intentionally unused here (no clock to compare against).
let _ = cycle;
if let Some(w) = cell.content {
self.set_content(row, w);
}
if let Some(q) = cell.qualia {
self.set_qualia(row, q);
}
if let Some(e) = cell.edge {
self.set_edge(row, e);
}
if let Some(m) = cell.meta {
self.set_meta(row, m);
}
if let Some(t) = cell.entity_type {
self.set_entity_type(row, t);
}
if let Some(t) = cell.temporal {
self.set_temporal(row, t);
}
if let Some(x) = cell.expert {
self.set_expert(row, x);
}
if let Some(s) = cell.sigma {
self.set_sigma(row, s);
}
WriteOutcome::Accepted
}
}

#[cfg(test)]
Expand Down
Loading
Loading