Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,32 @@
- Not yet: persistence (one Lance version per published batch, P-SPARSE-1
base + delta) — the next half of D-STO-1.

## Store, persistence half (D-STO-1, feature `lance`)

- `hubspo-store::log::Log`: every batch is one Lance version of an
append-only log (P-SPARSE-1's delta shape; nothing updated in place).
Records: `BORN` (row), `SET` (row, lane, present, value as the lane's 32
bits), `DELETE` (row), each stamped with version and owner.
- `commit(table, batch)`: check → append one Lance version → apply. Lance
version = table version; the log's version is checked before the append
(and after it). A refused batch writes nothing; an empty batch is
refused (`StoreError::EmptyBatch`).
- `replay(owner, specs)`: rebuilds the table batch by batch through the
same `Table::apply`; refuses a foreign owner, an undecodable record, and
a log whose Lance version is not its batch count.
- Tests (`cargo test -p hubspo-store --features lance`, CI job
`store-lance`, which installs `protoc`): 25 committed batches replay to
an identical image (deletes and negative values included); one Lance
version per batch; a second, stale writer is refused before it applies
anything; refused and empty batches leave the log at its version;
replay under another owner is refused.
Disable runs 2026-10-10, red: no pre-append version check, owner ignored
on replay, the I32 sign bit lost, deletes not logged, apply before the
append. Green, and kept on purpose: the post-append version check — it
catches only a writer that appends between the check and the write, a
race no sequential test can schedule.
- Not yet: folding the log into a base dataset (D-STO-5), object stores.

## Board view (D-QRY-3, board half)

- `hubspo-query::board::Board`: per (pipeline, stage), record count and
Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@

| D-id | Deliverable | Size | Status | Evidence |
|---|---|---|---|---|
| D-STO-1 | `hubspo-store` write lane: batch image → one Lance version per publish; owner-stamped (`mailbox_owner`) | L | In progress | resident half in PR: `hubspo-store::table` (owner-stamped atomic batches, one version each, `changed` plane); Lance persistence next |
| D-STO-1 | `hubspo-store` write lane: batch image → one Lance version per publish; owner-stamped (`mailbox_owner`) | L | In PR | resident half shipped (#22); persistence half in PR: `hubspo-store::log` (one Lance version per batch, replay, feature `lance`) |
| D-STO-2 | KV value store for variable-length properties; zero-copy lens reads | M | Queued | |
| D-STO-3 | Edge lanes + association labels (H-ADR-4); cardinality invariants as Quack `GroupReduce` | L | Queued | |
| D-STO-4 | Tenant layout + P-TEN-1 probe | M | Queued | P-TEN-1 passed 2026-10-10 (7.8 ms p99 at 10k tenants, local disk) |
Expand Down
2 changes: 1 addition & 1 deletion .claude/plans/hubspo-integration-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -1167,7 +1167,7 @@ below. Cross-repo follow-ups are tracked in `.claude/board/ISSUES.md`.
| OD-9 | Edition 2024 for HubSPO-rs | yes |
| OD-10 | Which HubSpot sandbox portal is the value oracle, and who holds its token | operator |
| OD-11 | lance-graph `ogar-consumer-preflight.md` still whitelists `lance =9.0.0 / lancedb =0.33.0`; lance-graph `CLAUDE.md` floats `12.*` / `0.39.*` | follow `CLAUDE.md`; fix the preflight upstream |
| OD-12 | How stalwart gets users and teams (`ISS-STALWART-DIRSIM-WIRING`): (a) build the dir-sim backend outside stalwart's config model, (b) dir-sim persistence + stalwart's schema generator first, (c) stalwart keeps its own directory and the CRM join key comes from spear's placement | (c) — approved 2026-10-10 The fork's `crates/directory/src/backend/dirsim/` exists behind feature `dir-sim` but nothing constructs it and the registry `Directory` enum has no variant for it (re-read 2026-10-10, `.claude/knowledge/comms-api-surface.md` §3). |
| OD-12 | How stalwart gets users and teams (`ISS-STALWART-DIRSIM-WIRING`): (a) build the dir-sim backend outside stalwart's config model, (b) dir-sim persistence + stalwart's schema generator first, (c) stalwart keeps its own directory and the CRM join key comes from spear's placement | (c) — approved 2026-10-10 The fork's `crates/directory/src/backend/dirsim/` exists behind feature `dir-sim` but nothing constructs it and the registry `Directory` enum has no variant for it (re-read 2026-10-10, `.claude/knowledge/comms-api-surface.md` §3). *Option to evaluate, added 2026-10-10:* (d) stalwart's own LDAP directory backend reading dir-sim through lance-graph #1453's read-only LDAP v3 handler (`lance-graph-dir-sim::ldap::Server`; the host owns the socket, so `hubspo-server` would host it). No stalwart code change. Open before it can replace (c): password checks go through the host's `Authority`, not dir-sim, and the served version is in memory until dir-sim persistence lands. |

---

Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,23 @@ jobs:
# identity shapes (Gate G0).
- name: Test
run: cargo test --workspace

# hubspo-store's Lance log (feature `lance`) needs `protoc` to build
# lance-encoding, so it runs in its own job and the default job stays fast.
store-lance:
name: hubspo-store (lance)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Install pinned toolchain
run: rustup show active-toolchain || rustup toolchain install

- name: Install protoc
run: sudo apt-get update && sudo apt-get install -y protobuf-compiler

- name: Clippy
run: cargo clippy -p hubspo-store --features lance --all-targets --no-deps -- -D warnings

- name: Test
run: cargo test -p hubspo-store --features lance
14 changes: 14 additions & 0 deletions crates/hubspo-store/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,17 @@ workspace = true
lance-graph-quack.workspace = true
lance-graph-mask-risc.workspace = true
lance-graph-contract.workspace = true

# One Lance version per published batch (D-STO-1, persistence half). Off by
# default: lance needs `protoc` to build, and only the `store-lance` CI job
# installs it.
lance = { workspace = true, optional = true }
arrow = { workspace = true, optional = true }
futures = { version = "0.3", optional = true }

[features]
lance = ["dep:lance", "dep:arrow", "dep:futures"]

[dev-dependencies]
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
tempfile = "3"
4 changes: 4 additions & 0 deletions crates/hubspo-store/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@
//! through (H-ADR-6).
//! - [`table`] (D-STO-1, resident half): one dataset's resident lanes, changed
//! only by whole owner-stamped batches, one version each.
//! - [`log`] (D-STO-1, persistence half, feature `lance`): every batch is one
//! Lance version of an append-only log; a table is rebuilt by replay.

pub mod catalog;
#[cfg(feature = "lance")]
pub mod log;
pub mod table;
Loading
Loading