Repository navigation
D-STO-1 (persistence half): one Lance version per batch, rebuilt by replay - #23
Merged
Merged
Conversation
…'s LDAP handler lance-graph #1453 merged a read-only LDAP v3 handler over a dir-sim projection. stalwart's own LDAP directory backend could read dir-sim through it with no stalwart code change. Recorded as an option to evaluate; the approved (c) stands until passwords (the host's Authority) and dir-sim persistence are settled. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
…eplay hubspo-store::log (feature lance): an append-only Lance log in P-SPARSE-1's delta shape. Log::commit checks the batch, appends it as exactly one Lance version (version checked before and after), then applies it; Log::replay rebuilds the table through the same Table::apply and refuses a foreign owner, an undecodable record, or a version that is not the batch count. Table gains check(), specs() and refuses an empty batch. New CI job store-lance installs protoc and runs the feature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
AdaWorldAPI
pushed a commit
that referenced
this pull request
Oct 10, 2026
Log::history(specs, row, col) returns every write of one property on one record, oldest first, with version and owner: the inserted value, each update, and the record's deletion. It runs one Quack program per log batch over the log's own Arrow buffers and decodes only the matches. The log's op and lane columns widen to u32 so the whole predicate is one mask-risc program over three U32 lanes. The log format shipped in #23 with no data written yet, so no migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
AdaWorldAPI
added a commit
that referenced
this pull request
Oct 10, 2026
…d cost (#24) * D-STO-5: property history read from the Lance log Log::history(specs, row, col) returns every write of one property on one record, oldest first, with version and owner: the inserted value, each update, and the record's deletion. It runs one Quack program per log batch over the log's own Arrow buffers and decodes only the matches. The log's op and lane columns widen to u32 so the whole predicate is one mask-risc program over three U32 lanes. The log format shipped in #23 with no data written yet, so no migration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z * D-STO-5: history timing probe; note why the sort is kept examples/probe_history.rs times one property's history over 1,000 versions of 1,000 updates each on 100k records, with no index on row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z * D-STO-5 probe: split raw scan from history; measure after compaction Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z * Board: D-STO-1 shipped; D-STO-5 history read with its measured cost History is correct but fails P-HIST-1's 20 ms budget: 336 ms fragmented, 57 ms compacted, almost all of it the raw scan. ISS-HIST-SCAN-COST names the levers (compaction after decoupling the version check, column projection, an index on row). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z * Never compact the log: readers rely on append order The append order is the version order, so history and replay drop their sorts (history's was inert under a disable run). Replay still refuses an out-of-order log as a batch whose version is not the next one. The probe drops its compaction arm; ISS-HIST-SCAN-COST and plan §5.2 record the decision, leaving projection and an index on row as the levers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z * History refuses an absent value in a required lane Replay refuses that record through Table::apply, so history must not report it as Set(None). Found by review on #24. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
hubspo-store::log::Log(behind thelancefeature) persists the residentTablefrom #22 as an append-only Lance log. It uses P-SPARSE-1's delta shape: nothing is updated in place, so there are no deletion vectors and the log is safe to compact.BORN(row),SET(row, lane, present, the lane's 32 bits) andDELETE(row). Every record is stamped with its version and owner.commit(table, batch): check the batch, append it as exactly one Lance version, then apply it to the table. The Lance version equals the table version. The log's version is compared before the append, which refuses a stale second writer, and again after it.replay(owner, specs): rebuilds the table batch by batch through the sameTable::apply. It refuses a foreign owner, an undecodable record, or a log whose Lance version is not its batch count.Tablechanges: it gainscheck()andspecs(), and it refuses an empty batch (StoreError::EmptyBatch), since an empty batch would be a version that changes nothing.store-lance, installsprotoc(lance-encoding needs it at build time) and runs clippy and the tests with the feature. The default job stays fast.The PR also records option (d) for OD-12 in the plan. lance-graph #1453 merged a read-only LDAP v3 handler over dir-sim, so stalwart's own LDAP directory backend could read dir-sim through it with no stalwart code change. The approved option (c) stands until two questions are settled: password checks (they go through the host's
Authority) and dir-sim persistence.Evidence
Tests (
cargo test -p hubspo-store --features lance):Disable runs:
The post-append check stays green when disabled, and I kept it on purpose. It only fires when another writer appends between the pre-check and the write, a race no sequential test can schedule. The source says so at the check.
Locally: workspace fmt, clippy (
--all-targets --no-deps -D warnings, default features and--features lance) and tests all pass. Rebased onmainafter #22; no other HubSPO-rs PR is open.🤖 Generated with Claude Code
https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
Generated by Claude Code