Repository navigation
D-STO-5: history as a fold over resident log records (0.72 ms, was 336 ms) - #25
Merged
Merged
Conversation
…replay One write costs about a million folds (lance-graph 2026-09-24), and a Lance scan is recovery cost (~320 ms over 1,000 fragments). So the log keeps every record it commits or replays resident, and history runs its Quack program over memory instead of scanning Lance. - Log owns the resident records; commit appends to them after the Lance append and refuses when they are not at the table's version. - replay is the one Lance read; on any error nothing becomes resident. - history is synchronous. A corrupt record cannot become resident (replay refuses it), so the per-match required-lane check is structural now. - Tests: a second log replayed from disk gives identical histories; a log without resident records refuses to commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
ISS-HIST-SCAN-COST resolved without an index or projection. Plan 5.2 records the decision: history folds over resident records; Lance is read once, by replay; only learning adds persistent state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
AdaWorldAPI
pushed a commit
that referenced
this pull request
Oct 10, 2026
…n doc Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
AdaWorldAPI
pushed a commit
that referenced
this pull request
Oct 14, 2026
…#22-#25 D-QRY-1 points at the D-CMR set algebra (relalg prototype, #1546), D-QRY-5 at the D-CMR-5 preimage hop, D-FLOW-1 at graph-flow-lance. New ask ISS-LG-GRAPHFLOW-LANCE-PATHS; the preflight pin issue is still open at c21bd165. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
#24 read property history by scanning the Lance log on every call. Measured, that took 336 ms, 320 ms of it the scan, against P-HIST-1's < 20 ms budget.
The rule this applies (lance-graph, 2026-09-24): one write costs about a million folds, and a Lance scan is recovery cost. So history should be a fold over records already in memory, not a read of Lance.
Logkeeps its records resident. Every record it commits or replays stays in memory, at 29 B each.replayis the one Lance read. On any error nothing becomes resident; the log keeps what it held before.commitappends to the resident records after the Lance append. It refuses withDivergedwhen the records are not at the table's version, because otherwise history would silently miss earlier batches.historyis synchronous now. It runs the same Quack program over memory and never opens the dataset.No index, no column projection, no Lance filter expression, and no compaction.
ISS-HIST-SCAN-COSTis resolved. Plan §5.2 records the decision: derived reads write nothing, and only learning adds persistent state, recorded after as one merged, auditable outcome.Evidence
Timing. Release,
examples/probe_history.rs, 100k records, 1,000 versions × 1,000 updates, 1.2M log records:Tests (
cargo test -p hubspo-store --features lance; the log suite dropped from 16 s to 0.44 s):Disable runs, all red:
row = rtermop = DELETEarmop = SETtermLocal: fmt, clippy (
-D warnings, default andlance) and the workspace tests pass. No other HubSPO-rs PR is open; the branch is current withmain.🤖 Generated with Claude Code
https://claude.ai/code/session_01PPcakjk7ioQsnasmNDtc5z
Generated by Claude Code