poef · Sep 18, 2026 · Sep 17, 2026 · Sep 17, 2026 · Sep 17, 2026 · Sep 17, 2026
diff --git a/.spiral/cycles/CYC-20260917-TTZ7C-4.md b/.spiral/cycles/CYC-20260917-TTZ7C-4.md
@@ -0,0 +1,80 @@
+# Cycle: Persisted Offset Indexes
+
+Branch: `spiral/CYC-20260917-TTZ7C-4-conversion-offsets`, created from clean
+`master` and verified on 2026-09-17.
+
+## Commitment And Evidence
+
+After verification found 181,723 of 181,724 persisted offsets incorrect in the
+local curriculum-store dataset, the maintainer clarified: "the offset index
+code is incorrect, i want to correct that code, in simplystore first."
+The ID index was correct. Inspection of SimplyStore's converter confirms that
+index creation precedes the final serialization, although custom index hooks
+can change record sizes. This request authorizes fixing that evidenced defect.
+
+## Goal And Scope
+
+Make conversion persist offsets matching the final OD-JSONTag output after
+custom indexing. Preserve the CLI, custom create hook, and disk format. Verify
+default conversion and hooks that grow, shrink, and add records, including
+multibyte text. Do not change curriculum-store data or publish a package in
+this cycle.
+
+On 2026-09-17 the maintainer extended the goal to command changesets after a
+worker probe reproduced stale offsets and a missing new-entity entry, saying
+"But it needs fixing ...". This is an explicit re-plan of the active cycle:
+both conversion and successful commands must publish offsets derived from
+their final serialized bytes. Preserve sparse changeset record numbers,
+pre-serialization custom mutations, integrity recording, and failure behavior.
+
+The maintainer subsequently approved routing finalization through the index
+module so curriculum-store's customized methods work out of the box. Add an
+optional finalization hook with a default fallback for existing wrappers;
+preserve the current create/update/load contracts. This is a further explicit
+scope refinement within the same active cycle.
+
+## Plan Continuity
+
+Deliberate deviation at the maintainer's request from the durability/integrity
+sequence in SRC-001, REQ-001, and DES-001. CYC-018 is accepted; this bounded
+downstream-discovered correction does not replace that roadmap. The adopted
+Muze engineering/library cultures apply; compatibility and simplicity are
+explicit constraints. The command extension follows the same demonstrated
+defect and now also serves the existing durability and handler-ordering goals.
+
+## Evaluation
+
+Implemented final offset generation from the completed serialization. All 22
+tests in `npm test` pass, including three conversion CLI regressions. The
+readback/writer pass also produced exact byte ranges for all 181,724 records of
+the downstream data file in a disposable directory, taking about five seconds;
+the source file's SHA-256 remained unchanged. Test-file lint and diff checks
+pass. EVD-20260917-TTZ7C-8 records the evidence and limits.
+
+The command correction is also implemented and verified: all 53 tests across
+startup/offset, handler, ACID, and crash/recovery suites pass. Both conversion
+and commands now use the final-byte scanner/writer. The 181,724-record temporary
+dataset check still passes, taking about 0.23 seconds including verification
+with the shared scanner. EVD-20260917-TTZ7C-12 records the current evidence.
+
+The index-module dispatch refinement is implemented and verified: all 59 tests
+pass, with asynchronous custom finalizers, failure propagation, and automatic
+default fallback covered. A disposable fixture using byte-identical downstream
+index sources and shared dependencies passed conversion and command offset
+checks along with parent/root/niveau checks. EVD-20260917-TTZ7C-16 records the
+evidence and the cross-checkout dependency limitation found by the first probe.
+No merge, package release, downstream installation, or downstream-file repair
+was performed in this cycle.
+
+## Human Acceptance
+
+Accepted by the maintainer on 2026-09-18: "And accept the current cycle for
+simplystore." Acceptance covers final-byte offset generation for conversion
+and command changesets, including optional custom finalization with the default
+fallback, as evaluated above and recorded in EVD-20260917-TTZ7C-16.
+
+The cycle branch was rechecked as
+`spiral/CYC-20260917-TTZ7C-4-conversion-offsets` when recording acceptance.
+The maintainer separately requested explicit delegation in curriculum-store
+and chose to retain its published SimplyStore dependency. That downstream
+change is outside this cycle; using the new hook there awaits a package release.
diff --git a/.spiral/cycles/CYC-20260917-TTZ7C-4.ttl b/.spiral/cycles/CYC-20260917-TTZ7C-4.ttl
@@ -0,0 +1,9 @@
+@prefix sd: <https://muze.nl/ns/spiral-developer#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix project: <https://github.com/simplyedit/simplystore/spiral#> .
+
+project:CYC-20260917-TTZ7C-4
+    a sd:Cycle ;
+    dcterms:identifier "CYC-20260917-TTZ7C-4" ;
+    sd:repositoryPath ".spiral/cycles/CYC-20260917-TTZ7C-4.md" ;
+    sd:status sd:Accepted .
diff --git a/.spiral/designs/DES-20260917-TTZ7C-10.md b/.spiral/designs/DES-20260917-TTZ7C-10.md
@@ -0,0 +1,33 @@
+# Design: Derive Offsets From Final Serialized Records
+
+Keep the legacy custom index hook in its existing pre-serialization position:
+it can change canonical state, including records outside the initial changed
+list. Derive physical offsets only from the final bytes, not that changed list.
+
+Extract the existing OD-JSONTag framing validator's loop into a shared scanner
+with an optional record visitor. Track global record numbers through `+N`
+skips, report payload byte ranges for emitted records, and preserve all current
+framing failures. The existing validation entry point delegates without a
+visitor or allocating a complete index. This avoids parsing entity bodies or
+resolving references to records absent from a sparse changeset.
+
+Add a serialized-byte writer to the existing offset module. It builds the same
+JSON mapping and atomically replaces the appropriate base or command sidecar
+using the already-installed write-file-atomic library. Empty changesets produce
+`{}`. Keep the old create/update methods for custom-hook compatibility; the
+final pass overwrites their provisional output.
+
+The command worker awaits this final write after the changeset write and before
+integrity recording, the after-write fault point, and returning success. Failure
+propagates through the existing failed-command path. A crash can still leave
+uncommitted files; existing status-driven recovery must ignore them. This is not
+a new atomic multi-file transaction or power-loss durability guarantee.
+
+Use the same writer in conversion, replacing its extra full parse. This retains
+the earlier conversion guarantee and avoids maintaining two finalization paths.
+
+Verify with command-worker regressions for edits, additions, Unicode, leading
+and interior skips, hook-introduced changes, successive changesets, empty
+changesets, and failed final writes. Compare disk bytes with an independent
+framing oracle. Run startup, handler, ACID, and crash/recovery suites because
+command completion now awaits an additional write.
diff --git a/.spiral/designs/DES-20260917-TTZ7C-10.ttl b/.spiral/designs/DES-20260917-TTZ7C-10.ttl
@@ -0,0 +1,24 @@
+@prefix sd: <https://muze.nl/ns/spiral-developer#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix project: <https://github.com/simplyedit/simplystore/spiral#> .
+
+project:DES-20260917-TTZ7C-10
+    a sd:Design ;
+    dcterms:identifier "DES-20260917-TTZ7C-10" ;
+    sd:repositoryPath ".spiral/designs/DES-20260917-TTZ7C-10.md" ;
+    sd:status sd:Active ;
+    sd:supersedes [
+        a sd:ArtifactReference ;
+        sd:artifact project:DES-20260917-TTZ7C-6 ;
+        sd:gitCommit "5a118df699914f42e541daccb43a96b409101af2"
+    ] ;
+    sd:satisfies [
+        a sd:ArtifactReference ;
+        sd:artifact project:REQ-20260917-TTZ7C-9 ;
+        sd:gitCommit "5a4fd47ae8b59468067fbe164294472550abb274"
+    ] ;
+    sd:satisfies [
+        a sd:ArtifactReference ;
+        sd:artifact project:REQ-20260917-TTZ7C-5 ;
+        sd:gitCommit "55629a9da8860b909fe465a1a828129a8c53edf2"
+    ] .
diff --git a/.spiral/designs/DES-20260917-TTZ7C-14.md b/.spiral/designs/DES-20260917-TTZ7C-14.md
@@ -0,0 +1,33 @@
+# Design: Optional Index Finalization With Legacy Fallback
+
+Add `finalize(serialized, meta, uuid = null)` to the default index module. Its
+default behavior delegates to the existing serialized offset writer. The
+argument is the exact OD-JSONTag output (a string for conversion or Uint8Array
+for commands), metadata retains the existing shape, and uuid is null for base
+conversion or the command ID for a changeset.
+
+Export one `finalizeIndex(index, serialized, meta, uuid = null)` dispatcher
+from that module. It invokes the configured index's finalize when present,
+otherwise the default. Preserve the configured object's method receiver and
+return its promise so both callers await completion and propagate rejection.
+Do not mutate or spread the configured object: frozen legacy wrappers and
+methods inherited through prototypes should remain usable.
+
+The converter and command worker import only this dispatcher, not the offset
+module. Existing create/update/load hooks retain their invocation and mutation
+semantics. No additional method is required in curriculum-store. A custom
+finalizer owns its finalization behavior and can explicitly delegate to the
+default to retain standard offsets while adding custom outputs; the dispatcher
+does not invoke both and duplicate writes. Serialized input is read-only by
+contract, and finalization must not mutate canonical state.
+
+Keep the existing position relative to data persistence, integrity recording,
+and command completion. The final-byte writer and sparse framing behavior do
+not change. This supplements DES-20260917-TTZ7C-10 with a compatibility boundary,
+not a general handler registration system. Legacy ID/offset writes from custom
+pre-serialization hooks remain possible for compatibility.
+
+Verify async completion, rejection, method receiver, custom replacement and
+delegation, and fallback for frozen/legacy wrappers. Run a disposable probe
+against curriculum-store's actual unchanged index module for both conversion
+and a command on a representative graph, then run existing lifecycle suites.
diff --git a/.spiral/designs/DES-20260917-TTZ7C-14.ttl b/.spiral/designs/DES-20260917-TTZ7C-14.ttl
@@ -0,0 +1,19 @@
+@prefix sd: <https://muze.nl/ns/spiral-developer#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix project: <https://github.com/simplyedit/simplystore/spiral#> .
+
+project:DES-20260917-TTZ7C-14
+    a sd:Design ;
+    dcterms:identifier "DES-20260917-TTZ7C-14" ;
+    sd:repositoryPath ".spiral/designs/DES-20260917-TTZ7C-14.md" ;
+    sd:status sd:Active ;
+    sd:satisfies [
+        a sd:ArtifactReference ;
+        sd:artifact project:REQ-20260917-TTZ7C-13 ;
+        sd:gitCommit "59214a11f730c9657024c2f338513c86b1088dad"
+    ] ;
+    sd:derivedFrom [
+        a sd:ArtifactReference ;
+        sd:artifact project:DES-20260917-TTZ7C-10 ;
+        sd:gitCommit "c6bfb419cf48e2248e1bb0a7ae3d8ec069d0a6c9"
+    ] .
diff --git a/.spiral/designs/DES-20260917-TTZ7C-6.md b/.spiral/designs/DES-20260917-TTZ7C-6.md
@@ -0,0 +1,31 @@
+# Design: Finalize Offsets After Conversion
+
+Superseded by DES-20260917-TTZ7C-10 when the maintainer extended the correction
+to command changesets. The shared framing scanner replaces the full readback
+below while preserving the final-byte offset guarantee.
+
+The converter owns the final output bytes. Its custom index hook can mutate
+canonical objects after calling the built-in index writer. Positions captured
+before that hook completes cannot describe the final serialized file.
+
+Keep invoking the existing custom hook once. Serialize its completed result,
+write the output, parse that exact serialization with a fresh lazy OD-JSONTag
+parser, then use the existing offset writer with the fresh result array. This
+overwrites provisional offsets, includes newly serialized records, and retains
+the current JSON sidecar and CLI formats. The extra pass is linear in the data.
+Inspection of the installed parser shows that reading its position symbol
+materializes the entity body, so the current offset writer adds a full readback
+pass. Conversion already materializes its input; this bounded repair accepts
+that cost and makes no laziness-preservation claim for offset finalization.
+
+Changing the writer alone cannot fix later mutations by a caller. Introducing
+a new create/finalize hook contract would expand the compatibility surface for
+this bounded repair. Reuse the existing parser/writer at the point where the
+final bytes are known. This design makes no new atomic multi-file publication
+or crash-safety claim for the conversion command.
+
+Verify through the actual CLI with an independent byte-framing oracle. Cover
+default indexing, a hook that calls base indexing before changing lengths and
+adding an anonymous record, and a custom hook that writes no offset sidecar.
+Include multibyte text and a reused output directory. Run the normal startup
+suite in addition to these conversion regressions.
diff --git a/.spiral/designs/DES-20260917-TTZ7C-6.ttl b/.spiral/designs/DES-20260917-TTZ7C-6.ttl
@@ -0,0 +1,19 @@
+@prefix sd: <https://muze.nl/ns/spiral-developer#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix project: <https://github.com/simplyedit/simplystore/spiral#> .
+
+project:DES-20260917-TTZ7C-6
+    a sd:Design ;
+    dcterms:identifier "DES-20260917-TTZ7C-6" ;
+    sd:repositoryPath ".spiral/designs/DES-20260917-TTZ7C-6.md" ;
+    sd:status sd:Superseded ;
+    sd:transforms [
+        a sd:ArtifactReference ;
+        sd:artifact project:DES-20260917-TTZ7C-6 ;
+        sd:gitCommit "d63d2dead6bfc4b79b4265afd54c4de106c9ef47"
+    ] ;
+    sd:satisfies [
+        a sd:ArtifactReference ;
+        sd:artifact project:REQ-20260917-TTZ7C-5 ;
+        sd:gitCommit "55629a9da8860b909fe465a1a828129a8c53edf2"
+    ] .
diff --git a/.spiral/evidence/EVD-20260917-TTZ7C-12.md b/.spiral/evidence/EVD-20260917-TTZ7C-12.md
@@ -0,0 +1,44 @@
+# Evidence: Command And Conversion Offset Finalization
+
+Verified on 2026-09-17 with Node v24.7.0.
+
+## Regression Evidence
+
+All six command-offset tests failed before implementation. They exposed offsets
+into the prior file, missing appended entities and hook-introduced mutations,
+incorrect successive changesets, stale no-op indexes, and a missing failure
+when the final sidecar could not be written.
+
+After implementation, all 53 tests passed:
+
+- `npm test`: 28 startup, conversion, and command-offset tests.
+- `npm run test:handlers`: 4 tests, including final offset-write failure causing
+  failed status with the previous committed state visible and reconstructable.
+- `npm run test:acid`: 5 atomicity, consistency, isolation, and retry tests.
+- `npm run test:durability`: 16 tests including crash boundaries, restart,
+  timeouts, retry safety, and integrity-enabled command completion.
+
+Offsets are checked against an independent disk-byte framing oracle, including
+UTF-8 text, leading and interior skips, newly allocated records, hook mutations,
+successive changeset files, and empty changesets. Replaying base plus changesets
+preserves the expected logical data.
+
+The shared writer was also run against the local downstream 181,724-record base
+file with output confined to a temporary directory. Every offset matched the
+independent byte scan. Writer plus verification took 0.230 seconds in this one
+local run. The temporary output was removed; downstream files were not written.
+
+Scoped ESLint checks on command-offset tests, handler tests, and recovery code
+passed. Staged Spiral validation and whitespace checks passed before the
+implementation commit. Tests requiring processes/sockets ran with sandbox
+escalation; provenance validation used the existing temporary Python environment
+with rdflib.
+
+## Limits
+
+Existing stale sidecars are not repaired automatically. The corrected package
+has not been released or installed in curriculum-store. The final writer makes
+successful commands await complete offsets; it does not make the changeset,
+offset sidecar, and integrity record one atomic filesystem transaction. Failed
+or interrupted commands may leave uncommitted derived files, which the existing
+status-driven recovery excludes. Human acceptance remains pending.
diff --git a/.spiral/evidence/EVD-20260917-TTZ7C-12.ttl b/.spiral/evidence/EVD-20260917-TTZ7C-12.ttl
@@ -0,0 +1,24 @@
+@prefix sd: <https://muze.nl/ns/spiral-developer#> .
+@prefix dcterms: <http://purl.org/dc/terms/> .
+@prefix project: <https://github.com/simplyedit/simplystore/spiral#> .
+
+project:EVD-20260917-TTZ7C-12
+    a sd:VerificationEvidence ;
+    dcterms:identifier "EVD-20260917-TTZ7C-12" ;
+    sd:repositoryPath ".spiral/evidence/EVD-20260917-TTZ7C-12.md" ;
+    sd:status sd:Active ;
+    sd:verifies [
+        a sd:ArtifactReference ;
+        sd:artifact project:IMP-20260917-TTZ7C-11 ;
+        sd:gitCommit "c6bfb419cf48e2248e1bb0a7ae3d8ec069d0a6c9"
+    ] ;
+    sd:verifies [
+        a sd:ArtifactReference ;
+        sd:artifact project:IMP-20260917-TTZ7C-7 ;
+        sd:gitCommit "c6bfb419cf48e2248e1bb0a7ae3d8ec069d0a6c9"
+    ] ;
+    sd:verifies [
+        a sd:ArtifactReference ;
+        sd:artifact project:REQ-20260917-TTZ7C-9 ;
+        sd:gitCommit "5a4fd47ae8b59468067fbe164294472550abb274"
+    ] .
diff --git a/.spiral/evidence/EVD-20260917-TTZ7C-16.md b/.spiral/evidence/EVD-20260917-TTZ7C-16.md
@@ -0,0 +1,55 @@
+# Evidence: Configured Index Finalization
+
+Verified on 2026-09-17 with Node v24.7.0.
+
+## Regression And Lifecycle Checks
+
+Before implementation, all five focused custom-finalization tests failed:
+conversion and commands ignored the configured finalizer, including rejection.
+After implementation all 59 tests passed:
+
+- `npm test`: 33 startup, conversion, and command-offset tests.
+- `npm run test:handlers`: 5 tests, including custom finalizer rejection keeping
+  the prior committed state visible and independently reconstructable.
+- `npm run test:acid`: 5 tests.
+- `npm run test:durability`: 16 crash/recovery, timeout, retry, and integrity tests.
+
+Focused tests verify asynchronous completion, receiver and argument preservation,
+explicit delegation to default offsets, replacement without an unconditional
+default write, rejection propagation, and fallback for frozen legacy wrappers.
+An additional direct dispatcher probe verified an inherited finalizer on a
+frozen object, preserving its receiver, default UUID, and returned promise value.
+
+Scoped ESLint on `src/index.mjs`, conversion and command offset tests, and handler
+tests passed. Staged provenance and whitespace checks passed before committing
+the implementation. Process/socket suites used sandbox escalation.
+
+## Curriculum-Store Compatibility
+
+A disposable miniature curriculum graph exercised the actual downstream
+`index.mjs`, parent, root, niveau, and utility source files copied byte-for-byte
+from `curriculum-restapi-dev/curriculum-store/src`. The fixture linked SimplyStore
+to this checkout and shared its JSONTag/OD-JSONTag dependencies. No source edits
+were made to the copied wrapper, which still has no `finalize` method.
+
+Conversion and a command that lengthened a child title both produced offsets
+matching an independent byte-framing oracle. Conversion preserved the expected
+parent, root, and niveau links; replaying the command changeset preserved the
+new title and niveau index. Temporary data was removed afterward.
+
+The initial direct cross-checkout probe failed before finalization because the
+installed old SimplyStore and this checkout resolved distinct OD-JSONTag symbol
+instances: the old offset writer could not see the parsed entity's position.
+The shared-dependency fixture avoids that test-environment mismatch and verifies
+unchanged wrapper source against the fixed dependency; it does not prove that
+mixing arbitrary installed dependency copies works. Installed downstream
+dependencies and data were not changed.
+
+## Limits
+
+This refines dispatch only; the previously verified final-byte scanner remains
+unchanged. Custom finalizers own their output and must delegate to retain default
+offset files. Read-only finalization inputs are a documented contract, not an
+enforced sandbox. Failure can leave uncommitted files, not a partial successful
+command. Existing stale sidecars are not repaired automatically. No release,
+downstream installation, merge, or human acceptance is claimed.