From bcf64e82a3a4290a1079d1029ec6bd86c67734bb Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 12:47:15 +0200 Subject: [PATCH 01/17] Record meta.schema isolate failure report Retain the maintainer's report and stable-identity direction for SRC-20260926-TTZ7C-61. Co-Authored-By: Claude Opus 5.5 --- .spiral/sources/SRC-20260926-TTZ7C-61.md | 23 +++++++++++++++++++++++ .spiral/sources/SRC-20260926-TTZ7C-61.ttl | 10 ++++++++++ 2 files changed, 33 insertions(+) create mode 100644 .spiral/sources/SRC-20260926-TTZ7C-61.md create mode 100644 .spiral/sources/SRC-20260926-TTZ7C-61.ttl diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.md b/.spiral/sources/SRC-20260926-TTZ7C-61.md new file mode 100644 index 0000000..bdf477a --- /dev/null +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.md @@ -0,0 +1,23 @@ +# meta.schema fails in isolated queries + +Retained source: maintainer conversation on 2026-09-26. + +The maintainer reported that curriculum-store, which uses SimplyStore 0.11 with +isolated-vm queries, fails when a query asks for `meta.schema`, "even in the +slowquery (10s) it timesout". The maintainer's hypothesis: "meta.schema is a +full graph. It might be that the host boundary doesn't preserve object +identity, so the graph results in an infinite walk." + +The assistant reproduced the failure and reported the schema as an acyclic +graph with shared objects rather than a cyclic one. The maintainer confirmed +the resulting understanding and the proposal to open a defect cycle, adding: + +> I think each object should get a stable id, not just for a single query. +> That id shouldn't be exposed to the query side. If an object already has a +> stable id, we can reuse that, but I don't mind if we don't. + +This settles the correction direction: identity for non-record query objects +is a stable identifier that outlives one query, and it stays private to the +host/guest bridge. Reusing an existing identifier is permitted, not required. +The maintainer also confirmed that the target is the pre-0.11 behavior: +JSONTag responses keep shared objects as links; JSON responses expand them. diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.ttl b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl new file mode 100644 index 0000000..1c88cf3 --- /dev/null +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl @@ -0,0 +1,10 @@ +@prefix sd: . +@prefix dcterms: . +@prefix project: . + +project:SRC-20260926-TTZ7C-61 a sd:Source ; + dcterms:identifier "SRC-20260926-TTZ7C-61" ; + sd:repositoryPath ".spiral/sources/SRC-20260926-TTZ7C-61.md" ; + sd:sourceAvailability sd:Retained ; + sd:provenanceConfidence sd:Explicit ; + sd:status sd:Active . From d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 12:47:57 +0200 Subject: [PATCH 02/17] Open schema identity defect cycle Record DEF-20260926-TTZ7C-62 (schema objects identified by path across the isolate) and open CYC-20260926-TTZ7C-63 as a deliberate roadmap deviation to correct it. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 27 +++++++++++++++++++++++ .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 9 ++++++++ .spiral/defects/DEF-20260926-TTZ7C-62.md | 27 +++++++++++++++++++++++ .spiral/defects/DEF-20260926-TTZ7C-62.ttl | 13 +++++++++++ 4 files changed, 76 insertions(+) create mode 100644 .spiral/cycles/CYC-20260926-TTZ7C-63.md create mode 100644 .spiral/cycles/CYC-20260926-TTZ7C-63.ttl create mode 100644 .spiral/defects/DEF-20260926-TTZ7C-62.md create mode 100644 .spiral/defects/DEF-20260926-TTZ7C-62.ttl diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md new file mode 100644 index 0000000..0e4e9a0 --- /dev/null +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -0,0 +1,27 @@ +# Schema identity across the query isolate + +Goal: restore object identity for non-record objects across the query isolate +boundary, so `meta.schema` queries work and shared schema objects keep their +JSONTag links. Correct DEF-20260926-TTZ7C-62. + +Status: Active. Human commitment: the maintainer confirmed the understanding +and defect cycle on 2026-09-26 and set the identity direction retained in +SRC-20260926-TTZ7C-61: each object gets a stable identifier that outlives a +single query and is not exposed to query code. + +Classification: deliberate deviation from ROADMAP.md. Object history remains +the next roadmap target. This defect is in accepted, integrated work from +CYC-20260924-TTZ7C-56 and blocks curriculum-store's SimplyStore 0.11 upgrade. + +Branch: spiral/CYC-20260926-TTZ7C-63-schema-identity, created from +authoritative master bd61d9c. Branch identity checked when opening the cycle. + +In scope: the identity rule in DES-20260924-TTZ7C-59, its implementation in the +query view/guest bridge, a regression with a shared and cyclic schema, and +measurement of the curriculum-store schema through the real runtime. Guest-side +read caching is in scope only if measurements show identity alone leaves the +query outside its limits. + +Non-goals: object history, schema editing, changes to data record identity, +query limits or grants, curriculum-store's own Spiral repair and unrelated +cleanup. diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl new file mode 100644 index 0000000..63c15b2 --- /dev/null +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -0,0 +1,9 @@ +@prefix sd: . +@prefix dcterms: . +@prefix project: . + +project:CYC-20260926-TTZ7C-63 a sd:Cycle ; + dcterms:identifier "CYC-20260926-TTZ7C-63" ; + sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; + sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; + sd:status sd:Active . diff --git a/.spiral/defects/DEF-20260926-TTZ7C-62.md b/.spiral/defects/DEF-20260926-TTZ7C-62.md new file mode 100644 index 0000000..4a68eab --- /dev/null +++ b/.spiral/defects/DEF-20260926-TTZ7C-62.md @@ -0,0 +1,27 @@ +# Schema objects lose identity across the query isolate + +Observed on master bd61d9c with curriculum-store's 92 KB `schema.jsontag`. +A query for `meta.schema` fails in both JSON and JSONTag modes. With default +limits it exhausted the 64 MiB isolate heap after about 2.5 s; the maintainer +observed the 10 s slow-query timeout. Smaller contexts succeed, but their +JSONTag responses expand shared objects instead of linking them: the +`kerndoelen` context returns 28,227 bytes where host serialization returns +6,957. + +The schema is acyclic. It contains 281 distinct objects and 734 shared +references. Treating every path as a distinct object expands it to 99,001 +objects. Host-side, the whole schema serializes to 49,844 bytes of JSONTag in +about 1 ms (4,012,890 bytes of JSON). + +Cause: DES-20260924-TTZ7C-59 identifies query objects by "record numbers plus +property paths (or schema paths)". A record number is a stable identity, and +od-jsontag stores every data object as its own record, so data arrays have one +owning path. Schema objects are ordinary JSONTag objects that may be reached +by many paths. The guest caches proxies by reference, so one schema object +becomes a distinct proxy per path: `===` fails, the JSONTag serializer cannot +emit links, and each path repeats its host reads. + +Earliest meaningful cause: the design assumed path uniqueness for schema +objects. Verification did not catch it because the query test schema is a +tree. The correction belongs in the design's identity rule, followed by the +implementation and a regression with a shared/cyclic schema. diff --git a/.spiral/defects/DEF-20260926-TTZ7C-62.ttl b/.spiral/defects/DEF-20260926-TTZ7C-62.ttl new file mode 100644 index 0000000..4abd918 --- /dev/null +++ b/.spiral/defects/DEF-20260926-TTZ7C-62.ttl @@ -0,0 +1,13 @@ +@prefix sd: . +@prefix dcterms: . +@prefix project: . + +project:DEF-20260926-TTZ7C-62 + a sd:Defect ; + dcterms:identifier "DEF-20260926-TTZ7C-62" ; + sd:repositoryPath ".spiral/defects/DEF-20260926-TTZ7C-62.md" ; + sd:status sd:Active ; + sd:provenanceConfidence sd:Evidenced ; + sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; + sd:observes [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "bd61d9cf612008e96a518fa30fca9e33c40fa82e" ] ; + sd:observes [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "bd61d9cf612008e96a518fa30fca9e33c40fa82e" ] . From 49f35739f1e33695223da9a651375df23aff01d1 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 12:49:16 +0200 Subject: [PATCH 03/17] Add failing schema identity regressions Baseline for DEF-20260926-TTZ7C-62: shared, linked and cyclic schema objects must keep identity across the query isolate, and JSONTag responses must match host serialization. Both tests fail on bd61d9c (identity false; deeply shared schema times out). Co-Authored-By: Claude Opus 5.5 --- test/query-isolate.mjs | 63 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 61 insertions(+), 2 deletions(-) diff --git a/test/query-isolate.mjs b/test/query-isolate.mjs index f65bb23..092e99f 100644 --- a/test/query-isolate.mjs +++ b/test/query-isolate.mjs @@ -29,10 +29,12 @@ async function open(t, options = {}) { !(property === 'age' && method === 'has') } `) + const { schema = '{"version":42,"nested":{"value":"ok"}}', ...limits } = + options const schemaFile = path.join(fixture.dir, 'schema.jsontag') - await fs.writeFile(schemaFile, '{"version":42,"nested":{"value":"ok"}}') + await fs.writeFile(schemaFile, schema) runtime = await StoreRuntime.open({ - ...fixture, access, schemaFile, maxWorkers: 1, ...options + ...fixture, access, schemaFile, maxWorkers: 1, ...limits }) async function response(body, extra = {}) { return runtime.runQuery({ path: '/', body, jsontag: false, ...extra }) @@ -248,3 +250,60 @@ test('outer query deadline replaces a stuck worker at the committed head', await failed assert.equal(await pool.run('query', {}, {timeout: 50}), 2) }) + +// Shared and cyclic references, as produced by in a schema file. +const graphSchema = '{"types":{' + + '"Node":{"label":"node",' + + '"children":["/schema/types/Node/"],' + + '"title":{"type":"string"}},' + + '"Leaf":{' + + '"title":"/schema/properties/title/"}},' + + '"contexts":{"a":{"root":"/schema/types/Node/"},' + + '"b":{"root":"/schema/types/Node/"}}}' + +test('schema objects keep identity across paths, links and cycles', + async t => { + const { query, response } = await open(t, {schema: graphSchema}) + assert.equal(await query( + 'meta.schema.types.Node.title === meta.schema.types.Leaf.title' + ), true) + assert.equal(await query( + 'meta.schema.types.Node.children[0] === meta.schema.types.Node' + ), true) + assert.equal(await query( + 'meta.schema.contexts.a.root === meta.schema.contexts.b.root' + ), true) + const host = JSONTag.parse(graphSchema) + assert.deepEqual( + await query('Reflect.ownKeys(meta.schema.types.Node).map(String)'), + Reflect.ownKeys(host.types.Node).map(String) + ) + assert.deepEqual(await query('meta.schema.types.Leaf'), + {title: {type: 'string'}}) + const tagged = await response('meta.schema', {jsontag: true}) + assert.equal(tagged.code, undefined, tagged.body) + assert.equal(tagged.body, JSONTag.stringify(host)) + const start = performance.now() + const cyclic = await response('meta.schema', {jsontag: false}) + assert.equal(cyclic.code, 422, cyclic.body) + assert.match(cyclic.body, /circular/i) + assert.ok(performance.now() - start < 2000) + }) + +test('widely shared schema objects serialize as links within the timeout', + async t => { + // Each level links twice to the next: 26 objects, 2^25 paths. + const levels = 25 + let schema = '{"value":"end"}' + for (let level = levels - 1; level >= 0; level--) { + const next = '"/schema/' + (level + 1) + '/"' + schema = '{' + + '"left":' + schema + ',"right":' + next + '}' + } + const { response } = await open(t, {schema}) + const start = performance.now() + const result = await response('meta.schema', {jsontag: true}) + assert.equal(result.code, undefined, result.body) + assert.equal(result.body, JSONTag.stringify(JSONTag.parse(schema))) + assert.ok(performance.now() - start < 1000) + }) From 07c6d6590dc2ad9d585df4bb3fe1f2ea2fb1c7ca Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 12:50:28 +0200 Subject: [PATCH 04/17] Identify schema query objects by stable handles Correct DES-20260924-TTZ7C-59 for DEF-20260926-TTZ7C-62: schema paths do not identify shared or cyclic schema objects. Number schema objects once by a deterministic walk, keep the handles private to the bridge, and keep record numbers as data identity. Co-Authored-By: Claude Opus 5.5 --- .spiral/designs/DES-20260924-TTZ7C-59.md | 14 +++++++++++--- .spiral/designs/DES-20260924-TTZ7C-59.ttl | 5 ++++- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.md b/.spiral/designs/DES-20260924-TTZ7C-59.md index 53549f1..312adf9 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.md +++ b/.spiral/designs/DES-20260924-TTZ7C-59.md @@ -15,10 +15,18 @@ and private synchronous callbacks. A disposable prototype scanned 10,000 records in about 362–430 ms through this view; the chosen boundary still warrants real integration measurements. -References identify record numbers plus property paths (or schema paths), not -host objects or paths on disk. Resolve them through the host parser for every +References identify stable host identities, not host objects or paths on disk. +A data object reuses its record number, and a data array its record number plus +property path: od-jsontag stores every object as a record, so an array has one +owning path. Schema objects and arrays may be shared or cyclic, so a path does +not identify them. Number them once per parsed schema by a deterministic walk +in property order; the same schema yields the same handles in every worker and +query. Handles remain private to the bridge and add no query-visible property, +symbol or value. Existing JSONTag `id` attributes are not reused because not +every schema object has one. Resolve references through the host for every operation so bounded record caches remain effective. Preserve guest identity -with weak references. Return scalars, tagged scalar copies and descriptions of +with weak references keyed by these references, so one host object is one +guest object and JSONTag serialization retains links. Return scalars, tagged scalar copies and descriptions of object/array views. Expose only ordinary property names and JSONTag's public metadata symbols. Do not forward parser-private symbols, prototypes, methods, raw buffers, source handles or isolated-vm References. Mutating the view throws; diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.ttl b/.spiral/designs/DES-20260924-TTZ7C-59.ttl index 244ac42..c778299 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.ttl +++ b/.spiral/designs/DES-20260924-TTZ7C-59.ttl @@ -5,4 +5,7 @@ project:DES-20260924-TTZ7C-59 a sd:Design ; dcterms:identifier "DES-20260924-TTZ7C-59" ; sd:repositoryPath ".spiral/designs/DES-20260924-TTZ7C-59.md" ; sd:satisfies [ a sd:ArtifactReference ; sd:artifact project:REQ-20260924-TTZ7C-58 ; sd:gitCommit "fdef01a844e14f50c0af1aac6e89461714736f6d" ] ; - sd:status sd:Active . + sd:status sd:Active ; + sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; + sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "1e2c7851a12ddc1b4c236d557a70479f411dfb8d" ] . From a2a7aaf207d666cd80a6d662a2e254414f1a1c02 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:08:30 +0200 Subject: [PATCH 05/17] Record schema id decision and caching scope Retain the maintainer's choice to preserve schema JSONTag ids for query workers, and bring guest-side read caching into CYC-20260926-TTZ7C-63 after measuring 2.7 s for the full schema as JSON. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 7 +++++++ .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 2 ++ .spiral/sources/SRC-20260926-TTZ7C-61.md | 10 ++++++++++ .spiral/sources/SRC-20260926-TTZ7C-61.ttl | 3 ++- 4 files changed, 21 insertions(+), 1 deletion(-) diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index 0e4e9a0..cc9f8ce 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -22,6 +22,13 @@ measurement of the curriculum-store schema through the real runtime. Guest-side read caching is in scope only if measurements show identity alone leaves the query outside its limits. +Scope refinement during Act: with stable handles, curriculum-store's full +schema returns as JSON in 2.7 s, above the 1 s default timeout, while the host +serializes it in 11 ms. Guest-side read caching is therefore in scope. JSONTag +responses failed because the schema reaches workers without its JSONTag ids; +per the maintainer's decision in SRC-20260926-TTZ7C-61, query workers receive +and parse the schema's original JSONTag text. + Non-goals: object history, schema editing, changes to data record identity, query limits or grants, curriculum-store's own Spiral repair and unrelated cleanup. diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl index 63c15b2..d611b97 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -6,4 +6,6 @@ project:CYC-20260926-TTZ7C-63 a sd:Cycle ; dcterms:identifier "CYC-20260926-TTZ7C-63" ; sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; + sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; sd:status sd:Active . diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.md b/.spiral/sources/SRC-20260926-TTZ7C-61.md index bdf477a..e025f8d 100644 --- a/.spiral/sources/SRC-20260926-TTZ7C-61.md +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.md @@ -21,3 +21,13 @@ is a stable identifier that outlives one query, and it stays private to the host/guest bridge. Reusing an existing identifier is permitted, not required. The maintainer also confirmed that the target is the pre-0.11 behavior: JSONTag responses keep shared objects as links; JSON responses expand them. + +## Schema metadata decision + +During the cycle the assistant reported that schema objects reach query workers +by structured clone, which keeps sharing but drops JSONTag metadata such as +`id`. Under VM2 JSONTag responses therefore used random UUID link ids from +Node's crypto, which the isolate does not provide. Offered either preserving +the schema's ids for query workers or matching 0.10's generated ids, the +maintainer chose "Preserve schema ids". Query responses should use the ids from +the schema file. Command workers are unchanged by this decision. diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.ttl b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl index 1c88cf3..0590dbe 100644 --- a/.spiral/sources/SRC-20260926-TTZ7C-61.ttl +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl @@ -7,4 +7,5 @@ project:SRC-20260926-TTZ7C-61 a sd:Source ; sd:repositoryPath ".spiral/sources/SRC-20260926-TTZ7C-61.md" ; sd:sourceAvailability sd:Retained ; sd:provenanceConfidence sd:Explicit ; - sd:status sd:Active . + sd:status sd:Active ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] . From d881fe1e1e81450182b64d4e35ebf4d4bb648860 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:09:08 +0200 Subject: [PATCH 06/17] Carry schema text to query workers and cache guest reads Extend the DES-20260924-TTZ7C-59 correction: query workers parse the schema's JSONTag text so ids survive, and guest proxies cache reads for the query's lifetime. Co-Authored-By: Claude Opus 5.5 --- .spiral/designs/DES-20260924-TTZ7C-59.md | 13 ++++++++++++- .spiral/designs/DES-20260924-TTZ7C-59.ttl | 4 ++-- 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.md b/.spiral/designs/DES-20260924-TTZ7C-59.md index 312adf9..bdc0711 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.md +++ b/.spiral/designs/DES-20260924-TTZ7C-59.md @@ -26,7 +26,18 @@ symbol or value. Existing JSONTag `id` attributes are not reused because not every schema object has one. Resolve references through the host for every operation so bounded record caches remain effective. Preserve guest identity with weak references keyed by these references, so one host object is one -guest object and JSONTag serialization retains links. Return scalars, tagged scalar copies and descriptions of +guest object and JSONTag serialization retains links. + +Structured cloning keeps shared objects but drops JSONTag metadata, including +the schema's `id`s that JSONTag links name. The runtime therefore passes the +loaded schema's original JSONTag text to each query worker, which parses it +once at initialization. Query responses use the schema file's ids. + +Data cannot change during a query: updates are serialized with queries and +each query has a fresh isolate. A guest proxy therefore caches its keys and +property reads for the query's lifetime, so repeatedly traversing shared +objects does not repeat host calls. Grants still apply on the first read. The +cache uses built-ins captured before query code runs. Return scalars, tagged scalar copies and descriptions of object/array views. Expose only ordinary property names and JSONTag's public metadata symbols. Do not forward parser-private symbols, prototypes, methods, raw buffers, source handles or isolated-vm References. Mutating the view throws; diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.ttl b/.spiral/designs/DES-20260924-TTZ7C-59.ttl index c778299..29d8d6c 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.ttl +++ b/.spiral/designs/DES-20260924-TTZ7C-59.ttl @@ -7,5 +7,5 @@ project:DES-20260924-TTZ7C-59 a sd:Design ; sd:satisfies [ a sd:ArtifactReference ; sd:artifact project:REQ-20260924-TTZ7C-58 ; sd:gitCommit "fdef01a844e14f50c0af1aac6e89461714736f6d" ] ; sd:status sd:Active ; sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "1e2c7851a12ddc1b4c236d557a70479f411dfb8d" ] . + sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "07c6d6590dc2ad9d585df4bb3fe1f2ea2fb1c7ca" ] . From be90dd46119e0ff22fb5a9eac396b9d0dd65666a Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:12:49 +0200 Subject: [PATCH 07/17] Keep schema identity across the query isolate Implement the corrected DES-20260924-TTZ7C-59 for DEF-20260926-TTZ7C-62: - number schema objects once and reference them by private handles; - pass the schema's JSONTag text to query workers so ids survive; - cache host reads per guest view for the query's lifetime. curriculum-store's schema now returns in 34 ms as JSONTag (matching host serialization) and 271 ms as JSON, where it previously exhausted the isolate or timed out. Co-Authored-By: Claude Opus 5.5 --- .../implementations/IMP-20260919-TTZ7C-33.md | 12 +++++ .../implementations/IMP-20260919-TTZ7C-33.ttl | 8 +-- docs/query-runtime.md | 6 ++- src/file-data.mjs | 10 ++-- src/query-guest.mjs | 40 ++++++++++++--- src/query-view.mjs | 49 ++++++++++++++++++- src/query-worker-module.mjs | 4 ++ src/store-runtime.mjs | 3 ++ 8 files changed, 114 insertions(+), 18 deletions(-) diff --git a/.spiral/implementations/IMP-20260919-TTZ7C-33.md b/.spiral/implementations/IMP-20260919-TTZ7C-33.md index cbe752c..1018d0a 100644 --- a/.spiral/implementations/IMP-20260919-TTZ7C-33.md +++ b/.spiral/implementations/IMP-20260919-TTZ7C-33.md @@ -239,3 +239,15 @@ Query tests, actual-worker performance measurement, runtime documentation and Node startup requirements are included. Native isolates remain inside Node worker threads; no OS-process memory/crash boundary or public safety certification is claimed. Trusted command/storage/integrity/recovery contracts remain effective. + +## Schema identity correction + +Corrects DEF-20260926-TTZ7C-62 per the revised DES-20260924-TTZ7C-59. The query +view numbers schema objects once per parsed schema by a deterministic walk and +describes them as private `['schema', handle]` references; record numbers +remain data identity. The loader also returns the schema's JSONTag text, which +the runtime passes to query workers so their `meta.schema` keeps its ids. +Command workers still receive the cloned schema. Guest proxies cache host +descriptions of keys, reads and existence checks for the query's lifetime +using built-ins captured before query code runs; tagged scalars are still +copied on every read. diff --git a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl index 42805ee..fc156ce 100644 --- a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl +++ b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl @@ -33,9 +33,9 @@ project:IMP-20260919-TTZ7C-33 sd:implementationLocation "scripts/benchmark-file-data.mjs" ; sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260923-TTZ7C-47 ; sd:gitCommit "83edb03bc406b9e651ef93bed69b87ece17dcf47" ] ; sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260923-TTZ7C-54 ; sd:gitCommit "c8a3fd9464abfd5e7b699a2752789d82af4da2da" ] ; - sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "1e2c7851a12ddc1b4c236d557a70479f411dfb8d" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "2595efb946993f1b088a430c46dd04927fa27868" ] ; - sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:REQ-20260924-TTZ7C-58 ; sd:gitCommit "fdef01a844e14f50c0af1aac6e89461714736f6d" ] ; + sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "bd507554a2fed6f77322ef38a98839f127d810db" ] ; + sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; sd:shapedBy [ a sd:ArtifactReference ; sd:artifact ; sd:gitCommit "8d4b2c738a413abd4cccca740ce958f486e5f7af" ] ; - sd:implementationLocation "src/query-isolate.mjs", "src/query-view.mjs", "src/query-guest.mjs", "test/query-isolate.mjs", "scripts/benchmark-query-runtime.mjs" ; + sd:implementationLocation "src/query-isolate.mjs", "src/query-view.mjs", "src/query-guest.mjs", "test/query-isolate.mjs", "scripts/benchmark-query-runtime.mjs", "docs/query-runtime.md" ; sd:implementationChangeKind sd:SemanticChange . diff --git a/docs/query-runtime.md b/docs/query-runtime.md index 69f5b42..1232c07 100644 --- a/docs/query-runtime.md +++ b/docs/query-runtime.md @@ -23,7 +23,11 @@ descriptors and property existence. Access modules can keep their trusted Node imports and closures. Record numbers and property paths let the host resolve views without pinning -every scanned record in memory. Tagged scalar values are copied into the guest; +every scanned record in memory. Schema objects may be shared or cyclic, so the +host numbers them once per schema and resolves views by that private handle. +One stored object is one view within a query: `===` holds across paths, and +JSONTag responses link shared objects using the schema file's ids. Each view +reads a property from the host once per query. Tagged scalar values are copied into the guest; changes to such copies do not modify stored data. Object and array mutations throw. Committed source updates still serialize with queries, and replacement workers initialize at the current committed head. diff --git a/src/file-data.mjs b/src/file-data.mjs index 265844e..d068012 100644 --- a/src/file-data.mjs +++ b/src/file-data.mjs @@ -453,12 +453,14 @@ async function readFileData(files, manifest) { ids = dataset.rebuildIds() } meta.index = { id: ids } + // Cloning to another thread drops JSONTag metadata such as ids, so + // also return the text for consumers that need the schema intact. + let schema = null if (files.schemaFile) { - meta.schema = JSONTag.parse( - fs.readFileSync(files.schemaFile, 'utf8') - ) + schema = fs.readFileSync(files.schemaFile, 'utf8') + meta.schema = JSONTag.parse(schema) } - return { sources, meta } + return { sources, meta, schema } } finally { dataset.close() diff --git a/src/query-guest.mjs b/src/query-guest.mjs index 348d7b7..f7576a9 100644 --- a/src/query-guest.mjs +++ b/src/query-guest.mjs @@ -16,9 +16,13 @@ export function prepare(read, validURL, initial) { const stringify = JSON.stringify const parse = JSONTag.parse const charCode = Function.prototype.call.bind(String.prototype.charCodeAt) - const cache = new Map() + const Cache = Map + const cache = new Cache() const getCached = cache.get.bind(cache) const setCached = cache.set.bind(cache) + const cacheHas = Function.prototype.call.bind(Map.prototype.has) + const cacheGet = Function.prototype.call.bind(Map.prototype.get) + const cacheSet = Function.prototype.call.bind(Map.prototype.set) const dereference = Function.prototype.call.bind(WeakRef.prototype.deref) const makeWeak = value => new WeakReference(value) const metadataKey = key => symbols.indexOf(key) @@ -62,11 +66,23 @@ export function prepare(read, validURL, initial) { } return key } + // Data cannot change during a query, so each host answer is read once. + // Shared objects are then traversed without repeating host calls. + // Descriptions are cached, not values: tagged copies stay fresh. + const reads = new Cache() + const exists = new Cache() + let keys + const readOnce = (answers, operation, encoded) => { + if (!cacheHas(answers, encoded)) { + cacheSet(answers, encoded, read(operation, reference, encoded)) + } + return cacheGet(answers, encoded) + } const proxy = new Proxy(target, { get(target, key, receiver) { const encoded = remoteKey(key) if (encoded !== -1) { - const result = read('get', reference, encoded) + const result = readOnce(reads, 'get', encoded) if (result.present) { return wrap(result.entry) } @@ -75,16 +91,24 @@ export function prepare(read, validURL, initial) { }, has(target, key) { const encoded = remoteKey(key) - return (encoded !== -1 && read('has', reference, encoded)) || + return (encoded !== -1 && readOnce(exists, 'has', encoded)) || Reflect.has(target, key) }, ownKeys() { - return read('keys', reference).map(key => { + if (!keys) { + keys = read('keys', reference) + } + const names = [] + for (let index = 0; index < keys.length; index++) { + const key = keys[index] if (typeof key === 'number') { - return symbols[key] + names[index] = symbols[key] } - return key - }) + else { + names[index] = key + } + } + return names }, getOwnPropertyDescriptor(target, key) { if (description.array && key === 'length') { @@ -94,7 +118,7 @@ export function prepare(read, validURL, initial) { if (encoded === -1) { return undefined } - const result = read('get', reference, encoded) + const result = readOnce(reads, 'get', encoded) if (!result.present) { return undefined } diff --git a/src/query-view.mjs b/src/query-view.mjs index bf060e9..ff17fd8 100644 --- a/src/query-view.mjs +++ b/src/query-view.mjs @@ -19,17 +19,60 @@ function property(key) { throw new Error('Invalid query property') } +// Schema objects may be shared or cyclic, so a path does not identify them. +// Number them once per parsed schema, walking in property order, so the same +// schema yields the same handles in every worker and query. Handles stay in +// the bridge; query code never sees them. +const numberedSchemas = new WeakMap() + +function numberSchema(schema) { + const objects = [] + const handles = new Map() + const pending = [schema] + while (pending.length) { + const value = pending.pop() + if (!value || typeof value !== 'object' || handles.has(value)) { + continue + } + handles.set(value, objects.length) + objects.push(value) + const keys = Object.keys(value) + for (let index = keys.length - 1; index >= 0; index--) { + pending.push(value[keys[index]]) + } + } + return { objects, handles } +} + +function schemaHandles(schema) { + if (!schema || typeof schema !== 'object') { + return { objects: [], handles: new Map() } + } + let numbered = numberedSchemas.get(schema) + if (!numbered) { + numbered = numberSchema(schema) + numberedSchemas.set(schema, numbered) + } + return numbered +} + export class QueryView { constructor(dataset, maxBytes) { this.dataset = dataset this.maxBytes = maxBytes + this.schema = schemaHandles(dataset.parser.meta.schema) } resolve(reference) { const [record, ...keys] = reference let value if (record === 'schema') { - value = this.dataset.parser.meta.schema + const handle = keys.shift() + if (!Number.isSafeInteger(handle) || + handle < 0 || handle >= this.schema.objects.length) { + throw new Error('Invalid query reference') + } + value = this.schema.objects[handle] } else { value = this.dataset.parser.getLineProxy(record) @@ -63,9 +106,13 @@ export class QueryView { return { tagged } } const record = value[getIndex] + const handle = this.schema.handles.get(value) if (Number.isSafeInteger(record)) { reference = [record] } + else if (handle !== undefined) { + reference = ['schema', handle] + } const result = { reference, array: Array.isArray(value) } if (result.array) { result.length = value.length diff --git a/src/query-worker-module.mjs b/src/query-worker-module.mjs index 342397e..d7b9043 100644 --- a/src/query-worker-module.mjs +++ b/src/query-worker-module.mjs @@ -22,6 +22,10 @@ export default { async init(task) { dataset?.close() dataset = new FileDataset(task.req.meta) + if (task.req.schema) { + // The cloned meta.schema has lost its JSONTag ids and types. + dataset.parser.meta.schema = JSONTag.parse(task.req.schema) + } if (task.req.access) { const access = await import(task.req.access) dataset.parser.meta.access = access.default diff --git a/src/store-runtime.mjs b/src/store-runtime.mjs index 724f350..6ad4b23 100644 --- a/src/store-runtime.mjs +++ b/src/store-runtime.mjs @@ -175,6 +175,7 @@ export class StoreRuntime { this.mechanisms = mechanisms this.sources = [] this.meta = {} + this.schema = null this.status = new Map() this.commandQueue = [] this.serializeAcceptance = serialWriter() @@ -248,6 +249,7 @@ export class StoreRuntime { initializeLoadedState(inspection, loaded) { this.sources = loaded.sources this.meta = loaded.meta + this.schema = loaded.schema this.status = new Map( inspection.commands.map(command => { return [command.id, command.history.at(-1)] @@ -279,6 +281,7 @@ export class StoreRuntime { req: { sources: this.sources, meta: this.meta, + schema: this.schema, access: this.configuration.access, limits: { memoryLimit: this.configuration.queryMemoryLimit, From 8638f1f67f9a1b6509117be7c016ff61afd98c6d Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:13:57 +0200 Subject: [PATCH 08/17] Test schema handle stability across views and parses Evidence for the maintainer's requirement in SRC-20260926-TTZ7C-61 that schema identity outlives a single query. Rewrap the query runtime note. Co-Authored-By: Claude Opus 5.5 --- docs/query-runtime.md | 8 ++++---- test/query-isolate.mjs | 20 ++++++++++++++++++++ 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/docs/query-runtime.md b/docs/query-runtime.md index 1232c07..530576b 100644 --- a/docs/query-runtime.md +++ b/docs/query-runtime.md @@ -27,10 +27,10 @@ every scanned record in memory. Schema objects may be shared or cyclic, so the host numbers them once per schema and resolves views by that private handle. One stored object is one view within a query: `===` holds across paths, and JSONTag responses link shared objects using the schema file's ids. Each view -reads a property from the host once per query. Tagged scalar values are copied into the guest; -changes to such copies do not modify stored data. Object and array mutations -throw. Committed source updates still serialize with queries, and replacement -workers initialize at the current committed head. +reads a property from the host once per query. Tagged scalar values are copied +into the guest; changes to such copies do not modify stored data. Object and +array mutations throw. Committed source updates still serialize with queries, +and replacement workers initialize at the current committed head. This differs from the initial engine benchmark's raw-record bridge. Keeping the existing parser and grants on the host avoids moving authority into the hostile diff --git a/test/query-isolate.mjs b/test/query-isolate.mjs index 092e99f..3d80a62 100644 --- a/test/query-isolate.mjs +++ b/test/query-isolate.mjs @@ -4,6 +4,7 @@ import path from 'node:path' import test from 'node:test' import JSONTag from '@muze-nl/jsontag' import StoreRuntime from '../src/store-runtime.mjs' +import { QueryView } from '../src/query-view.mjs' import WorkerPool from '../src/workerPool.mjs' import { makeServerFixture } from './durability-helpers.mjs' @@ -307,3 +308,22 @@ test('widely shared schema objects serialize as links within the timeout', assert.equal(result.body, JSONTag.stringify(JSONTag.parse(schema))) assert.ok(performance.now() - start < 1000) }) + +test('schema handles are stable across views and separately parsed schemas', + () => { + const references = schema => { + const view = new QueryView({ parser: { meta: { schema } } }, 1024) + return [ + schema, + schema.types.Node.title, + schema.types.Leaf.title, + schema.contexts.b.root + ].map(value => view.describe(value, []).reference) + } + const schema = JSONTag.parse(graphSchema) + const first = references(schema) + assert.deepEqual(first[0], ['schema', 0]) + assert.deepEqual(first[1], first[2]) + assert.deepEqual(references(schema), first) + assert.deepEqual(references(JSONTag.parse(graphSchema)), first) + }) From 15f600e2994be0103645ed310d5bc8d4bffa4e18 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:15:15 +0200 Subject: [PATCH 09/17] Record schema identity evidence; cycle ready for evaluation EVD-20260926-TTZ7C-64: 205 passing tests, failing baselines, and curriculum-store's schema at 34 ms (JSONTag) / 271 ms (JSON). Record the DEF-20260926-TTZ7C-62 correction and retained discoveries. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 15 ++++++ .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 2 +- .spiral/defects/DEF-20260926-TTZ7C-62.md | 10 ++++ .spiral/defects/DEF-20260926-TTZ7C-62.ttl | 3 +- .spiral/evidence/EVD-20260926-TTZ7C-64.md | 59 ++++++++++++++++++++++ .spiral/evidence/EVD-20260926-TTZ7C-64.ttl | 10 ++++ 6 files changed, 97 insertions(+), 2 deletions(-) create mode 100644 .spiral/evidence/EVD-20260926-TTZ7C-64.md create mode 100644 .spiral/evidence/EVD-20260926-TTZ7C-64.ttl diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index cc9f8ce..17a1620 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -32,3 +32,18 @@ and parse the schema's original JSONTag text. Non-goals: object history, schema editing, changes to data record identity, query limits or grants, curriculum-store's own Spiral repair and unrelated cleanup. + +## Evaluation ready + +The goal can be judged. Schema objects keep identity across the isolate by +stable private handles, JSONTag responses keep the schema file's ids and links, +and guest views read each host property once per query. EVD-20260926-TTZ7C-64 +records 205 passing tests, failing baselines, and curriculum-store's full schema +returning in 34 ms as JSONTag and 271 ms as JSON through the actual runtime, +with record-query benchmarks unchanged within variation. + +Retained for next-cycle planning, not absorbed: command workers still receive +the schema without JSONTag ids; guest JSONTag serialization of query-built +shared objects without ids needs `crypto`, which the isolate lacks; +curriculum-store's own Spiral state needs repair. Human evaluation, acceptance +and integration remain separate decisions. diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl index d611b97..d8d6ee3 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -7,5 +7,5 @@ project:CYC-20260926-TTZ7C-63 a sd:Cycle ; sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; sd:status sd:Active . diff --git a/.spiral/defects/DEF-20260926-TTZ7C-62.md b/.spiral/defects/DEF-20260926-TTZ7C-62.md index 4a68eab..214639c 100644 --- a/.spiral/defects/DEF-20260926-TTZ7C-62.md +++ b/.spiral/defects/DEF-20260926-TTZ7C-62.md @@ -25,3 +25,13 @@ Earliest meaningful cause: the design assumed path uniqueness for schema objects. Verification did not catch it because the query test schema is a tree. The correction belongs in the design's identity rule, followed by the implementation and a regression with a shared/cyclic schema. + +## Correction + +DES-20260924-TTZ7C-59 was corrected at 07c6d65 and d881fe1; implementation +be90dd4 and test 8638f1f realize it. Schema objects are identified by stable +private handles, query workers parse the schema's JSONTag text so ids survive +(the maintainer's decision in SRC-20260926-TTZ7C-61), and guest views cache +host reads per query. EVD-20260926-TTZ7C-64 records the regressions, 205 +passing tests and curriculum-store's schema returning in 34 ms (JSONTag) and +271 ms (JSON). diff --git a/.spiral/defects/DEF-20260926-TTZ7C-62.ttl b/.spiral/defects/DEF-20260926-TTZ7C-62.ttl index 4abd918..f819787 100644 --- a/.spiral/defects/DEF-20260926-TTZ7C-62.ttl +++ b/.spiral/defects/DEF-20260926-TTZ7C-62.ttl @@ -10,4 +10,5 @@ project:DEF-20260926-TTZ7C-62 sd:provenanceConfidence sd:Evidenced ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; sd:observes [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "bd61d9cf612008e96a518fa30fca9e33c40fa82e" ] ; - sd:observes [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "bd61d9cf612008e96a518fa30fca9e33c40fa82e" ] . + sd:observes [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "bd61d9cf612008e96a518fa30fca9e33c40fa82e" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] . diff --git a/.spiral/evidence/EVD-20260926-TTZ7C-64.md b/.spiral/evidence/EVD-20260926-TTZ7C-64.md new file mode 100644 index 0000000..cc8d536 --- /dev/null +++ b/.spiral/evidence/EVD-20260926-TTZ7C-64.md @@ -0,0 +1,59 @@ +# Schema identity across the query isolate + +Verifies the DEF-20260926-TTZ7C-62 correction at 8638f1f (implementation +be90dd4, design d881fe1). Node 24.7.0, isolated-vm 6.2.0, Intel Core i7-1195G7. + +## Regressions + +Three tests in `test/query-isolate.mjs` cover the defect: + +- shared, linked and cyclic schema objects compare `===` across paths; the + guest sees the same own keys as the host, including JSONTag attributes; + a JSONTag response of `meta.schema` equals host `JSONTag.stringify`; a JSON + response of the cyclic schema fails promptly as circular, like host JSON; +- a schema with 26 objects and 2^25 paths returns as JSONTag, equal to host + serialization, within the 1 s default timeout; +- schema handles are equal for shared objects, across views and across + separately parsed copies of the schema, so identity outlives one query. + +Baseline commit 49f3573 recorded the first two failing on the old +implementation (identity false; "Query execution timed out"). Restoring the +pre-fix `src/` at the final tree makes all three fail; the fixed tree passes. + +## Full suite + +The EVD-20260924-TTZ7C-60 command, all 14 test files, passes 205 tests with +no failures or skips (202 there, plus these three). Focused ESLint on the +changed query files passes. `git diff --check` passes for every commit. + +## curriculum-store schema + +Actual runtime path, slow pool, curriculum-store's 92 KB `schema.jsontag` +(281 objects, 734 shared references): + +| Query | Before | After identity | After caching | +|---|---|---|---| +| `meta.schema`, JSONTag | 64 MiB exhausted, 2.4 s | 86 ms | 34 ms, 49,844 bytes | +| `meta.schema`, JSON | 64 MiB exhausted, 2.6 s | 2,806 ms | 271 ms, 4,012,890 bytes | +| `kerndoelen`, JSONTag | 28,227 bytes (expanded) | 6,957 bytes | 11 ms | +| `referentiekader`, JSON | 442 ms | 267 ms | 33 ms | + +Response sizes equal host serialization of the same schema. The JSON response +is large because JSON cannot express sharing; that is not a query-engine cost. +Measurements used a disposable store with the copied schema file. + +## Record queries + +`scripts/benchmark-query-runtime.mjs` medians (ms) compared with +EVD-20260924-TTZ7C-60: 1,000 scan 50.7 (48.7), filter/sort/project 49.8 +(52.9), serialize 84.9 (81.3); 10,000 scan 483.6 (453.3), filter/sort/project +467.6 (497.9), serialize 86.2 (81.0). Differences are within run-to-run +variation; read caching neither helps nor harms single-pass scans. + +## Limits + +Command workers still receive the structured-cloned schema without JSONTag +ids; this cycle changed only query workers. Guest JSONTag serialization of +query-constructed shared objects without ids still needs `crypto`, which the +isolate lacks. That is retained as a discovery, not corrected here. No +performance claim is made beyond these warm-cache measurements. diff --git a/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl b/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl new file mode 100644 index 0000000..6dbf141 --- /dev/null +++ b/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl @@ -0,0 +1,10 @@ +@prefix sd: . +@prefix dcterms: . +@prefix project: . + +project:EVD-20260926-TTZ7C-64 a sd:VerificationEvidence ; + dcterms:identifier "EVD-20260926-TTZ7C-64" ; + sd:repositoryPath ".spiral/evidence/EVD-20260926-TTZ7C-64.md" ; + sd:verifies [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "8638f1f67f9a1b6509117be7c016ff61afd98c6d" ] ; + sd:verifies [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] ; + sd:status sd:Active . From eb3a13c5aff090d5b6fd23528b79fadef03b5332 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 13:29:40 +0200 Subject: [PATCH 10/17] Correct schema identity cycle provenance Causal-chain review before acceptance: - DES-20260924-TTZ7C-59 is constrained by DEF-20260926-TTZ7C-62; sd:changeCausedBy is reserved for implementations. - IMP-20260919-TTZ7C-33's transition also cites SRC-20260926-TTZ7C-61. - EVD-20260926-TTZ7C-64's intermediate timings included schema ids. - Project context records CYC-56 as integrated and CYC-63 as active. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 11 +++++++++-- .spiral/designs/DES-20260924-TTZ7C-59.ttl | 2 +- .spiral/evidence/EVD-20260926-TTZ7C-64.md | 6 ++++-- .spiral/implementations/IMP-20260919-TTZ7C-33.ttl | 1 + .spiral/project-context.md | 9 +++++++-- 5 files changed, 22 insertions(+), 7 deletions(-) diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index 17a1620..dee89a4 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -45,5 +45,12 @@ with record-query benchmarks unchanged within variation. Retained for next-cycle planning, not absorbed: command workers still receive the schema without JSONTag ids; guest JSONTag serialization of query-built shared objects without ids needs `crypto`, which the isolate lacks; -curriculum-store's own Spiral state needs repair. Human evaluation, acceptance -and integration remain separate decisions. +curriculum-store's own Spiral state needs repair; nine older non-implementation +artifacts use `sd:changeCausedBy`, which the ontology reserves for +implementations, and the validators do not flag it. Human evaluation, +acceptance and integration remain separate decisions. + +A full causal-chain review before acceptance corrected this cycle's own use of +that relation on DES-20260924-TTZ7C-59 (now `sd:constrainedBy`), clarified the +EVD-20260926-TTZ7C-64 timing column, added SRC-20260926-TTZ7C-61 as a cause of +the IMP-20260919-TTZ7C-33 transition, and refreshed the stale project context. diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.ttl b/.spiral/designs/DES-20260924-TTZ7C-59.ttl index 29d8d6c..4b34ab6 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.ttl +++ b/.spiral/designs/DES-20260924-TTZ7C-59.ttl @@ -6,6 +6,6 @@ project:DES-20260924-TTZ7C-59 a sd:Design ; sd:repositoryPath ".spiral/designs/DES-20260924-TTZ7C-59.md" ; sd:satisfies [ a sd:ArtifactReference ; sd:artifact project:REQ-20260924-TTZ7C-58 ; sd:gitCommit "fdef01a844e14f50c0af1aac6e89461714736f6d" ] ; sd:status sd:Active ; - sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; + sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "07c6d6590dc2ad9d585df4bb3fe1f2ea2fb1c7ca" ] . diff --git a/.spiral/evidence/EVD-20260926-TTZ7C-64.md b/.spiral/evidence/EVD-20260926-TTZ7C-64.md index cc8d536..1f3b6f1 100644 --- a/.spiral/evidence/EVD-20260926-TTZ7C-64.md +++ b/.spiral/evidence/EVD-20260926-TTZ7C-64.md @@ -31,7 +31,7 @@ changed query files passes. `git diff --check` passes for every commit. Actual runtime path, slow pool, curriculum-store's 92 KB `schema.jsontag` (281 objects, 734 shared references): -| Query | Before | After identity | After caching | +| Query | Before | After identity and schema ids | After caching | |---|---|---|---| | `meta.schema`, JSONTag | 64 MiB exhausted, 2.4 s | 86 ms | 34 ms, 49,844 bytes | | `meta.schema`, JSON | 64 MiB exhausted, 2.6 s | 2,806 ms | 271 ms, 4,012,890 bytes | @@ -40,7 +40,9 @@ Actual runtime path, slow pool, curriculum-store's 92 KB `schema.jsontag` Response sizes equal host serialization of the same schema. The JSON response is large because JSON cannot express sharing; that is not a query-engine cost. -Measurements used a disposable store with the copied schema file. +Measurements used a disposable store with the copied schema file. With stable +handles alone, JSONTag responses failed with "crypto is not defined" until +query workers received the schema's JSONTag text. ## Record queries diff --git a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl index fc156ce..c82669d 100644 --- a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl +++ b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl @@ -36,6 +36,7 @@ project:IMP-20260919-TTZ7C-33 sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] ; sd:transforms [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "bd507554a2fed6f77322ef38a98839f127d810db" ] ; sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; + sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; sd:shapedBy [ a sd:ArtifactReference ; sd:artifact ; sd:gitCommit "8d4b2c738a413abd4cccca740ce958f486e5f7af" ] ; sd:implementationLocation "src/query-isolate.mjs", "src/query-view.mjs", "src/query-guest.mjs", "test/query-isolate.mjs", "scripts/benchmark-query-runtime.mjs", "docs/query-runtime.md" ; sd:implementationChangeKind sd:SemanticChange . diff --git a/.spiral/project-context.md b/.spiral/project-context.md index 3cef4b5..1f56086 100644 --- a/.spiral/project-context.md +++ b/.spiral/project-context.md @@ -28,8 +28,13 @@ clean lint, copied-example verification and actual worker timings (about 453 ms for a 10,000-record scan). The maintainer accepted the cycle on 2026-09-24 at bd8f8eb3ee617c5f9e9198f234725ef7ab5b4201 and requested version 0.11.0. The published od-jsontag 0.5.0 package also passed all 202 tests and copied-example -checks. Acceptance closes this cycle; integration remains pending. No next cycle -is active. +checks. The cycle was integrated into `master` at bd61d9c (pull request #27). + +Active cycle: CYC-20260926-TTZ7C-63, schema identity across the query isolate, +on spiral/CYC-20260926-TTZ7C-63-schema-identity. It corrects +DEF-20260926-TTZ7C-62, where curriculum-store's shared schema exhausted query +limits, as a deliberate deviation from the roadmap; object history remains the +next roadmap target. Queries must have no import, Node filesystem/network/database authority. Preserve existing grants and file-backed ordering. Queryable history follows; UI can wait. From 269a729823d135f48e4570e6e12af1e818bfcaf8 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:06:06 +0200 Subject: [PATCH 11/17] Extend identity cycle to all query data Retain the maintainer's request for object identity across all query data and the decision to derive output ids from record numbers. Shared arrays and the jsontag fallback bug are deferred. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 10 ++++++++++ .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 2 +- .spiral/sources/SRC-20260926-TTZ7C-61.md | 21 +++++++++++++++++++++ .spiral/sources/SRC-20260926-TTZ7C-61.ttl | 2 +- 4 files changed, 33 insertions(+), 2 deletions(-) diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index dee89a4..2196850 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -33,6 +33,16 @@ Non-goals: object history, schema editing, changes to data record identity, query limits or grants, curriculum-store's own Spiral repair and unrelated cleanup. +## Goal extension + +On 2026-09-26, after evaluation-ready evidence for the schema, the maintainer +extended the goal to object identity for all query data (SRC-20260926-TTZ7C-61): +JSONTag responses must link shared objects without an `id`, using ids derived +from record numbers for stored objects and per-response ids for objects a query +builds; tagged scalars keep identity within a query. Shared arrays and the +jsontag fallback bug remain out of scope. The cycle is back in Act; the +evaluation-ready section below describes the earlier schema-only state. + ## Evaluation ready The goal can be judged. Schema objects keep identity across the isolate by diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl index d8d6ee3..2ebdd9f 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -7,5 +7,5 @@ project:CYC-20260926-TTZ7C-63 a sd:Cycle ; sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "eb3a13c5aff090d5b6fd23528b79fadef03b5332" ] ; sd:status sd:Active . diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.md b/.spiral/sources/SRC-20260926-TTZ7C-61.md index e025f8d..173d6c3 100644 --- a/.spiral/sources/SRC-20260926-TTZ7C-61.md +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.md @@ -31,3 +31,24 @@ Node's crypto, which the isolate does not provide. Offered either preserving the schema's ids for query workers or matching 0.10's generated ids, the maintainer chose "Preserve schema ids". Query responses should use the ids from the schema file. Command workers are unchanged by this decision. + +## Identity across all query data + +After the schema correction the maintainer raised: "this fixes meta.schema. +But is the same problem not also in the normal data? That data should also +work with graphs, so object identity should be fixed across the whole system, +not just meta.schema". + +The assistant probed stored data through the runtime. Records keep identity +across paths and cycles, because the record number is a stable identity. Three +gaps remained: JSONTag responses fail for shared objects without an `id` +(stored or query-built), because JSONTag writes a generated id onto the object +using `crypto`; shared arrays are stored inline by od-jsontag and return as +distinct arrays even on the host; tagged scalars are fresh copies on each read +in the isolate, while the host returns one object. + +The maintainer decided: "I think using the record number is probably fine. And +yes, fix 1 and 3. We'll address 2 later, and jsontag also later." Output ids +for shared stored objects without an `id` derive from their record number. +Shared-array identity (an od-jsontag storage-format question) and the jsontag +`createId` fallback bug are deferred. diff --git a/.spiral/sources/SRC-20260926-TTZ7C-61.ttl b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl index 0590dbe..b3365bd 100644 --- a/.spiral/sources/SRC-20260926-TTZ7C-61.ttl +++ b/.spiral/sources/SRC-20260926-TTZ7C-61.ttl @@ -8,4 +8,4 @@ project:SRC-20260926-TTZ7C-61 a sd:Source ; sd:sourceAvailability sd:Retained ; sd:provenanceConfidence sd:Explicit ; sd:status sd:Active ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] . + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] . From 213742efdd64028c6741fa97fc772482dba83800 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:06:46 +0200 Subject: [PATCH 12/17] Design output ids and scalar identity for query data DES-20260924-TTZ7C-59: serialize shared objects without ids using record-derived or per-response output ids, supplied only during serialization, and copy tagged scalars once per query. Co-Authored-By: Claude Opus 5.5 --- .spiral/designs/DES-20260924-TTZ7C-59.md | 16 +++++++++++++++- .spiral/designs/DES-20260924-TTZ7C-59.ttl | 4 ++-- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.md b/.spiral/designs/DES-20260924-TTZ7C-59.md index bdc0711..9e72af3 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.md +++ b/.spiral/designs/DES-20260924-TTZ7C-59.md @@ -37,7 +37,21 @@ Data cannot change during a query: updates are serialized with queries and each query has a fresh isolate. A guest proxy therefore caches its keys and property reads for the query's lifetime, so repeatedly traversing shared objects does not repeat host calls. Grants still apply on the first read. The -cache uses built-ins captured before query code runs. Return scalars, tagged scalar copies and descriptions of +cache uses built-ins captured before query code runs. A tagged scalar is copied +once per view property and query, so repeated reads return the same object, +as they do on the host. Child views are not retained by that cache, so scans +do not pin every visited record. + +JSONTag links a shared object by its `id`, and generates and writes one when +the object has none. Views are read-only and the isolate has no `crypto`. The +serializer therefore first finds, by the same traversal JSONTag uses, the +objects that will be linked but have no `id`. A view receives an output id +derived from its stable identity: `~` for records, as od-jsontag writes +references, or `~schema-`. A query-built object receives +`~query-`, numbered per response. A derived id that is already a stored +`id` gains a numeric suffix. View ids exist only while serializing, through +the attribute read JSONTag performs; query code cannot observe them and the +host is never asked to write. Return scalars, tagged scalar copies and descriptions of object/array views. Expose only ordinary property names and JSONTag's public metadata symbols. Do not forward parser-private symbols, prototypes, methods, raw buffers, source handles or isolated-vm References. Mutating the view throws; diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.ttl b/.spiral/designs/DES-20260924-TTZ7C-59.ttl index 4b34ab6..30161cf 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.ttl +++ b/.spiral/designs/DES-20260924-TTZ7C-59.ttl @@ -7,5 +7,5 @@ project:DES-20260924-TTZ7C-59 a sd:Design ; sd:satisfies [ a sd:ArtifactReference ; sd:artifact project:REQ-20260924-TTZ7C-58 ; sd:gitCommit "fdef01a844e14f50c0af1aac6e89461714736f6d" ] ; sd:status sd:Active ; sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "07c6d6590dc2ad9d585df4bb3fe1f2ea2fb1c7ca" ] . + sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "269a729823d135f48e4570e6e12af1e818bfcaf8" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] . From 832afc19533e730013c43491a23d8cb41bba97f8 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:07:21 +0200 Subject: [PATCH 13/17] Point design lineage at its immediate predecessor DES-20260924-TTZ7C-59 at 213742e revised the version from eb3a13c, not d881fe1. Co-Authored-By: Claude Opus 5.5 --- .spiral/designs/DES-20260924-TTZ7C-59.ttl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.spiral/designs/DES-20260924-TTZ7C-59.ttl b/.spiral/designs/DES-20260924-TTZ7C-59.ttl index 30161cf..3ab4c89 100644 --- a/.spiral/designs/DES-20260924-TTZ7C-59.ttl +++ b/.spiral/designs/DES-20260924-TTZ7C-59.ttl @@ -8,4 +8,4 @@ project:DES-20260924-TTZ7C-59 a sd:Design ; sd:status sd:Active ; sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; sd:constrainedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "269a729823d135f48e4570e6e12af1e818bfcaf8" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] . + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "eb3a13c5aff090d5b6fd23528b79fadef03b5332" ] . From 083542cb0df7c603b81878cc86b36a2e2dc1d1be Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:08:08 +0200 Subject: [PATCH 14/17] Add failing data identity regressions Shared stored and query-built objects without ids must link in JSONTag responses with record-derived or per-response ids, hidden from queries; tagged scalars must keep identity within a query. Both fail on 832afc1 (crypto is not defined; scalar identity false). Co-Authored-By: Claude Opus 5.5 --- test/durability-helpers.mjs | 3 ++- test/query-isolate.mjs | 52 +++++++++++++++++++++++++++++++++++++ 2 files changed, 54 insertions(+), 1 deletion(-) diff --git a/test/durability-helpers.mjs b/test/durability-helpers.mjs index 16c709f..d96d8d6 100644 --- a/test/durability-helpers.mjs +++ b/test/durability-helpers.mjs @@ -43,7 +43,8 @@ export async function makeServerFixture(t, options = {}) { await fs.writeFile(commandStatus, '') await fs.writeFile( datafile, - serialize(JSONTag.parse(options.initialData || '{"persons":[]}')) + serialize(options.initialValue ?? + JSONTag.parse(options.initialData || '{"persons":[]}')) ) await fs.writeFile( commandsFile, diff --git a/test/query-isolate.mjs b/test/query-isolate.mjs index 3d80a62..ec99717 100644 --- a/test/query-isolate.mjs +++ b/test/query-isolate.mjs @@ -327,3 +327,55 @@ test('schema handles are stable across views and separately parsed schemas', assert.deepEqual(references(schema), first) assert.deepEqual(references(JSONTag.parse(graphSchema)), first) }) + +test('shared data without ids links by record in JSONTag responses', + async t => { + const shared = { value: 1 } + const initialValue = { + a: shared, b: shared, list: [shared, shared], alone: { value: 2 }, + named: JSONTag.parse('{"value":3}') + } + initialValue.also = initialValue.named + const fixture = await makeServerFixture(t, { initialValue }) + const runtime = await StoreRuntime.open({ ...fixture, maxWorkers: 1 }) + t.after(() => runtime.close()) + const response = body => { + return runtime.runQuery({ path: '/', body, jsontag: true }) + } + const result = await response('data') + assert.equal(result.code, undefined, result.body) + const data = JSONTag.parse(result.body) + assert.equal(data.a, data.b) + assert.equal(data.list[0], data.a) + assert.equal(data.also, data.named) + const id = JSONTag.getAttribute(data.a, 'id') + assert.match(id, /^~\d+$/) + assert.notEqual(id, '~1') + assert.equal(JSONTag.getAttribute(data.alone, 'id'), undefined) + assert.equal((await response('data')).body, result.body) + const hidden = await runtime.runQuery({ + path: '/', jsontag: false, + body: '[JSONTag.getAttribute(data.a, "id") ?? null]' + }) + assert.deepEqual(JSON.parse(hidden.body), [null]) + const built = await response( + 'const y = {}; ({m: y, n: y, a: data.a, b: data.b})' + ) + assert.equal(built.code, undefined, built.body) + const copy = JSONTag.parse(built.body) + assert.equal(copy.m, copy.n) + assert.equal(copy.a, copy.b) + assert.equal(JSONTag.getAttribute(copy.m, 'id'), '~query-1') + assert.equal(JSONTag.getAttribute(copy.a, 'id'), id) + }) + +test('tagged scalars keep identity within a query only', async t => { + const { query } = await open(t) + assert.equal(await query( + 'data.persons[0].url === data.persons[0].url' + ), true) + assert.equal(await query(`const url = data.persons[0].url + url.extra = 1 + data.persons[0].url.extra`), 1) + assert.equal(await query('data.persons[0].url.extra ?? null'), null) +}) From 92a077373e5a983eab24a64feb475ecb33603850 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:15:26 +0200 Subject: [PATCH 15/17] Link shared query data without ids and keep scalar identity Implement DES-20260924-TTZ7C-59 for SRC-20260926-TTZ7C-61: - before JSONTag serialization, assign output ids to shared objects without an id: ~ for views, supplied only while serializing, and ~query- for query-built objects; avoid stored ids; - copy each tagged scalar once per view property and query. Co-Authored-By: Claude Opus 5.5 --- .../implementations/IMP-20260919-TTZ7C-33.md | 13 +++ .../implementations/IMP-20260919-TTZ7C-33.ttl | 9 +- docs/query-runtime.md | 17 ++- src/query-guest.mjs | 109 +++++++++++++++++- test/query-isolate.mjs | 9 +- 5 files changed, 139 insertions(+), 18 deletions(-) diff --git a/.spiral/implementations/IMP-20260919-TTZ7C-33.md b/.spiral/implementations/IMP-20260919-TTZ7C-33.md index 1018d0a..45c3041 100644 --- a/.spiral/implementations/IMP-20260919-TTZ7C-33.md +++ b/.spiral/implementations/IMP-20260919-TTZ7C-33.md @@ -251,3 +251,16 @@ Command workers still receive the cloned schema. Guest proxies cache host descriptions of keys, reads and existence checks for the query's lifetime using built-ins captured before query code runs; tagged scalars are still copied on every read. + +## Identity for all query data + +Extends the correction per the maintainer's request in SRC-20260926-TTZ7C-61. +Before JSONTag serialization, the guest repeats JSONTag's reference traversal +and assigns output ids to objects that will be linked but have no `id`: views +receive `~` (or `~schema-`) only through the attribute read +while serializing, and query-built objects receive `~query-` as an ordinary +attribute. Candidates that match a stored id or an id already assigned gain a +numeric suffix. Views record their reference in a private WeakMap. Tagged +scalar copies are cached per view property for the query's lifetime; child +views are not retained. Test fixtures accept an `initialValue` object so shared +objects without ids can be stored. diff --git a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl index c82669d..e252c36 100644 --- a/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl +++ b/.spiral/implementations/IMP-20260919-TTZ7C-33.ttl @@ -33,10 +33,9 @@ project:IMP-20260919-TTZ7C-33 sd:implementationLocation "scripts/benchmark-file-data.mjs" ; sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260923-TTZ7C-47 ; sd:gitCommit "83edb03bc406b9e651ef93bed69b87ece17dcf47" ] ; sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260923-TTZ7C-54 ; sd:gitCommit "c8a3fd9464abfd5e7b699a2752789d82af4da2da" ] ; - sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "bd507554a2fed6f77322ef38a98839f127d810db" ] ; - sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "a2a7aaf207d666cd80a6d662a2e254414f1a1c02" ] ; + sd:implements [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "832afc19533e730013c43491a23d8cb41bba97f8" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "eb3a13c5aff090d5b6fd23528b79fadef03b5332" ] ; + sd:changeCausedBy [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "269a729823d135f48e4570e6e12af1e818bfcaf8" ] ; sd:shapedBy [ a sd:ArtifactReference ; sd:artifact ; sd:gitCommit "8d4b2c738a413abd4cccca740ce958f486e5f7af" ] ; - sd:implementationLocation "src/query-isolate.mjs", "src/query-view.mjs", "src/query-guest.mjs", "test/query-isolate.mjs", "scripts/benchmark-query-runtime.mjs", "docs/query-runtime.md" ; + sd:implementationLocation "src/query-isolate.mjs", "src/query-view.mjs", "src/query-guest.mjs", "test/query-isolate.mjs", "test/durability-helpers.mjs", "scripts/benchmark-query-runtime.mjs", "docs/query-runtime.md" ; sd:implementationChangeKind sd:SemanticChange . diff --git a/docs/query-runtime.md b/docs/query-runtime.md index 530576b..4240689 100644 --- a/docs/query-runtime.md +++ b/docs/query-runtime.md @@ -25,11 +25,18 @@ imports and closures. Record numbers and property paths let the host resolve views without pinning every scanned record in memory. Schema objects may be shared or cyclic, so the host numbers them once per schema and resolves views by that private handle. -One stored object is one view within a query: `===` holds across paths, and -JSONTag responses link shared objects using the schema file's ids. Each view -reads a property from the host once per query. Tagged scalar values are copied -into the guest; changes to such copies do not modify stored data. Object and -array mutations throw. Committed source updates still serialize with queries, +One stored object is one view within a query: `===` holds across paths and +cycles. Each view reads a property from the host once per query. Tagged scalar +values are copied into the guest once per property and query, so repeated reads +return the same copy; changes to a copy do not modify stored data or reach the +next query. Object and array mutations throw. + +JSONTag responses link an object reached more than once by its `id`. A shared +object without one gets an output id: `~` for stored records, which +stays the same across queries and updates, or `~query-` for objects the +query built, numbered per response. A numeric suffix avoids a stored `id`. +Queries never see output ids. od-jsontag stores arrays inline, so an array +shared between records is returned as separate arrays. Committed source updates still serialize with queries, and replacement workers initialize at the current committed head. This differs from the initial engine benchmark's raw-record bridge. Keeping the diff --git a/src/query-guest.mjs b/src/query-guest.mjs index f7576a9..5995a77 100644 --- a/src/query-guest.mjs +++ b/src/query-guest.mjs @@ -24,6 +24,18 @@ export function prepare(read, validURL, initial) { const cacheGet = Function.prototype.call.bind(Map.prototype.get) const cacheSet = Function.prototype.call.bind(Map.prototype.set) const dereference = Function.prototype.call.bind(WeakRef.prototype.deref) + const Weak = WeakMap + const weakHas = Function.prototype.call.bind(WeakMap.prototype.has) + const weakGet = Function.prototype.call.bind(WeakMap.prototype.get) + const weakSet = Function.prototype.call.bind(WeakMap.prototype.set) + const assign = Object.assign + const isArray = Array.isArray + const getAttribute = JSONTag.getAttribute + const setAttribute = JSONTag.setAttribute + const attributesSymbol = symbols[1] + // Views and their references, and output ids assigned while serializing. + const references = new Weak() + const outputIds = new Weak() const makeWeak = value => new WeakReference(value) const metadataKey = key => symbols.indexOf(key) const immutable = () => { @@ -68,23 +80,56 @@ export function prepare(read, validURL, initial) { } // Data cannot change during a query, so each host answer is read once. // Shared objects are then traversed without repeating host calls. - // Descriptions are cached, not values: tagged copies stay fresh. + // A tagged scalar is copied once, so repeated reads return the same + // object; child views are not kept here, so scans do not pin them. const reads = new Cache() const exists = new Cache() + let copies let keys const readOnce = (answers, operation, encoded) => { if (!cacheHas(answers, encoded)) { - cacheSet(answers, encoded, read(operation, reference, encoded)) + const answer = read(operation, reference, encoded) + if (answer?.entry) { + // Inherited properties must not change what an entry is. + setPrototype(answer.entry, null) + } + cacheSet(answers, encoded, answer) } return cacheGet(answers, encoded) } + const entryValue = (encoded, entry) => { + if (!('tagged' in entry)) { + return wrap(entry) + } + if (!copies) { + copies = new Cache() + } + if (!cacheHas(copies, encoded)) { + cacheSet(copies, encoded, wrap(entry)) + } + return cacheGet(copies, encoded) + } + // While serializing, a linked view without an id reports its output + // id through the attribute read JSONTag performs. + const outputAttributes = encoded => { + const attributes = {} + const result = readOnce(reads, 'get', encoded) + if (result.present) { + assign(attributes, entryValue(encoded, result.entry)) + } + attributes.id = weakGet(outputIds, proxy) + return attributes + } const proxy = new Proxy(target, { get(target, key, receiver) { const encoded = remoteKey(key) + if (key === attributesSymbol && weakHas(outputIds, proxy)) { + return outputAttributes(encoded) + } if (encoded !== -1) { const result = readOnce(reads, 'get', encoded) if (result.present) { - return wrap(result.entry) + return entryValue(encoded, result.entry) } } return Reflect.get(target, key, receiver) @@ -123,7 +168,7 @@ export function prepare(read, validURL, initial) { return undefined } return { - value: wrap(result.entry), + value: entryValue(encoded, result.entry), enumerable: result.enumerable, configurable: true, writable: false @@ -136,9 +181,64 @@ export function prepare(read, validURL, initial) { preventExtensions: immutable }) setCached(key, makeWeak(proxy)) + weakSet(references, proxy, reference) return proxy } + // JSONTag links an object reached twice by its id, and writes a generated + // id onto it when it has none. Views are read-only and the isolate has no + // crypto, so assign those ids first, following JSONTag's own traversal. + function assignOutputIds(value) { + const seen = new Weak() + const assigned = new Cache() + let generated = 0 + const unused = candidate => { + let id = candidate + let suffix = 0 + while (cacheHas(assigned, id) || read('hasId', null, id)) { + suffix++ + id = candidate + '-' + suffix + } + cacheSet(assigned, id, true) + return id + } + const identify = value => { + const reference = weakGet(references, value) + if (reference && typeof reference[0] === 'number' && + reference.length === 1) { + weakSet(outputIds, value, unused('~' + reference[0])) + } + else if (reference && reference[0] === 'schema') { + weakSet(outputIds, value, unused('~schema-' + reference[1])) + } + else { + generated++ + setAttribute(value, 'id', unused('~query-' + generated)) + } + } + const visit = value => { + if (isArray(value)) { + for (const item of value) { + visit(item) + } + } + else if (value && typeof value === 'object') { + if (weakHas(seen, value)) { + if (!getAttribute(value, 'id')) { + identify(value) + } + } + else { + weakSet(seen, value, true) + for (const key in value) { + visit(value[key]) + } + } + } + } + visit(value) + } + const root = wrap(initial.root) const meta = { index: { @@ -207,6 +307,7 @@ export function prepare(read, validURL, initial) { let body if (taggedResponse) { parseAllObjects(result) + assignOutputIds(result) body = stringifyTagged(result) } else { diff --git a/test/query-isolate.mjs b/test/query-isolate.mjs index ec99717..e9e2329 100644 --- a/test/query-isolate.mjs +++ b/test/query-isolate.mjs @@ -336,9 +336,10 @@ test('shared data without ids links by record in JSONTag responses', named: JSONTag.parse('{"value":3}') } initialValue.also = initialValue.named + let runtime + t.after(() => runtime?.close()) const fixture = await makeServerFixture(t, { initialValue }) - const runtime = await StoreRuntime.open({ ...fixture, maxWorkers: 1 }) - t.after(() => runtime.close()) + runtime = await StoreRuntime.open({ ...fixture, maxWorkers: 1 }) const response = body => { return runtime.runQuery({ path: '/', body, jsontag: true }) } @@ -348,9 +349,9 @@ test('shared data without ids links by record in JSONTag responses', assert.equal(data.a, data.b) assert.equal(data.list[0], data.a) assert.equal(data.also, data.named) + // The shared object is record 1; the stored id "~1" forces a suffix. const id = JSONTag.getAttribute(data.a, 'id') - assert.match(id, /^~\d+$/) - assert.notEqual(id, '~1') + assert.equal(id, '~1-1') assert.equal(JSONTag.getAttribute(data.alone, 'id'), undefined) assert.equal((await response('data')).body, result.body) const hidden = await runtime.runQuery({ From fa6c791991381beb5fac40103dbc9dba15233ee8 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:16:17 +0200 Subject: [PATCH 16/17] Record data identity evidence; extended cycle ready for evaluation EVD-20260926-TTZ7C-64: regressions, 207 passing tests, stable output ids across updates, and about 4% added cost on large scans. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 10 ++++++ .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 2 +- .spiral/evidence/EVD-20260926-TTZ7C-64.md | 38 ++++++++++++++++++++++ .spiral/evidence/EVD-20260926-TTZ7C-64.ttl | 3 ++ 4 files changed, 52 insertions(+), 1 deletion(-) diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index 2196850..f36d586 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -64,3 +64,13 @@ A full causal-chain review before acceptance corrected this cycle's own use of that relation on DES-20260924-TTZ7C-59 (now `sd:constrainedBy`), clarified the EVD-20260926-TTZ7C-64 timing column, added SRC-20260926-TTZ7C-61 as a cause of the IMP-20260919-TTZ7C-33 transition, and refreshed the stale project context. + +## Evaluation ready (extended goal) + +The extended goal can be judged. Stored records keep identity across paths and +cycles; JSONTag responses link shared objects without an `id` as `~`, +stable across queries and updates, or `~query-` for query-built objects, +without exposing those ids to queries; tagged scalars keep identity within a +query. EVD-20260926-TTZ7C-64 records the regressions, 207 passing tests and +about 4% added cost on large scans. Shared-array identity and the jsontag +fallback bug remain deferred, with the earlier retained discoveries. diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl index 2ebdd9f..b528928 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -7,5 +7,5 @@ project:CYC-20260926-TTZ7C-63 a sd:Cycle ; sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "eb3a13c5aff090d5b6fd23528b79fadef03b5332" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "269a729823d135f48e4570e6e12af1e818bfcaf8" ] ; sd:status sd:Active . diff --git a/.spiral/evidence/EVD-20260926-TTZ7C-64.md b/.spiral/evidence/EVD-20260926-TTZ7C-64.md index 1f3b6f1..d7d0f98 100644 --- a/.spiral/evidence/EVD-20260926-TTZ7C-64.md +++ b/.spiral/evidence/EVD-20260926-TTZ7C-64.md @@ -59,3 +59,41 @@ ids; this cycle changed only query workers. Guest JSONTag serialization of query-constructed shared objects without ids still needs `crypto`, which the isolate lacks. That is retained as a discovery, not corrected here. No performance claim is made beyond these warm-cache measurements. + +## Identity for all query data + +Verifies the extension at 92a0773 (design 832afc1). + +A runtime probe of stored data before the extension showed records already +keep identity across paths and cycles, and a 26-record DAG with 2^25 paths and +ids returns as JSONTag in about 110 ms, equal to host serialization. It also +showed the gaps now corrected: shared objects without an `id` failed as JSONTag +("crypto is not defined"; with a query-supplied crypto, "dataspace is +immutable"), and tagged scalars were distinct copies per read. + +Two regressions cover the correction and failed at baseline 083542c: + +- shared stored objects without ids link as `~` in JSONTag responses; + a record whose derived id matches a stored `id` (`~1`) receives `~1-1`; + unshared objects receive no id; repeated responses are identical; queries + cannot read output ids; query-built shared objects receive `~query-`, + and mixed query-built/stored results link both; +- tagged scalars are identical within a query, and a change to the copy does + not reach the next query. + +A command probe showed output ids stable across updates: a shared record kept +`~1` through two modifying commands, and an object shared by a command received +`~2` and stayed linked. The 2^25-path DAG without ids now returns in 7 ms. +Shared arrays still return as separate arrays, as od-jsontag stores them +inline; that is deferred by the maintainer. + +The full 14-file suite passes 207 tests. One earlier run failed the +timing-based "hanging command times out unsafe" command test; it passed three +isolated reruns and the next full run, and exercises no query code. + +Benchmark, three interleaved rounds on the same machine against 083542c +(medians, ms): 10,000 scan 510 → 534, filter/sort/project 495 → 514, +serialize 89 → 92; 1,000-record results are within variation. The machine ran +slower than in the earlier measurements above. Tracking each view's identity +costs about 4% on large scans. curriculum-store's schema remains 38 ms as +JSONTag and 302 ms as JSON. diff --git a/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl b/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl index 6dbf141..e847228 100644 --- a/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl +++ b/.spiral/evidence/EVD-20260926-TTZ7C-64.ttl @@ -7,4 +7,7 @@ project:EVD-20260926-TTZ7C-64 a sd:VerificationEvidence ; sd:repositoryPath ".spiral/evidence/EVD-20260926-TTZ7C-64.md" ; sd:verifies [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "8638f1f67f9a1b6509117be7c016ff61afd98c6d" ] ; sd:verifies [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "d881fe1e1e81450182b64d4e35ebf4d4bb648860" ] ; + sd:verifies [ a sd:ArtifactReference ; sd:artifact project:IMP-20260919-TTZ7C-33 ; sd:gitCommit "92a077373e5a983eab24a64feb475ecb33603850" ] ; + sd:verifies [ a sd:ArtifactReference ; sd:artifact project:DES-20260924-TTZ7C-59 ; sd:gitCommit "832afc19533e730013c43491a23d8cb41bba97f8" ] ; + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:EVD-20260926-TTZ7C-64 ; sd:gitCommit "eb3a13c5aff090d5b6fd23528b79fadef03b5332" ] ; sd:status sd:Active . From 2158aac6a8f2438f8e9555ad80446d7189b9fd42 Mon Sep 17 00:00:00 2001 From: Auke van Slooten Date: Sat, 26 Sep 2026 14:20:47 +0200 Subject: [PATCH 17/17] Accept object identity cycle Record the maintainer's acceptance of CYC-20260926-TTZ7C-63 at fa6c791 and update the project context. Integration remains a separate decision. Co-Authored-By: Claude Opus 5.5 --- .spiral/cycles/CYC-20260926-TTZ7C-63.md | 17 ++++++++++++- .spiral/cycles/CYC-20260926-TTZ7C-63.ttl | 4 +-- .spiral/project-context.md | 31 +++++++++++------------- 3 files changed, 32 insertions(+), 20 deletions(-) diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.md b/.spiral/cycles/CYC-20260926-TTZ7C-63.md index f36d586..386861c 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.md +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.md @@ -4,7 +4,7 @@ Goal: restore object identity for non-record objects across the query isolate boundary, so `meta.schema` queries work and shared schema objects keep their JSONTag links. Correct DEF-20260926-TTZ7C-62. -Status: Active. Human commitment: the maintainer confirmed the understanding +Status: Accepted on 2026-09-26. Human commitment: the maintainer confirmed the understanding and defect cycle on 2026-09-26 and set the identity direction retained in SRC-20260926-TTZ7C-61: each object gets a stable identifier that outlives a single query and is not exposed to query code. @@ -74,3 +74,18 @@ without exposing those ids to queries; tagged scalars keep identity within a query. EVD-20260926-TTZ7C-64 records the regressions, 207 passing tests and about 4% added cost on large scans. Shared-array identity and the jsontag fallback bug remain deferred, with the earlier retained discoveries. + +## Human acceptance + +On 2026-09-26, after reviewing the extended evaluation, the maintainer +instructed: "Accepted." + +This accepts the cycle at fa6c791991381beb5fac40103dbc9dba15233ee8: stable +schema handles, schema JSONTag text for query workers, per-query read caching, +record-derived and per-response output ids, and per-query tagged scalar +identity, with the evidence and limits in EVD-20260926-TTZ7C-64. Deferred: +shared-array identity, the jsontag `createId` fallback bug, schema ids in +command workers, the nine older `sd:changeCausedBy` uses and curriculum-store's +Spiral repair. No version change was requested. Acceptance authorizes proposing +integration; merging remains a separate decision. Branch identity was verified +before recording acceptance. diff --git a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl index b528928..d39f5b9 100644 --- a/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl +++ b/.spiral/cycles/CYC-20260926-TTZ7C-63.ttl @@ -7,5 +7,5 @@ project:CYC-20260926-TTZ7C-63 a sd:Cycle ; sd:repositoryPath ".spiral/cycles/CYC-20260926-TTZ7C-63.md" ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:SRC-20260926-TTZ7C-61 ; sd:gitCommit "bcf64e82a3a4290a1079d1029ec6bd86c67734bb" ] ; sd:derivedFrom [ a sd:ArtifactReference ; sd:artifact project:DEF-20260926-TTZ7C-62 ; sd:gitCommit "d34a97f400e5708d0a7aa23d4f601f3ab51ae7d6" ] ; - sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "269a729823d135f48e4570e6e12af1e818bfcaf8" ] ; - sd:status sd:Active . + sd:transforms [ a sd:ArtifactReference ; sd:artifact project:CYC-20260926-TTZ7C-63 ; sd:gitCommit "fa6c791991381beb5fac40103dbc9dba15233ee8" ] ; + sd:status sd:Accepted . diff --git a/.spiral/project-context.md b/.spiral/project-context.md index 1f56086..7c5deca 100644 --- a/.spiral/project-context.md +++ b/.spiral/project-context.md @@ -18,23 +18,20 @@ Project causal-graph namespace: `https://github.com/simplyedit/simplystore/spira Spiral core source: `.spiral-core/`, git submodule for `https://github.com/muze-labs/spiral-developer.git`, currently checked out at `8d4b2c738a413abd4cccca740ce958f486e5f7af`. -Latest accepted cycle: CYC-20260924-TTZ7C-56, isolated-vm query execution, on -spiral/CYC-20260924-TTZ7C-56-isolated-vm. The maintainer selected isolated-vm after -capability and performance probes and explicitly authorized continuation. -SRC-20260924-TTZ7C-57 retains the commitment. This continues ROADMAP section 2. -The branch implementation now uses isolated-vm with host-authorized read-only -views and fresh per-query isolates. EVD-20260924-TTZ7C-60 records 202 passing tests, -clean lint, copied-example verification and actual worker timings (about 453 ms -for a 10,000-record scan). The maintainer accepted the cycle on 2026-09-24 at -bd8f8eb3ee617c5f9e9198f234725ef7ab5b4201 and requested version 0.11.0. -The published od-jsontag 0.5.0 package also passed all 202 tests and copied-example -checks. The cycle was integrated into `master` at bd61d9c (pull request #27). - -Active cycle: CYC-20260926-TTZ7C-63, schema identity across the query isolate, -on spiral/CYC-20260926-TTZ7C-63-schema-identity. It corrects -DEF-20260926-TTZ7C-62, where curriculum-store's shared schema exhausted query -limits, as a deliberate deviation from the roadmap; object history remains the -next roadmap target. +Latest accepted cycle: CYC-20260926-TTZ7C-63, object identity across the query +isolate, on spiral/CYC-20260926-TTZ7C-63-schema-identity, accepted on 2026-09-26 +at fa6c791. It corrects DEF-20260926-TTZ7C-62 (curriculum-store's shared schema +exhausted query limits) and, at the maintainer's request, identity for all query +data: stable private schema handles, schema ids in query workers, record-derived +output ids for shared objects without an `id`, and per-query tagged scalar +identity. EVD-20260926-TTZ7C-64 records 207 passing tests and about 4% added +cost on large scans. It was a deliberate deviation from the roadmap; object +history remains the next roadmap target. Integration is pending; no cycle is +active. + +The preceding cycle, CYC-20260924-TTZ7C-56 (isolated-vm query execution, SimplyStore +0.11.0), is accepted and integrated at bd61d9c (pull request #27). + Queries must have no import, Node filesystem/network/database authority. Preserve existing grants and file-backed ordering. Queryable history follows; UI can wait.