feat(sparql): burn-down Wave 3 — within-ledger FROM (D-3), UPDATE graph-management (D-5/6), equality/EBV/promotion (D-12, #1447), USING+GRAPH over-delete (#1441), triple-term syntax (−145 tests)#1462
Open
aaj3f wants to merge 26 commits into
Conversation
SPARQL CONSTRUCT templates containing blank nodes ([ ], _:a, or an RDF collection ( )) evaluated to an empty @graph: template blank nodes lower to variables the WHERE clause never binds, so instantiate_row dropped every template triple at the (Some,Some,Some) guard. Record the template blank-node variables (those whose registry name keeps the `_:` prefix the term lowerer assigns) on ConstructTemplate.bnode_vars at lowering, and mint a fresh blank node per solution row in the CONSTRUCT formatter -- shared across a row's template triples, distinct across rows. constructlist rides the same path via PR-1's collection desugaring. Greens W3C construct-3, construct-4 (data-r2) and constructlist (data-sparql11); their skip-register entries are removed suite-driven.
Add SPARQL CONSTRUCT regression tests proving a blank-node template mints a fresh blank per solution -- one node per match, linking subject / predicate / object within a row -- for both the anonymous [ ] (construct-3 shape) and labeled _:a (construct-4 shape) forms.
…nting
Adversarial review found the per-solution CONSTRUCT blank labels were minted
as b{n}, which overlaps other blank-node producers' label spaces (BNODE() emits
_:b{hex}; SERVICE passes through _:b0). Safe today only because every real
Fluree producer uses the reserved fdb- prefix and external SERVICE is rejected,
but a mixed template could otherwise silently merge a minted blank with a data
blank — and the isomorphism-based W3C CONSTRUCT suite cannot catch that. Mint
into a reserved cst{n} prefix so the output is provably disjoint.
Recognize TRIPLE/SUBJECT/PREDICATE/OBJECT/isTRIPLE as arity-validated function-call builtins that lower to not_implemented (burn-down D-1). They are contextual function-call identifiers, not reserved keywords: the lexer emits a single generic TripleTermFn token (carrying the canonical name) and the expression parser resolves it via FunctionName::parse and validates arity (TRIPLE/3, the rest/1). Greens the 3 bucket-B entries of SPARQL12_SYNTAX_TRIPLE_TERMS_POSITIVE (expr-tripleterm-03/04/05).
Accept bare `<<( s p o )>>` triple-term values in the subject, object, VALUES, and BIND positions the RDF 1.2 grammar allows, lowering them to not_implemented (burn-down D-1). A shared parse_triple_term_value enforces the grammar guardrails so the (empty) negative register stays rejected: no path/collection/reified-triple inside a term; a value-context term (VALUES/BIND) may not have a blank-node or nested-triple-term subject, while a pattern-context term may; a bare `<<( )>> .` is not a statement. BIND `<<( )>>` reuses the TRIPLE(...) function path. Greens the 24 bucket-C entries of SPARQL12_SYNTAX_TRIPLE_TERMS_POSITIVE (register now empty). Two pr-w2a unit tests that asserted the pre-D-1 parse rejection are updated to the accept-then-defer behavior; the rdf:reifies reifier path keeps rejecting nested triple-term subjects.
A single-child group graph pattern was always unwrapped to its bare child
(parse_group_graph_pattern), erasing the `{ }` scope boundary. When the child is
a FILTER, BIND, or nested Group that boundary is load-bearing: the lowerer only
wraps *Group* children as uncorrelated subqueries, so an unwrapped
`{ FILTER(?v) }` or `OPTIONAL { { ... FILTER(?title) } }` lands in the enclosing
scope and the FILTER wrongly sees an enclosing-scope variable.
Keep a single-child group wrapped when the child is scope-sensitive
(GraphPattern::is_scope_sensitive: Filter/Bind/Group); the existing lowerer path
then routes it through an uncorrelated subquery evaluated in an independent
scope. Relax the lowerer's nested-Group invariant assertion accordingly.
Greens W3C filter-nested-2 and dawg-optional-filter-005-not-simplified. Adds
JSON-LD parity regression tests (it_query_filter_scope.rs) for the shared
filter-scope semantics guarded only on the SPARQL surface by the W3C submodule.
A correlation variable a sub-SELECT binds only via OPTIONAL/UNION is not self-produced, so subquery.rs excluded it from the hash join key and the merge kept the parent's value while silently dropping the subquery's conflicting binding — W3C var-scope-join-1 returned rows where the natural join on that variable must eliminate them. Partition correlation_vars into join_keys (self-produced, unchanged hash path) and reconcile_vars (the rest). Reconcile vars are (a) excluded from the per-row seed so the subquery binds them independently instead of pinning them to the parent value, and (b) checked at the merge: a row whose reconcile var is bound on both sides to different terms is dropped (SPARQL 18.4 compatible-mapping join). self_produced_vars and the join_key hash path are untouched, so BSBM-shape correlated subqueries (self-produced correlation) stay byte-identical. Greens W3C join-scope-1. The seed exclusion also brings the JSON-LD surface's correlated sub-SELECT into parity: the SPARQL nested group lowers to an uncorrelated subquery (join-mode), where the merge reconcile suffices; JSON-LD `["query", ...]` is correlated (per-row seeded), where the merge check alone is a no-op without the seed change. Adds the JSON-LD parity test for the join-scope shape.
…he update harness The UpdateEvaluationTest guard rejected any mf:result that parsed to no ut:data/ut:graphData/ut:result, on the assumption that a deliberate empty-store expectation always carries ut:result ut:success. That is false: the W3C graph-management tests (DROP ALL, the update-silent/* cases) express "expected: empty store" as a bare mf:result [] . Distinguish a present-but- empty result node (legitimate empty expectation) from an entirely absent mf:result (a real manifest mis-parse) via a new Test.result_present flag, and fire the guard only on the latter.
…P/CREATE/COPY/MOVE/ADD)
Add grammar/AST/parse + execution for the SPARQL 1.1 Update graph-management
operations, plus SILENT/INTO. The verbs were already lexed; this is AST +
recursive-descent parse (parse/query/{mod,update}.rs, ast/update.rs) lowering
to a new Txn graph-management directive (transact/ir.rs) executed by a
whole-graph scan primitive at staging time (transact/stage.rs::stage_graph_mgmt):
- CLEAR/DROP GRAPH <g> | DEFAULT scan the g_id and emit retractions; CLEAR/DROP
ALL|NAMED iterate GraphRegistry::iter_entries. DROP is semantically CLEAR
(D-6): the registry is additive-only and an emptied graph is
harness-indistinguishable from a dropped one.
- COPY/MOVE/ADD scan the source graph and re-home its flakes into the
destination by rewriting only the flake's g (datatype/lang/list-index ride
along), composed over the CLEAR primitive (COPY/MOVE clear the destination
first, MOVE clears the source afterward). Set-difference against the
destination avoids assert+retract of the same fact.
- CREATE and SILENT LOAD lower to an empty no-op transaction (Fluree cannot
represent an empty named graph, D-6). Non-SILENT remote LOAD is a documented
divergence (D-5): the embedded transact path has no HTTP client. No W3C eval
test requires a real fetch.
The dispatch is a single Option check at the top of stage(); the ordinary
insert/upsert/update path and pr-u2's multi-op staging stay byte-identical.
Query-surface parity (compliance case 2): expose the new capability on the
non-SPARQL transact surface via Txn::clear_graph/drop_graph/copy_graph and
GraphTransactBuilder::clear_graph/drop_graph/copy_graph, with a parity test in
it_named_graphs.rs.
Also correct pr-u2's cross-operation blank-node scope check: it over-collected
from Modify (INSERT/DELETE ... WHERE) templates, wrongly rejecting the approved
positive tests insert-where-same-bnode/-2 (template bnodes are per-operation,
per-solution; reuse across operations yields distinct nodes). The scope rule
applies to ground DATA forms only (INSERT DATA / DELETE DATA), keeping the
syntax-update-54 negative test rejected.
Register: remove the 68 SPARQL11_UPDATE + 23 double-registered
SPARQL11_SYNTAX_UPDATE_1 graph-management entries (91 lines) that green.
Empty-named-graph model (D-6 second half) resolved as documented divergence:
bindings#graph and agg-empty-group-count-graph stay registered with a
divergence comment (Fluree models a graph as existing iff a flake carries it).
`v.as_array().map(|a| a.len())` trips clippy::redundant_closure_for_method_calls, which is deny at the workspace level, so `cargo clippy --all-targets` failed to compile the grp_graphsource test target. `as_array()` yields `Option<&Vec<Value>>`, so neither `<[_]>::len` (expects &[_]; E0631) nor `Vec::len` (no inherent len) works as a method reference — use an explicit match instead.
A SPARQL FROM / FROM NAMED clause naming graphs within the queried ledger (the ledger alias -> the default graph; registered named-graph IRIs -> named graphs) now builds a within-ledger DataSet over the one snapshot and runs through the shared dataset execution path, reusing DatasetOperator, policy and reasoning. Cross-ledger clause IRIs are rejected (use the connection path). Greens the 12 dawg-dataset tests + constructwhere04; the harness pre-loads clause-referenced files as named graphs. The delegation created a mutual async recursion between query_with_options and query_dataset_with_options (whose single-ledger fast path delegates back), which made the fluree-db-api type-check pathologically slow (30+ min) and defeated Send inference. Broken by giving query_dataset_with_options a concrete boxed return (-> Pin<Box<dyn Future + Send>>), which resolves the opaque-recursive auto-trait cycle. R2RML + within-ledger FROM is a documented unsupported combination (the R2RML dataset path is non-Send and runs from Send-required spawn contexts) and is rejected rather than silently mis-executed. Actions decision D-3 (within-ledger datasets, Option A engine construction).
`DELETE {..} USING <g> WHERE { GRAPH <h> {..} }` over-deleted: the explicit
`GRAPH <h>` block reached graph h even though `USING <g>` (no `USING NAMED <h>`)
scopes the WHERE dataset to default=g with an empty named-graph set. Per SPARQL
1.1 §3.1.3 the GRAPH block must then match nothing ("the GRAPH clause does not
override the USING clause").
`stream_where_into_accumulator` treated an empty `USING NAMED` set as "no
restriction" and registered every named graph in the runtime dataset, so the
`GRAPH <h>` probe found h and generated retractions against the default-graph
DELETE template. Restrict the WHERE-visible named graphs to exactly the
`USING NAMED` set whenever any `USING`/`USING NAMED` clause is present; keep the
ambient all-named-graphs dataset only when no USING clause is present at all, so
a plain `DELETE WHERE { GRAPH <g> {..} }` stays byte-identical.
Greens W3C dawg-delete-using-02a/06a and adds a JSON-LD `fromNamed` graph-scoped
delete-where parity test.
…egates Predicate-grouped COUNT fast paths (StatsCountByPredicateOperator, PredicateObjectCountFirsts) tagged the count xsd:long, but SPARQL COUNT is xsd:integer (§18.5.1.6) — matching the materialized and streaming finalize paths. Greens agg02. SUM/AVG silently skipped bound non-numeric group members and averaged over the numeric subset; per §18.5 a non-numeric member is a type error that unbinds the whole aggregate. Poison both the materialized (aggregate.rs) and streaming (group_aggregate.rs) paths; the streaming numeric hot path is kept first so it stays byte-identical. Greens agg-err-01.
…lang match CONCAT silently skipped a non-string (or unbound) argument, coercing e.g. an xsd:integer away; per §17.4.3.5 a non-string argument is a type error, so the whole result is unbound (concat02). Language tags are case-insensitive (RDF 1.1 §3.3), and a language-tagged literal's datatype is rdf:langString even though the SPARQL results format writes only xml:lang. Normalize both in the harness result comparison so a produced en-US / no-datatype literal matches an expected en-us / rdf:langString fixture (strlang03-rdf11, dawg-langMatches-1/2/3/basic).
Numeric promotion collapsed xsd:float into xsd:double and produced xsd:decimal
for double+decimal. Add a first-class ComparableValue::Float(f32) so the XPath
lattice integer<decimal<float<double holds: float o {integer,decimal,float} =
float, float o double = double, double o decimal = double (op:numeric-*).
xsd:float is stored as an f64 tagged only by its datatype (coerce.rs), so
eval_to_comparable recovers the float type with a single datatype check on the
xsd:double literal path; the Long/String fast paths stay byte-identical.
Greens type-promotion-03/04/05/21/29/30 and expr-ops
{add,subtract,multiply,divide}-numbers-cast, unplus-2, unminus-2.
FILTER(?v) routed a bare variable through a bool-returning EBV where every literal except xsd:boolean false was truthy and unbound was false. Per §17.2.2 the EBV of numeric-zero / empty-string is false, and a language-tagged or foreign-datatype literal, an IRI, an ill-typed literal, or an unbound variable is a type error (a demotable Comparison error, so a FILTER excludes the row and a BIND leaves the variable unbound). Route the bare-variable path through a fallible binding_effective_bool; the lenient From<&Binding>/From<ComparableValue> EBVs stay for the Cypher structural surfaces (lists/maps/paths/rels). Greens dawg-bev-1..6. not-not stays registered: its "z"^^xsd:boolean VALUES row is stored coerced to Boolean(false), losing its ill-typedness before EBV — blocked on ill-typed-literal preservation (D6 / PR-X3).
`=`/`!=` dropped datatypes: a foreign-datatype literal ("zzz"^^:t) compared
equal to a plain "zzz", and incomparable operands returned false/true rather
than a type error. Carry a foreign string datatype into the ComparableValue at
the eval boundary (the Long / xsd:double / xsd:string / lang fast paths stay
byte-identical; language tags are deliberately NOT carried, to keep the string
builtins working) and route `=`/`!=` through a three-valued RDFterm-equal
(§17.4.1.7): value equality with numeric promotion and plain = xsd:string; a
resource is never equal to a literal; two differently-typed recognized values
are known-unequal; an unrecognized / ill-typed datatype that might still denote
the same value is a type error excluding the row for both `=` and `!=`.
Also correct AND/OR to SPARQL three-valued logic (§17.2): false / true dominate
over an operand error instead of aborting on the first one (open-cmp-02).
Greens eq-2-1, eq-2-2, open-eq-04. The rest of the equality cluster stays
registered on separate blockers (typed-literal constant datatypes, blank-node
output identity, temporal timezone semantics, scan-path constraints).
… deferral notes JSON-LD-surface regressions alongside the SPARQL ones (the W3C submodule guards only the SPARQL surface): numeric-promotion result datatype (D4), datatype-aware EBV (D-EBV), RDFterm-equal value equality vs known-unequal (D5), CONCAT non-string type error (D11), AVG non-numeric poison (agg-err-01), plus wave-2-failing discriminating regressions a revert would fail: foreign / ill-typed datatype vs plain string is a type error under both = and != , and three-valued OR (true dominates an operand error). The predicate-grouped COUNT xsd:integer tests (agg02) are confirmatory-only, not discriminating: a memory ledger routes through the general group_aggregate finalize (already xsd:integer on wave-2) and bypasses the indexed StatsCountByPredicateOperator fast path where the xsd:long bug lived; the fast path is detected only for the bare `GROUP BY ?p COUNT(?o)` shape (no post-aggregation bind), and xsd:long/xsd:integer both render as bare JSON numbers, so the datatype can't be observed without dropping off the fast path. The revert-catching coverage is the greened W3C agg02 test against indexed storage (reviewer finding #4b). Realigns two pre-existing float-division tests (grp_query / grp_query_sparql) to the typed-literal rendering now that D4 keeps the result xsd:float instead of widening it to xsd:double. Also updates register comments to name PR-X2 as decision-owner for the deferred cluster with the real root cause (scan-path BGP object-match for eq-graph/open-eq-02/quotes; temporal for date-1/eq-dateTime; blank-node output identity for open-eq-07..12; typed-constant lowering for open-eq-05/06). fmt normalization of the eval files.
# Conflicts: # testsuite-sparql/tests/registers/mod.rs
# Conflicts: # fluree-db-transact/src/lower_sparql_update.rs
# Conflicts: # testsuite-sparql/tests/registers/mod.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave-3 SPARQL W3C burn-down — within-ledger datasets, UPDATE graph-management, the equality/EBV/promotion lattice, and SPARQL 1.2 triple-term syntax
This is the capstone wave of the W3C SPARQL test-suite burn-down: seven thematically-coherent PRs integrated onto
burndown/wave-2. It greens 145 W3C tests to a suite that is 0-failed / 0-stale in both directions (cd testsuite-sparql && cargo test→36 passed), shrinking the skip register from 376 to 207 entry lines (343 → 198 unique tests). The wave is perf-neutral: the integrated A/B vsburndown/wave-2is within the 5% budget on all ten measured scenarios (worst +2.34%). The seven PRs are PR-G2 (within-ledgerFROM/FROM NAMED), PR-W1 (nested-group FILTER scope + sub-SELECT correlation), PR-W2BC (SPARQL 1.2 triple-term syntax), PR-W2 (CONSTRUCT blank-node instantiation), PR-U3 (UPDATE graph-management verbs), PR-U6 (USING + explicit-GRAPH delete scoping), and PR-X2 (the equality/EBV/numeric-promotion lattice).Stacks on the Wave-2 PR (#1454) — this PR branches from
burndown/wave-2; review and merge it after #1454.Reviewer quick reference
End-user API / contract changes. SPARQL
SELECT … FROM <g> FROM NAMED <n>against a single ledger's graphs now returns results instead of being rejected (PR-G2). Seven SPARQL UPDATE graph-management verbs —LOAD/CLEAR/DROP/CREATE/COPY/MOVE/ADD(+SILENT/INTO) — now parse and execute, and the capability is exposed on the transact-builder surface (Txn::clear_graph/drop_graph/copy_graph/clear_default_graph,GraphTransactBuilder::clear_graph/drop_graph/copy_graph) (PR-U3). SPARQL/JSON-LD=/!=, effective-boolean-value,AND/OR, numeric promotion,CONCAT, andSUM/AVGare now spec-strict value semantics (PR-X2 — DELIBERATE SEMANTICS CHANGE (D-12), 2-reviewer sign-off requested) — this can return fewer rows for queries that relied on the old permissive behaviour; see the migration notes.DELETE … USING <g> WHERE { GRAPH <h> … }now scopes the explicit GRAPH block to theUSING NAMEDset (deletes fewer rows — a data-loss fix) (PR-U6). ACONSTRUCTwhose template has a blank node now emits triples instead of an empty graph (PR-W2). SPARQL 1.2 triple-term syntax (TRIPLE/SUBJECT/PREDICATE/OBJECT/isTRIPLEand bare<<( s p o )>>values) now parses/validates and defers evaluation with a cleannot_implemented(PR-W2BC). PR-W1 has no API surface change.Performance verdict: WITHIN BUDGET. Integrated A/B vs
burndown/wave-2,FLUREE_BENCH_PROFILE=full(n=100): all ten scenarios within the 5% budget, worst +2.34% (queryq5). The whole wave — including the u3/u6 transact/staging path — is perf-neutral. Full table below.Production bugs found & fixed (beyond W3C compliance). (1)
USING-scoped delete with an explicitGRAPHblock over-deleted from the default/other graph — a real data-loss bug, Closes #1441 (PR-U6). (2) Predicate-groupedCOUNTmis-taggedxsd:long;SUM/AVGsilently averaged over the numeric subset of a mixed group;AND/ORaborted the whole expression on the first operand error;CONCATsilently coerced non-strings — Closes #1447 (PR-X2). (3) A JSON-LD correlated sub-SELECT returned an extra non-conformant row when a correlation variable was bound only via an inner OPTIONAL (PR-W1). (4) PR-U2's cross-operation blank-node scope check over-rejected valid multi-op requests, and the W3C harness rejected a legitimate baremf:result []empty-store expectation (both fixed in PR-U3).What this wave does (per PR)
PR-G2 — within-ledger
FROM/FROM NAMED(D-3, Option A). Replaces the single-ledger dataset-clause rejection on the buffered query paths with within-ledger dataset construction: each clause IRI is resolved against the ledger's own graph registry (alias → default graph, registered named-graph IRI → that g_id), assembled into a single-snapshotDataSetDb, and executed through the existingDatasetOperator/dataset_query.rspath. A clause IRI that is not a graph in this ledger is still rejected (cross-ledger goes through the connection path). R2RML + within-ledgerFROMis a documented unsupported combination (the R2RML dataset path is non-Send). Also fixes a mutual-async-recursion compile trap the delegation created —query_dataset_with_optionsgets a concrete boxed return (Pin<Box<dyn Future … + Send>>), droppingcargo checkon the crate from 30+ min to ~7s.PR-W1 — nested-group FILTER scope + sub-SELECT correlation. Two independent evaluation fixes. Family A: a single-child
{ }group whose child is scope-sensitive (FILTER/BIND/nestedGroup) stays wrapped as an uncorrelated subquery rather than being flattened into the enclosing scope (so a nested FILTER no longer sees an enclosing-scope variable). Family B: a sub-SELECT correlation variable bound only via OPTIONAL/UNION is reconciled at merge (excluded from the per-row seed so it binds independently, then dropped on a conflicting double-binding) — the compatible-mapping join SPARQL §18.4 requires. Common self-produced-correlation and BGP/OPTIONAL paths (incl. BSBM) stay byte-identical.PR-W2BC — SPARQL 1.2 triple-term syntax (D-1, accept-then-defer). The lexer emits a single contextual
TripleTermFntoken for the five function names (not reserved keywords), the expression parser resolves + arity-validates them, and a newparse_triple_term_valueaccepts<<( s p o )>>in the subject/object/VALUES/BINDpositions RDF 1.2 allows; lowering rejects all deferred forms withnot_implemented. The §2 negative-suite guardrails (no path/collection/reified-triple inside a term; value-context vs pattern-context subject rules; bare<<( )>> .is not a statement) keep every negative test rejected. No new evaluable capability — this is syntax acceptance only.PR-W2 — CONSTRUCT per-solution blank-node instantiation. A CONSTRUCT template blank node (
[ … ],_:a, or a PR-1 collection cell) previously lowered to a never-bound variable and produced an empty graph. The fix records template blank-node variables at lowering (ConstructTemplate.bnode_vars) and mints a fresh blank node per solution row in the CONSTRUCT output path — shared within a row, distinct across rows — under a reservedcst{n}label prefix provably disjoint from stored (fdb-),BNODE()(b{hex}), and SERVICE blanks (a soundness hardening the isomorphism-based suite cannot catch).PR-U3 — SPARQL UPDATE graph-management verbs (D-5, D-6). Grammar + AST + parse + lowering to a new
GraphMgmtOpTransaction-IR directive + a whole-graph-scan staging primitive (stage_graph_mgmt). A named graph is a reservedGraphId; the execution primitive is a scan of a graph's flakes — clone-as-retraction for CLEAR/DROP, clone-with-rewritten-gfor COPY/MOVE/ADD (datatype/lang/list-index ride along verbatim). CLEAR/DROP ALL|NAMED resolve their target set from the registry; COPY/MOVE compose over CLEAR with a set-difference so overlapping triples are never simultaneously asserted and retracted. The dispatch is a singleif txn.graph_mgmt.is_some()check at the top ofstage(), before any hot-path setup, so ordinary insert/upsert/update staging is byte-identical.PR-U6 — USING + explicit-GRAPH dataset scoping (class F, #1441). SPARQL 1.1 §3.1.3: when
USING/USING NAMEDis present it defines the WHERE dataset exactly — the WHERE-visible named graphs are precisely theUSING NAMEDset (empty under a plainUSING <g>). The bug treated an emptyUSING NAMEDset as "no restriction" and registered every named graph, soDELETE { … } USING <g3> WHERE { GRAPH <g2> { … } }matchedg2and over-deleted. The fix restricts the runtime dataset's named graphs to exactly theUSING NAMEDset whenever anyUSING/USING NAMEDclause is present; the no-USINGpath (plainDELETE WHERE { GRAPH <g> … }) stays byte-identical. Prepare-time only; SELECT queries never reach it.PR-X2 — equality / EBV / numeric-promotion lattice (D-12). DELIBERATE SEMANTICS CHANGE — 2-reviewer sign-off requested. Five IR/engine value-semantics fixes on the shared
fluree-db-queryevaluator: D4 numeric promotion (ComparableValuegains first-classFloat(f32)sointeger < decimal < float < doubleholds andxsd:floatno longer widens toxsd:double); D-EBV (a fallible, datatype-aware bare-variable effective-boolean-value — numeric-zero/empty-string falsy; lang-tagged/foreign-datatype/IRI/ill-typed/unbound → a type error excluding the row); D5/D7 (datatype-aware three-valuedrdf_term_equalfor=/!=, with §17.2 three-valuedAND/OR); D11 (CONCATtype-errors on a non-string argument); and aggregates (predicate-groupedCOUNT→xsd:integer;SUM/AVGpoison-to-unbound on a non-numeric member). The common int-int/iri-iri fast paths are kept first; the added per-row cost is one well-known-Sid tag-check on the xsd:double and xsd:string literal paths plus theEqOutcomewrapping on=/!=, measured neutral (query_hot_bsbm_bi/f2−0.51% on X2's own standalone branch-vs-wave-2 n=100 A/B; the whole-wave integrated f2 is +1.80% in the perf table below — both well inside the 5% budget, and the difference between the two is measurement, not an X2 cost, since no other wave-3 PR touches the FILTER/eval hot path f2 exercises).Decisions actioned in this wave
Per the standing transparency rule (ROADMAP §4), each decision point, the options weighed, and why the adopted option was chosen.
D-1 — SPARQL 1.2 triple-term syntax: accept-then-defer (PR-W2BC). Options: (4) accept + arity-validate the syntax, lower to
not_implemented[adopted]; (3) reject-on-principle as a documented divergence; (2) desugar<<( )>>to the reifier model (dominated — a triple term is a value, a reifier is a resource, soisTRIPLE/SUBJECT/term-equality would silently diverge). Chosen Option 4: it makes Fluree parse standard SPARQL 1.2 today at zero engine/perf risk — exactly what a syntax-conformance suite rewards — while honestly deferring evaluation to the Option-1 first-class-value epic. The trade-off (a runtimenot_implementedwhere there used to be a parse error) is named explicitly; failing fast at parse time is Option 3, a genuine either/or the team owns.D-3 — within-ledger
FROM/FROM NAMEDdatasets (PR-G2). Options: (A) engine within-ledgerDataSetreusing the runtime dataset path [adopted]; (B) harness ledger-per-graph withquery_connection_sparql(zero engine change). Both green the same 13 tests, so the tie-breakers are product value, reuse, and perf — all favour A. A fixes the actual user-facing gap (a real user can query one ledger's named graphs), extends #1279's within-ledger registry the natural way (a named graph is a g_id), and keeps a single snapshot (no cross-ledger provenance stamping, binary store stays enabled); B only satisfies the harness and spins up N ledgers per query.D-5 — remote (non-SILENT) LOAD: documented divergence (PR-U3). Options: (a) parse LOAD, execute
SILENTas a swallowed no-op, error on non-SILENT [adopted]; (b) add an opt-in HTTP fetch hook into the transact path. Chosen (a): it costs zero W3C coverage (the two eval tests LOAD a non-resolvable source under SILENT and expect the store unchanged; the syntax tests are parse-only), whereas (b) would drag an HTTP client and its failure/security surface into the embedded write path for no test benefit (ingesting a known document already has a first-class path — the insert API).D-6 — DROP ≡ CLEAR + the empty-named-graph model (PR-U3). First half — DROP is implemented as CLEAR semantics now (the
GraphRegistryis additive-only, and the two are indistinguishable to the harness, which compares/lists only non-empty graphs). Second half (a spotlighted product call) — whether a CLEAR'd/CREATE'd zero-triple named graph is enumerable byGRAPH ?g/FROM NAMED: Option A (persist a graph-existence record decoupled from flakes across storage/commit/query, greensbindings#graph+agg-empty-group-count-graph, −2) vs Option B (documented permanent divergence — a graph exists iff a flake carries its g_id) [adopted]. Chosen B: the capability is a query/dataset-model feature orthogonal to the UPDATE verbs, it is the same model fact that makes DROP≡CLEAR correct, and Option A's blast radius (three subsystems, a durable "empty graph" concept Fluree deliberately lacks) is disproportionate to −2 tests. Both tests stay registered with the full rationale.D-12 — SPARQL strict-type-error mode vs unified strictness (PR-X2). Options: (a) a per-surface strictness flag threaded through evaluation; (b) one unified strict value-semantics path shared by all surfaces [adopted for equality/EBV/promotion]; (c) leave lenient. Chosen (b): equality/EBV/promotion are value semantics, correct on both the SPARQL and JSON-LD surfaces, and the parity tests assert the strict behaviour on both. Deliberate JSON-LD extensions are preserved by reading the binding directly rather than the surface (the lenient
From<&Binding>EBVs stay for Cypher structural truthiness; PR-X1 already actioned the D2bDATATYPE/LANG@idcarve-out).D-11 (lexical-form preservation) is NOT actioned this wave — its tests (
group01, thesameTerm/distinct/dawg-strlexical set) remain registered and are owned by the future PR-X3.Register entries removed (145 unique; suite-driven, both directions)
SPARQL10_QUERY_EVALdawg-dataset-01..12b(12) +SPARQL11_CONSTRUCTconstructwhere04.SPARQL10_QUERY_EVALfilter-nested-2,dawg-optional-filter-005-not-simplified,join-scope-1.SPARQL12_SYNTAX_TRIPLE_TERMS_POSITIVE→ empty (basic/bnode/compound/expr/inside/nested/subject/update-tripleterm +compound-all).SPARQL10_QUERY_EVALconstruct-3,construct-4+SPARQL11_CONSTRUCTconstructlist.SPARQL11_UPDATEgraph-management eval tests (add/clear/copy/create/drop/load/move+-silentvariants, the 4 same-bnode joint tests) and the 23 double-registeredSPARQL11_SYNTAX_UPDATE_1test_*(both copies deleted per the standing rule, so 23 tests remove 46 lines; that register is now empty). Per-PR unique counts sum to the 145 headline (13 + 3 + 27 + 3 + 68 + 2 + 29); in raw register lines the wave removes 169.SPARQL11_UPDATEdawg-delete-using-02a,dawg-delete-using-06a(empties the last class-F cluster).agg02,agg-err-01,concat02,strlang03-rdf11,dawg-langMatches-1/2/3/basic,type-promotion-03/04/05/21/29/30,{add,subtract,multiply,divide}-numbers-cast,unplus-2,unminus-2,dawg-bev-1..6,eq-2-1,eq-2-2,open-eq-04.Query-surface parity
Per
docs/contributing/sparql-compliance.md§"Query Surface Parity" (JSON-LD only; Cypher excluded — we do not own the openCypher grammar): every IR/engine-level fix carries a JSON-LD regression test alongside the SPARQL one, since the W3C submodule guards only the SPARQL surface. PR-X2 (value semantics), PR-W1 (scope/correlation — incl. the JSON-LD-only correlated-subselect bug fix), and PR-G2 (within-ledger dataset, a single construction over both surfaces) are IR/engine (case a) with JSON-LD tests init_query_expression_semantics.rs,it_query_filter_scope.rs, andit_query_dataset.rs. PR-U3 exposes the new graph-management capability on the transact-builder surface in the same PR (it_named_graphs.rs::test_graph_mgmt_transact_builder_parity); a JSON-LD document key syntax is a deliberately-deferred surface-design decision. PR-U6 is a SPARQLUSINGsurface fix with a JSON-LDfromNamedguard test on the shared machinery (the symmetric JSON-LDfrom-without-fromNamedrestriction is a scoped follow-up). PR-W2 (CONSTRUCT bnode) and PR-W2BC (parse-only) are SPARQL-only surface (case c) — the FQL construct form has no blank-node template syntax, and accept-then-defer introduces nothing newly executable.Performance (integrated A/B vs
burndown/wave-2@ 3040d8e)FLUREE_BENCH_PROFILE=full FLUREE_BENCH_SCALE=small(real n=100;query_hot_bsbm.rs:166setssample_sizefrom the profile, so the env var is the real knob). Query benches whole-bench interleaved; insert/import benches per-scenario fine-interleaved (wave-2↔wave-3 back-to-back) to kill cross-run drift; self-gated on rustc=0; contended runs discarded. Δ% = (wave-3 − wave-2)/wave-2 (positive = wave-3 slower); budget 5% atsmall.All within budget. An initial coarse-interleaved pass showed two apparent breaches (jsonld/100nodes +7.33%, import single_threaded +12.31%) that were foreign-build contention landing in the longer wave-3 windows, not a code regression: no wave-3 change touches the data-insert/bulk-import hot path (fluree-db-core + fluree-graph-turtle untouched; u3's shared-
stage()change is a single per-transaction guard, byte-identical for inserts; u6's is behindsparql_where), and the clean fine-interleaved re-run collapsed both to −1.28% / −0.72% with tight CIs.Deferrals (staying registered, with per-entry pointer comments)
The equality cluster's remaining tests are blocked outside the equality/EBV/promotion lattice and stay registered with X2-owned notes: the D5b scan-path
EncodedLitdatatype-carry family (eq-4,open-eq-02/05/06,dawg-lang-3,quotes-3/4,eq-graph-1/2/4— bare-BGP object term-match, deliberately not risking the binary-index hot path), blank-node OUTPUT-identity isomorphism (open-eq-07/08/10/11/12), temporal timezone semantics (eq-dateTime,date-1),dawg-langMatches-4(LANG-of-IRI error propagation),not-not(ill-typed literal preservation, → PR-X3), andagg-count-rows-distinct(COUNT(DISTINCT *)needs a newCountDistinctAllIR aggregate + whole-row group plumbing — a perf-neutral post-wave-3 follow-up). The D-6 empty-named-graph divergence keepsbindings#graph+agg-empty-group-count-graphregistered (PR-U3). D-11 lexical-form tests (group01,sameTerm/distinct/dawg-str) are PR-X3's.nested-opt-1/2(correlated-OPTIONAL independence) is PR-W1-OPT;join-combo-2+optional-complex-2/3/4are PR-G1 (GRAPH-variable) territory.Verification & harness/CI notes
Verified: full W3C suite
36 passed; 0 failed; 0 ignored(0 stale, both directions);-p fluree-db-query -p fluree-db-api -p fluree-db-transact -p fluree-db-sparql0-failed (32 bins; fluree-db-query 1178, grp_query 351 / grp_query_sparql 282 / grp_misc 243);cargo fmt --all --checkclean;cargo clippy --all-targets --features native -- -D warnings0 warnings.Harness notes for reviewers:
testsuite-sparqlis an EXCLUDED separate cargo workspace — run the W3C suite withcd testsuite-sparql && cargo test, not top-levelcargo test. A freshgit worktree adddoes NOT populate thetestsuite-sparql/rdf-testssubmodule — rungit submodule update --initbefore the suite or every manifest spuriously fails.fluree-db-apiuses grouped test bins (grp_query,grp_query_sparql,grp_misc, …), so scope with the full-p fluree-db-api; a per-file--test <name>matches nothing. The pre-existingresolve_shapes_source_g_idsdead-code warning influree-db-api/src/tx.rs:506is base noise (untouched by this wave; does not fire under clippy--all-targets).Migration / changelog
AND/ORapply three-valued dominance;CONCATandSUM/AVGraise type errors rather than coercing; numeric promotion preservesxsd:float(soxsd:float(10)/4serializes as a typedxsd:floatliteral, not a bare number). Queries relying on the old permissive behaviour may return fewer rows.GraphTransactBuilder/Txnclear_graph/drop_graph/copy_graph.DELETE … USING <g> WHERE { GRAPH <h> … }now follows §3.1.3 — theGRAPH <h>block matches only when<h>is inUSING NAMED; updates relying on the old over-reach should addUSING NAMED <h>.CONSTRUCTtemplates with blank nodes now emit triples (a bug fix aligning with §16.2; no migration).<<( )>>values now parse/validate and produce a lower-timenot_implementedwhere they previously produced a parse error.Issues closed on merge
Closes #1441 (PR-U6 — the USING + explicit-GRAPH delete over-delete data-loss bug; PR-U6 greens
dawg-delete-using-02a/06a, fully resolving it — the issue has no JSON-LD component). Closes #1447 (PR-X2 — aggregate / equality / EBV conformance). The symmetric JSON-LDfrom-without-fromNamedrestriction PR-U6 identified is a separate scoped follow-up (file a new issue if it needs tracking), not part of #1441.Mechanics: the
Closeskeywords fire only when this PR merges into the repository's DEFAULT branch. In the stacked burn-down flow (Wave 3 →burndown/wave-2→burndown/wave-1→main), merging this PR intoburndown/wave-2does NOT close #1441 or #1447 — GitHub auto-closes them only once the stack collapses tomainand this PR is retargeted onto it. Until then the issue references stay linked but open.