Skip to content

feat(driver-sql): compile $field to column-to-column comparison (#5222) - #7582

Merged
os-zhuang merged 2 commits into
mainfrom
claude/issue-5222-field-ref-pushdown
Aug 11, 2026
Merged

feat(driver-sql): compile $field to column-to-column comparison (#5222)#7582
os-zhuang merged 2 commits into
mainfrom
claude/issue-5222-field-ref-pushdown

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Closes #5222

The second half of #5041. FieldReferenceSchema ({ $field: 'col' }) is declared in the spec and genuinely PRODUCED — compileCelToFilter emits it whenever a CEL permission/RLS rule compares one field to another — but its only implementation was the in-memory evaluator. #5041 measured that, replaced a bare TypeError (and, inside $in lists, a silent zero-row answer) with a loud INVALID_FILTER / 400, and deliberately left the capability itself to this issue. Until now one permission rule had two behaviours, chosen by whether the query reached a database.

What compiles now

The six scalar comparison operators — $eq / $ne / $gt / $gte / $lt / $lte, including the array-triple authorings that lower to them — compile the reference to a real column reference (knex ?? identifier binding):

{ amount: { $gt: { $field: 'budget' } } }   // → where "amount" > "budget"

Nothing that worked before changes. The refusal gate was narrowed, never removed.

The refused arm, and why each entry is there

All keep the ADR-0112 envelope (INVALID_FILTER / 400):

Position Why
Dotted paths (account.owner_id) Maintainer ruling: v1 is same-table columns only. No JOIN planning, no alias contract.
Undeclared columns, either side The value lands in a SQL identifier position — declared-only enumeration, refused at COMPILE time. Federated/external tables (ADR-0015) refuse wholesale, since this driver does not own their column set.
The tenant-isolation column, either side A privilege-escalation comparison surface. Closed on both sides because the operands of = commute — a ban one swap away is not a ban.
Cross-class comparisons The two paths genuinely disagree; see below.
Columns with no scalar stored form multiple: true / JSON columns, and formula fields (virtual — no column at all).
$in / $nin / $between list members The memory evaluator does not resolve a reference inside a list either (resolveValue returns an array unchanged), so there is no correct semantics for SQL to be equivalent TO.
The string operators A column-side LIKE pattern cannot be metacharacter-escaped portably, and an unescaped one is the %-matches-every-row bypass.
Bare { field: { $field: 'other' } } What parseFilterAST(['a', '=', { $field: 'b' }]) lowers to. The memory evaluator answers false for it rather than reading it as an equality, so compiling it would open a divergence in the change that closes one. The message now names $eq instead of falling through to a generic operator list.

Equivalence is proven, not asserted

A cross-path conformance suite runs each supported shape through matchesFilterCondition and through SQL push-down against the same seeded rows, holding both to the same declared id list — a third, independent statement of the semantics, so a case fails loudly when both paths drift together.

The NULL rows are the point. Three-valued SQL against a two-valued JS matcher is the one place these paths can genuinely diverge, so the fixture carries every NULL arrangement two columns can be in: target NULL, referent NULL, and BOTH NULL. Every emitted predicate is therefore written TOTAL — { a: { $eq: { $field: 'b' } } } matches a row where both columns are NULL (a plain a = b drops it), and $not over any cross-field leaf is its exact complement, so the #5146 negation rewrite needs no guard.

Coverage: both SQL drivers (driver-sqlite-wasm inherits the compiler but executes through its own sql.js dialect, which binds the identifier list itself), across the full driver axis — SQLite always, live Postgres and MySQL when provisioned.

Reverse verification

Predicted in writing before running, then measured:

Break Predicted Measured
Remove the cross-field branch every supported case red, refusal arm green 28 supported red — and 12 refusal cases red too, because their boundary MESSAGES come from the new validation, not the old blanket refusal. Prediction corrected.
Break the tenant guard (target side only) exactly the target-side pins red exactly 2 red, both naming the tenant column; the referent-side pin stayed green, proving it does not cover the commuted spelling
Remove the dotted-path refusal not a hole — falls through to declared-only confirmed: still refused, envelope intact, only the message degrades to not a declared field
Emit a naive A = B for $eq $eq, self-reference, $not $eq, $or, De Morgan red; memory assertion stays green exactly 7 red, every one reporting "SQL push-down disagreed" — the divergence isolated to the SQL side
Remove the class check cross-class refusals red, and the shape compiles 4 red with "expected the driver to refuse this filter, but it resolved"

The last one corrected the change. Measuring it showed the divergence is directional and the corpus had tested the wrong direction: { stage: { $gt: { $field: 'amount' } } } (text target, numeric referent) returns four rows on SQLite and none in memory — SQLite orders by storage class, so every TEXT sorts above every INTEGER, while JS coerces 'won' > 10 to a NaN comparison. The mirrored spelling happens to agree. The divergent direction is now the corpus's headline case, and the code comment states which cell was measured divergent instead of claiming all of them are.

Notes for review

  • The corpus deliberately does not live in packages/spec/src/data. That directory is the (driver × case-set) matrix check-driver-conformance.mjs scores, and every case-set there obliges every driver to import it or carry a ledger entry — promoting it would enrol driver-memory, driver-mongodb and driver-turso REMOTE as DEBT for a capability this issue's ruling scoped to SQL push-down. It is exported from @objectstack/driver-sql instead, which is also what lets driver-sqlite-wasm run the identical corpus.
  • @objectstack/formula is added as a devDependency of both driver packages — the cross-path suite needs the reference evaluator. No runtime dependency added.
  • service-analytics's read-scope-sql / filter-normalizer are independent emitters with their own mirrored comparand gate and still refuse $field. Out of scope here (this issue's rulings are about the driver), but it means a CEL rule lowered through the analytics faces still gets a 400 — reported to the PM for its own card.

Generated by Claude Code

claude added 2 commits August 11, 2026 06:22
`FieldReferenceSchema` (`{ $field: 'col' }`) is declared in the spec and really
is produced — `compileCelToFilter` emits it for a field-to-field comparison in a
CEL permission/RLS rule — but its only implementation was the in-memory
evaluator. #5041 measured that and installed a loud refusal (INVALID_FILTER/400,
replacing a bare TypeError and a silent zero-row answer inside $in lists),
deliberately leaving the capability to this issue. Until now one permission rule
had two behaviours, chosen by whether the query reached a database.

The six scalar comparison operators now compile the reference to a real column
reference. The refusal gate is NARROWED, never removed — dot paths, undeclared
columns, the tenant-isolation column (both sides, because = commutes),
cross-class comparisons, list members and the string family all keep the
ADR-0112 envelope.

Every emitted predicate is written TOTAL across NULLs, so it agrees with the
two-valued in-memory evaluator rather than with three-valued SQL: both columns
NULL satisfies $eq, and $not over a cross-field leaf is its exact complement.
A cross-path conformance suite proves that row for row on both SQL drivers.

Closes #5222
@vercel

vercel Bot commented Aug 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 11, 2026 6:24am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/driver-sql.

8 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-sql)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-sql)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-sql)
  • content/docs/plugins/anatomy.mdx (via @objectstack/driver-sql)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/driver-sql)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-sql)

1 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-sql)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down)

2 participants