fix(metadata-protocol): dotted-path SORT hint prescribes a stored field, not a formula (#6924)

[os-zhuang]

Fixes #6924

1. The repro, first — the premise HOLDS for formula, and is FALSIFIED for rollup

This card was dispatched verification-first: it was promoted to pm:queue on "same defect
class as #6673", but the promotion gate the first triage comment set — one measured repro —
had never been taken. The card is honest that its core claim is inferred from architecture,
not observed. So the measurement came first.

Method. A real SqlDriver (better-sqlite3, on-disk) and a real InMemoryDriver wired to
a real ObjectQL engine, plus ObjectStackProtocolImplementation on top for the ingress
half. An object carrying a formula field (sort_key, expression record.title) and a
summary field (child_total, count). Five rows inserted C A E B D so that "sorted" and
"insertion order" are distinguishable byte for byte. The formula field is named directly
in orderBy — non-dotted, so assertSortFieldsExist lets it through and the driver really
sees it.

Predictions were written down before the run (P1 formula degrades silently; P2 summary
has a real column and sorts; P3 memory driver same as SQL; P4 the ingress gate does not
refuse it).

Result — driver-sql (better-sqlite3):

repro_contact physical columns: ["id","created_at","updated_at","title","seq","account_id","child_total"]
formula col `sort_key` present:  false
summary col `child_total` present: true

CONTROL   orderBy title asc         -> ["A","B","C","D","E"]        a real column really sorts
CONTROL   orderBy title desc        -> ["E","D","C","B","A"]
BASELINE  no sort                   -> ["C","A","E","B","D"]        insertion order

FORMULA   orderBy sort_key asc      -> ["C","A","E","B","D"]   5 rows, 200
                       sort_key values -> ["C","A","E","B","D"]
FORMULA   orderBy sort_key desc     -> ["C","A","E","B","D"]   direction-blind

RAW SQL   order by sort_key         -> sqlite: "no such column: sort_key"

PROTOCOL  ?sort=sort_key            -> 200, 5 records, ["C","A","E","B","D"]
PROTOCOL  ?orderBy=["sort_key"]     -> 200, 5 records, ["C","A","E","B","D"]

Result — driver-memory:

CONTROL   orderBy title asc         -> ["A","B","C","D","E"]
BASELINE  no sort                   -> ["C","A","E","B","D"]
FORMULA   orderBy sort_key asc      -> ["C","A","E","B","D"]
FORMULA   orderBy sort_key desc     -> ["C","A","E","B","D"]

P1, P3 and P4 confirmed. The premise holds for formula. Sorting by a formula field
returns 200, every row present, in insertion order, identically for asc and desc —
which is what makes it a dropped sort rather than a coincidence. SqlDriver.createColumn
returns early for formula (case 'formula': return; // Virtual — no column), the engine
evaluates the expression only after the driver returns (applyFormulaPlan), sqlite rejects
the ORDER BY, and the #3821 unknown-column backstop retries without the sort. The response
even carries the sort_key values in plain view — C, A, E, B, D — in a column the caller
asked to be sorted ascending.

The ingress gate does not refuse it either, so the hint pointed the author at a shape that
lands them back inside the exact silent degradation #4226/#4256 exist to stop, one step
after being refused for it.

But the card's rollup/summary diagnosis is FALSIFIED. The card asserted summary is
"likewise excluded from real-column treatment". Measured, it is not:

summary col child_total present: true            (SqlDriver: `case 'summary': col = table.float(name)`)
orderBy child_total desc  -> ["E","D","C","B","A"]  values [5,4,3,2,1]
raw stored column         -> E:5  D:4  C:3  B:2  A:1

A summary field gets a real, maintained column and ORDER BY on it genuinely works.
rollup is still dropped from the hint, but for a different and weaker reason: it aggregates
child records (count/sum/min/max/avg), so it cannot carry a looked-up parent's
column such as account.company_name onto the queried object. Wrong tool, not a broken one.
That distinction is recorded in the code comment rather than smoothed over, because
"unmaterializable" would have been a false claim about summary.

2. The fix

assertSortFieldsExist's hint, before:

Denormalise the value onto 'OBJECT' (a formula or rollup field that copies it into a real column) and sort by that.

after:

Denormalise the value onto 'OBJECT' (a stored field, written when the source changes) and sort by that. Not a formula field: it is virtual, no driver materialises a column for one, and ORDER BY on it is silently dropped.

(OBJECT stands in for the interpolated object name — GitHub's body sanitizer eats an
angle-bracket placeholder, so it is spelled out here.)

Prescription first, trap second. "Stored" is #6673's vocabulary, deliberately reused
rather than re-invented: that PR landed "copy the value onto a stored text field" /
"mirror it onto a stored text field" for the identical correction on the SEARCH axis. The two
axes now say the same word. The trap clause is carried here — and not in #6673 — because the
measured failure is silent, and because the docs on this axis were still teaching the wrong
answer until this PR (#6673's docs corpus had already moved to "stored").

3. This overturns a recorded decision

#4256 (closed completed, same filer) explicitly proposed and got this exact wording as
its chosen remedy for dotted-path sort. This is not a leftover sweep. The reason to overturn
it is narrow and measured: the remedy prescribes something the platform cannot materialize,
so a refusal whose whole purpose is to stop a silent degradation was routing the author
straight back into it. The rationale is written into assertSortFieldsExist's doc comment,
with the measurement, so the next reader finds the overturn where the decision lives.

#4256's own changeset (.changeset/sort-dotted-path-rejected.md, still pending — the repo
has ~1550 unreleased changesets) also describes the old prescription. It is deliberately
left untouched: it is an accurate record of what that PR shipped. This PR's changeset names
it and states that it supersedes its prescription, so the release compiler has the link. Flagging
it for the PM rather than editing another change's record.

4. File surface — exactly as dispatched, no deviations

File	Change
packages/metadata-protocol/src/protocol.ts	assertSortFieldsExist hint text + the doc comment recording the overturn and the measurement. Nothing else in the file — the saveMetaItem region (#6190, ~:7xxx) is untouched.
packages/objectql/src/query-expression-conformance.test.ts	test-only: the pin of this exact string travels with the string it pins. Nothing else in packages/objectql.
content/docs/protocol/objectql/query-syntax.mdx	"Sorting on Related Fields" corrected to "stored", plus a second callout stating the formula trap and the rollup distinction with the measured numbers.
.changeset/sort-hint-prescribes-stored-field.md	new — the error message is user-visible.

content/docs/releases/ untouched. packages/spec untouched.

5. Reverse verification — direction predicted before running

Prediction (written before the fix existed): reverting the protocol.ts hint to the old
wording turns the new pinned assertion RED, and only that one — the fix is a string, the pin
is of that string, so this is the plain before-green/after-red direction with no inversion.

Result — as predicted. Fix taken out with a patch file (git diff > …patch +
git checkout --; never git stash, which shares one stack across every worktree),
metadata-protocol rebuilt, suite re-run:

❯ src/query-expression-conformance.test.ts (91 tests | 1 failed)
  × the dotted rejection names the relationship it tried to cross and prescribes a STORED field

AssertionError: expected 'Query parameter \'sort\' sorts by \'p…' to match /a stored field/
+ Received:
"… Denormalise the value onto 'showcase_task' (a formula or rollup field that copies it
 into a real column) and sort by that."

Tests  1 failed | 90 passed (91)

1 red, 90 green, and the failure prints the old hint verbatim. Fix restored via git apply;
suite back to 91/91. No prediction missed.

The pin was also strengthened while moving, per the rejection-envelope rule: it now asserts
status: 400 and code: 'INVALID_SORT' alongside the wording, instead of the previous
bare .rejects.toThrow(/…/). Note it can not assert not.toContain('formula') the way
#6673's pins do — the new text names formula as the trap. It asserts the stronger pair
instead: /a stored field/ must be present, /formula or rollup/ must be gone, and
/Not a formula field/ must be present, so a reword that keeps the dead end in a subordinate
clause still fails.

6. Verification

• pnpm --filter @objectstack/metadata-protocol test → 65 files, 813 tests passed
• pnpm --filter @objectstack/objectql test → 161 files, 2770 tests passed
• pnpm --filter @objectstack/objectql typecheck → tsc --noEmit clean
  (metadata-protocol declares no typecheck script — it carries a ledger entry)
• eslint --no-inline-config on both changed source files → exit 0
• Family gates run locally, all PASS: check:doc-authoring, check:docs-audit-scope,
  check:empty-changeset, check:error-code-casing, check:route-envelope,
  check:engine-double-contract, check:adr-links
• node scripts/check-nul-bytes.mjs → OK (6467 files); plus a targeted control-byte
  self-scan of the four changed files → no matches

7. Out of scope, filed separately — NOT absorbed here

Filed as #6994 (unassigned, for PM triage; routing suggestion domain:engine-core).

The repro surfaced a genuine engine-side defect that is not hint text: a non-dotted
orderBy naming a formula field is accepted by assertSortFieldsExist (a formula field is
in gate.known), reaches the driver, and answers 200 in arbitrary order — the same silent
degradation, on a shape no gate covers. The full measurement above is repeated on that issue,
along with the three candidate remedies (refuse at ingress / refuse at the driver /
materialize) and the wrinkle that an ingress refusal must name the same "stored field" remedy
this PR just landed, or the two doors disagree again.

Whether to refuse, and where, is a domain:engine-core decision — so it is filed rather than
decided here.

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 9, 2026 8:11am

[github-actions]

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/metadata-protocol.

4 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

• content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata-protocol)
• content/docs/kernel/services-checklist.mdx (via @objectstack/metadata-protocol)
• content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/metadata-protocol)
• content/docs/releases/v9.mdx (via @objectstack/metadata-protocol)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

[os-zhuang]

ACCEPT — PM step-7 review (domain:metadata seat, session session_01W6bLax4KMrSfnE1ydFU8Dw). Marking ready + auto-merge (SQUASH). The changeset question below is resolved as A, no escalation needed.

The dispatch's whole point was the unmeasured premise, and it was measured. This card reached pm:queue through a maintainer-authorized one-off pass that cleared an earlier triage gate ("one measured repro") by analogy rather than by measurement, so it was dispatched verification-first. The repro was taken on a real SqlDriver (better-sqlite3) and a real InMemoryDriver: a non-dotted orderBy on a formula-typed field returns 200 with all 5 rows in insertion order, byte-identical for asc and desc — SqlDriver.createColumn returns early for formula, sqlite answers "no such column", and the #3821 backstop retries without the ORDER BY. The hint was routing authors into exactly the silent degradation #4226/#4256 exist to prevent. Premise holds.

And half the card's argument is wrong, which the dev reported instead of banking. The filing claimed rollup/summary is "likewise excluded from real-column treatment". Measured: false — a summary field gets a real, maintained float column and ORDER BY on it genuinely works (orderBy child_total desc → E D C B A over 5 4 3 2 1). It is still dropped from the hint, but for a weaker and completely different reason: it aggregates child records, so it cannot carry a looked-up parent's column. That correction lives in the code comment rather than being smoothed away — which matters, because the next reader would otherwise inherit a false claim about summary from a PR that "fixed" the paragraph.

Verified against GitHub:

• CI on cb18f29: 26 check runs, zero non-success — 25 completed: success including ESLint (the job carrying the family gates) and TypeScript Type Check, plus Build Docs and Check Documentation Links (this PR carries a docs change, so those matter here), Check Changeset, Spec property liveness, Test Core 1-3/3, Dogfood gates, Temporal Conformance; 1 skipped (Console Pin Gate) by design.
• Both surfaces moved together, which was the one thing distinguishing this card from The searchable-fields hints prescribe a "text/formula" mirror — the formula half never works, so the tool output now contradicts the docs that quote it #6673: the hint now prescribes "a stored field, written when the source changes" with an explicit "Not a formula field" clause, reusing The searchable-fields hints prescribe a "text/formula" mirror — the formula half never works, so the tool output now contradicts the docs that quote it #6673's stored vocabulary so the SORT and SEARCH axes agree — and query-syntax.mdx, which taught the same denormalization, moved in the same PR. Fixing one side alone would have relocated the contradiction rather than removed it.
• The PR body states plainly that this overturns sort 的点号路径（?sort=account.company_name）仍然静默降级为「不排序」——#4226 收口后唯一漏网的 sort 形态 #4256's recorded wording choice and why, as required — sort 的点号路径（?sort=account.company_name）仍然静默降级为「不排序」——#4226 收口后唯一漏网的 sort 形态 #4256 chose that phrasing deliberately and closed completed.
• Reverse verification with direction predicted first (plain before-green / after-red; no inversion, since the fix is a string and the pin is of that string): fix removed via patch file — git diff + git checkout --, never git stash — package rebuilt, suite re-run, exactly 1 red with the assertion naming the old text. No prediction missed.

The changeset question — resolved as A, and it is a lane-principle call, not a judgement call

.changeset/sort-dotted-path-rejected.md (from #4256, unreleased) still describes the prescription this PR overturns, and with ~1550 unreleased changesets the two will very likely compile into the same release — so the notes would carry a prescription and its retraction.

A stands: leave both. A changeset is a record of what a PR actually shipped, not a live instruction; editing another change's record to tidy a release note trades an honest history for a cosmetic one. This lane already holds the adjacent rule — release notes are written centrally at release time, and a PR's only inputs to them are its own changeset — so reaching into #4256's changeset would be this seat making a release-compilation decision from the wrong end. This PR's changeset names #4256 and states that it supersedes its prescription, which gives the release compiler the link it needs. If the release-notes author would rather not print the retracted advice, that is a one-line edit they can make at compile time with full context, which is where the judgement belongs.

Out-of-scope finding, correctly not absorbed

#6994 filed: a non-dotted orderBy naming a formula field passes assertSortFieldsExist (formula fields are in gate.known), reaches the driver, and answers 200 in arbitrary order — the same silent degradation as the dotted form, on a shape no gate covers at all. Measured on both driver-sql and driver-memory, filed unassigned with three candidate remedies and a domain:engine-core routing suggestion. The dispatch asked for exactly this: report the engine-side question, do not absorb it. The hint text is this card; whether the platform should refuse such a sort is not.

---

Generated by Claude Code