`engine.find()` still drops a `formula` ORDER BY silently — decide whether the engine refuses or keeps its internal-caller tolerance

[os-zhuang]

Filed from #6994's implementation (PR pending). Recorded, not claimed. Routing suggestion: domain:engine-core — this is a contract decision about engine.find, not a gate fix.

What #6994 closed, and what it did not

#6994 refuses a non-dotted orderBy naming a formula field at ingress (assertSortFieldsExist, 400 INVALID_SORT). That covers everything reaching findData: the REST list route, POST /data/:object/query, the export route (its $orderby funnels through findData), and the RPC dispatcher.

It does not cover a caller that reaches engine.find() directly — hooks, flows, reports, expand sub-reads. Measured on the #6994 branch with the fix in place, one script, real SqlDriver (better-sqlite3, on-disk) + real ObjectQL + the real protocol on top, five rows inserted C A E B D, sort_key a formula over record.title:

PROTOCOL  {"sort":"sort_key"}          REFUSED 400 INVALID_SORT: … a formula field on 'repro_contact' …
PROTOCOL  {"orderBy":["sort_key"]}     REFUSED 400 INVALID_SORT
PROTOCOL  {"sort":"-sort_key"}         REFUSED 400 INVALID_SORT

FORMULA   orderBy sort_key asc         ["C","A","E","B","D"]   engine.find direct — still silent
FORMULA   orderBy sort_key desc        ["C","A","E","B","D"]
   asc === desc (byte-identical)?      true

Both halves in one run: the door refuses, the direct path still answers 200 in arbitrary order. PR for #6994 carries a test explicitly labelled RECORD OF A KNOWN HOLE pinning this — it records the unfixed behaviour and says it should go red the day this issue is closed.

The decision, not the patch

This is deliberately not filed as "add the same check to the engine", because the engine's posture here is documented and load-bearing in the opposite direction. assertProjectionFieldsExist states it in the protocol source: the engine's tolerance "guards INTERNAL callers (hooks, flows, expand sub-reads, registry-less hosts) that never pass through this ingress". planFormulaProjection acts on the same posture — it drops virtual formula names from ast.fields rather than refusing them. So making engine.find refuse is a new posture, not an extension of an existing one.

Options, all real:

1. engine.find refuses an unmaterializable orderBy. One place, every driver and every caller covered. Cost: reverses a documented tolerance; any internal caller that currently sorts by a formula field starts throwing where it used to silently under-deliver. Needs a sweep of internal callers first.
2. engine.find drops it the way it drops virtual names from the projection, but observably — e.g. via the existing DroppedFieldsEvent channel rather than silently. Keeps the tolerance, removes the invisibility. Cheaper, and consistent with planFormulaProjection's existing behaviour.
3. Leave it at ingress. The external door is the one an author can hit by hand; internal callers are tsc-covered code we own. Cost: the two halves of one contract answer differently, which is exactly the split REST 列表：sort / select / expand 指向不存在的字段时被静默丢弃（filter 轴已收口，这三条轴还没有） #4226 was filed about on the other axis.

⚠️ One trap worth writing down before anyone reaches for it: the engine could sort post-hoc, after applyFormulaPlan evaluates the formulas. That looks like "make formula sorting work" and is a trap — driver.find has already applied limit/offset, so post-sorting reorders an arbitrary page. It would appear to work on small result sets and be wrong the moment pagination is involved.

Also worth deciding here: one spelling of "virtual"

"Which field types materialise no column" is currently spelled in six places, none of them packages/spec: driver-sql's fieldHasColumn and createColumn, driver-turso's remote-transport, objectql's planFormulaProjection and search-companion, plugin-audit's VIRTUAL_FIELD_TYPES — and now metadata-protocol's UNMATERIALIZED_SORT_TYPES (#6994 added the sixth deliberately, local and documented, rather than opening a cross-package spec edit for one string). Note the spec's COMPUTED_VALUE_TYPES (formula/summary/autonumber) is not this set — it is the write contract, and summary/autonumber both get real columns and sort correctly. Consolidating is a cross-package change (#6298 cross-seat declaration) and is a candidate to fold into whichever option above wins.

Refs: #6994 (the ingress half, PR pending), #6924 (the dotted hint text), #6673 (the SEARCH axis), #3821 (the backstop that swallows it), #4226 / #4256 (the family).

[claude]

Triage: needs-user-decision.

• Why the decision box: the card is explicitly the decision fork spun out of A non-dotted orderBy naming a formula field answers 200 in arbitrary order — the sort is silently dropped (measured on driver-sql + driver-memory) #6994 — whether engine.find() refuses an unmaterializable orderBy, drops it observably (DroppedFieldsEvent), or keeps the documented internal-caller tolerance. Option 1 reverses a documented, load-bearing posture (assertProjectionFieldsExist's stated tolerance for internal callers; planFormulaProjection acts on the same posture); the three options land in different packages. Queueing or routing now would pre-decide it.
• No domain:* yet, deliberately: options 1/2 land in packages/objectql (domain:engine-core), option 3 keeps the fix at ingress in packages/metadata-protocol (domain:metadata). Routing follows the ruling.
• Premise verified on origin/main @ ac244ad: the ingress half has landed — assertSortFieldsExist at packages/metadata-protocol/src/protocol.ts:4868 (consumed at :5586), planFormulaProjection at packages/objectql/src/engine.ts:620 still drops virtual names from the projection while leaving sort untouched. The card's "PR pending" note is already one hop stale in the right direction; the decision itself is untouched.
• Dedup: A non-dotted orderBy naming a formula field answers 200 in arbitrary order — the sort is silently dropped (measured on driver-sql + driver-memory) #6994 (open, pm:dispatched, domain:metadata) is the ingress half this card was deliberately split from — distinct dispatch entries, filer cross-referenced; repo search for INVALID_SORT/formula found no third card.
• target:v17: not applied. The author-reachable door now refuses (A non-dotted orderBy naming a formula field answers 200 in arbitrary order — the sort is silently dropped (measured on driver-sql + driver-memory) #6994); the remaining hole is internal callers under a documented tolerance — no user-visible shipped-surface failure today, fails all four blocking classes.

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

---

Generated by Claude Code

[os-zhuang]

Maintainer ruling (2026-08-10, directed in session session_01BPWqbmEFU8gJepBJTHESXd): refuse at the public boundary.

A formula field in ORDER BY that the engine cannot apply is a 4xx refusal with guidance prose — never a silent drop. This is the same direction as the analytics dataset refusal envelope and the #6924 sort-hint prescription. The internal-caller tolerance survives only if a measured internal call site actually relies on it, and then behind a pinned internal path, not on the public API. Implementer: enumerate internal callers first and put the count in the PR.

needs-user-decision → pm:queue; unlabelled for domain — triage seat routes it (landing site suggests domain:engine-core).

---

Generated by Claude Code

[os-zhuang]

Triage (routing per the 2026-08-10 maintainer ruling above): domain:engine-core appended; pm:queue already restored by the ruling comment.

• Why engine-core: the ruled direction — refuse at the public boundary, with the internal-caller tolerance surviving only behind a pinned internal path if a measured call site relies on it — lands the refusal in engine.find() itself, i.e. packages/objectql (planFormulaProjection verified at engine.ts:620 in the earlier triage pass; the ingress half assertSortFieldsExist is already live in metadata-protocol). packages/objectql ⇒ domain:engine-core, as both the card's own routing suggestion and the ruling comment anticipated. The 2026-08-09 deliberate non-route ("routing follows the ruling") is hereby resolved by the ruling.
• Implementer constraint carried from the ruling: enumerate internal callers first and put the count in the PR; the trap note in the body (post-hoc sort reorders an arbitrary page under pagination) stands.
• Dedup/premise: unchanged from the 08-09 triage comment — A non-dotted orderBy naming a formula field answers 200 in arbitrary order — the sort is silently dropped (measured on driver-sql + driver-memory) #6994 is the ingress half, distinct dispatch entry; no third card. Premise verified then on ac244ad, ruling is newer than that check.
• target:v17: not applied — unchanged from the 08-09 grading (author-reachable door already refuses; the remaining hole is internal callers).

本评论来自分诊座位 Routine（#5474 试点），不构成认领。

---

Generated by Claude Code