Merge branch 'main' into claude/issue-6743-migrate-plan-readonly-open

Author: os-project-manager

.changeset/action-param-default-value-contract.md

@@ -0,0 +1,49 @@
+---
+"@objectstack/spec": minor
+---
+
+fix(spec): an action param's `defaultValue` is validated against the param's own value contract (#6970)
+
+`ActionParamSchema.defaultValue` was `z.unknown().optional()`, so a default that
+could never satisfy its own param was accepted at authoring time with no warning,
+prefilled into the dialog control, and refused only at submit — on a field the
+user never touched, by a message that named the param but not the author's
+default as the cause.
+
+The default is now checked at parse time through the **same** `valueSchemaFor`
+the dispatcher already runs at submit (ADR-0104 D2, `validateActionParams`) —
+one rule set, two moments, no second vocabulary. `datetime` was the loudest
+instance (`'2026-08-10T15:00'`, a wall clock `datetime-local` happily displays
+and `InstantValueSchema` refuses), but the hole was every type: `number` +
+`'abc'`, `select` + a non-member, a `multiple` param + a scalar.
+
+The rejection names the param, its type, the offending literal, and why it
+matters:
+
+```
+Action param "start" (datetime): the default "2026-08-10T15:00" cannot satisfy
+this param's own value contract — expected an ISO-8601 instant with explicit
+zone (e.g. 2026-03-15T14:30:00.000Z). The dialog would PREFILL this value and
+the submit would then be refused with that same message (ADR-0104 D2), for a
+field the user never touched …
+```
+
+**Acceptance tightening — what is NOT judged.** The gate only answers what the
+declaration itself can answer, because an authoring gate that guesses rejects
+valid metadata. A param with no `type` of its own keeps an open value shape (the
+same default `validateActionParams` applies to an unresolvable type); a
+field-backed param that inherits its arity or its option set is not held to
+either; and `null` / `''` defaults are skipped exactly as the dispatcher's own
+presence check skips them.
+
+**Stock compatibility.** Already-stored action metadata carrying a nonconforming
+default keeps loading and keeps working: the read path (`DatabaseLoader.rowToData`)
+replays the ADR-0087 conversion chain but runs no Zod validation, and
+`MetadataManager.validate` is deliberately a structural check only. Authoritative
+spec validation lives on the WRITE path (`protocol.saveMetaItem`) and is surfaced
+on reads as the advisory `_diagnostics` envelope — which now reports the
+nonconforming default instead of staying silent about it. So this is loud at
+authoring, non-fatal at rest, and no conversion is owed: there is no mechanical
+rewrite for "the author meant some other instant", and inventing one would pick a
+timezone the metadata never declared (the ambiguity #5061 refused to resolve
+consumer-side).

.changeset/adr0094-d5r-overlay-direction-retired.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/plugin-security": patch
+---
+
+ADR-0094 D5-R: retire the "customize packaged permission sets through an ADR-0005 env
+overlay" direction (2026-07-14), and make the ADR text and the
+`permission-set-projection.ts` header agree with what is enforced.
+
+`#6483` (PR #6608) rolled `permission` back to `allowOrgOverride: false`, so a metadata
+write against a **code-declared (artifact-backed)** permission set is refused with 403
+`NOT_OVERRIDABLE` — ADR-0005's security row ("overlays would create silent privilege
+drift") is enforced again. The supported channel for those sets is the one ADR-0086
+always named: edit the package and re-publish. Environment authoring survives on the
+`allowRuntimeCreate` tier, for sets whose definition lives only in `sys_metadata`
+(data-door creations, and package sets authored + published through the metadata door);
+that tier edits the single stored definition in place and is deliberately **not**
+described as a re-route of the retired overlay channel.
+
+No behaviour change: the four production write points keep their current dispositions.
+The refusal is left to the producer — `plugin-security` does not re-derive
+artifact-backing to pre-empt it — and the two write points that catch a failed metadata
+write (the `restore` leg and the boot backfill) keep reporting on the durability channel.
+What changes is prose, plus test coverage that can now see the gate: the suite's protocol
+stub models ADR-0005's tier gate, so the four cases that pinned the retired direction no
+longer pass for want of a stub that could refuse.

.changeset/aggregation-node-distinct-retired.md

@@ -0,0 +1,71 @@
+---
+'@objectstack/spec': major
+'@objectstack/objectql': major
+---
+
+refactor(spec,objectql)!: retire `AggregationNode.distinct` — one face honoured it, five ignored it, and the same query answered two plausible numbers (#6815, ADR-0049)
+
+<!-- adr-0087: registered aggregation-node-distinct-retired -->
+
+**FROM → TO:** `{ function: 'count', field: 'x', distinct: true, alias: 'a' }` →
+`{ function: 'count_distinct', field: 'x', alias: 'a' }` — the deduplicating spelling
+every backend computes, lowered to `COUNT(DISTINCT x)` on both SQL faces since #6409.
+`{ function: 'sum' | 'avg' | 'min' | 'max', …, distinct: true }` → delete the key; there is
+no replacement, because no SQL backend ever computed `SUM(DISTINCT …)` here and the
+in-memory fallback was the only thing that did. `distinct: false` → delete the key; it
+selected the behaviour that is now the only behaviour.
+
+`AggregationNode.distinct` was read by exactly ONE of the six faces that consume an
+`aggregations[]` entry. `objectql`'s in-memory fallback (`in-memory-aggregation.ts`)
+deduplicated the values before applying the function; `SqlDriver.aggregate`, the Turso
+`RemoteTransport.aggregate`, `driver-mongodb`'s `buildAggregationStage`, `driver-memory`'s
+`computeAggregate` and `service-analytics`' `AGGREGATE_SQL` all ignored it. So
+`{ function: 'sum', field: 'amount', distinct: true }` returned a deduplicated sum when the
+engine fell back in memory and an ordinary sum on every SQL datasource — one query, two
+numbers, chosen by which backend answered. The engine picks that path per query (a driver
+without native aggregation, a non-UTC date bucket, a partial SQL driver), so the number
+could move under a dashboard with nothing changing in the query.
+
+That is the divergence class #6203 and #5907 each closed on the aggregate axis, still open
+on this key, and it is worse to leave: both answers are plausible NUMBERS rather than a
+refusal, so nothing surfaced it. It survived the #4286 sweep of this same schema because
+that sweep asked which members no executor reads — the wrong question for a key whose
+defect is *which* executor reads it.
+
+REMOVE rather than ENFORCE, per the maintainer ruling of 2026-08-09: `count_distinct`
+already covers the only deduplicating spelling with measured demand and took ADR-0049's
+enforce leg in #6409, while `SUM(DISTINCT …)` / `AVG(DISTINCT …)` are near-universally a
+modelling mistake and would have to be lowered across five faces — two of them frozen under
+#5499 — to buy it.
+
+The retirement kit:
+
+- **Tombstone, not deletion** (`retiredKey()`): `AggregationNodeSchema` is not `.strict()`,
+  so a plain delete would let existing queries parse clean and lose the key in silence
+  (#3733, ADR-0104) — trading a divergent flag for an ignored one. Authoring it is now a
+  `tsc` error at the call site and a parse error carrying the prescription. One tombstone
+  covers every aggregation door: `QuerySchema.aggregations` and
+  `EngineAggregateOptionsSchema.aggregations` both reuse that one schema by reference.
+- **ADR-0087 D3 `SemanticMigration`** (`aggregation-node-distinct-retired`) plus the exact
+  `RETIRED_KEYS_BY_MAJOR[17]` entry `data/AggregationNode:distinct`. No D2 conversion,
+  deliberately: `QueryAST` is a request surface — the client SDK builder's output and the
+  `POST /data/:object/query` body — never stored in stack metadata, so there is no source
+  for `os migrate meta` to rewrite. That is the disposition every other `data.query.*`
+  retirement in this major already takes (#4286).
+- `objectql`'s in-memory fallback loses its `collectValues` dedupe limb — the whole runtime
+  cost of the removal. **The observable numbers change on that one path, and that is the
+  point:** a `sum`/`avg` that used to be deduplicated there now answers what every SQL face
+  has always answered for the same query. Verify against the SQL answer, not against the
+  pre-upgrade fallback answer — the two disagreed.
+- Measured blast radius inside the fallback, narrower than the key suggests: only `sum` and
+  `avg` ever changed answer. `count` returned from its own branch before reaching the
+  dedupe, `count_distinct` fed the values into a `Set` (dedupe-then-`Set` is `Set`), and
+  dedupe does not move `min`/`max`.
+- `POST /api/v1/data/:object/query` answers `400 VALIDATION_FAILED` with a `fields[]` entry
+  at `aggregations.<i>.distinct` instead of serving a number — the #3899 entry validation
+  descending into the array, pinned in the REST request-schema conformance gate.
+- Liveness ledger (`query.json` `aggregations.children.distinct` → `dead`, README counts),
+  generated baselines (`authorable-surface/data.json` gains `[RETIRED]`),
+  `spec-changes.json`, the upgrade guide and the reference docs regenerated.
+
+`count_distinct` is untouched and remains the live deduplicating spelling.

.changeset/api-runtime-create-withdrawn.md

@@ -0,0 +1,77 @@
+---
+'@objectstack/spec': major
+'@objectstack/metadata-protocol': major
+'@objectstack/objectql': major
+---
+
+refactor(spec)!: `api` is code-only — withdraw a runtime create door the endpoint matcher could never read (#5488, ADR-0049 remove side)
+
+<!-- adr-0087: registered api-runtime-create-withdrawn -->
+
+**FROM → TO:** `PUT /api/v1/meta/api/{name}` (200 "Saved") → declare the endpoint as a
+stack artifact (`**/*.api.ts`, or `defineStack({ apis })`) and ship it through
+`publishPackage`. The runtime write now answers **403 `NOT_CREATABLE`**, in `?mode=draft`
+as well as direct-active. The artifact route is **unchanged** — a `**/*.api.ts` file valid
+before this release is valid after it, byte for byte.
+
+`DEFAULT_METADATA_TYPE_REGISTRY`'s `api` entry declared `allowRuntimeCreate: true` and the
+runtime never honoured it. Measured on a real showcase boot (`objectstack dev --fresh`, 47
+plugins):
+
+```
+PUT /api/v1/meta/api/e8_backdoor   → 200 {"success":true,…,"message":"Saved …"}
+GET /api/v1/apps/showcase/backdoor → 404      (anonymous AND authenticated)
+```
+
+…and **no** `[EndpointMatcher] … EXCLUDED` line anywhere in the boot log: the endpoint was
+not gated out, it was never in the index at all. The serving criterion belongs to
+`IMetadataService.matchEndpoint` → `EndpointMatcher` → `MetadataManager.listForIndex('api')`,
+which reads the manager's own registry plus its registered loaders
+(`["filesystem","memory"]` on dev/serve). A runtime write lands in `sys_metadata`, which is
+in neither. So the declaration promised a capability that could not exist.
+
+A declared-but-unhonoured capability is ADR-0049 false compliance, and "answers Saved, then
+404s forever" is its most dangerous shape for the AI authors ADR-0033 targets. The
+maintainer ruled REMOVE on 2026-08-07 rather than converge the read path: making the matcher
+read `sys_metadata` re-opens cache, invalidation, tenancy and the ADR-0110 D3
+miss-vs-outage distinction on a new read path, and there is no business pull for
+Studio-authored endpoints today — 17.x serves declarative endpoints through stack artifacts,
+which is what showcase uses (#5040 E8, LIVE).
+
+## The retirement kit
+
+- **`allowRuntimeCreate: false`** on the `api` registry entry. With `allowOrgOverride`
+  already `false`, the type is now **code-only** — the `job` / `agent` / `capability` shape —
+  so the existing #5086 inlet refuses before persistence, on every kernel, with
+  `code: 'NOT_CREATABLE'`, `status: 403` and a prescription derived from the entry's own
+  `filePatterns[0]`. No new refusal mechanism was written for this.
+- **`gateApiDraftsForPublish` is retired** (`metadata-protocol`), together with its nine
+  tests and the `PUBLISH_DRAFTS_NAMESPACE_REMEDY` string only it appended. It landed two
+  days earlier in PR #5279 and is removed **deliberately and on the record**, not lost in a
+  refactor: it gated a draft→active promotion into a state the matcher can never read, and
+  with the inlet closed no `api` draft can exist for it to judge. The in-place comment at
+  its old call site carries the reasoning.
+- **The `metadata-plugin.zod.ts` decision block is rewritten as a recorded overturn.** It
+  used to record CODE-ONLY as "considered and rejected"; its three bullets are kept verbatim
+  with what became of each, so the reversal is auditable rather than silently contradicted.
+- **The `api` create seed is removed** and `api` joins `KNOWN_UNSEEDED`. A pre-filled "New
+  API Endpoint" form whose save can only 403 is the UI half of the same false compliance.
+- **Pins, not deletions.** The two #5271 tripwire pins that asserted
+  `allowRuntimeCreate: true` are **replaced** by retirement pins asserting the new verdict —
+  their comments predicted this exact consequence, and both predictions were correct. Every
+  rejection case asserts `code` **and** `status` (ADR-0112 envelope), never `toThrow()`
+  alone (#6142).
+
+## What did NOT change
+
+`validateApiEndpointDeclarations` / `identityFreeEndpointGateFailure` remain the one judge
+of what is servable, on the route that serves: the stack schema, `publishPackage` (#5189),
+and again at load in `buildEndpointIndex` (PR #5203). ADR-0121's "publish REJECTS" ruling is
+intact. `deleteMetaItem` stays ungated so pre-existing rows can be cleaned up, and
+`OS_METADATA_WRITABLE=api` remains the single operator escape hatch — note it unlocks the
+**write** only; the endpoint still will not be served, which is why it is a diagnostic
+rather than a workaround.
+
+**Re-entry path**, recorded by the ruling: if #2657 Part B promotes `apis` to a registered
+type **with a real consumption path**, the flag and the publish gate come back together —
+implementation first, declaration second.

.changeset/audit-consume-bound-previous.md

@@ -0,0 +1,65 @@
+---
+"@objectstack/plugin-audit": patch
+---
+
+fix(plugin-audit): consume the engine's bound `ctx.previous` and record one normalised view on both sides of the diff (#6656)
+
+`plugin-audit` used to fetch its own pre-image. `captureBefore`, registered on
+`beforeUpdate` / `beforeDelete`, issued a `ql.findOne` for the target row and
+stashed it on `ctx.__previous`, because `HookContext.previous` was "officially
+typed but not always populated by the engine itself". That is no longer true on
+any path this plugin registers for, so the read is retired and the writer reads
+the contract value.
+
+**The read that goes away** (measured with a counting driver on the audited
+object, `driver.findOne` per write):
+
+| write | before | after |
+|:--|--:|--:|
+| single-id `update()` | 2 | 1 |
+| single-id `delete()` | 2 | 1 |
+| predicate `update()`, 3 matched rows | 3 | 0 |
+| predicate `delete()`, 3 matched rows | 3 | 0 |
+
+The predicate column is the larger half and was pure waste. #5574 binds
+`input.id` on every per-row *before* context, which defeated the handler's own
+`if (!id) return` bulk guard — so it read every matched row, and every result
+was discarded, because `__previous` landed on the per-row *before* context while
+the per-row *after* contexts (the ones the writer actually runs on) never saw
+it. The engine's own matched-row read is untouched and still serves both phases,
+so the ledger is unchanged.
+
+**What the ledger records changes, and deliberately.** The two sides of an audit
+diff came from two different pipelines: `before` through the engine's read path
+(credentials masked, formulas hydrated, file references resolved) and `after`
+from the raw write result. That asymmetry — not the redundant read — is why a
+write that touched one field recorded phantom "changes" for every secret, file
+and formula field on the record. Retiring the read makes both sides
+same-source; the writer now also gives them one view, so the surface levels
+upward rather than down to raw store contents:
+
+- **Credential fields are masked on both sides.** Single-id delete `old_value`
+  still reads `••••••••` for a `secret` field — that face is byte-identical.
+  Change detection still runs on the raw values, so rotating a secret is still
+  recorded as a change; only the recorded values are masked.
+- **A pre-existing leak is closed.** The stored `secret:` ref was already
+  reaching `sys_audit_log.new_value` on every create and update, and a
+  `password` field — which ADR-0100 stores in cleartext at rest — was landing
+  there **in plaintext**, in the audit ledger and in the `sys_activity` summary
+  rendered in the record feed. Both now record the mask.
+- **Virtual (`formula`) fields leave the full snapshots.** `ctx.result` carries
+  hydrated formulas (#5504) and the raw pre-image structurally cannot, so
+  create `new_value` would have described a field delete `old_value` could
+  never carry. Only genuinely virtual fields are dropped: `autonumber` and
+  `summary` are stored columns present and equal on both sides, and they stay
+  in the snapshot.
+
+Two consequences worth naming, both narrowing single-id delete to what bulk
+delete already did: its `old_value` now records a file field's stored id rather
+than the resolved `{id, name, size, url}` object, and drops formula values. An
+object whose label field is a formula falls back to the record id in the
+`sys_activity` label on delete for the same reason.
+
+No audit coverage is removed: the plugin keeps its `afterInsert` / `afterUpdate`
+/ `afterDelete` registrations, which is what holds the engine's pre-image demand
+gates open, and every one of them keeps the `excludeObjects` face from #5860.