refactor(spec)!: api is code-only — withdraw a runtime create door the endpoint matcher could never read (#5488)

.changeset/api-runtime-create-withdrawn.md

@@ -0,0 +1,77 @@
+---
+'@objectstack/spec': major
+'@objectstack/metadata-protocol': major
+'@objectstack/objectql': major
+---
+
+refactor(spec)!: `api` is code-only — withdraw a runtime create door the endpoint matcher could never read (#5488, ADR-0049 remove side)
+
+<!-- adr-0087: registered api-runtime-create-withdrawn -->
+
+**FROM → TO:** `PUT /api/v1/meta/api/{name}` (200 "Saved") → declare the endpoint as a
+stack artifact (`**/*.api.ts`, or `defineStack({ apis })`) and ship it through
+`publishPackage`. The runtime write now answers **403 `NOT_CREATABLE`**, in `?mode=draft`
+as well as direct-active. The artifact route is **unchanged** — a `**/*.api.ts` file valid
+before this release is valid after it, byte for byte.
+
+`DEFAULT_METADATA_TYPE_REGISTRY`'s `api` entry declared `allowRuntimeCreate: true` and the
+runtime never honoured it. Measured on a real showcase boot (`objectstack dev --fresh`, 47
+plugins):
+
+```
+PUT /api/v1/meta/api/e8_backdoor   → 200 {"success":true,…,"message":"Saved …"}
+GET /api/v1/apps/showcase/backdoor → 404      (anonymous AND authenticated)
+```
+
+…and **no** `[EndpointMatcher] … EXCLUDED` line anywhere in the boot log: the endpoint was
+not gated out, it was never in the index at all. The serving criterion belongs to
+`IMetadataService.matchEndpoint` → `EndpointMatcher` → `MetadataManager.listForIndex('api')`,
+which reads the manager's own registry plus its registered loaders
+(`["filesystem","memory"]` on dev/serve). A runtime write lands in `sys_metadata`, which is
+in neither. So the declaration promised a capability that could not exist.
+
+A declared-but-unhonoured capability is ADR-0049 false compliance, and "answers Saved, then
+404s forever" is its most dangerous shape for the AI authors ADR-0033 targets. The
+maintainer ruled REMOVE on 2026-08-07 rather than converge the read path: making the matcher
+read `sys_metadata` re-opens cache, invalidation, tenancy and the ADR-0110 D3
+miss-vs-outage distinction on a new read path, and there is no business pull for
+Studio-authored endpoints today — 17.x serves declarative endpoints through stack artifacts,
+which is what showcase uses (#5040 E8, LIVE).
+
+## The retirement kit
+
+- **`allowRuntimeCreate: false`** on the `api` registry entry. With `allowOrgOverride`
+  already `false`, the type is now **code-only** — the `job` / `agent` / `capability` shape —
+  so the existing #5086 inlet refuses before persistence, on every kernel, with
+  `code: 'NOT_CREATABLE'`, `status: 403` and a prescription derived from the entry's own
+  `filePatterns[0]`. No new refusal mechanism was written for this.
+- **`gateApiDraftsForPublish` is retired** (`metadata-protocol`), together with its nine
+  tests and the `PUBLISH_DRAFTS_NAMESPACE_REMEDY` string only it appended. It landed two
+  days earlier in PR #5279 and is removed **deliberately and on the record**, not lost in a
+  refactor: it gated a draft→active promotion into a state the matcher can never read, and
+  with the inlet closed no `api` draft can exist for it to judge. The in-place comment at
+  its old call site carries the reasoning.
+- **The `metadata-plugin.zod.ts` decision block is rewritten as a recorded overturn.** It
+  used to record CODE-ONLY as "considered and rejected"; its three bullets are kept verbatim
+  with what became of each, so the reversal is auditable rather than silently contradicted.
+- **The `api` create seed is removed** and `api` joins `KNOWN_UNSEEDED`. A pre-filled "New
+  API Endpoint" form whose save can only 403 is the UI half of the same false compliance.
+- **Pins, not deletions.** The two #5271 tripwire pins that asserted
+  `allowRuntimeCreate: true` are **replaced** by retirement pins asserting the new verdict —
+  their comments predicted this exact consequence, and both predictions were correct. Every
+  rejection case asserts `code` **and** `status` (ADR-0112 envelope), never `toThrow()`
+  alone (#6142).
+
+## What did NOT change
+
+`validateApiEndpointDeclarations` / `identityFreeEndpointGateFailure` remain the one judge
+of what is servable, on the route that serves: the stack schema, `publishPackage` (#5189),
+and again at load in `buildEndpointIndex` (PR #5203). ADR-0121's "publish REJECTS" ruling is
+intact. `deleteMetaItem` stays ungated so pre-existing rows can be cleaned up, and
+`OS_METADATA_WRITABLE=api` remains the single operator escape hatch — note it unlocks the
+**write** only; the endpoint still will not be served, which is why it is a diagnostic
+rather than a workaround.
+
+**Re-entry path**, recorded by the ruling: if #2657 Part B promotes `apis` to a registered
+type **with a real consumption path**, the flag and the publish gate come back together —
+implementation first, declaration second.

docs/protocol-upgrade-guide.md

@@ -411,6 +411,9 @@ The same descriptor loses a key in this step, and the pairing is the point (#674
 - **`notification-list-cursor-retired`** — `api.listNotifications cursor — the key on BOTH halves of GET /api/v1/notifications (ListNotificationsRequestSchema and ListNotificationsResponseSchema) and the cursor argument of the client SDK call client.notifications.list(). The same entry covers the limit default: the request schema no longer declares default(20)` → a larger `limit` — the route answers the newest N notifications and has no page 2. There is no replacement for `cursor`, deliberately: nothing ever minted one, so no caller holds a value to carry over. Callers that looped on it were re-reading the first window and should read one window sized to what they display (the Console bell polls exactly this way). For the removed `limit` default, send the number you want explicitly if you were relying on 20 — omitting it takes the server window, which is 50 on the platform inbox and clamped into 1..200, and has been since before the declaration existed
   - Why not automatic: One capability, both halves, never half-deleted (maintainer ruling 2026-08-07, Option A, ruled jointly with #6363). `cursor` was declared on the request and on the response and honoured on neither: the dispatcher domain reads `read` / `type` / `limit` and nothing else, and no emit site has ever written the response key. It was worse than inert because it had a shipped PRODUCER — the SDK appended it to the query string — so a caller paginating by the published contract looped on page 1 forever, with no error and no 400. Measured over a real boot with 60 unread before the removal: page2 === page1, both parsing green against the response schema, which is why no conformance gate could see it. This is `data.query.cursor` (#4286, `query-cursor-retired`) one layer up, with the same verdict for the same reason, down to deleting the SDK producer alongside the key. A first-class inbox cursor, if one is ever designed, will be a response-minted opaque token — a different API — so keeping this one preserved a wrong design rather than a roadmap. The `limit` default goes with it because the FICTION WAS THE MECHANISM, not the number: no request path parses a query string through this schema (#3899 wired the catalog's requestSchema to the real entry for BODIES only), so `.default(20)` never stamped anything onto anything, and the server has always applied its own 50. Re-spelling 20 as 50 — the other arm the ruling allowed — would have kept a declaration that does not execute and merely made it coincide with the implementation until someone moved the clamp; `.optional()` plus prose is true about both the schema and the server. No constraint (`.int()` / `.max(200)`) is declared either, because the service CLAMPS an out-of-range limit rather than refusing it, and declaring a rejection the wire does not perform is the same defect mirrored. Route 2, and the split is worth stating exactly because the two halves of the bookkeeping go different ways. There IS a tombstone: both schemas are non-strict, so a bare deletion would have made Zod SILENTLY STRIP whatever a caller kept sending — a clean parse and a parameter that never takes effect, which is this issue's own defect re-created one layer down (#3733, ADR-0104). So `cursor` is `retiredKey()` on both halves, typed `never` for tsc and raising the prescription at any parse, and both keys are registered in RETIRED_KEYS_BY_MAJOR[17]. There is NO D2 conversion: a conversion rewrites an authored source or a stored `sys_metadata` row, and these two shapes are HTTP-only — nobody authors a `ListNotificationsRequest` and nothing persists one. Request AND response shapes: two semantic TODOs for API callers, no stack conversion — the same disposition `BatchOptions.validateOnly` (#4052) and the `AnalyticsQueryRequest` envelope keys already take in this major. The `limit` default is declared separately and mechanically, in DEFAULT_CHANGES_BY_MAJOR[17] (#4666), whose `from`/`to` fingerprints are re-derived on every build. ADR-0049 / ADR-0078, #6361.
   - Done when: No caller sends `cursor` to `GET /api/v1/notifications` and no SDK call site passes it: `client.notifications.list({ cursor })` is a `tsc` error (TS2353, excess property), which is the enforced channel — the removal is loud at compile time for every TypeScript consumer. Reading `response.cursor` no longer type-checks either, and always answered `undefined` before. ⚠️ Behaviour on the wire is deliberately UNCHANGED and must be verified as such: a request still carrying `?cursor=…` is IGNORED, not refused — the domain reads three named query keys and no route validates this query against a schema, so an unknown key has never produced a 400 and does not start doing so here. The declaration stopped promising what the wire never did; the wire did not change. `unreadCount` is untouched (#6363) and still reports the total across the whole matching inbox rather than the window. A caller that omitted `limit` receives the same 50 rows it always received.
+- **`api-runtime-create-withdrawn`** — `PUT /api/v1/meta/api/{name} (runtime-authored `api` endpoints, draft and active alike)` → Declare the endpoint as a stack artifact (`**/*.api.ts`, or `defineStack({ apis })`) and ship it through `publishPackage`
+  - Why not automatic: The `api` registry entry declared `allowRuntimeCreate: true` and the runtime never honoured it. Measured on a real showcase boot (#5488): `PUT /api/v1/meta/api/e8_backdoor` answered 200 with `{"success":true,…,"message":"Saved …"}`, and the declared route then answered 404 forever — with NO `[EndpointMatcher] … EXCLUDED` line, because the endpoint was never in the index to be excluded from. The serving criterion belongs to `IMetadataService.matchEndpoint` -> `EndpointMatcher` -> `MetadataManager.listForIndex('api')`, which reads the manager's registry plus its registered loaders (`["filesystem","memory"]` on dev/serve); a runtime write lands in `sys_metadata`, which is in neither. A declared capability the runtime does not honour is ADR-0049 false compliance, and a write that answers "Saved" and then 404s forever is its most dangerous shape for the AI authors ADR-0033 targets. The maintainer ruled REMOVE on 2026-08-07 rather than converge the read path, because making the matcher read `sys_metadata` re-opens cache, invalidation, tenancy and the ADR-0110 D3 miss-vs-outage distinction on a new read path, and there is no business pull for Studio-authored endpoints today (zero `.api.*` artifacts author them at runtime; showcase uses the artifact route, #5040 E8 LIVE). There is NO D2 conversion, for the reason this list exists: nothing in an authored source spells this key. `allowRuntimeCreate` is a PLATFORM registry value, not an authorable one, and the artifact route it points authors toward is untouched — a `**/*.api.ts` file valid before this change is valid after it, byte for byte. What changed is a runtime HTTP verdict, so it is one semantic TODO for operators and Studio callers rather than a stack conversion — the same disposition `BatchOptions.validateOnly` (#4052) takes. Consequently `gateApiDraftsForPublish` (PR #5279) is retired with it: it gated a promotion into a state the matcher can never read, and with the inlet closed no `api` draft can exist for it to judge. Re-entry is recorded in the ruling: if #2657 Part B promotes `apis` to a registered type WITH A REAL CONSUMPTION PATH, the flag flips back then — implementation first, declaration second. ADR-0049 / ADR-0121, #5488 (subsumes #5311).
+  - Done when: No caller creates or updates an `api` item through the runtime metadata API. `PUT /api/v1/meta/api/{name}` answers 403 with `code: "NOT_CREATABLE"` and a body naming both flags (`allowRuntimeCreate=false, allowOrgOverride=false`) and the prescription `Declare it in source (**/*.api.ts) and redeploy` — in `?mode=draft` as well as direct-active, because the gate runs before the draft/publish branch and does not read `mode`. ⚠️ Verify the artifact route is UNAFFECTED, which is the whole point of the change: a stack declaring `apis:` still compiles, still passes `validateApiEndpointDeclarations` at publish (`publishPackage`, #5189) and at load (`buildEndpointIndex`, PR #5203), and its endpoints still SERVE — that route was always the only one that served. An operator who genuinely needs the runtime door back on one deployment sets `OS_METADATA_WRITABLE=api`, the same single escape hatch `job` / `agent` / `capability` use; note that this unlocks the WRITE only, and the endpoint still will not be served, which is why it is a diagnostic and not a workaround. Any `api` rows already sitting in `sys_metadata` from before this change were never served either; they can be deleted (`deleteMetaItem` is deliberately not gated by this refusal, so repair stays possible).
 - **`import-run-automations-declared-default-corrected`** — `api.ImportRequest runAutomations — the declared default of the key on BOTH import bodies, POST /api/v1/data/:object/import (ImportRequest) and its async twin POST /api/v1/data/:object/import/jobs (CreateImportJobRequest, which IS the same schema object). It was declared default(false) and described as "off by default for bulk"; it is now default(true), which is what the server has always done` → an explicit runAutomations: false on any import request that is meant to load rows without firing triggers/hooks. That spelling is unchanged and has always been the only one the server read — what changes is that omitting the key now DECLARES what it already DID. Callers who want automations on need write nothing
   - Why not automatic: A DECLARATION corrected to match a runtime that did not move — the inverse of a behaviour flip, and registered here for the reason protocol 12's `rest-requireauth-default-flip` and this major's `action-descriptor-resume-authority-default-flip` are: whether a given import was meant to fire triggers is a judgment no transform can make, so the prescription is a TODO rather than a rewrite. The server decides in import-prepare.ts with `body?.runAutomations !== false`, i.e. an omitted flag runs automations, and has since #2922 — automations always ran on import historically (the engine ignored the flag entirely before then), so opt-out was made the explicit act, matching platform convention. The schema said the opposite in both machine-readable and human-readable form, and both SHIPPED: `.default(false)` in `@objectstack/spec`'s JSON Schema, and the describe prose in the published reference tables for both defs. ⚠️ Nothing in this repo reconciled the two and NO deployed caller changes behaviour: no request path parses an import body through this schema — the route reads the raw body, and the sole reference to `CreateImportJobRequestSchema` is the declarative `ImportJobApiContracts` catalog entry, a declaration and not a parse. That is exactly why this needed a ruling rather than a docs edit: the divergence was unobservable in-tree and observable only to a consumer OUTSIDE it. A client or SDK that validated its request through the published schema materialised `runAutomations: false` from the declared default and sent it explicitly, and the server honoured it — so the same request body produced opposite behaviour depending on whether the caller validated before sending, with the validating caller silently losing its triggers. Nothing rejected it, nothing warned, and the reference page told an author the wrong thing in the other direction. There is deliberately NO schema tombstone and no D2 conversion: no key is removed, and an HTTP request body is neither authored nor persisted — the same disposition `notification-list-cursor-retired` (#6361) takes for the sibling default on this major, and `batch-options-validate-only-retired` before it. The declared move itself is recorded mechanically, per key, in DEFAULT_CHANGES_BY_MAJOR[17] (#4666), whose `from`/`to` fingerprints are re-derived on every build. Maintainer ruling 2026-08-09 (#6704, disposition A: the spec follows the runtime). ADR-0049 / ADR-0078.
   - Done when: Every import request of yours that must NOT fire triggers sends `runAutomations: false` explicitly, rather than omitting the key and trusting the old declared default. The check is worth doing precisely where it looks unnecessary: if you build the body by parsing it through `ImportRequestSchema` (or the published JSON Schema) and then send the PARSED object, your bulk loads were running with automations OFF and will now run with them ON — that is the only class whose behaviour changes, and it changes toward what an unvalidated caller always got. ⚠️ Behaviour on the wire is deliberately UNCHANGED and should be verified as such: a body that omits `runAutomations` fired triggers before this change and fires them after, and `runAutomations: false` turns them off before and after. Nothing starts being refused — the route never validated this body against the schema and does not begin to. `dryRun` is unaffected and still runs NO automations whatever the flag says (#6037).