Skip to content

fix(objectql,driver-mongodb): declare the tenant index in indexes[], so a registry-backed object stops reporting itself invalid (#6810) - #6812

Merged
os-zhuang merged 4 commits into
mainfrom
claude/issue-6810-organization-id-indexed-key
Aug 8, 2026
Merged

fix(objectql,driver-mongodb): declare the tenant index in indexes[], so a registry-backed object stops reporting itself invalid (#6810)#6812
os-zhuang merged 4 commits into
mainfrom
claude/issue-6810-organization-id-indexed-key

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Fixes #6810

applySystemFields provisioned the injected organization_id with
indexed: opts.multiTenant. indexed is not a FieldSchema key#2377 /
ADR-0049 removed it because a field-level index flag built no index — and
FieldSchema is a strictObject, so a field carrying it is rejected by name.
registerObject runs applySystemFields before storing and
getItem('object', …) serves that post-injection document, so the key reached
/meta, where decorateMetadataItem re-parsed the served body and stamped the
verdict on it.

Direction A per the PM ruling on the issue: stop stamping a key the schema
rejects, declare the tenant index where every other index in this system is
declared (the object's indexes[]), and teach the single consumer to read it
there. B (strip-before-serve) and C (re-declare indexed on
FieldSchema) were not needed — the premise held, see the table below.

Premise verification (the falsifiable part, checked first)

# Premise Verdict Evidence
a indexes[] can express what indexed: opts.multiTenant meant holds IndexSchema is { name?, fields[], unique? } (packages/spec/src/data/object.zod.ts). multiTenant: true{ fields: ['organization_id'] }; unique defaults to false, so a plain lookup index needs no key at all. multiTenant: falsedeclare nothing, which is what the PM predicted: nothing filters by organization on an unwalled stack, so an index declared false and an index not declared are the same fact, and indexes[] spells it as absence.
b driver-mongodb reaching the index through indexes[] produces the same DDL outcome holds Old: field.indexedcreateIndex({organization_id: 1}, { name: 'idx_organization_id' }). New: the declaration generates idx_<fields.join('_')>, i.e. the same name and same key spec, so a re-synced collection finds its existing index rather than building a second one. Pinned in mongodb-schema-declared-indexes.test.ts ("byte-identical to what the retired flag built"). name is deliberately not set in the declaration: SQL index names are schema-global and must be table-qualified (buildIndexNameidx_<table>_organization_id), Mongo's are per-collection — one hardcoded name cannot be right for both, so each driver derives its own.
c no other reader of field.indexed exists holds, with two fixture sites the filer's count did not include Repo-wide grep for the exact key: the only runtime reader was driver-mongodb/src/mongodb-schema.ts:87, as filed. Three objectql test files asserted the flag (registry.test.ts, registry-tenancy-posture.test.ts, injected-system-columns-parity.test.ts) and driver-mongodb's README + mongodb-driver.test.ts used it in a fixture (email: { type: 'email', indexed: true }) — that fixture was silently relying on a key the spec rejects, so it is rewritten to indexes: [{ fields: ['email'] }], producing the same idx_email.

A note the premise check turned up, since it changes what "same DDL outcome"
means at the fleet level: driver-sql — which every walled deployment runs —
never read the flag at all. It only ever materialized indexes[]. So the
tenant index has not existed on any SQL deployment regardless of what
indexed: true said, and this PR is the first time the intent is actually
enforced there. Expect it to appear as ordinary index drift on existing tables
(idx_<table>_organization_id), created by os migrate apply or the
autoMigrate: 'safe' path in dev, like any other declared index. Called out in
the changeset.

What changed

No packages/spec / FieldSchema change — direction C would restore exactly
the declared-but-unenforced key #2377 removed.

The #6562 tripwire, flipped

#6562 (PR #6811) merged into main while this branch was in flight — its
tripwire file did not exist at my base commit, so origin/main was merged in
(one conflict, in registry.ts, resolved onto #6562's new shared
TENANT_SCOPE_FIELD_DEF) and all three pinned lines flipped to the post-fix
truth. The old expectations are quoted in place in the test comments, so the
reversal reads as a record rather than a rewrite (PD #13). Nothing was deleted.

divergences(registryBacked, overlayBacked)   ['organization_id.indexed'] → []
registryBacked._diagnostics                  valid: false                → { valid: true }
registryBacked.…organization_id.indexed      === multiTenant             → undefined,
                                                                           read off indexes[]

One new residual is recorded there rather than left to be rediscovered: the
declaration does not converge the way the field set does — the overlay-backed
answer is rebuilt from the stored body, which declares no indexes. It is inert on
that surface (a driver materializes from the REGISTERED schema, never from a
served document — the same reasoning #6562 used to leave the flag at the
injection site) and both answers parse green either way. Asserted explicitly, so
the day a served-document consumer of indexes[] appears, there is a line that
says so.

Acceptance pins

Driven through the real SchemaRegistry + the real
ObjectStackProtocolImplementation — the filer's exact measurement, inverted:

Pin Where
_diagnostics: { valid: true } at both /meta exits, in both tenancy modes registry-tenant-index-declaration.test.ts
No served field carries indexed — asserted across every injected column, not just the tenant one same
The tenant index is actually declared ([{ fields: ['organization_id'] }]), not merely the key removed same
multiTenant=false declares no tenant index and still parses green same
The served field is otherwise byte-identical to the pre-fix one, minus the key same
The index is actually created: createIndex({organization_id: 1}, { name: 'idx_organization_id' }) mongodb-schema-declared-indexes.test.ts

The Mongo DDL assertion is driven against a fake Db on purpose: this package's
mongodb-memory-server suite is opt-in (it downloads a server binary, #5517), so
a DDL pin parked there would not run on any ordinary CI lane — which is exactly
the lane that has to notice a regression. Its fake engine opens both write verbs
with assertEngineDeleteDispatch / assertEngineUpdateDispatch, as
check:engine-double-contract requires.

Reverse verification — predicted, then run

Prediction written before reverting; both source files (registry.ts,
mongodb-schema.ts) restored to origin/main with the tests kept at HEAD.

File Predicted Actual
objectql/registry-tenant-index-declaration.test.ts 9 fail / 3 pass 9 fail / 3 pass
objectql/registry.test.ts 2 fail 2 fail
objectql/registry-tenancy-posture.test.ts 4 fail / 3 pass 4 fail / 3 pass
objectql/injected-system-columns-parity.test.ts 1 fail 1 fail
driver-mongodb/mongodb-schema-declared-indexes.test.ts 4 fail / 2 pass 4 fail / 2 pass
total 20 failing 20 failing

Signature-level, the reverted run reproduces the filer's measurement verbatim at
both exits and in both tenancy modes:

AssertionError: expected { valid: false, errors: [ { …(3) } ] } to deeply equal { valid: true }
+   "errors": [
+     {
+       "code": "unrecognized_keys",
+       "message": "Unrecognized key(s) on this field: `indexed`.
+   • never a FieldSchema key; a field-level index flag built no index (#2377). Declare
+     the index in the object's `indexes[]`. …",
+       "path": "fields.organization_id",
+     },
+   ],
+   "valid": false,

The three surviving-green cases are the ones that should survive: multiTenant=false
declares no index either way, an author's own tenant index is not duplicated either
way, and a systemFields: false object is untouched either way.

Fix restored, working tree verified clean against HEAD afterwards.

Gates — enumerated fresh from .github/workflows/lint.yml, run one by one

All green locally, on a full build first.

  • pnpm lint
  • Lint job checks (32): slot-lookup · query-options-erasure ·
    verify-stand-in · nul-bytes · doc-authoring · docs-audit-scope ·
    role-word · quick-reference-counts · adr-anchors · org-identifier ·
    authz-resolver · service-providers · route-envelope ·
    error-code-casing · wildcard-fallthrough · meta-type-normalized ·
    init-service-contract · durability-log-level · startup-registry-verdict ·
    objectui-changeset · release-notes · release-body · node-version ·
    workflow-status-functions · shard-attestation · published-files ·
    engine-double-contract · kernel-hook-pairs · resume-authority-declared ·
    driver-memory-census · merge-driver · spec-parsed-alias — all ✅
  • Type-check job: check:type-check-coverage ✅ · check:driver-conformance ✅ ·
    check:stall-guard ✅ · spec tsc --noEmit ✅ ·
    spec check:generated --reconcile-only ✅ · spec check:skill-docs ✅ ·
    spec check:spec-changes ✅ · spec check:upgrade-guide ✅ ·
    spec check:authorable-surface ✅ · spec check:docs ✅ ·
    spec check:skill-refs ✅ · check:skill-frame-sync ✅ ·
    check:skill-compatibility ✅ · spec check:react-blocks ✅ ·
    turbo typecheck (120/120) ✅ · check:type-check-debt ✅ (informational
    "can be lowered" lines only; nothing above its recorded number)
  • Full suite: turbo run test135/135 tasks, exit 0

check:engine-double-contract was the one gate that went red on the first pass —
the new fake engine's delete/update were not routed through the producer's
dispatch predicates. Fixed in 915359e, re-run green.

origin/main was merged twice during this work (5e247fd for #6562, then
6de592c for #6809/driver-sql). The second merge was clean with no overlap
against this diff; the overlap packages (objectql, driver-sql,
driver-mongodb, metadata-protocol, metadata-core) were rebuilt and re-tested
after it — 19/19 ✅. packages/spec did not move on either side, so no generated
artifact is in play.


Generated by Claude Code

claude added 2 commits August 8, 2026 20:17
…#6810)

`applySystemFields` provisioned the injected `organization_id` column with
`indexed: opts.multiTenant`. `indexed` is not a `FieldSchema` key — #2377 /
ADR-0049 removed it because a field-level index flag built no index — and
`FieldSchema` is a `strictObject`, so a field carrying it is rejected by name.

`registerObject` runs `applySystemFields` before storing and
`getItem('object', …)` serves that post-injection document, so the key reached
`/meta`, where `decorateMetadataItem` re-parsed the served body and stamped
`_diagnostics: { valid: false, errors: [{ path: 'fields.organization_id',
code: 'unrecognized_keys' }] }` on every registry-backed object — both tenancy
modes, both read exits. That is the channel Studio renders invalid-metadata
banners from and an AI author reads to judge its own document, so the platform
was reporting a defect on its own column and drowning real authoring errors.

The tenant index is now declared in the object's `indexes[]`, where every other
index in this system is declared: `{ fields: ['organization_id'] }` on a
multi-tenant stack, nothing at all on a single-tenant one (absence is what
`indexed: false` meant). `driver-mongodb` — the sole reader of the retired flag
— reads declared indexes instead, generating the same index name it used to, so
a re-synced collection finds its existing `idx_organization_id`. `driver-sql`
already materialized `indexes[]`, so this is the first time the intent is
enforced there at all.

No `FieldSchema` change: re-declaring `indexed` would restore exactly the
declared-but-unenforced key #2377 removed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JE8Bbwb8qhau3yNtP3ftLJ
#6562 (PR #6811) landed while this branch was in flight and factored the tenant
column into `TENANT_SCOPE_FIELD_DEF`. Conflict resolved onto that shared table:
the definition is now spread VERBATIM, with the `indexed` key gone and the
tenant index declared in `indexes[]`.

`protocol-meta-effective-schema.test.ts` carries the three-line tripwire #6562
pinned in both directions so this fix could not be forgotten. All three flip to
the post-fix truth, annotated with #6810 and with the old expectation quoted in
place so the reversal reads as a record, not a rewrite:

  divergences(...)            ['organization_id.indexed'] → []
  registryBacked._diagnostics valid: false                → { valid: true }
  organization_id.indexed     === multiTenant             → undefined, read
                                                            off `indexes[]`

One residual is recorded rather than left to be rediscovered: the DECLARATION
does not converge the way the field set does — the overlay-backed answer is
rebuilt from the stored body, which declares no indexes. Inert on that surface
(drivers materialize from the REGISTERED schema, never a served document) and
both answers parse green either way.

`metadata-core`'s `injected-system-columns.ts` header described the stamped key
as live; corrected to describe it as closed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JE8Bbwb8qhau3yNtP3ftLJ
@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 8, 2026 9:00pm

Request Review

@github-actions github-actions Bot added the size/l label Aug 8, 2026
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 3 package(s): @objectstack/driver-mongodb, @objectstack/metadata-core, @objectstack/objectql.

17 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/metadata-lifecycle.mdx (via @objectstack/metadata-core, @objectstack/objectql)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/driver-mongodb)
  • content/docs/data-modeling/formulas.mdx (via packages/objectql)
  • content/docs/deployment/migration-from-objectql.mdx (via @objectstack/objectql)
  • content/docs/deployment/vercel.mdx (via @objectstack/objectql)
  • content/docs/getting-started/glossary.mdx (via @objectstack/driver-mongodb)
  • content/docs/kernel/runtime-services/examples.mdx (via packages/objectql)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/driver-mongodb, @objectstack/objectql)
  • content/docs/kernel/services.mdx (via @objectstack/objectql)
  • content/docs/permissions/authentication.mdx (via @objectstack/objectql)
  • content/docs/plugins/index.mdx (via @objectstack/objectql)
  • content/docs/plugins/packages.mdx (via @objectstack/driver-mongodb, @objectstack/metadata-core, @objectstack/objectql)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/objectql)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/driver-mongodb, packages/objectql)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/objectql)
  • content/docs/releases/implementation-status.mdx (via @objectstack/driver-mongodb, @objectstack/objectql)
  • content/docs/releases/v12.mdx (via @objectstack/metadata-core)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Aug 8, 2026
claude added 2 commits August 8, 2026 20:51
…tch guards (#6810)

`check:engine-double-contract` pins every engine double to ObjectQL's own
`delete`/`update` dispatch predicates — a fake looser than the producer is how
#4434 shipped a dead REST route with its suite green. The fake in
`registry-tenant-index-declaration.test.ts` now routes both verbs through
`assertEngineDeleteDispatch` / `assertEngineUpdateDispatch` from
`@objectstack/metadata-core`, matching the pinned fake in
`protocol-meta-effective-schema.test.ts` next to it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JE8Bbwb8qhau3yNtP3ftLJ
@os-zhuang
os-zhuang marked this pull request as ready for review August 8, 2026 22:25
@os-zhuang
os-zhuang added this pull request to the merge queue Aug 8, 2026
Merged via the queue into main with commit e13fd91 Aug 8, 2026
26 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-6810-organization-id-indexed-key branch August 8, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants