Skip to content

feat(store): configured namespaces with stores of their own, placed by a resolver - #663

Open
yzabelle-go wants to merge 6 commits into
mainfrom
yzabellego/fd-5441-p2-sdk-restricted-namespace
Open

yzabelle-go wants to merge 6 commits into
mainfrom
yzabellego/fd-5441-p2-sdk-restricted-namespace

Conversation

@yzabelle-go

@yzabelle-go yzabelle-go commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

FD-5441 phase 2. A deployment can now keep some entities, and everything about them, in stores of their own. A [namespaces.<name>] table names a document store and an object store, and a [namespace_resolver] says which ids are in which namespace. Ids derived from one (<id>__…) follow it.

What goes to a namespace's stores:

  • its entities;
  • everything a run of one of its tasks records and writes: run records, conversations, trajectories, snapshots, recordings, and any entity or object the run creates.

Objects go under a <name>/ key prefix, so namespaces can share a bucket.

Reads that no id places, such as lists or lookups by instance id, see only the shared store and the running task's namespace, unless a service opens more with reading_namespaces(...). That lets a service show a namespace to its members only.

A read by id is not scoped by reading_namespaces: it goes straight to the store that holds the id. So who may read restricted data is the hub's gate to decide, not this layer's.

Writes ask the resolver for a fresh answer and are refused if it can't give one. A run in a namespace never writes a new version of a shared entity, never updates, replaces or deletes a shared entity in place (as an @local run can't; it can still read it), never updates a shared run record, and never writes objects outside its namespace's store. An entity a run creates whose id the resolver places nowhere (such as <instance>__recording) keeps every later version, document and objects, in that run's namespace: anything else that writes it, a write outside the run or a run in another namespace, is refused.

This does nothing until a [namespaces] table is configured. Task steps need no edits: get_object_store() was already run-scoped for @local runs and now is for configured namespaces too. TableNamespaceResolver, which lists ids in config, is the in-tree resolver, used by the tests.

Limits:

  • Docker images for a namespace's envs still go to the shared image store.
  • A snapshot needs its env and its universe in the same namespace.

Validation:

  • New tst/unit/store/configured_namespace_test.py (152 tests): routing, fail-closed writes, guards, objects, an artifact round trip, and the document-store conformance suite in three modes.
  • The full unit suite passes locally, except 3 tests that need Linux tools or click in a subprocess's Python, and 1 that hangs on this machine's git credential helper. All 4 behaved the same before this change.
  • CI ran unit, integration-local, integration-local-slow (12 shards) and integration-local-gateway (5 shards); all pass.
  • Not run: a real Mongo or S3 backend.

Part of a three-PR set for restricted storage:

Merge order: this first, then a release. The other two don't need this one to merge, but they do nothing until it is released and pinned.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The changes appear safe to merge.

Summary

This PR adds separate document and object stores for configured namespaces, with a resolver deciding where each entity belongs.

  • Configured namespaces keep selected entities and task records in separate document stores.
  • Namespace tasks keep their objects in their own object stores.
  • Snapshot capture requires its environment and universe to share a namespace.

The previous findings are addressed. No new actionable issues were found in the changes since the previous review.

Reviews (9) · Last reviewed commit: "fix(store): keep a run-created id's late..." · Reviewed by Greptile

yzabelle-go and others added 2 commits November 13, 2026 11:42
An @<name>/ id in a namespace configured under [namespaces.<name>] (document and
object tables) keeps to that namespace's stores: its entities by id, what a run of
its tasks records, and every entity and object that run writes, objects under a
<name>/ key prefix. Reads no id places see the configured store and the running
task's namespace, plus only the namespaces reading_namespaces() opens, so a service
decides who reads which. A shared entity may not name a namespace's ids, a run never
writes a new version of a shared entity, and a run's objects never go to the
shared store. Routing is inert with no [namespaces] table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A [namespace_resolver] (module:Class, namespace_of(entity_id, fresh=)) now says which
ids are in which configured namespace, instead of an @<name>/ id prefix. Ids derived
from a placed id (<id>__...) follow it (id_bases). Writes ask the resolver fresh and
fail when it raises; a run asks once where its task is, an enclosing run's namespace
winning. Configured stores refuse every @ id again, key_segment is unchanged from main,
and the reference guard (a shared document naming a namespaced id) is gone: with a
resolver it would cost a lookup per string written. TableNamespaceResolver lists ids
in config, for tests and small deployments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yzabelle-go
yzabelle-go requested a review from a team as a code owner November 13, 2026 17:01
Comment thread src/agent_env/config/runtime.py Resolved
Comment thread src/agent_env/store/routing.py Resolved
yzabelle-go and others added 3 commits November 13, 2026 12:08
…d every store a multi-id filter names

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A run in a configured namespace no longer updates, replaces or deletes an
entity in the configured store; it still reads it. Matches @Local, whose run
can't change a configured-store record either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/agent_env/store/routing.py Resolved
Comment thread src/agent_env/store/ids.py Resolved
An entity a run creates whose id the resolver places nowhere, such as an
instance's recording, now takes later versions, document and objects, only
from a run in that namespace. A write outside the run or from a run in
another namespace is refused before any bytes are uploaded, so it can't land
in the shared store. Also names the namespace name length limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant