Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
cf262c2
feat(db)!: key component relations by name; rewrite migration 017
aaaaahaaaaa Sep 7, 2026
8366ee4
feat(db)!: RelationStore binds by name through Relation.accepts
aaaaahaaaaa Sep 8, 2026
c0b1fde
docs(core): docstring corrections parked from the phase 1 review
aaaaahaaaaa Sep 8, 2026
c22a4f5
fix(db): sibling rows come from Source.sibling_bindings; locks on eve…
aaaaahaaaaa Sep 8, 2026
beebb7d
feat(db): component store reads relations by name; guards through on_…
aaaaahaaaaa Sep 8, 2026
0fa57d8
feat(db): hydrate relations by name; parented targets become referenc…
aaaaahaaaaa Sep 8, 2026
711b8e6
feat(api)!: relations are addressed by name; org-wide list filters by…
aaaaahaaaaa Sep 8, 2026
0931530
feat(toolkit): lineage by asset kinds; generic bind_relation and unbi…
aaaaahaaaaa Sep 8, 2026
4717464
feat(agent): bind source connections and destinations by relation name
aaaaahaaaaa Sep 8, 2026
49e3efa
refactor(scheduler)!: hooks match on the watches relation; the DAG jo…
aaaaahaaaaa Sep 8, 2026
5d9f544
chore(platform): migration round trip verified; sweep retired names a…
aaaaahaaaaa Sep 8, 2026
90be2f0
feat(agent): expose bind_relation and unbind_relation on the collecti…
aaaaahaaaaa Sep 8, 2026
6b04683
fix(db): a drifted child surfaces as ComponentDriftError; single-rela…
aaaaahaaaaa Sep 8, 2026
580be73
test(agent): a source without connection relations rejects a connection
aaaaahaaaaa Sep 8, 2026
c116786
style(agent): drop the em-dashes from the collection agent's description
aaaaahaaaaa Sep 8, 2026
030392d
fix(core)!: an optional inferred upstream detaches when its target is…
aaaaahaaaaa Sep 8, 2026
910d5e7
fix(db)!: on_delete alone decides whether a referrer blocks a delete
aaaaahaaaaa Sep 8, 2026
14c4b72
test(db): hydration tests read bindings through the relation attributes
aaaaahaaaaa Sep 9, 2026
af1459e
fix(db): the relation store reads kinds, keys and local as properties
aaaaahaaaaa Sep 9, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/extending/components.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,9 +80,9 @@ destinations a source writes to, a job's targets, an upstream asset.
| `kind` | The component kind, or kinds, the relation may point at. |
| `key` | The keys it narrows to: an exact key, `source.asset`, `*.asset`, a list, or `""` for any key of those kinds. |
| `many` | Whether it binds several components at once. |
| `optional` | Whether it may stay unbound. Says nothing about data. |
| `optional` | Whether it may stay unbound, or be emptied once bound. Says nothing about data, and nothing about deletion. |
| `default` | A zero-argument factory producing the value an unbound relation resolves to. |
| `on_delete` | What deleting the target does to the referrer: `block` (default, for consumption relations) or `detach` (for orchestration pointers such as a job's targets or a hook's watches). |
| `on_delete` | What deleting a bound target does, and the only field that decides it: `block` (the default) refuses the deletion while this relation holds the target, `detach` lets it through and drops the binding. Orchestration pointers (a job's `targets`, a hook's `watches`) and inferred optional upstreams declare `detach`; consumption relations keep `block`, an optional one included. |

Two attributes are derived, not declared: `name` is stamped from the attribute the relation is
declared under, and `target` is the class the `il.Relation(cls)` shorthand was written with,
Expand Down
5 changes: 4 additions & 1 deletion docs/guide/dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,10 @@ class Shop(il.Source):

The parameter name is the relation name and the bare asset key it expects, so
`orders: il.Upstream` declares `il.Relation("asset", "orders")`, a sibling of the same source
instance. A `None` default makes the relation **optional**, meaning its wiring may be absent:
instance. A `None` default makes the relation **optional**, meaning its wiring may be absent, and
declares `on_delete="detach"` with it: a parameter that tolerates a missing leg has no claim on
the upstream, so deleting that upstream is allowed and simply drops the wiring, where a required
upstream refuses the deletion while it is bound.

```py
@il.asset(partitioning=il.TimePartitionConfig(column="date"))
Expand Down
3 changes: 3 additions & 0 deletions docs/guide/resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,9 @@ class WarehouseDestination(il.Destination):
`il.Relation(ReportingConfig)` is shorthand for `il.Relation(kind="config", key="reporting_config")`
with the class kept as the relation's `target`, which is what makes a fallback possible.
`optional=True` allows the relation to stay unbound; `default=` is a zero-argument factory.
Neither says anything about deletion: `on_delete` alone decides that, and a resource relation
keeps its `block` default whether it is optional or not, so a bound connection, config or
destination cannot be deleted while a component consumes it.

## Fallbacks

Expand Down
9 changes: 6 additions & 3 deletions packages/interloper-agent/src/interloper_agent/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ def _catalog_tools() -> list:
description=(
"The organisation's collection of component instances: lists their sources, connections, and "
"destinations, checks connection health, sets up new connections via the app's secure form "
"(never collecting credentials in chat), creates sources conversationally — resolving "
"provider-backed options like the account to use through an existing connection — and edits "
"existing components (rename, config changes, a source's enabled assets)."
"(never collecting credentials in chat), creates sources conversationally (resolving "
"provider-backed options like the account to use through an existing connection), edits "
"existing components (rename, config changes, a source's enabled assets), and binds or "
"unbinds a component's relations by name (a source's connection, a job's targets)."
),
instruction=with_current_time(COLLECTION_INSTRUCTION),
tools=[
Expand All @@ -131,6 +132,8 @@ def _catalog_tools() -> list:
collection.create_source,
collection.create_sources,
collection.update_component,
collection.bind_relation,
collection.unbind_relation,
collection.create_job,
interaction.request_user_selection,
interaction.request_confirmation,
Expand Down
10 changes: 9 additions & 1 deletion packages/interloper-agent/src/interloper_agent/prompts.py
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@
6. Recap with request_confirmation (what and how many; accounts, assets,
connection, destination — "None" when none), then create_sources on
confirm. Report per-account failures and any unresolved cross-source
requirements (those are wired in the app).
requirements, and offer to bind them once the user names the upstream.
7. Offer a schedule: recap the job (name, cadence in words, targets) and
create_job on confirm.

Expand All @@ -124,6 +124,14 @@
just the delta. Connection credentials are never edited in chat — offer a
rename, or point the user to the app (or a fresh connection via the secure
form) for credential changes.

Bind what a component points at with bind_relation, and detach it with
unbind_relation: both take the component's id, the relation name its class
declares (list_components names the component, the catalog specialist the
names it declares), and the target's id. A single-valued name repoints, so
it needs no unbind first; a non-optional one cannot be emptied, only
repointed. Recap the component, the name, and old → new, and act only on
confirmation.
""" + PRESENTATION

CATALOG_CONSULT_INSTRUCTION = """\
Expand Down
134 changes: 105 additions & 29 deletions packages/interloper-agent/src/interloper_agent/tools/collection.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ def update_component(
the stored config is kept; pass null to reset a field to its default.
Connection configs hold credentials and are never edited here: the user
changes those in the app (renaming a connection is fine). Rebinding
relations (a source's connection, a job's targets) also happens in the
app, not through config.
relations (a source's connection, a job's targets) is not config either:
use bind_relation and unbind_relation.

Args:
component_id: UUID of the component, from list_components.
Expand Down Expand Up @@ -144,6 +144,41 @@ def update_component(
return {"status": "error", "error": str(e)}


# -- Relations (generic over kinds) -----------------------------------------------


def bind_relation(
component_id: str,
name: str,
dst_id: str,
tool_context: ToolContext | None = None,
) -> dict[str, Any]:
# Thin ADK wrapper: the implementation (and LLM-facing docstring, adopted
# below) lives in the shared toolkit, which the MCP server deliberately
# does not register, since it writes.
return toolkit_collection.bind_relation(toolkit_ctx(tool_context), component_id, name, dst_id).model_dump(
mode="json"
)


bind_relation.__doc__ = toolkit_collection.bind_relation.__doc__


def unbind_relation(
component_id: str,
name: str,
dst_id: str,
tool_context: ToolContext | None = None,
) -> dict[str, Any]:
# Thin ADK wrapper: see bind_relation above.
return toolkit_collection.unbind_relation(toolkit_ctx(tool_context), component_id, name, dst_id).model_dump(
mode="json"
)


unbind_relation.__doc__ = toolkit_collection.unbind_relation.__doc__


# -- Connection operations (kind-specific by nature) ------------------------------


Expand Down Expand Up @@ -501,46 +536,87 @@ def _source_relations(
source_key: str,
connection_id: str | None,
destination_ids: list[str] | None,
) -> tuple[dict[str, list[tuple[UUID, str]]] | None, dict[str, Any] | None]:
"""Bind the connection into the definition's resource slot, plus destinations.
) -> tuple[dict[str, list[UUID]] | None, dict[str, Any] | None]:
"""Bind the connection into the definition's named relation, plus destinations.

Finds the relations the definition declares whose ``kind`` includes
``"connection"``. A given connection binds under whichever one it fits
(an empty ``key`` accepts any connection, otherwise the connection's key
must appear in the relation's key list); every non-optional connection
relation left unbound afterwards is an error naming the relation and the
key it expects. Destinations bind under the fixed ``"destinations"`` name
once each one is confirmed to belong to the organisation.

Args:
store: The database store, for looking up the connection and
destination rows.
org_id: Organisation UUID the connection and every destination must
belong to.
defn: The source's catalog definition, carrying its declared
``relations`` (name to relation dict: ``kind``, ``key``, ``many``,
``optional``, ``on_delete``, ``name``).
source_key: The source definition's catalog key, named in error
messages.
connection_id: UUID of the connection to bind, or ``None`` to leave
every connection relation unbound.
destination_ids: UUIDs of the destinations to attach, or ``None``.

Returns:
``(relations, None)`` or ``(None, error_response)``.
``(relations, None)``, ``relations`` mapping relation name to the
UUIDs bound under it (ready for ``ComponentStore.create``'s
``relations`` argument), or ``(None, error_response)``.
"""
slots = (((defn.get("relations") or {}).get("resource") or {}).get("slots")) or {}
relations_defn = defn.get("relations") or {}
connection_relations = {
name: relation
for name, relation in relations_defn.items()
if "connection" in (relation["kind"] if isinstance(relation["kind"], list) else [relation["kind"]])
}

connection = None
if connection_id is not None:
connection = store.components.get(UUID(connection_id), kind="connection")
if connection.org_id != org_id:
return None, {"status": "error", "error": f"Connection '{connection_id}' not found"}
resource_bindings: list[tuple[UUID, str]] = []
for slot_name, spec in slots.items():
expected = spec.get("key")
if connection is not None and (not expected or connection.key == expected):
resource_bindings.append((connection.id, slot_name))
connection = None
elif spec.get("required"):

bindings: dict[str, list[UUID]] = {}
if connection is not None:
name = next(
(
relation_name
for relation_name, relation in connection_relations.items()
if not relation.get("key")
or connection.key in ([relation["key"]] if isinstance(relation["key"], str) else relation["key"])
),
None,
)
if name is None:
return None, {
"status": "error",
"error": f"Connection '{connection.key}' does not fit any relation of '{source_key}'",
}
bindings[name] = [connection.id]

for name, relation in connection_relations.items():
if not relation.get("optional") and name not in bindings:
expected = relation.get("key") or "connection"
return None, {
"status": "error",
"error": (
f"'{source_key}' requires a '{expected or 'connection'}' in slot '{slot_name}' — "
f"'{source_key}' requires a '{expected}' as '{name}'; "
"pick one from the collection or set one up first"
),
}
if connection is not None:
return None, {
"status": "error",
"error": f"Connection '{connection.key}' does not fit any slot of '{source_key}'",
}

destination_bindings: list[tuple[UUID, str]] = []
destination_bindings: list[UUID] = []
for dest_id in destination_ids or []:
dest = store.components.get(UUID(dest_id), kind="destination")
if dest.org_id != org_id:
return None, {"status": "error", "error": f"Destination '{dest_id}' not found"}
destination_bindings.append((dest.id, ""))
destination_bindings.append(dest.id)
bindings["destinations"] = destination_bindings

return {"resource": resource_bindings, "destination": destination_bindings}, None
return bindings, None


def _unresolved_requirements(defn: dict[str, Any], row: Any) -> list[str]:
Expand Down Expand Up @@ -576,8 +652,8 @@ def create_source(
name: Display name — default to the label of the chosen account /
discriminator option.
config: Values for the definition's config schema (e.g. account_id).
connection_id: UUID of the connection to bind required when the
definition declares a required connection slot.
connection_id: UUID of the connection to bind; required when the
definition declares a required connection relation.
asset_keys: Child asset keys to enable; omit to enable all.
destination_ids: Destination UUIDs to attach (optional).
"""
Expand Down Expand Up @@ -619,8 +695,8 @@ def create_source(
"key": row.key,
"name": row.name,
"asset_count": len(row.children),
"connection_bound": bool(relations["resource"]),
"destination_count": len(relations["destination"]),
"connection_bound": connection_id is not None,
"destination_count": len(relations["destinations"]),
},
"unresolved_requirements": _unresolved_requirements(defn, row),
}
Expand Down Expand Up @@ -765,12 +841,12 @@ def create_job(
org_id = get_org_id(tool_context)
store = get_store()

targets: list[tuple[UUID, str]] = []
targets: list[UUID] = []
for source_id in target_source_ids:
source = store.components.get(UUID(source_id), kind="source")
if source.org_id != org_id:
return {"status": "error", "error": f"Source '{source_id}' not found"}
targets.append((source.id, ""))
targets.append(source.id)
if not targets:
return {"status": "error", "error": "target_source_ids must name at least one source"}

Expand All @@ -787,7 +863,7 @@ def create_job(
"lookback": lookback,
"offset": offset,
},
relations={"target": targets},
relations={"targets": targets},
)
except (ConfigError, CatalogKeyError) as e:
return {"status": "error", "error": str(e)}
Expand Down
21 changes: 21 additions & 0 deletions packages/interloper-agent/tests/test_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,26 @@
import datetime
from typing import cast

from google.adk.agents.base_agent import BaseAgent
from google.adk.agents.readonly_context import ReadonlyContext

from interloper_agent import agent as agent_module


def _tool_names(agent: BaseAgent) -> set[str]:
"""The names an agent's registered tools carry.

Args:
agent: The agent whose tool list is read.

Returns:
One name per registered tool: a plain function's ``__name__``, or a
tool object's own ``name`` (an ``AgentTool``, say).
"""
tools = getattr(agent, "tools", [])
return {getattr(tool, "__name__", None) or getattr(tool, "name", "") for tool in tools}


def test_with_current_time_appends_now():
provider = agent_module.with_current_time("BASE")
text = provider(cast(ReadonlyContext, None))
Expand All @@ -32,3 +47,9 @@ def test_all_agents_carry_the_current_time():
text = instruction(cast(ReadonlyContext, None))
assert isinstance(text, str)
assert "Current date and time:" in text


def test_the_collection_agent_registers_the_relation_write_tools():
# The toolkit's only write functions: exposed here, never on the
# deliberately read-only MCP server.
assert {"bind_relation", "unbind_relation"} <= _tool_names(agent_module.collection_agent)
Loading
Loading