Add config-driven agent output policies#5
Conversation
Agents can declare `output_policies` (module:function refs) in the controller config. The global controller writes them to Redis per agent (agent:<name>:output_policies), and the local controller resolves and runs them automatically after the agent produces a result — no manual on_done registration by the caller. Policies are fire-and-forget side effects: the framework passes the output to each configured policy and ignores its return value, so policies observe/react (audit, notify, enforce externally) but do not modify the result. Framework only finds and runs; all decision logic lives inside the policy.
- Isolate policy loading (Redis read + JSON parse) inside _run_output_policies so a malformed stored value or Redis blip cannot propagate and let _execute_locally overwrite a good agent result. - Add tests exercising the real examples/policies/finance.py through the actual resolution + dispatch path, plus a load-failure isolation test.
Extract _policy_context and set request_id/service/function by direct assignment instead of setdefault, so a caller-controlled request context (from baggage) cannot spoof the service/request_id/function a policy sees. Add tests covering the override and the None-context case.
There was a problem hiding this comment.
Pull request overview
This PR introduces a config-driven agent output policy mechanism: after an agent method returns, the local controller automatically loads and runs configured module:function policy callables for that agent, with failures isolated from the main result write-back path.
Changes:
- Add
output_policiesconfiguration (YAML) and persist per-agent policy refs into Redis from the global controller. - Add lazy-loading + importlib-based resolution and automatic post-execution dispatch of output policies in the local controller.
- Add unit tests covering policy loading, dispatch semantics, failure isolation, caching, and example-policy integration; wire these into the test runner.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| ventis/templates/config/global_controller.yaml | Documents the new optional output_policies agent config. |
| ventis/controller/local_controller.py | Loads policy refs from Redis, resolves callables, and runs output policies after agent execution. |
| ventis/controller/global_controller.py | Writes configured per-agent output policy refs into Redis for consumption by local controllers. |
| tests/test_output_policy.py | Adds self-contained unit tests for output policy loading/dispatch and example-policy resolution. |
| tests/run_tests.sh | Runs the new output-policy unit tests as part of the test script. |
| examples/policies/finance.py | Provides example output policies for auditing and sensitive-token alerting. |
| examples/config/global_controller.yaml | Demonstrates configuring output policies for FinanceAgent. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ctx = dict(context or {}) | ||
| ctx["request_id"] = request_id | ||
| ctx["service"] = service | ||
| ctx["function"] = function | ||
| return ctx |
| # Run any output policies bound to this agent, automatically. | ||
| # Policies react to the result (audit / notify / enforce); they do | ||
| # not modify it, so ordering between them doesn't affect the output. | ||
| policy_ctx = self._policy_context(context, request_id, service, function) | ||
| self._run_output_policies(service, result, policy_ctx) | ||
|
|
| for redis_client in targets: | ||
| redis_client.set(f"agent:{name}:output_policies", policies_json) |
- _policy_context coerces a non-mapping context to empty instead of letting dict() raise (a non-dict context from baggage could otherwise abort _execute_locally and clobber a successful result). - _run_output_policies hands policies a deepcopy of the output so a policy that mutates a dict/list in place cannot change what is serialized and written back; uncopyable outputs skip policies safely. - Add tests for non-dict context coercion and in-place mutation safety.
|
Thanks @copilot — went through the three comments: ✅ Accepted — non-dict ✅ Accepted — in-place mutation of the result ( ❌ Declined — isolating per-Redis write failure in Fixes pushed in 55fcbf9. Full suite: 14/14. |
What
Adds an agent output policy mechanism: after an agent produces a result, the framework automatically runs the policies configured for that agent — no manual
on_donefrom the caller.output_policies(a list ofmodule:functionrefs) inglobal_controller.yaml.GlobalController._write_output_policies()writes them to Redis per agent (agent:<name>:output_policies), mirroring_write_resource_specs/_load_and_write_policies.LocalController._run_output_policies()resolves the refs viaimportliband invokes each one in_execute_locally, right after the agent returns. Policy exceptions, unresolvable refs, and policy-loading failures (bad Redis value / Redis blip) are all isolated (logged, skipped) so they can never corrupt the agent's result write-back.The framework only finds and runs the configured policies; the decision logic lives entirely inside each policy.
Risk / deliberate limitation
Output policies are fire-and-forget side effects. The framework passes the agent output to each policy but ignores the return value, and isolates any exception a policy raises. Consequences:
This is an intentional simplification chosen over a chained/transform model, which was rejected because chaining makes correctness depend on YAML ordering (easy to misconfigure). If transform (rewrite the result) or gate (deny/abort) semantics are needed later, the return-value / exception contract must be explicitly extended — the hook is already placed before the result is written to Redis to leave room for that.
Test
tests/test_output_policy.py(self-contained, stubs gRPC/Redis) — 10 cases: all policies see the same output, return value ignored, order-independent, exception isolated, unresolvable ref skipped, loading failure isolated, no-config no-op, resolution cached, and two real-module tests that resolve + run the actualexamples/policies/finance.pythrough the real dispatch path. Wired intotests/run_tests.sh.TODO (follow-up PR — not included here)
Because
ventis build/deploydoesn’t ship the policies/directory into the agent Docker image, import policies.financefails inside the container and output policies never run there; they only work in local (non-container) runs for now.